Vision-based object counting method, apparatus and device

Abstract

A vision-based object counting method includes: identifying P candidate regions in an image; extracting a contour of a candidate object in a candidate region among the P candidate regions; determining whether a difference between the contour of the candidate object in the candidate region and a contour of a template object is less than a second threshold value; if the difference between the contour of the candidate object in the candidate region and the contour of the template object is less than the second threshold value, segmenting the contour of the candidate object in the candidate region into a first segment set; determining whether a proportion of segments in the first segment set that can be matched to at least one segment in a second segment set in the first segment set is greater than a third threshold value; if the proportion of the segments in the first segment set that can be matched to the at least one segment in the second segment set in the first segment set is greater than the third threshold value, extracting key points in the contour of the candidate object in the candidate region, and counting a target object according to the extracted key points.

Description

Technical Field

[0001] This invention relates to the field of object counting technology, and specifically to a vision-based object counting method, apparatus, and device. Background Technology

[0002] In numerous fields such as industrial automation, quality inspection, smart warehousing, document digitization, and biological microscopic analysis, the rapid and accurate identification and counting of specific shapes or objects in images or video streams is a fundamental and crucial task. For example, in the electronics manufacturing industry, it's necessary to count the number and location of specific components on circuit boards; in the packaging industry, it's necessary to detect the number and orientation of specified markings on printed materials; and in agriculture, it may be necessary to count and estimate the posture of collected fruit images. Traditional solutions mainly rely on manual visual inspection, a method that is inefficient, subjective, prone to fatigue, and unable to meet the high precision and speed requirements of modern industry. Summary of the Invention

[0003] This invention provides a vision-based object counting method, apparatus, and device that can achieve object counting with high robustness, efficiency, and accuracy.

[0004] Firstly, a vision-based object counting method is provided, the method comprising: in an image to be detected... Identify P candidate regions, where each of the P candidate regions includes candidate objects whose contours differ from the contours of the template object by less than a first threshold, where P is a positive integer, and the template object belongs to the target object; extract the candidate regions from the P candidate regions. The outline of the candidate object is used to determine the candidate region. The candidate region is determined by whether the difference between the outline of the candidate object and the outline of the template object is less than a second threshold, wherein the second threshold is less than the first threshold. Let P be any candidate region among the candidate regions; if the candidate region If the difference between the outline of the candidate object and the outline of the template object is less than the second threshold, then the candidate region is... The contour of the candidate object is segmented into a first set of segments, which includes M segments, where M is a positive integer greater than 1. It is determined whether the proportion of segments in the first set that can match at least one segment in a second set is greater than a third threshold. The second set of segments is obtained by segmenting the contour of the template object and includes M segments. If the proportion of segments in the first set that can match at least one segment in the second set is greater than the third threshold, then the candidate region is extracted. The key points in the contour of the candidate object are used to obtain the key point set. Based on the set of key points Key points and template key point set The key point pairs in the template are used to count the key point relationships between key points in the target object. The key point is that it is extracted from the outline of the template object.

[0005] In one possible implementation, the method further includes: acquiring an image containing the template object. From the image Extract the outline of the template object; obtain the first global shape context feature of the template object based on the features of multiple outline points in the outline of the template object; determine the candidate region. Whether the difference between the outline of the candidate object and the outline of the template object is less than a second threshold includes: based on the candidate region The candidate region is obtained by analyzing the features of multiple contour points in the contour of the candidate object. The second global shape context feature of the candidate object; the candidate region is determined based on the first global shape context feature and the second global shape context feature. Whether the difference between the outline of the candidate object and the outline of the template object is less than a second threshold.

[0006] In one possible implementation, determining whether the proportion of segments in the first segment set that can be matched with at least one segment in the second segment set is greater than a third threshold includes: calculating the segments in the first segment set... The fragment features, wherein the fragment The fragment features include: the fragment The segment chord length feature, segment orientation histogram, segment shape context, or any one or more of these; based on the segment Based on the fragment features, determine whether there exists a fragment in the second fragment set that matches the fragment. The fragment that matches the fragment.

[0007] In one possible implementation, the extraction of the candidate region... The key points in the contour of the candidate object are used to obtain the key point set. Includes: obtaining the candidate region Points in the contour of the candidate object ,point and ,in, and Located at point Both sides, and points and The interval between them There are 10 points, of which 10 are points. It is a positive integer; calculate the point. Time The first vector, and the calculation Time The second vector; calculate the cosine value between the first vector and the second vector; determine the point based on the cosine value. This is the key point.

[0008] In one possible implementation, the key point set... Key points and template key point set The key point pair relationship between key points in the data includes: obtaining the set of key points. Key points The direction of the first contour tangent at the location, and the acquisition of the template key point set. Key points The direction of the second contour tangent at the location; calculate the key points. and key points The first angle between the chord formed and the tangent direction of the second contour; obtain the set of key points. Key points The direction of the third contour tangent at the location, and the acquisition of the template key point set. Key points The direction of the fourth contour tangent at the location; calculate the key points. and key points The second included angle between the chord formed and the tangent direction of the fourth contour; the key point pair relationship is determined based on the tangent direction of the first contour, the tangent direction of the second contour, the tangent direction of the third contour, the tangent direction of the fourth contour, the first included angle, and the second included angle.

[0009] In one possible implementation, the method further includes: based on the set of key points The set of key points in the template The pose parameters of the candidate object are determined by identifying keypoints that have keypoint pairs with each other, wherein the pose parameters include translation and rotation.

[0010] Secondly, a vision-based object counting device is provided, the device comprising: a recognition unit for identifying objects in an image to be detected. The system identifies P candidate regions, where each of the P candidate regions includes a candidate object whose contour differs from the contour of the template object by less than a first threshold, where P is a positive integer, and the template object belongs to the target object; a first extraction unit is used to extract the candidate regions from the P candidate regions. The outline of the candidate object is used to determine the candidate region. The candidate region is determined by whether the difference between the outline of the candidate object and the outline of the template object is less than a second threshold, wherein the second threshold is less than the first threshold. For any candidate region among the P candidate regions; a segmentation unit, used to segment if the candidate region If the difference between the outline of the candidate object and the outline of the template object is less than the second threshold, then the candidate region is... The contour of the candidate object is segmented into a first set of segments, which includes M segments, where M is a positive integer greater than 1. A judgment unit is used to determine whether the proportion of segments in the first set that can match at least one segment in a second set is greater than a third threshold, wherein the second set of segments is obtained by contour segmentation of the template object and includes M segments. A second extraction unit is used to extract the candidate region if the proportion of segments in the first set that can match at least one segment in the second set is greater than the third threshold. The key points in the contour of the candidate object are used to obtain the key point set. ; Counting unit, used to count the key points set Key points and template key point set The key point pairs in the template are used to count the key point relationships between key points in the target object. The key point is that it is extracted from the outline of the template object.

[0011] Thirdly, a computing device is provided, including a memory and a processor; the memory stores computing instructions; the processor executes the computing instructions, causing the computing device to perform the method provided in the first aspect.

[0012] Fourthly, a computer-readable storage medium is provided, including computer program instructions that, when executed by a computing device, perform the method provided in the first aspect.

[0013] Fifthly, a computer program product is provided, including a computer program that, when executed by a processor, implements the method provided in the first aspect.

[0014] The solution provided by this invention adopts a multi-level matching method to determine whether a candidate object is a target object, and then counts the target objects, which has high robustness, counting accuracy and counting efficiency. Attached Figure Description

[0015] Figure 1 This is a flowchart of an object counting method provided by the present invention;

[0016] Figure 2 This is a flowchart of an object counting method provided by the present invention;

[0017] Figure 3 This is a schematic diagram illustrating the construction of a multi-level template feature library provided by the present invention;

[0018] Figure 4 This is a schematic diagram of the structure of an object counting device provided by the present invention;

[0019] Figure 5 This is a schematic diagram of the structure of a computing device provided by the present invention. Detailed Implementation

[0020] The following examples are for illustrative purposes only and should not be considered as limiting the scope of the invention.

[0021] Before further describing specific embodiments of the present invention, it should be understood that the scope of protection of the present invention is not limited to the specific embodiments described below; it should also be understood that the terminology used in the embodiments of the present invention is for describing specific embodiments and not for limiting the scope of protection of the present invention; in the specification and claims of the present invention, unless otherwise expressly stated in the text, the singular forms "a", "an" and "this" include the plural forms.

[0022] When numerical ranges are given in the embodiments, it should be understood that, unless otherwise stated in the present invention, both endpoints of each numerical range and any value between the two endpoints may be selected. Unless otherwise defined, all technical and scientific terms used in this invention have the same meaning as commonly understood by one of ordinary skill in the art. In addition to the specific methods, apparatus, and materials used in the embodiments, based on the knowledge of the prior art possessed by one of ordinary skill in the art and the description of this invention, any prior art methods, apparatus, and materials similar to or equivalent to those described, apparatus, and materials in the embodiments of this invention may be used to implement the present invention.

[0023] With the rapid development of computer vision technology, image processing-based automatic counting methods have gradually become mainstream. The core objective of these methods is to separate foreground objects from complex backgrounds, identify object instances similar to preset templates (i.e., "registered graphics"), and accurately obtain the spatial position (coordinates) and rotation angle (direction) of each instance in the image coordinate system. Position and angle information are crucial for subsequent operations such as mechanical grasping, automatic sorting, and assembly alignment. Therefore, a robust, efficient, and accurate graphic counting and pose estimation method has extremely high industrial application value.

[0024] Various machine vision solutions have been applied to image recognition and counting tasks, among which the mainstream solutions can be divided into two main categories: feature matching-based methods and deep learning-based object detection methods.

[0025] The core process of a feature-matching-based solution (taking feature point matching as an example) is as follows:

[0026] First, template registration is performed: a clear and standard image to be detected is manually selected as the template. Feature point detection algorithms (such as SIFT, SURF, ORB, etc.) are used to extract a set of key points and their corresponding feature descriptors from the template image. Each key point describes a stable and unique structure (such as a corner or patch) of a local region of the image, while the descriptor is a high-dimensional vector used to characterize the texture, gradient, and other appearance information of the region surrounding the key point. Simultaneously, a reference point (usually the geometric center of the image or a significant feature point) and a reference direction (such as the major axis direction or the direction of one side of the image) are manually or semi-automatically defined in the template image.

[0027] Secondly, in the real-time detection phase, for each input image to be detected, the same feature point detection algorithm is used to extract keypoints and descriptors. Subsequently, feature matching pairs are established between keypoints in the input image and template keypoints using similarity metrics between descriptors (such as Euclidean distance and Hamming distance). This process typically employs nearest neighbor search and ratio testing to filter out reliable matching pairs.

[0028] Assume the keypoint set of the template image T is Its corresponding descriptor subset is The keypoint set of the input image I is: The subset is described as For each keypoint in the input image In the template description subset Searching for its descriptor The two template descriptors that are closest and second closest to each other and Calculate the nearest distance. and the second closest distance ratio If the ratio is less than a preset threshold... (Usually taken as 0.7-0.8), then it is considered and A match was found, forming a matching pair. .

[0029] After obtaining a set of initial matching pairs Next, geometric constraints are needed to filter out erroneous matches (outside points) and estimate the pose of the figure. The most commonly used method is the random sample consensus algorithm. Its basic idea is to randomly select a minimum sample set from the matching set M (e.g., 2 pairs of matching points are needed for similarity transformations, and more are needed for affine or projective transformations), and calculate a hypothetical transformation matrix. Then, this transformation matrix is ​​used to test all other matching points, and the number of interior points that conform to the transformation model is counted. This process is repeated multiple times, and finally, the transformation model with the largest number of interior points is selected. This model describes the geometric mapping from the template coordinate system to the region in the input image where the target instance is located.

[0030] Based on the optimal transformation model Using the reference points defined in the template and their corresponding interior point set, the position and angle of the target in the input image can be calculated. pass Transform to the input image coordinate system to obtain the position of the target center. For angle calculations, the reference direction vector defined in the template is typically used. (For example, a vector pointing from a reference point to another feature point of the graph) via Transformation yields the direction vector in the input image. (Note: For transformations involving rotation, usually only take...) (The rotating part in the middle). Then calculate Angle relative to the horizontal axis of the image This represents the angle of the target. Finally, iterate through all instances that pass geometric verification and whose spatial locations do not overlap (or pass non-maximum suppression), outputting a list of their coordinates and angles to complete the counting.

[0031] The deep learning-based object detection scheme (taking a single-stage detector as an example) transforms the image counting problem into an object detection problem. Its core process is as follows:

[0032] During the training phase, a large amount of labeled dataset needs to be prepared. In each training image, all target shapes to be detected are bounded by a rotated bounding box (usually represented as...). ,in The coordinates of the center point, For the width and height of the frame, The bounding box is labeled with either the rotation angle or the horizontal bounding box (where the angle information may be regressed or classified in other ways). Then, a deep learning detection network architecture (such as YOLO series, SSD, RetinaNet, etc.) is selected for training. The network learns a mapping function from the input image to the target bounding box and class probability through the backpropagation algorithm.

[0033] Taking a variant of YOLOv5 as an example, the network segments the input image into... There are 3 grid cells. Each grid cell is responsible for predicting targets whose center point falls within that grid cell. For each grid cell, the network outputs B predicted bounding boxes, each containing 5 basic elements: the offset of the bounding box center relative to the top-left corner of that grid cell. Width and height of the bounding box (Transformation relative to a predefined anchor frame size), and a target confidence level. At the same time, the grid cells will also output the conditional probabilities of C categories. Ultimately, a complete prediction of a target is calculated using the following formula:

[0034]

[0035]

[0036]

[0037]

[0038] in, It is the coordinate of the top left corner of the grid. It is the sigmoid function. These are the preset width and height of the anchor frame. (Regarding angles...) Some networks will directly output it as the regression target. And through linear or periodic functions (such as...) The decoding process is then performed. Finally, a non-maximum suppression algorithm is used to filter out predicted boxes with high overlap and low confidence, resulting in a final list of detection results, which includes the category, confidence, center coordinates, width, height, and angle of each target.

[0039] During the inference phase, the image to be detected is directly input into the trained neural network, and the position and angle of all targets can be obtained after forward propagation, without the need for complex feature extraction and matching processes.

[0040] The solutions in related technologies still face a series of inherent defects in actual industrial scenarios, which limit their performance boundaries and scope of application.

[0041] Feature-matching-based schemes have the following drawbacks:

[0042] Sensitive to image quality and deformation: When the image is blurred, has low contrast, strong lighting changes, or contains nonlinear distortions (such as perspective distortion or specular reflection), the repeatability of feature point extraction algorithms drops sharply. Furthermore, when the target image itself has sparse or highly repetitive textures (such as smooth circular objects or regularly arranged grids), it is difficult to extract enough discriminative feature points, leading to matching failures.

[0043] Mismatches and Ambiguity: In complex backgrounds or scenarios with numerous similar interfering objects, matching methods based on local descriptor similarity are prone to mismatches. While the RANSAC algorithm can filter out some geometrically inconsistent outliers, it requires an excessive number of iterations when the proportion of inliers is too low, resulting in low computational efficiency and the potential for incorrect transformation models. For symmetrical or periodic figures, the matching results contain inherent ambiguity, which may lead to incorrect angle estimations (such as 180-degree confusion).

[0044] Computational efficiency and real-time performance bottlenecks: Feature point extraction and descriptor computation are computationally intensive operations, especially for floating-point descriptors such as SIFT and SURF. Full-image dense feature extraction from high-resolution images is significantly time-consuming. Simultaneously, the feature matching stage requires extensive distance calculations and searches; when the number of templates and image feature points is large, the time complexity is high, making it difficult to meet the demands of high-speed online detection.

[0045] Template dependency and complex parameter tuning: This method heavily relies on the quality and representativeness of a single template image. It suffers from insufficient generalization ability when dealing with similar images that exhibit appearance differences (such as color variations or slight wear). Furthermore, multiple parameters, including feature point detection threshold, matching ratio threshold, RANSAC iteration count, and error tolerance, require repeated manual tuning based on specific scenarios, resulting in poor robustness and ease of use.

[0046] Deep learning-based object detection schemes have the following drawbacks:

[0047] Data dependency and high annotation costs: Deep learning models are data-driven, requiring large amounts of diverse and precisely labeled training data to achieve ideal performance. In industrial applications, acquiring sample images covering all possible defects, lighting conditions, occlusions, and background variations is costly. Especially for the precise annotation of rotated targets (rotation bounding boxes), the annotation process is more cumbersome and time-consuming than that of horizontal bounding boxes.

[0048] Limited model generalization ability: Well-trained models typically perform well in scenarios similar to the distribution of their training data. However, once the target image changes significantly (e.g., a completely new model or material), the background environment changes drastically, or the imaging equipment is replaced, the model's performance may deteriorate severely, requiring the collection of new data and retraining of the model, resulting in poor flexibility.

[0049] Insufficient detection performance for small and dense targets: Although deep learning detectors are powerful, their detection performance (especially the accuracy of localization and angle estimation) will be significantly reduced for targets that are too small or dense targets that are close to each other and overlap, which can easily lead to missed detections or false merging.

[0050] Poor interpretability and difficulties in edge deployment: Deep learning models are like "black boxes," and their decision-making processes are difficult to interpret. Debugging and root cause analysis are challenging when false positives or false negatives occur. Furthermore, high-performance deep learning models typically require GPU acceleration, posing challenges for deployment on resource-constrained embedded or edge devices.

[0051] This invention aims to provide a vision-based object counting method that combines the interpretability and low data dependency of traditional methods with the powerful representational capabilities of deep learning methods. The objectives of this method include:

[0052] 1. Design a graphical representation and matching mechanism that is more robust to image quality degradation (blurring, low contrast) and sparse texture targets.

[0053] 2. A strategy is proposed that can effectively eliminate mismatches and angular ambiguity, especially in complex backgrounds and similar interference scenarios, and improve matching confidence.

[0054] 3. Build a more computationally efficient workflow that is better suited for real-time or near-real-time processing, reducing reliance on high-performance computing hardware.

[0055] 4. Reduce the method's dependence on a large amount of labeled data, and achieve "small sample" learning and matching capabilities that can work efficiently with only one or a few template images.

[0056] 5. Improve the accuracy of counting and angle estimation for densely arranged, partially occluded, and target graphics with significant scale and viewpoint changes.

[0057] 6. Provides clearer intermediate results and decision-making basis, enhancing the interpretability and debuggability of the entire system.

[0058] The vision-based object counting method provided by this invention can not only count objects in shape but also estimate their pose. This method is a multi-level adaptive shape context matching approach for shape counting and pose estimation. It abandons excessive reliance on local texture features, instead extracting more discriminative and robust features from the global and semi-global geometric contours and distribution structures of the shape. By constructing a multi-level matching and verification framework from coarse to fine and from global to local, it significantly improves computational efficiency and robustness while maintaining accuracy.

[0059] This method can be executed by any device, equipment, cluster, platform, or system with data processing capabilities.

[0060] like Figure 1 As shown, the method may include: step 201, in the image to be detected In step 202, P candidate regions are identified, where each of the P candidate regions includes candidate objects whose contours differ from the contours of the template object by less than a first threshold, where P is a positive integer, and the template object belongs to the target object; step 202, candidate regions are extracted from the P candidate regions. The outline of the candidate object is used to determine the candidate region. The candidate region is determined by whether the difference between the outline of the candidate object and the outline of the template object is less than a second threshold, wherein the second threshold is less than the first threshold. Let P be any candidate region among the candidate regions; Step 203, if the candidate region If the difference between the outline of the candidate object and the outline of the template object is less than the second threshold, then the candidate region is... The contour of the candidate object is segmented into a first set of segments, which includes M segments, where M is a positive integer greater than 1; Step 204: Determine whether the proportion of segments in the first set that can match at least one segment in the second set is greater than a third threshold, wherein the second set is obtained by contour segmentation of the template object and includes M segments; Step 205: If the proportion of segments in the first set that can match at least one segment in the second set is greater than the third threshold, then the candidate region is extracted. The key points in the contour of the candidate object are used to obtain the key point set. Step 206, based on the set of key points Key points and template key point set The key point pairs in the template are used to count the key point relationships between key points in the target object. The key point is that it is extracted from the outline of the template object.

[0061] In some embodiments, such as Figure 2 As shown, the execution steps of this method may include:

[0062] Step S101: Template Registration and Multi-level Feature Library Construction. Input one or more high-quality template images, perform preprocessing, extract their multi-level shape context features, and construct a structured template feature library.

[0063] Step S102: Input Image Preprocessing and Initial Screening of Candidate Regions. Adaptive preprocessing is performed on the input image to be detected, and a set of candidate regions that may contain the target is generated using a fast region proposal method.

[0064] Step S103: Cascaded shape matching. For each candidate region, matching and hypothesis generation are performed using different levels of features from the template feature library, from coarse to fine, to quickly eliminate a large number of negative samples.

[0065] Step S104: Dense Correspondence and Precise Pose Calculation. For candidate hypotheses obtained through cascaded matching, dense correspondences are established using the finest level of features, and the sub-pixel precision position and angle are solved using an optimization algorithm.

[0066] Step S105: Hypothesis Fusion and Result Output. Clustering and non-maximum suppression are performed on all calculated pose hypotheses to eliminate duplicate detections. Finally, a list of center coordinates and rotation angles of all unique target images is output.

[0067] Next, an example of the vision-based object counting method provided by this invention will be introduced.

[0068] In execution Figure 1 Before the method shown, you can first execute... Figure 2 Step S101, namely template registration and multi-level feature library construction, involves inputting one or more high-quality template images, preprocessing them, extracting their multi-level shape context features, and constructing a structured template feature library. The relevant implementation method is as follows.

[0069] The template processing flow begins with the input template image. First, adaptive binarization and contour extraction are performed. A local adaptive thresholding algorithm (such as the Sauvola algorithm) is used instead of a global thresholding algorithm to handle potential uneven illumination in the template image. For grayscale template images... Its pixel Local threshold The calculation is as follows:

[0070]

[0071] in, and They are respectively based on Centered on, with radius The mean and standard deviation of grayscale values ​​within a local window. It is the maximum value of the dynamic range of the standard deviation. This is a correction parameter (usually a positive decimal). It is calculated by comparing the grayscale value of each pixel with... By comparison, a more robust binary image is obtained. Subsequently, from Extract the outermost contour , This represents the number of outline points.

[0072] Next is the calculation of multi-level shape context features. Traditional shape context describes contour points at a single scale, while this embodiment introduces a multi-level pyramid structure. Next, combined with... Figure 3 This will be explained in detail. Figure 3 The relevant English terms and their corresponding Chinese translations are as follows.

[0073] template image → template image

[0074] adaptive binarization

[0075] contour extraction → contour extraction

[0076] global shape context

[0077] shape context histogram → shape context histogram

[0078] Zernike rotation → Zernike rotation normalization

[0079] Local contour fragments (LCF)

[0080] chord-length ratio

[0081] directional histogram

[0082] segment feature set

[0083] Keypoint pair relativity (KPSR) → Spatial relationship between keypoint pairs

[0084] Relationship between segments

[0085] keypoint pair feature set

[0086] Multi-level template feature library.

[0087] Level 1: Global Shape Context (GSC). This level aims to capture the overall outline distribution of the graphic. First, the outline... Perform uniform resampling, with a fixed number of points. (e.g., 100 points) to get .for Each reference point in A shape contexthistogram is constructed using this as the origin. The entire 360-degree space is divided into... Each angular sector is divided into logarithmic distances along its radius. There are several intervals. Therefore, for non-reference points... Its relative polar coordinates Can be mapped to a certain Intervals. Count the number of all other contour points falling into each log-polar coordinate interval, forming a... dimensional histogram vector That is, a point The global shape context. The global shape context of a point can be referred to as the global shape context feature of that point. To obtain a global description of the graphic, all contour points are calculated. mean vector . The profile is sensitive to rotation. To achieve rotation invariance, normalization is performed using the principal directions of the profile. The Zernike moments of the profile are calculated to obtain a more stable estimate of the rotation angle. First, the second-order central moments of the profile are calculated. The direction of the outline It can be calculated using the following formula:

[0088]

[0089] Rotate all contour point coordinates around the contour centroid After adjusting the angle, recalculate. This rotation normalized This refers to the globally rotation-invariant shape context features of the template. Simultaneously, record... The inherent direction of the template.

[0090] Level Two: Local Contour Fragments (LCF). To handle occlusion and local deformation, embodiments of this invention describe the graphics at a medium scale. The original contour... Divided into A series of consecutive segments of approximately equal length For each segment Three complementary features are extracted: (1) Chord-length ratio: the ratio of the Euclidean distance between the two endpoints of the segment to the original arc length of the segment, describing the curvature of the segment. (2) Directional histogram: the direction angle of the line connecting each two adjacent points on the segment is calculated, and its histogram is plotted. (3) Segment feature: using only the points within the segment, its local shape context histogram is calculated (scale is smaller than GSC). These three features are concatenated to obtain the description vector of each segment. Simultaneously, the center point of each segment is recorded after normalization (rotation). The position in the contour coordinate system (after).

[0091] Level 3: Keypoint Pair Relationship (KPSR). At the finest level, this embodiment of the invention detects stable keypoints on the contour, such as points of maximum curvature and inflection points. A detection method based on a chord length function is employed. For points on the contour... Take it before and after. Calculate each of the points. and arrive The vectors, the angle between the two vectors The cosine value is defined as the curvature response at that point. .exist Local maxima and greater than the threshold The location is identified as a key point. Assuming it is detected... Several key points form a set. For each pair of key points Define its associated descriptor For a quadruple: .in, It is the normalized distance between two points (divided by the length of the principal axis of the graph); and They are points and The tangent direction of the contour at that location (also after global rotation) (Normalization) It is the chord formed by connecting two points and The angle between the tangents at the point. This point-to-point relationship-based description is highly robust to noise and local occlusion.

[0092] Finally, the template feature library It is structured data, containing: 1) Global GSC features and inherent direction ;2) Features of M local segments and its central location; 3) The location of each key point and its constituent parts A set of point pairs descriptors .

[0093] In this embodiment of the invention, the template image refers to the image of the template object of the target object, that is, the template image includes the template object.

[0094] A template feature library can also be called a multi-level template feature library.

[0095] The vision-based object counting method provided in this application embodiment can count target objects using the multi-level template feature library constructed above.

[0096] First, in step 201, in the image to be detected... P candidate regions are identified, where each of the P candidate regions includes a candidate object whose contour differs from the contour of the template object by less than a first threshold, where P is a positive integer, and the template object belongs to the target object. Additionally, the template object can be simply referred to as a template.

[0097] An image can be any object related to the target object; it includes the target object, or in other words, it is an image that includes the target object. Images can be captured using imaging equipment such as industrial cameras. .

[0098] In some embodiments, the target object may be a fruit, such as an apple or an orange. In some embodiments, the target object may be an industrial product, such as a tire.

[0099] Candidate regions can be rectangular candidate boxes. In step 201, adaptive contrast enhancement is first performed. A contrast-limited adaptive histogram equalization method is used to enhance details in dark areas while suppressing noise amplification. Then, candidate regions are quickly generated using saliency detection or edge density maps. This embodiment of the invention employs an improved edge box algorithm: first, the gradient magnitude map of the image is calculated, and then the consistency of the sum of edge intensities and the spatial distribution of edge points within any rectangular region is quickly calculated using an integral map. By setting a low threshold, hundreds of rectangular candidate boxes that may contain similar contour structures can be quickly generated. This significantly reduces the search space for subsequent fine-grained matching. If the consistency between the sum of edge intensity and the spatial distribution of edge points within the rectangular region is greater than this threshold, it indicates that the difference between the sum of edge intensity and the spatial distribution of edge points within the rectangular region is less than the preset first threshold, or in other words, the difference between the sum of edge intensity within the rectangular region and the outline of the template object is less than the first threshold.

[0100] After obtaining P candidate regions, a cascaded matcher can be used to identify whether candidate objects within these regions belong to the target object. The cascaded matcher consists of three sequentially connected stages, with increasing complexity at each stage, eliminating most negative samples early on. These stages will be described in detail below.

[0101] In step 202, candidate regions are extracted from the P candidate regions. The outline of the candidate object is used to determine the candidate region. The candidate region is determined by whether the difference between the outline of the candidate object and the outline of the template object is less than a second threshold, wherein the second threshold is less than the first threshold. It is any candidate region among the P candidate regions.

[0102] In some embodiments, the method further includes: acquiring an image containing the template object. From the image The outline of the template object is extracted; based on the features of multiple outline points in the outline of the template object, the first global shape context feature of the template object is obtained. The relevant implementation method can be found in the description of step S101 above, and will not be repeated here.

[0103] In step 202, based on the candidate region The candidate region is obtained by analyzing the features of multiple contour points in the contour of the candidate object. The second global shape context feature of the candidate object; the candidate region is determined based on the first global shape context feature and the second global shape context feature. Whether the difference between the outline of the candidate object and the outline of the template object is less than a second threshold.

[0104] Step 202 involves fast global feature filtering. For each candidate region... Extract its internal contour (using the same method as the template, employing adaptive binarization), and calculate its globally rotation-invariant shape context features. .calculate Template features Between distance:

[0105] in This is a very small quantity to prevent division by zero. If If a region deviates significantly from the overall shape of the template (based on a preset threshold), it is considered too different and is directly rejected. The set of candidate regions that pass this stage is denoted as... . The second threshold is a preset threshold, which is less than the first threshold.

[0106] In step 203, if the candidate region If the difference between the outline of the candidate object and the outline of the template object is less than the second threshold, then the candidate region is... The outline of the candidate object is segmented into a first set of segments, which includes M segments, where M is a positive integer greater than 1.

[0107] And, in step 204, it is determined whether the proportion of fragments in the first fragment set that can be matched with at least one fragment in the second fragment set is greater than a third threshold, wherein the second fragment set is obtained by contour segmentation of the template object, and the second fragment set includes M fragments.

[0108] In some embodiments, step 204 includes: calculating the segments in the first segment set. The fragment features, wherein the fragment The fragment features include: the fragment The segment chord length feature, segment orientation histogram, segment shape context, or any one or more of these; based on the segment Based on the fragment features, determine whether there exists a fragment in the second fragment set that matches the fragment. The fragment that matches the fragment.

[0109] Steps 203 and 204 involve local segment voting verification. For For each candidate region, the consistency of its local structure needs to be verified. The contour of the candidate region is also divided into M segments. For each segment of the candidate region... Calculate its local segment features Then, among all M fragments of the template object, search for those that match... Most similar fragments Similarity is measured by the cosine distance between feature vectors. If the distance between the most similar features is less than a threshold... If a candidate fragment is found to correspond to the t-th fragment in the template, then this candidate fragment is considered to potentially correspond to the t-th fragment in the template. Record this possible correspondence. After traversing all candidate fragments, a sparse set of correspondences is obtained. Since the contours of the template and candidate regions are segmented sequentially, a correct match should exhibit sequential consistency (i.e., the mapping relationship between template fragment index t and candidate fragment index q should be approximately monotonic). This embodiment of the invention designs a sequential consistency voting algorithm. Find an offset... , so that satisfaction The most corresponding pairs. This is the most votes. This reflects the degree of matching of local structures. If ( For example If a region passes this stage, the candidate is rejected. The set of regions that pass this stage is denoted as... This stage not only filters out more negative samples, but also provides an initial coarse rotation estimate for each candidate region. This is because the found fragment correspondence implicitly contains angle alignment information.

[0110] In step 205, if the proportion of segments in the first segment set that can match at least one segment in the second segment set is greater than a third threshold, then the candidate region is extracted. The key points in the contour of the candidate object are used to obtain the key point set. .

[0111] In some embodiments, step 205 includes: obtaining the candidate region. Points in the contour of the candidate object ,point and ,in, and Located at point Both sides, and points and The interval between them There are 10 points, of which 10 are points. It is a positive integer; calculate the point. Time The first vector, and the calculation Time The second vector; calculate the cosine value between the first vector and the second vector; determine the point based on the cosine value. This is the key point. The relevant implementation method can be found in the above description of step S101, and will not be repeated here.

[0112] In step 206, based on the set of key points Key points and template key point set The key point pairs in the template are used to count the key point relationships between key points in the target object. The key point is that it is extracted from the outline of the template object.

[0113] In some embodiments, step 206 includes: obtaining the key point set. Key points The direction of the first contour tangent at the location, and the acquisition of the template key point set. Key points The direction of the second contour tangent at the location; calculate the key points. and key points The first angle between the chord formed and the tangent direction of the second contour; obtain the set of key points. Key points The direction of the third contour tangent at the location, and the acquisition of the template key point set. Key points The direction of the fourth contour tangent at the location; calculate the key points. and key points The second included angle between the chord formed and the tangent direction of the fourth contour; the key point pair relationship is determined based on the tangent direction of the first contour, the tangent direction of the second contour, the tangent direction of the third contour, the tangent direction of the fourth contour, the first included angle, and the second included angle.

[0114] Step 206 includes a geometric verification of key point pair relationships. For Extract the set of key points on the contour of the candidate region. The goal is to establish With template key point set A reliable correspondence between them. Instead of directly matching descriptors of individual keypoints (as they are susceptible to noise), we match keypoint pair relationships. For candidate regions, we also compute their set of keypoint pair descriptors. Then, relation matching is performed between the point-pair descriptor subset of the template and the candidate point-pair descriptor subsets. Specifically, for a point-pair descriptor subset of the template... and a candidate point-pair descriptor If the following conditions are met:

[0115]

[0116] Then it is considered that the points are paired. and point pairs Match successful. This actually means "from..." arrive "Spatial-directional relationship" and "from" arrive The relationships are highly consistent. For each successful point pair match, a vote is cast for each of its four constituent points (two template points and two candidate points). For example, a match... Will for and Each of these two potential points corresponds to an additional vote. After traversing all possible point pair combinations, we obtain the corresponding vote for each potential point. votes The number of votes selected exceeded the threshold. The points correspond to each other, forming an initial set of point correspondences. Because point-to-point matching has strong geometric constraints, the resulting point correspondences are highly reliable. Through the candidate regions obtained at this stage, the existence of a target instance can be preliminarily confirmed, and a set of high-quality point correspondences is obtained. The set of regions obtained at this stage is denoted as […]. That is, the hypothesis set.

[0117] Among them, the target instance is the target object instance, that is, the target object. After confirming that the candidate object in the candidate area is the target object, the target object can be counted.

[0118] In some embodiments, the key point set can be used as a basis. The set of key points in the template The pose parameters of the candidate object are determined by identifying keypoints that have keypoint pairs with each other, wherein the pose parameters include translation and rotation.

[0119] In this embodiment, dense correspondence and accurate pose calculation can be performed, as detailed below.

[0120] for Each hypothesis in the model has: 1) a set of high-confidence sparse keypoint correspondences ;2) A rough rotation estimate obtained from the second stage The goal of this stage is to solve for the precise pose parameters: translation. Rotation and possible isotropic scaling Suppose that the image undergoes a similarity transformation within the image plane.

[0121] First, using The transformation parameters are initially estimated using the least squares method. Let the coordinates of the key points of the template be... The corresponding candidate keypoint coordinates are Minimize the following errors:

[0122]

[0123] in It is a rotation matrix. This is a standard similarity transformation parameter estimation problem with a closed-form solution.

[0124] However, sparse keypoint mapping may still have some errors, and ultimately, the pose of the graphic reference points (such as the centroid) is needed. Therefore, this embodiment introduces an iterative dense contour point matching optimization process. Using the initially estimated transformation parameters, the entire contour of the template is... Transform to the candidate image coordinate system to obtain the initially aligned contour. Then, in the candidate region outline and Establish the nearest correspondence between them: For Each point on, in Find the nearest point on the upper boundary and vice versa, forming a set of bidirectional dense point correspondences. Based on this dense correspondence, the transformation parameters are optimized again using least squares (or the point-to-surface error metric in the more robust iterative nearest-point algorithm) to minimize the distance between corresponding points. This process can be iterated 2-3 times, making the alignment increasingly accurate.

[0125] After the final iteration, the optimal similarity transformation parameters are obtained. So, the reference point predefined in the template. Coordinates in the input image for:

[0126]

[0127] The final output angle of the target image It needs to be combined with the inherent direction of the template. :

[0128]

[0129] here It is the angle calculated during the matching process that rotates the template (normalized to its inherent orientation) to align with the candidate region, plus... Then the absolute orientation of the graphic in the original image was restored.

[0130] Then, hypothesis fusion and result output are performed, as follows.

[0131] Because candidate regions may overlap, and a target may be covered by multiple slightly different hypotheses, it is necessary to consider all... The pose assumptions calculated in the solution are fused. This embodiment uses a clustering method based on pose spatial distance. Two assumptions are defined. and Distance between for:

[0132]

[0133] in This is a weighting factor used to balance the contributions of position and angle errors. Using mean-shift clustering or DBSCAN clustering algorithms, all clusters with distances less than a threshold are selected. The assumption is that all targets are grouped into one class. Here, "one class" refers to a target object or an instance of a target object. For each class, the weighted median of the poses of all its members (the weights can be the matching score or the number of inliers) is taken as the final output target pose for that class. Finally, the pose results for all classes are output to complete the counting.

[0134] In addition, the coarse rotation estimate obtained in the second stage It can be used for the following purposes:

[0135] 1. Providing reliable initial values ​​for pose optimization avoids ambiguity and local optima. Pose calculation (especially rotation angle θ) is a periodic, non-convex optimization problem. For symmetrical or approximately symmetrical shapes (such as rectangles or crosses), the objective function often has multiple local optima (e.g., differing by 180° or 90°). If optimization starts directly from zero or random initial values, the algorithm is prone to converge to erroneous local optima, leading to "flipped" ambiguities in angle estimation. θ_init is derived based on the sequential correspondence of local contour segments. Although not precise, it is close to the true rotation angle, with errors typically limited to tens of degrees. Using this as the initial value for optimization constrains the search space to a small neighborhood of the true solution, significantly improving convergence speed and fundamentally avoiding ambiguities caused by symmetry.

[0136] 2. Enhancing Geometric Constraints for Keypoint Pair Matching: In the third stage, "Geometric Verification of Keypoint Pair Relationships," point pair descriptors contain orientation components (e.g., ψ_a, ψ_b). If the rotation is unknown, the orientation tolerance threshold needs to be relaxed during matching, which can easily introduce mismatches. By using θ_init to pre-rotate and compensate for the orientation of candidate keypoints, the orientation components of the candidate descriptors can be roughly aligned with the template descriptors. Therefore, a stricter orientation threshold can be used during relation matching, significantly improving the selectivity and reliability of point pair matching and providing a cleaner sparse corresponding point set ℳ_init for subsequent pose calculation.

[0137] 3. The results of each stage of cascade matching, which serves as the basis for internal consistency verification and hypothesis screening, should corroborate each other. θ_init should be essentially consistent with the rotation angle θ′ initially obtained through the correspondence of sparse keypoints. If the difference between the two exceeds a reasonable range (e.g., > 30°), it indicates that the hypothesis may have matching contradictions or noise interference, and can be eliminated early. Furthermore, in the final hypothesis fusion stage, θ_init can serve as a rapid clustering feature to help distinguish different instances that are geographically close but have significantly different orientations, thus improving fusion efficiency.

[0138] The embodiments of the present invention also provide the overall architecture and key components of "multi-level adaptive shape context matching", including the method for constructing multi-level shape feature representation and structured feature library, and the process and algorithm for cascaded shape matching.

[0139] The methods for multi-level shape feature representation and the construction of structured feature libraries include:

[0140] The specific process of adaptive binarization and contour extraction of template graphics;

[0141] A method for extracting Global Rotation Invariant Shape Context (GSC) features, particularly a specific implementation combining Zernike moments with rotational normalization to simultaneously obtain discriminative and rotational invariance;

[0142] The definition of Local Contour Fragment Features (LCF), segmentation strategies, and feature description methods that fuse fragment chord length, orientation histogram, and local context;

[0143] The definition and extraction method of Keypoint Pair Spatial Relationship (KPSR) descriptors, including stable keypoint detection based on chord length function, and the specific form of describing point pair relations using quadruples (ρ,ψ_a,ψ_b,χ);

[0144] The process and algorithm for cascaded shape matching include:

[0145] A threshold determination method for fast filtering using global GSC features;

[0146] A specific algorithm is used to perform sequential consistency voting using local LCF features to verify local structures and estimate coarse rotations.

[0147] Relationship matching is performed using KPSR descriptors, and high-confidence sparse points are generated through a voting mechanism.

[0148] In summary, the solution provided by this invention employs a multi-level matching method to determine whether a candidate object is a target object, and then counts the target objects, exhibiting high robustness, counting accuracy, and counting efficiency. The advantages are explained below.

[0149] 1. Excellent robustness

[0150] 11. Insensitive to texture and lighting: This invention is entirely based on contour shape information, fundamentally avoiding the dependence of traditional feature point methods on rich textures. It can work stably even when the target surface is smooth, textureless, or has highly repetitive textures, as long as its contours are clearly discernible.

[0151] 12. Anti-blurring and low contrast: Adaptive binarization and geometric contour-based features are more tolerant of image blurring and global contrast degradation compared to traditional feature points that rely on image gradients.

[0152] 13. Resistance to local occlusion and deformation: Multi-level feature design, especially Local Fragment Features (LCF) and Keypoint Pair Relationships (KPSR), enables the method to reliably match using information from the visible parts even when the target is partially occluded or undergoes non-rigid deformation. Global features filter out regions with excessively large overall shape differences, while local features are responsible for verification and refinement.

[0153] 2. High precision and strong discriminative power

[0154] 21. Eliminate angular ambiguity: By combining the global principal direction of the contour (Zernike moment) and the relative rotation calculated by matching, the target angle can be uniquely determined, effectively solving the 180-degree or 90-degree ambiguity problem of symmetrical figures (such as rectangles and crosses).

[0155] 22. Dense Correspondence Optimization: Advancing from sparse keypoint matching to dense contour point iterative optimization, achieving sub-pixel level positioning accuracy and more accurate angle estimation.

[0156] 23. Relationship matching suppresses false detections: Based on the key point matching mechanism for spatial relationships, a strong geometric constraint is introduced, which can effectively distinguish interference objects that are similar locally but have different overall spatial structures, greatly reducing the false detection rate in complex backgrounds.

[0157] 3. High efficiency and real-time potential:

[0158] 31. Cascaded fast filtering: More than 80%-90% of irrelevant image regions can be quickly eliminated in the first and second stages with extremely low computational cost, avoiding full-image search in the most time-consuming key matching and density optimization stages.

[0159] 32. Controllable feature computation: The computational complexity of features such as shape context is linearly related to the number of contour points, and can be reduced by fixing the number of resampling points. This allows for control, far less computation of dense SIFT features in high-resolution images.

[0160] 33. No GPU acceleration required: The entire algorithm consists of explicit image processing and geometric calculation steps, without relying on large-scale neural network forward propagation. It can run efficiently on the CPU and is more suitable for cost-sensitive embedded deployments.

[0161] 4. Few-shot learning and strong generalization ability:

[0162] 41. Works with only one or a few templates: Only one or a few standard template images are needed to build a feature library, without the need to collect and label massive amounts of training data, which greatly reduces the implementation threshold and cycle.

[0163] 42. Robust to intra-class variations: Shape-based features are insensitive to changes in color and internal texture; as long as the outlines are basically consistent, they can be correctly identified. For slight perspective distortions, the similarity transformation model can still approximate the shape well, and stable results can be obtained through multi-level matching.

[0164] 43. High interpretability: Each step of the entire matching process has a clear geometric meaning and intermediate results (such as contours, key points, matching pairs, alignment errors). When missed or false detections occur, engineers can intuitively analyze the reasons (e.g., whether the contour extraction is complete, whether the key point detection is stable), which facilitates debugging and optimization.

[0165] 5. Suitable for dense and small targets:

[0166] 51. Initial screening of candidate regions: Region proposal methods such as Edge Boxing can effectively locate dense regions that may contain the target.

[0167] 52. Contour-based differentiation: Even if targets are closely adjacent, as long as their contours can be separated at the pixel level, this method can perform independent matching and pose estimation based on their respective contours.

[0168] 53. Multi-scale characteristics: By controlling the number of points for contour resampling and the granularity of log-polar coordinate partitioning, the method can adapt to targets of different scales. Running this method on the input image pyramid can further improve multi-scale detection capabilities.

[0169] This invention provides a vision-based object counting device 400. For example... Figure 4 As shown, the device 400 includes:

[0170] Recognition unit 410, used for identifying images to be detected P candidate regions are identified in the process, wherein each of the P candidate regions includes a candidate object whose contour differs from the contour of the template object by less than a first threshold, where P is a positive integer, and the template object belongs to the target object.

[0171] The first extraction unit 420 is used to extract candidate regions from the P candidate regions. The outline of the candidate object is used to determine the candidate region. The candidate region is determined by whether the difference between the outline of the candidate object and the outline of the template object is less than a second threshold, wherein the second threshold is less than the first threshold. It can be any candidate region among the P candidate regions;

[0172] Segmentation unit 430, used if the candidate region If the difference between the outline of the candidate object and the outline of the template object is less than the second threshold, then the candidate region is... The outline of the candidate object is segmented into a first set of segments, which includes M segments, where M is a positive integer greater than 1;

[0173] The judgment unit 440 is used to determine whether the proportion of the fragments in the first fragment set that can be matched with at least one fragment in the second fragment set is greater than a third threshold, wherein the second fragment set is obtained by segmenting the contour of the template object, and the second fragment set includes M fragments.

[0174] The second extraction unit 450 is configured to extract the candidate region if the proportion of segments in the first segment set that can be matched with at least one segment in the second segment set is greater than a third threshold. The key points in the contour of the candidate object are used to obtain the key point set. ;

[0175] Counting unit 460, used for counting based on the set of key points Key points and template key point set The key point pairs in the template are used to count the key point relationships between key points in the target object. The key point is that it is extracted from the outline of the template object.

[0176] The functions of each functional unit of device 400 can also be found in the above description. Figure 1 The implementation details of the methods shown are not repeated here.

[0177] See Figure 5 This invention also provides a computing device 500, including a memory 510 and a processor 520; wherein the memory 510 stores computing instructions; and the processor 520 executes the computing instructions, causing the computing device 500 to perform... Figure 1 The method shown.

[0178] This invention also provides a computer-readable storage medium including computer program instructions, which, when executed by a computing device, enable the computing device to perform... Figure 1 The method shown.

[0179] This invention also provides a computer program product, including a computer program, which, when executed by a processor, implements the following: Figure 1 The method shown.

[0180] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A vision-based object counting method, characterized in that, The method includes: In the image to be detected P candidate regions are identified in the process, wherein each of the P candidate regions includes a candidate object whose contour differs from the contour of the template object by less than a first threshold, where P is a positive integer, and the template object belongs to the target object. Extract candidate regions from the P candidate regions The outline of the candidate object is used to determine the candidate region. The candidate region is determined by whether the difference between the outline of the candidate object and the outline of the template object is less than a second threshold, wherein the second threshold is less than the first threshold. It can be any candidate region among the P candidate regions; If the candidate region If the difference between the outline of the candidate object and the outline of the template object is less than the second threshold, then the candidate region is... The outline of the candidate object is segmented into a first set of segments, which includes M segments, where M is a positive integer greater than 1; Determine whether the proportion of segments in the first segment set that can be matched with at least one segment in the second segment set is greater than a third threshold, wherein the second segment set is obtained by contour segmentation of the template object and the second segment set includes M segments; If the proportion of segments in the first segment set that can match at least one segment in the second segment set is greater than a third threshold, then the candidate region is extracted. The key points in the contour of the candidate object are used to obtain the key point set. ; Based on the set of key points Key points and template key point set The key point pairs in the template are used to count the key point relationships between key points in the target object. The key point is that it is extracted from the outline of the template object.

2. The method according to claim 1, characterized in that, The method further includes: Obtain an image containing the template object. ; From the image Extract the outline of the template object; Based on the features of multiple contour points in the contour of the template object, the first global shape context feature of the template object is obtained; The determination of the candidate region Whether the difference between the contour of the candidate object and the contour of the template object is less than a second threshold includes: According to the candidate region The candidate region is obtained by analyzing the features of multiple contour points in the contour of the candidate object. The second global shape context feature of the candidate object; The candidate region is determined based on the first global shape context feature and the second global shape context feature. Whether the difference between the outline of the candidate object and the outline of the template object is less than a second threshold.

3. The method according to claim 1 or 2, characterized in that, The step of determining whether the proportion of segments in the first segment set that can be matched with at least one segment in the second segment set is greater than a third threshold includes: Calculate the fragments in the first fragment set The fragment features, wherein the fragment The fragment features include: the fragment The segment chord length feature, segment orientation histogram, segment shape context, or any one or more of these; According to the fragment Based on the fragment features, determine whether there exists a fragment in the second fragment set that matches the fragment. The fragment that matches the fragment.

4. The method according to any one of claims 1-3, characterized in that, Extracting the candidate region The key points in the contour of the candidate object are used to obtain the key point set. include: Obtain the candidate region Points in the contour of the candidate object ,point and ,in, and Located at point Both sides, and points and The interval between them There are 10 points, of which 10 are points. It is a positive integer; Calculation points Time The first vector, and the calculation Time The second vector; Calculate the cosine value between the first vector and the second vector; Determine the point based on the cosine value. This is the key point.

5. The method according to any one of claims 1-4, characterized in that, According to the set of key points Key points and template key point set Keypoint pairs in the text include: Obtain the set of key points Key points The direction of the first contour tangent at the location, and the acquisition of the template key point set. Key points The direction of the second contour tangent at the location; Calculate key points and key points The first angle between the chord formed and the tangent direction of the second contour; Obtain the set of key points Key points The direction of the third contour tangent at the location, and the acquisition of the template key point set. Key points The direction of the fourth contour tangent at the location; Calculate key points and key points The second angle between the chord formed and the tangent direction of the fourth contour; The key point pair relationship is determined based on the first contour tangent direction, the second contour tangent direction, the third contour tangent direction, the fourth contour tangent direction, the first included angle, and the second included angle.

6. The method according to any one of claims 1-5, characterized in that, The method further includes: Based on the set of key points The set of key points in the template The pose parameters of the candidate object are determined by identifying keypoints that have keypoint pairs with each other, wherein the pose parameters include translation and rotation.

7. A vision-based object counting device, characterized in that, The device includes: Recognition unit, used for identifying images to be detected P candidate regions are identified in the process, wherein each of the P candidate regions includes a candidate object whose contour differs from the contour of the template object by less than a first threshold, where P is a positive integer, and the template object belongs to the target object. The first extraction unit is used to extract candidate regions from the P candidate regions. The outline of the candidate object is used to determine the candidate region. The candidate region is determined by whether the difference between the outline of the candidate object and the outline of the template object is less than a second threshold, wherein the second threshold is less than the first threshold. It can be any candidate region among the P candidate regions; Segmentation unit, used if the candidate region If the difference between the outline of the candidate object and the outline of the template object is less than the second threshold, then the candidate region is... The outline of the candidate object is segmented into a first set of segments, which includes M segments, where M is a positive integer greater than 1; The judgment unit is used to determine whether the proportion of fragments in the first fragment set that can be matched with at least one fragment in the second fragment set is greater than a third threshold, wherein the second fragment set is obtained by segmenting the contour of the template object, and the second fragment set includes M fragments. The second extraction unit is configured to extract the candidate region if the proportion of segments in the first segment set that can be matched with at least one segment in the second segment set is greater than a third threshold. The key points in the contour of the candidate object are used to obtain the key point set. ; A counting unit is used to count the key points. Key points and template key point set The key point pairs in the template are used to count the key point relationships between key points in the target object. The key point is that it is extracted from the outline of the template object.

8. A computing device, characterized in that, Including memory and processor; The memory stores calculation instructions; The processor executes the computation instructions, causing the computing device to perform the method of any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, It includes computer program instructions, which, when executed by a computing device, cause the computing device to perform the method of any one of claims 1-6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-6.

Images