A reference point detection method and device for an electronic device and a readable storage medium

Abstract

The present application belongs to the technical field of 3D AOI industrial vision detection, and relates to a fiducial point detection method and device for electronic devices and a readable storage medium. A fiducial point to-be-positioned region image is collected, the fiducial point to-be-positioned region image is detected, and at least one coarse positioning circular region where the fiducial point is located is obtained. Based on the radius of each coarse positioning circular region and the radius of a standard fiducial point, the positioning score of each coarse positioning circular region is calculated. The coarse positioning circular region with a positioning score greater than a preset positioning score threshold is taken as a target coarse positioning circular region. The edge points of the target coarse positioning circular region are extracted multiple times, the edge points extracted each time are fitted, and multiple fitting circles corresponding to the target coarse positioning circular region are obtained. The difference between the radius of each fitting circle and the radius of the standard fiducial point is calculated, the fitting circle with the smallest difference is taken as the best fitting circle, and thus the fiducial point of the electronic device is obtained, effectively improving the detection precision and detection efficiency of the fiducial point.

Description

Technical Field

[0001] This invention relates to the field of 3D AOI industrial vision inspection technology, and in particular to a method, apparatus and computer-readable storage medium for detecting reference points of electronic devices. Background Technology

[0002] With the rapid development of electronic technology, electronic components are becoming smaller and smaller, and their packaging density is constantly increasing. This has brought unprecedented challenges to Automatic Optical Inspection (AOI) visual inspection. In the current context of Industry 4.0 and smart manufacturing, the size of components on PCBs has shrunk from the traditional millimeter level to the micrometer level. Passive components such as 0201 or even 01005 packages have become the norm. The widespread application of high-density packaging technologies such as chip-scale packaging (CSP) and ball grid array (BGA) has enabled the number of components on a single PCB to reach thousands or even tens of thousands.

[0003] In AOI (Automated Optical Inspection) systems for electronic components, the reference point acts as the "origin" and "navigation beacon" of the entire inspection system. It provides CAD (Computer-Aided Design) information for the standard fit position of components during inspection. However, errors such as motion stop errors and board bending can cause discrepancies between the CAD position of the component and its position in the image. Therefore, it is necessary to record the coordinates of the current reference point after scanning the entire large image, and then re-detect the reference point's position during inspection. Next, an affine transformation matrix is ​​constructed using the current reference point's position and the reference point's position during modeling. Finally, this matrix is ​​applied to the standard CAD position of the component to align it with the position in the image during acquisition, allowing for component-level inspection. Therefore, the positioning accuracy of the reference point has the following impacts on AOI inspection of electronic components: 1. Impact on inspection accuracy: The cumulative error in component position detection amplifies with increasing distance from the reference point; the originally precisely aligned inspection area may shift to a non-target area, leading to missed or false detections; for fine-pitch devices (such as 0.4mm pitch QFPs), even a few pixels of offset can cause detection failure. 2. Increased false alarms: Correctly placed components are misjudged as poorly placed due to coordinate misalignment; genuine defects may be missed, reducing detection reliability; and increasing the workload and production costs of subsequent manual re-inspection. 3. Impact on production efficiency: Frequent false alarms can interrupt the production line, affecting capacity; repeated calibration and debugging are required, extending equipment setup time; and the stability and consistency of automated production are reduced.

[0004] During AOI (Automated Optical Inspection) testing, due to the small field of view of the camera, multiple sets of FOV-level images are typically acquired. Because AOI operates on a stop-and-shoot basis and due to the instability of the light source, images from different FOVs will exhibit differences in color, transparency, and grayscale. Furthermore, the different materials and manufacturing methods of electronic components result in various morphological characteristics of the reference points. In addition, oxidation, wear, and dirt can also alter the surface condition of the reference points. These issues make the positioning of reference points significantly challenging, potentially leading to substantial deviations from the actual location, thus exacerbating false positives and impacting production efficiency. Therefore, improving the positioning accuracy of reference points is essential.

[0005] Existing technologies have proposed reference point detection methods based on template matching, fixed color and threshold segmentation, and texture features. However, in template matching-based reference point detection, the algorithm assumes that the target image and the template image are highly consistent in shape, size, and grayscale distribution. In actual production, the reference point's morphology can change due to factors such as differences in screen printing processes, PCB substrate deformation, and changes in lighting angle, making the rigid template unable to adapt to these changes. Furthermore, differences in surface reflectivity of different batches of electronic devices and changes in workshop lighting conditions can alter the grayscale distribution of the reference point, causing grayscale-related template matching algorithms to fail. Therefore, to improve detection robustness, multiple templates (templates with different scales, angles, and lighting conditions) need to be established, leading to a decrease in real-time detection performance and making it difficult to meet the requirements of high-speed production lines. When performing detection based on fixed color and threshold segmentation, the low contrast between the reference point and the substrate (especially the light-colored reference point on the green solder mask layer) makes it difficult for traditional global thresholding or adaptive thresholding to accurately segment the reference point area. Non-uniform lighting in industrial environments can produce gradual shadows on the surface of electronic devices, resulting in significant differences in grayscale values ​​of the same reference point in different areas. This makes it impossible for fixed thresholding to adapt to such changes, affecting the detection results based on fixed thresholds. In addition, limitations of screen printing processes and the micro-roughness of electronic device surfaces can cause jagged, burr-like, or blurred edges on the reference points. Simple threshold segmentation cannot obtain clear edge contours, and noise such as reflective spots, stains, and scratches on the surface of electronic devices can easily be confused with the real reference points after binarization, affecting the accuracy of reference point detection. When performing detection based on texture features, traditional texture features (such as LBP, GLCM, and Gabor filters) mainly describe local statistical characteristics. For targets like reference points with well-defined geometric shapes, they cannot fully characterize their spatial structure features and edge distribution patterns. Furthermore, the window size for texture feature calculation directly affects the feature representation capability. If the window is too small, it cannot capture complete reference point features, while if the window is too large, it will introduce background interference. Introducing multi-scale and multi-directional texture feature extraction requires a large number of convolution operations, which seriously affects the real-time performance of detection. Moreover, the manufacturing processes of different electronic devices can change the surface micro-texture characteristics, making this method unstable in cross-process and cross-manufacturer applications. At the same time, the lack of effective utilization of the geometric regularity of reference points results in low accuracy of reference point detection.

[0006] In summary, existing 2D detection algorithm-based methods for detecting reference points in electronic devices all have limitations to varying degrees when facing the task of detecting reference points in modern high-density electronic devices: template matching methods are limited by rigid matching mechanisms and are difficult to adapt to process changes and environmental interference; color threshold segmentation methods perform poorly under low contrast and uneven lighting conditions; texture feature methods are computationally complex and lack geometric constraints. These problems are further amplified when the device spacing is reduced to the sub-millimeter level, and any tiny positioning deviation may lead to serious false detections or missed detections, failing to meet the comprehensive requirements of modern electronic manufacturing for detection accuracy, stability, and real-time performance. Summary of the Invention

[0007] Therefore, the technical problem to be solved by the present invention is to overcome the problem that the existing reference point detection methods for electronic devices cannot meet the comprehensive requirements of electronic manufacturing for detection accuracy, stability and real-time performance.

[0008] To address the aforementioned technical problems, this invention provides a method for detecting reference points in electronic devices, comprising: Acquire images of the area to be located for the reference point, detect the image of the area to be located for the reference point, and obtain at least one coarse circular region where the reference point is located. Based on the radius of each coarse positioning circular region and the radius of the standard reference point, the positioning score of each coarse positioning circular region is calculated; the coarse positioning circular regions with positioning scores greater than the preset positioning score threshold are taken as the target coarse positioning circular regions. The edge points of the target coarsely localized circular region are extracted multiple times, and the edge points extracted each time are fitted to obtain multiple fitted circles corresponding to the target coarsely localized circular region. Calculate the difference between the radius of each fitted circle and the radius of the standard reference point, and take the fitted circle with the smallest difference as the best fitted circle, thus obtaining the reference point of the electronic device.

[0009] Preferably, the image of the region to be located for the reference point is detected to obtain at least one coarse circular region where the reference point is located, including: The spot detection algorithm is used to detect the image of the reference point to be located, and it is determined whether there is at least one coarse circular region with a positioning score greater than a preset positioning score threshold in the detection result. If it exists, the coarse positioning circular region with a positioning score greater than the preset positioning score threshold is taken as the target coarse positioning circular region; if it does not exist, the whole image circle finding algorithm is used to perform secondary detection on the image of the reference point to be located, and it is determined whether there is at least one coarse positioning circular region with a positioning score greater than the preset positioning score threshold in the secondary detection result. If it exists, the coarse positioning circular region with a positioning score greater than the preset positioning score threshold is taken as the target coarse positioning circular region; if it does not exist, the color contour analysis algorithm is used to perform three detections on the image of the reference point to be located region.

[0010] Preferably, the image of the region to be located using a blob detection algorithm is used to detect the reference point, including: The image of the reference point to be located is preprocessed, and a Gaussian-scale spatial pyramid is constructed on the preprocessed image of the reference point to be located. Different thresholds are used to binarize the reference point to be located images at different scales to obtain binarized images at different scales, and the geometric features of the connected components in each binarized image are obtained. The position stability coefficient of each connected component is calculated based on the ratio of the variance of the position coordinates of each connected component in different binarized images to the preset position stability reference variance. The scale stability coefficient of each connected component is calculated based on the proportion of the scale span of each connected component to the difference between the maximum and minimum scales of the Gaussian scale space pyramid. The response stability coefficient of each connected component is calculated based on the ratio of the average response intensity to the standard deviation of the response intensity in different binarized images. The stability score of each connected component is obtained by weighting the position stability coefficient, scale stability coefficient, and response stability coefficient of each connected component. Connected components with stability scores greater than a preset score threshold are selected as coarsely located circular regions.

[0011] Preferably, a full-image circle-finding algorithm is used to perform secondary detection on the image of the region to be located for the reference point, including: Calculate the gradient magnitude of each pixel in the image of the region to be located, and obtain the gradient magnitude image; In the gradient magnitude image, select pixels whose gradient magnitude is greater than the gradient magnitude of each pixel in the surrounding preset neighborhood as initial anchor points; Using each initial anchor point as the anchor point to be linked, a search is performed along the vertical direction of the gradient to obtain pixels that meet the search conditions. These pixels are then used as link points to connect to the anchor points to be linked, forming continuous edge segments. The newly added link points in each continuous edge segment are used as anchor points to be linked. The search is performed along the vertical direction of the gradient to update each continuous edge segment until there are no link points that meet the search conditions, thus obtaining the continuous edge segments of each initial anchor point. Fit each continuous edge segment to obtain the corresponding circular shape; verify the false alarm count of all circular shapes. If the verification is successful, each circular shape is used as the coarsely located circular region. The search conditions include a gradient magnitude greater than the gradient magnitude of the anchor to be linked, a gradient direction difference between the gradient and the anchor to be linked less than a preset difference, and a distance between the gradient and the anchor to be linked less than a preset distance threshold.

[0012] Preferably, the image of the region to be located using the color contour analysis algorithm is subjected to three detections, including: Based on all pixels in the image of the reference point to be located, the coefficient of variation and outliers of the image of the reference point to be located are calculated; the image of the reference point to be located is scaled to obtain the image of the reference point to be located at different scales. When the coefficient of variation is less than the preset coefficient of variation threshold, the standard deviation statistical method is used to extract the color range of the region where the reference point is located in the image of the reference point to be located at different scales. When the outlier exceeds the preset outlier threshold, the median absolute deviation statistical method is used to extract the color range of the region where the reference point is located in the image of the reference point to be located at different scales. When the coefficient of variation is greater than or equal to the preset coefficient of variation threshold and the outlier is less than or equal to the preset outlier threshold, the interquartile range statistical method is used to extract the color range of the region where the reference point is located in the image of the reference point to be located at different scales. Calculate the confidence score of the color interval of the region where the reference point is located at different scales, and take the color interval of the region where the reference point has the highest confidence score as the standard interval; The image of the region to be located is segmented based on the standard interval to obtain the reference point region and the non-reference point region; the contour is extracted from the reference point region, and the geometric features of the extracted contour are compared with the radius of the standard reference point to obtain the coarse positioning circular region.

[0013] Preferably, extracting the edge points of the target coarsely localized circular region multiple times includes: Centered on the center of the target coarse positioning circular area, set multiple rotating rectangles with preset widths and heights at preset angle intervals along the edge of the target coarse positioning circular area, which serve as edge point extraction calipers; Using calipers extracted from each edge point, pixels on the edge of the target coarsely located circular area are sampled at a fixed step size; the gray values ​​of pixels at the same sampling position of all edge point calipers are averaged to obtain the projected average gray value sequence. The gradient sequence is obtained by performing differential calculation on the projected average grayscale sequence; multiple gradient thresholds are set, and effective gradient points in the gradient sequence are filtered based on each gradient threshold. If there are multiple consecutive effective gradient points for a gradient threshold, the pixel point corresponding to the gradient peak among the multiple effective gradient points is taken as the target pixel point, and the position of the target pixel point is taken as the edge sampling position of the edge point extraction caliper under the gradient threshold. Based on the pixel points collected at the edge sampling positions of the calipers at each edge point under each gradient threshold, a set of edge points for the target coarsely localized circular region is obtained.

[0014] Preferably, after selecting the pixel corresponding to the gradient peak among multiple effective gradient points as the target pixel, the method further includes: Select multiple pixels within a preset distance of the target pixel, perform quadratic function fitting on the multiple pixels, and calculate the extreme points of the fitted quadratic function. Calculate the sum of the gradient peak and extreme points among multiple effective gradient points, and calculate the set angle of the caliper, the cosine value of the set angle, and the product of the sum for each edge point; The sum of the x-coordinate of the center point of the caliper extracted from each edge point and the product of the product yields the x-coordinate of the edge sampling position of the caliper extracted from each edge point. The sum of the ordinate of the center point of the caliper extracted from each edge point and the product of the ordinate is used to obtain the ordinate of the edge sampling position of the caliper extracted from each edge point.

[0015] Preferably, after selecting the fitted circle with the smallest difference as the best fitted circle, the process further includes: Based on the fitted circle, the radius and center of the best-fit circle, calculate the deviation of the radial distance from each edge point to the best-fit circle. Based on the deviation of the radial distance from each edge point to the best-fit circle, the sum of the deviations of the radial distances from all edge points to the best-fit circle is obtained, thus obtaining the deviation distance between the edge points and the best-fit circle. If the deviation distance is less than the preset minimum tolerance distance, and the number of edge points whose radial distance to the best-fit circle is less than the preset minimum tolerance distance is greater than the preset minimum tolerance number, then the reference point position of the electronic device is obtained based on the best-fit circle.

[0016] The present invention also provides a reference point detection device for electronic devices, comprising: The coarse positioning module is used to acquire images of the area to be positioned of the reference point, detect the images of the area to be positioned of the reference point, and obtain at least one coarse positioning circular area where the reference point is located. The coarse positioning circular region filtering module is used to calculate the positioning score of each coarse positioning circular region based on the radius of each coarse positioning circular region and the radius of the standard reference point; and to select coarse positioning circular regions whose positioning scores are greater than the preset positioning score threshold as target coarse positioning circular regions. The edge extraction and fitting module is used to extract the edge points of the target coarsely located circular region multiple times, and fit the edge points extracted each time to obtain multiple fitted circles corresponding to the target coarsely located circular region. The reference point acquisition module is used to calculate the difference between the radius of each fitted circle and the radius of the standard reference point, and selects the fitted circle with the smallest difference as the best fitted circle, thereby obtaining the reference point of the electronic device.

[0017] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the above-described reference point detection method for electronic devices.

[0018] The reference point detection method for electronic devices provided in this application has the following beneficial effects: This application directly acquires the image of the reference point to be located area, without directly detecting the entire image of the electronic device, thus narrowing the detection range and reducing background interference. Then, based on the circular shape of the reference point, the circular area is used as the core detection target. First, a coarse localization screening is performed on the image of the reference point to be located area. This avoids the problem of grayscale algorithm failure caused by differences in silkscreen printing and lighting changes, as it does not rely on the consistency of grayscale distribution between the reference point and the template. After obtaining multiple coarse localization circular areas based on shape characteristics, the radius of the standard reference point is used as a benchmark to evaluate the localization of each coarse localization circular area, selecting the target coarse localization circular area whose radius is closest to the standard reference point. Then, by extracting the edge points of the target coarse localization circular area and fitting the extracted edge points, a fine fitting of the region where the reference point is located is achieved. Specifically, edge points are extracted multiple times within a small target coarse localization circular area. Even if the contrast between the reference point and the substrate is low, the edges can be accurately captured in the local area, and multiple extractions are performed. Edge point extraction can offset the deviation caused by a single extraction. Finally, multiple edge points are fitted to obtain multiple fitted circles. The fitted circle whose radius is closest to the radius of the standard reference point is taken as the best fitted circle, thereby achieving accurate positioning of the reference point area. This application directly uses the circular geometric features of the reference point as the core of detection. The target is locked through radius screening and edge fitting. It does not rely on the local statistical characteristics of texture. It takes into account the characteristic that the reference point has a clear geometric shape and avoids the defect that texture features cannot characterize spatial structure. At the same time, it does not require setting multi-scale and multi-directional texture extraction windows, nor does it require a lot of convolution operations. Detection can be completed by coarse positioning screening and multiple rounds of edge fitting. It is suitable for general reference point sub-pixel positioning when there are changes in lighting, shape changes, and complex positioning backgrounds. It effectively solves the problems of insufficient positioning accuracy and low robustness caused by the diversity, randomness, and low contrast of the shape in the reference point positioning process, and meets the requirements of accuracy and detection efficiency in the detection process. Attached Figure Description

[0019] To make the content of this invention easier to understand, the invention will be further described in detail below with reference to specific embodiments and accompanying drawings, wherein: Figure 1 A flowchart of the reference point detection method for electronic devices provided in this application; Figure 2 This application provides a coarsely localized circular region detected using a blob detection algorithm; wherein, Figure 2 Image (a) in the image is the region to be located using the first type of reference point. Figure 2 Image (b) is a schematic diagram of the coarsely located circular region obtained by detecting the image of the region to be located using the first type of reference point. Figure 2 Image (c) in the image represents the region to be located using the second type of reference point. Figure 2 (d) in the figure is a schematic diagram of the coarse positioning circular region obtained by detecting the image of the region to be located by the second type of reference point; Figure 2 Image (e) in the image represents the region to be located using the third type of reference point. Figure 2 (f) in the figure is a schematic diagram of the coarse positioning circular region obtained by detecting the image of the region to be located by the third reference point; Figure 3 This application provides a coarsely located circular region detected using a full-image circle-finding algorithm; wherein, Figure 3 Image (a) in the image is the region to be located using the first type of reference point. Figure 3 Image (b) is a schematic diagram of the coarsely located circular region obtained by detecting the image of the region to be located using the first type of reference point. Figure 3 Image (c) in the image represents the region to be located using the second type of reference point. Figure 3 (d) in the figure is a schematic diagram of the coarse positioning circular region obtained by detecting the image of the region to be located by the second type of reference point; Figure 3 Image (e) in the image represents the region to be located using the third type of reference point. Figure 3 (f) in the figure is a schematic diagram of the coarse positioning circular region obtained by detecting the image of the region to be located by the third reference point; Figure 4 This application provides a coarsely located circular region detected using a color contour analysis algorithm; wherein, Figure 4 Image (a) in the image is the region to be located using the first type of reference point. Figure 4 Image (b) is a schematic diagram of the coarsely located circular region obtained by detecting the image of the region to be located using the first type of reference point. Figure 4 Image (c) in the image represents the region to be located using the second type of reference point. Figure 4 (d) in the figure is a schematic diagram of the coarse positioning circular region obtained by detecting the image of the region to be located by the second type of reference point; Figure 4 Image (e) in the image represents the region to be located using the third type of reference point. Figure 4 (f) in the figure is a schematic diagram of the coarse positioning circular region obtained by detecting the image of the region to be located by the third reference point; Figure 5A schematic diagram of edge points before and after differential filter processing provided in this application; Figure 6 This is a schematic diagram illustrating the edge point extraction and localization effect provided in this application; Figure 7 A flowchart of the reference point testing operation for the electronic device provided in this application; Figure 8 The embodiments of this application are based on Figure 7 The diagram illustrates the detection results obtained through the process shown; where... Figure 8 Image (a) in this embodiment is an image of the area to be located for a reference point. Figure 8 (b) in the text is a pair Figure 8 (a) is a schematic diagram of sub-pixel edge points obtained by detection. Figure 8 (c) in the text is a pair Figure 8 (a) is a schematic diagram of the benchmark positioning results obtained through detection. Figure 8 In this embodiment, (d) represents the second type of reference point image to be located. Figure 8 (e) in the text is a pair Figure 8 A schematic diagram of sub-pixel edge points obtained by detecting (d) in the diagram. Figure 8 (f) in the middle is a pair Figure 8 A schematic diagram of the benchmark positioning results obtained by detecting (d) in the diagram. Figure 8 In this embodiment, (g) represents the third type of reference point image to be located. Figure 8 (h) in the text represents the pair Figure 8 A schematic diagram of sub-pixel edge points obtained by detecting (g) in the diagram. Figure 8 (i) in the text represents the pair Figure 8 A schematic diagram of the benchmark positioning results obtained by detecting (g) in the diagram. Figure 8 In this embodiment, (j) represents the fourth type of reference point image to be located. Figure 8 (k) in the figure represents the pair Figure 8 A schematic diagram of sub-pixel edge points obtained by detecting (j) in the diagram. Figure 8 (l) in the middle represents the pair Figure 8 A schematic diagram of the benchmark positioning results obtained by detecting (j) in the diagram; Figure 9 This is a schematic diagram illustrating the effect of benchmark detection and positioning provided in an embodiment of this application. Detailed Implementation

[0020] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, so that those skilled in the art can better understand and implement the present invention. However, the embodiments described are not intended to limit the present invention.

[0021] Please see Figure 1 , Figure 1 The diagram shows a flowchart of the reference point detection method for electronic devices provided in this application. The method specifically includes: S10: Acquire an image of the area to be located for the reference point, detect the image of the area to be located for the reference point, and obtain at least one coarse positioning circular area where the reference point is located.

[0022] Specifically, since electronic devices (such as PCB boards) are fed into the board along the x-axis or y-axis during inspection, and there will only be slight fluctuations in the other direction, the detection range of the reference point can be limited to the area along the vertical direction of board entry, thereby effectively reducing the occurrence of locating similar reference points.

[0023] S20: Calculate the positioning score of each coarse positioning circular region based on the radius of each coarse positioning circular region and the radius of the standard reference point; take the coarse positioning circular region with a positioning score greater than the preset positioning score threshold as the target coarse positioning circular region.

[0024] Specifically, by calculating the error between the radius of the coarse positioning circular area and the radius of the standard reference point, the positioning score can be calculated for each coarse positioning circular area. The calculation formula is: , in, Indicates the radius of the coarse positioning circular area; This represents the radius of the standard reference point; this calculation formula can both assess whether the coarse positioning circular area was successfully detected and reflect the matching accuracy between the detected coarse positioning circular area and the standard reference point.

[0025] S30: Extract the edge points of the target coarsely located circular region multiple times, and fit the edge points extracted each time to obtain multiple fitted circles corresponding to the target coarsely located circular region.

[0026] S40: Calculate the difference between the radius of each fitted circle and the radius of the standard reference point, and take the fitted circle with the smallest difference as the best fitted circle, thereby obtaining the reference point of the electronic device.

[0027] Further, in step S10, the image of the region to be located for the reference point is detected to obtain at least one coarsely positioned circular region where the reference point is located, including: Step 1-1: Use the blob detection algorithm to detect the image of the reference point to be located, and determine whether there is at least one coarse positioning circular region in the detection result with a positioning score greater than the preset positioning score threshold.

[0028] Step 1-2: If it exists, the coarse positioning circular region with a positioning score greater than the preset positioning score threshold is taken as the target coarse positioning circular region; if it does not exist, the whole-image circle-finding algorithm is used to perform secondary detection on the image of the reference point to be located, and it is determined whether there is at least one coarse positioning circular region with a positioning score greater than the preset positioning score threshold in the secondary detection results.

[0029] Steps 1-3: If it exists, the coarse positioning circular region with a positioning score greater than the preset positioning score threshold is taken as the target coarse positioning circular region; if it does not exist, the color contour analysis algorithm is used to perform three detections on the image of the reference point to be located region.

[0030] Three different detection algorithms are used to coarsely locate the area where the reference point is located. First, the speckle detection algorithm is used to search for speckle areas in the image that conform to the circle. If the search fails, the whole image circle search algorithm is used to search for circular areas. If the first two algorithms cannot locate the area where the reference point is located, color is extracted and contour analysis is performed to search for areas in the image that conform to the contour features of the reference point.

[0031] Blob detection is a multi-scale image analysis technique used to identify connected regions in an image that share similar set features. This algorithm preprocesses the input image and constructs a Gaussian scale-space pyramid to capture features at different scales. It then performs binarization using different thresholds at multiple scale levels, extracts the geometric features of connected components, establishes temporally consistent connections for candidate blobs through scale-space tracking, and finally selects stable regions that persist across multiple scale levels and exhibit high response intensity based on stability metrics.

[0032] Specifically, step 1-1 uses a blob detection algorithm to detect the image of the region to be located for the reference point, including: Step 1-1-1: Preprocess the image of the reference point to be located region, and construct a Gaussian scale spatial pyramid on the preprocessed reference point to be located region image.

[0033] Specifically, preprocessing includes grayscale conversion and noise filtering of the image of the reference point to be located; constructing a Gaussian scale spatial pyramid can obtain feature information of the image of the reference point to be located at different scales.

[0034] Step 1-1-2: Use different thresholds to binarize the reference point to be located region images at different scales to obtain binarized images at different scales, and obtain the geometric features of the connected components in each binarized image.

[0035] Specifically, the geometric characteristics of a connected component can be the area, roundness, or convexity of the connected region, etc.

[0036] Step 1-1-3: Calculate the position stability coefficient of each connected component based on the ratio of the variance of the position coordinates of each connected component in different binarized images to the preset position stability reference variance.

[0037] Specifically, position stability coefficient The calculation formula is: , in, This represents the variance of the position coordinates of connected components in different binarized images; This represents the reference variance for the stability of the preset position.

[0038] The positional stability coefficient is used to evaluate the consistency of the detected positional coordinates of the same spot at different scales. The smaller the variance of the positional coordinates of the connected components in different binarized images, the more stable the spot is.

[0039] Step 1-1-4: Calculate the scale stability coefficient of each connected component based on the proportion of the scale span of each connected component to the difference between the maximum and minimum scales of the Gaussian scale space pyramid.

[0040] Specifically, scale stability coefficient The calculation formula is: , in, Indicates the scale span of the connected components; This represents the maximum scale of the Gaussian-scale spatial pyramid; This represents the smallest scale of the Gaussian-scale spatial pyramid.

[0041] The scale stability coefficient is used to assess the ability of a spot to persist across multiple scale levels. The larger the scale span, the more stable the spot is.

[0042] Step 1-1-5: Calculate the response stability coefficient of each connected component based on the ratio of the average response intensity to the standard deviation of the response intensity in different binarized images.

[0043] Specifically, the response stability coefficient The calculation formula is: , in, This represents the average response intensity of the connected components in different binarized images; This represents the standard deviation of the response intensity of connected components in different binarized images; To prevent constants with a denominator of 0.

[0044] The scale stability coefficient is used to assess the ability of a spot to persist across multiple scale levels. The larger the scale span, the more stable the spot is.

[0045] Step 1-1-5: Calculate the response stability coefficient of each connected component based on the ratio of the average response intensity to the standard deviation of the response intensity in different binarized images.

[0046] The response stability coefficient is used to evaluate the consistency of the detection response intensity of a spot. The larger the coefficient, the more stable and stronger the response of the spot.

[0047] Step 1-1-6: Based on the weighted sum of the position stability coefficient, scale stability coefficient, and response stability coefficient of each connected component, obtain the stability score of each connected component.

[0048] Specifically, stability score The calculation formula is: , in, , , These are the weights of the location stability coefficient, scale stability coefficient, and response stability coefficient, respectively.

[0049] Step 1-1-7: Select connected components with stability scores greater than a preset score threshold as coarsely located circular regions.

[0050] In blob detection algorithms, selecting stable blob regions is crucial. This application evaluates the stability of blob regions by combining three indicators, ultimately using high-quality blobs with stable positions, consistent scale, and strong responses as reference points for coarse localization. Figure 2 The image shows a coarsely located circular region detected using a blob detection algorithm provided in this application; wherein, Figure 2 Image (a) in the image is the region to be located using the first type of reference point. Figure 2 Image (b) is a schematic diagram of the coarsely located circular region obtained by detecting the image of the region to be located using the first type of reference point. Figure 2 Image (c) in the image represents the region to be located using the second type of reference point. Figure 2 (d) in the figure is a schematic diagram of the coarse positioning circular region obtained by detecting the image of the region to be located by the second type of reference point; Figure 2 Image (e) in the image represents the region to be located using the third type of reference point. Figure 2 (f) in the figure is a schematic diagram of the coarse positioning circular region obtained by image detection of the region to be located by the third reference point.

[0051] Since the area where the reference point is located has a near-circular feature, this application introduces a full-image circle-finding algorithm when the speckle detection algorithm fails to roughly locate the area where the reference point is located. This algorithm uses the Prewitt or Sobel operator to calculate the gradient of the input image, identifies the local maximum point of the gradient magnitude through anchor point detection, and then links the edges along the vertical direction of the gradient from the anchor point to form continuous edge segments. Geometric shape fitting is performed on the effective edge segments (for example, ellipse fitting uses the least squares method to solve the general ellipse equation). Finally, the reliability of the detection results is ensured by statistical verification of the number of false alarms. This algorithm has high detection accuracy and processing speed, can effectively handle noise interference, has low parameter sensitivity, and is suitable for real-time vision applications such as industrial inspection.

[0052] Specifically, in steps 1-2, the whole-image circle-finding algorithm is used to perform secondary detection on the image of the region to be located, including: Step 1-2-1: Calculate the gradient magnitude of each pixel in the image of the region to be located, and obtain the gradient magnitude image.

[0053] Step 1-2-2: Select pixels in the gradient magnitude image whose gradient magnitude is greater than the gradient magnitude of each pixel in the surrounding preset neighborhood as initial anchor points.

[0054] Steps 1-2-3: Using each initial anchor point as the anchor point to be linked, search along the vertical direction of the gradient to obtain pixels that meet the search conditions. These pixels are then used as link points to connect to the anchor points to be linked, forming continuous edge segments.

[0055] Where, the gradient is perpendicular to the direction The formula is expressed as: , in, This indicates the gradient direction of the anchor point to be linked.

[0056] The search criteria include a gradient magnitude greater than the gradient magnitude of the anchor point to be linked, a gradient direction difference between the two anchor points less than a preset difference, and a distance between the two anchor points less than a preset distance threshold. These criteria ensure that the searched points are spatially continuous, preventing jump points from being included in continuous edge segments. Specifically, the formula for the spatial continuity constraint is: ,in,( , ) represents the coordinates of the anchor point to be linked. , () represents the coordinates of the link point; Typically set to This indicates that connectivity to eight neighboring domains is allowed.

[0057] Step 1-2-4: Take the newly added link points in each continuous edge segment as anchor points to be linked, and search along the gradient vertical direction to update each continuous edge segment until there are no link points that meet the search conditions, thus obtaining the continuous edge segments of each initial anchor point.

[0058] Step 1-2-5: Fit each continuous edge segment to obtain the corresponding circular shape; verify the false alarm count of all circular shapes. If the verification is successful, each circular shape is used as the coarsely located circular region.

[0059] Specifically, the least squares method is used to fit a circle-like shape for the point set on the continuous edge segment. The minimum error function of its fit is: ,in, , , , , , These represent the coefficients of the general equation of an ellipse.

[0060] Furthermore, false alarm count verification is a statistical verification method used to evaluate whether the detected geometric shape is real or a random phenomenon caused by random noise. Its core idea is: assuming the edge points in the image are randomly distributed, calculate the probability of observing the current detection result under random conditions. If this probability is small, the detection result is considered meaningful, meaning the detected geometric shape is real. For the detection of circular-like features, the binomial distribution is calculated as follows: , , in, This represents the probability that exactly k points out of n points fall on the ellipse. This represents the total number of ellipses that may exist in the image of the area to be located at the reference point; , These represent the width and height of the image of the region to be located at the reference point, respectively. Represents the minimum area of ​​the ellipse; This represents the number of points in the image of the region to be located that are fitted with all the ellipses; Represents the number of points in a given ellipse; This represents the probability that a random point falls on the ellipse.

[0061] Figure 3 The image shown is a coarsely located circular region detected using the full-image circle-finding algorithm provided in this application; wherein, Figure 3 Image (a) in the image is the region to be located using the first type of reference point. Figure 3Image (b) is a schematic diagram of the coarsely located circular region obtained by detecting the image of the region to be located using the first type of reference point. Figure 3 Image (c) in the image represents the region to be located using the second type of reference point. Figure 3 (d) in the figure is a schematic diagram of the coarse positioning circular region obtained by detecting the image of the region to be located by the second type of reference point; Figure 3 Image (e) in the image represents the region to be located using the third type of reference point. Figure 3 (f) in the figure is a schematic diagram of the coarse positioning circular region obtained by image detection of the region to be located by the third reference point.

[0062] Furthermore, in steps 1-3, the color contour analysis algorithm is used to perform three detections on the image of the reference point to be located, including: Step 1-3-1: Based on all pixels in the image of the reference point to be located, calculate the coefficient of variation and outliers of the image of the reference point to be located; scale the image of the reference point to be located to obtain the image of the reference point to be located at different scales.

[0063] For example, reference point images of the region to be located at three scales of 1.0x, 0.8x, and 0.6x can be obtained. In other embodiments, reference point images of the region to be located at other scales can also be obtained, and this application does not limit this.

[0064] Step 1-3-2: When the coefficient of variation is less than the preset coefficient of variation threshold, the standard deviation statistical method is used to extract the color range of the region where the reference point is located in the image of the reference point to be located at different scales.

[0065] Step 1-3-3: When the outlier is greater than the preset outlier threshold, the median absolute deviation statistical method is used to extract the color range of the region where the reference point is located in the image of the reference point to be located at different scales.

[0066] Step 1-3-4: When the coefficient of variation is greater than or equal to the preset coefficient of variation threshold and the outlier is less than or equal to the preset outlier threshold, the interquartile range statistical method is used to extract the color range of the region where the reference point is located in the image of the reference point to be located at different scales.

[0067] Specifically, when the color consistency of the image is high (i.e., coefficient of variation CV < 0.1), the high-precision standard deviation statistical method is used. When the outliers are high (i.e., |mean - median| > 0.5σ), the median absolute deviation (MAD) statistical method with higher anti-interference performance is used. In general, only the most robust interquartile range (IQR) statistical method is needed to ensure the stability of color extraction.

[0068] Step 1-3-5: Calculate the confidence score of the color interval of the region where the reference point is located at different scales, and take the color interval of the region where the reference point has the highest confidence score as the standard interval.

[0069] Step 1-3-6: Segment the image of the region to be located based on the standard interval to obtain the reference point region and the non-reference point region; extract the contour of the reference point region, compare the geometric features of the extracted contour with the radius of the standard reference point, and select the coarse positioning circular region.

[0070] Specifically, the geometric features of the contour can be the roundness of the contour and the minimum enclosing circle of the contour points. By comparing the geometric features of the contour points with the radius of the standard reference point, the most likely coarse positioning circular area can be extracted.

[0071] Figure 4 The image shows a coarsely located circular region detected using a color contour analysis algorithm provided in this application; wherein, Figure 4 Image (a) in the image is the region to be located using the first type of reference point. Figure 4 Image (b) is a schematic diagram of the coarsely located circular region obtained by detecting the image of the region to be located using the first type of reference point. Figure 4 Image (c) in the image represents the region to be located using the second type of reference point. Figure 4 (d) in the figure is a schematic diagram of the coarse positioning circular region obtained by detecting the image of the region to be located by the second type of reference point; Figure 4 Image (e) in the image represents the region to be located using the third type of reference point. Figure 4 (f) in the figure is a schematic diagram of the coarse positioning circular region obtained by image detection of the region to be located by the third reference point.

[0072] Furthermore, after obtaining the target coarsely located circular region, the sub-pixel extraction of the reference point edge points is achieved by using a circular caliper algorithm with multi-gradient thresholds. The algorithm process includes: inputting the center and radius of the circle to generate a radial caliper, then performing bilinear interpolation sampling, projection averaging, gradient filtering, peak detection, and sub-pixel precise positioning, and finally outputting the edge points.

[0073] Specifically, step S30 involves repeatedly extracting the edge points of the target coarsely localized circular region, including: Step 2-1: Using the center of the target coarse positioning circular area as the center, set multiple rotating rectangles with preset widths and heights at preset angle intervals along the edge of the target coarse positioning circular area, as edge point extraction calipers.

[0074] Specifically, the extension direction of each radial caliper is perpendicular to the tangent direction of the target coarse positioning circular area. The calculation formulas for the circumferential points, starting points, and ending points of the generated edge point extraction calipers are expressed as follows: , , , in, This indicates that, using the center of the coarsely positioned circular area as a reference, the edge point extraction caliper is used at a radius r. The coordinates of a point on the circumference in the direction; The center of the edge point extraction caliper is represented by r; the radius of the edge point extraction caliper is represented by r. This indicates the starting point for edge point extraction using calipers; This indicates the height of the caliper used to extract edge points; This represents the parameters that control the direction of caliper extension extracted from the edge points; This indicates that the angle of the caliper is extracted from the edge points, with the center of the circular area where the target is roughly located as the center.

[0075] Step 2-2: Using the edge point extraction caliper, the pixels at the edge of the target coarsely positioned circular area are sampled at a fixed step size; the gray values ​​of the pixels at the same sampling position of all edge point extraction calipers are arithmetically averaged to obtain the projected average gray value sequence.

[0076] Steps 2-3: Perform differential calculation on the projected average grayscale sequence to obtain the gradient sequence; set multiple gradient thresholds, and filter the effective gradient points in the gradient sequence based on each gradient threshold. If there are multiple consecutive effective gradient points for a gradient threshold, the pixel point corresponding to the gradient peak among the multiple effective gradient points is taken as the target pixel point, and the position of the target pixel point is taken as the edge sampling position of the edge point extraction caliper under the gradient threshold.

[0077] It should be noted that multiple consecutive valid gradient points refer to the existence of at least two adjacent gradient values ​​in the gradient sequence that are greater than the gradient threshold, meaning that these two adjacent gradient values ​​are both valid gradient points.

[0078] For example, if calipers are extracted from N edge points, and each point generates M sampling points according to a fixed step size, then the caliper... Grayscale sequence generated by extracting calipers from edge points. for: The projected average grayscale sequence refers to the arithmetic mean of the sampled values ​​at the same sampling point in the grayscale sequence generated by extracting calipers from all edge points, resulting in a one-dimensional projected average grayscale sequence.

[0079] Specifically, steps 2-2 and 2-3 utilize multiple edge point extraction calipers to perform image subpixel sampling, projection averaging, and differential filter processing. Image subpixel sampling effectively eliminates the "step effect," resulting in smoother edge responses and improved accuracy in subsequent subpixel positioning, while also addressing the issue of non-integer coordinate sampling. Projection averaging projects a two-dimensional matrix to one dimension, reducing interference from outliers on the localization of reference points. The differential filter further emphasizes edge positions, suppresses flat areas, reduces the impact of random noise, distinguishes edges in different directions, and requires only simple addition and subtraction operations, exhibiting high real-time performance. Figure 5 The diagram shown is a schematic of the edge points before and after processing by the differential filter provided in this application.

[0080] Steps 2-4: Extract the pixel points collected at the edge sampling positions of the caliper based on each edge point under each gradient threshold to obtain a set of edge points for the target coarsely localized circular region.

[0081] Specifically, filtering effective gradient points based on gradient thresholds means selecting points in the gradient sequence that are greater than the gradient threshold as effective gradient points. One gradient threshold can filter out a set of effective gradient points, thereby extracting the edge sampling position of the caliper corresponding to a set of edge points. By setting multiple gradient thresholds and filtering multiple times, multiple sets of edge points can be extracted. By fitting each set of edge points, multiple fitted circles can be obtained. Finally, the fitted circle whose radius is closest to the radius of the standard reference point is selected.

[0082] Furthermore, after taking the pixel corresponding to the gradient peak as the target pixel, by performing quadratic function fitting on multiple pixels near the target pixel and using the mathematical difference method to solve for the true extreme value position of the continuous function, the positioning accuracy of the edge sampling position can be further improved from ±0.5 pixels to ±0.1 pixels. Therefore, after taking the pixel corresponding to the gradient peak among multiple effective gradient points as the target pixel in steps 2-3, the following steps are also included: Step 2-3-1: Select multiple pixels within a preset distance from the target pixel, perform quadratic function fitting on the multiple pixels, and calculate the extreme points of the fitted quadratic function.

[0083] For example, for three points near the target pixel: (-1, a), (0, b), and (1, c), fit a quadratic function. Finally, the formula for calculating the extreme point can be obtained as follows: .

[0084] Step 2-3-2: Calculate the sum of the gradient peak and extreme points among multiple effective gradient points, and calculate the set angle of the caliper extraction for each edge point, the cosine value of the set angle, and the product of the sum.

[0085] Step 2-3-3: Based on the sum of the x-coordinate of the center point of the caliper extracted from each edge point and the product of the x-coordinate, the x-coordinate of the edge sampling position of the caliper extracted from each edge point is obtained.

[0086] Steps 2-3-4: Based on the sum of the ordinate of the center point of the caliper extracted from each edge point and the product of the ordinate, the ordinate of the edge sampling position of the caliper extracted from each edge point is obtained.

[0087] For example, the x-coordinate of the edge sampling position of the edge point extraction caliper is represented as: The ordinate of the edge sampling position of the edge point extraction caliper is represented as: ,in, and These represent the x and y coordinates of the center point of the edge point extraction caliper, respectively, as follows: Figure 6 The image shown is a schematic diagram illustrating the edge point extraction and localization effect provided in this application.

[0088] Furthermore, after selecting the fitted circle with the smallest difference as the best fitted circle, the following steps are also taken: Step 3-1: Based on the fitted circle, the radius of the best-fit circle, and the center of the circle, calculate the deviation of the radial distance from each edge point to the best-fit circle.

[0089] Step 3-2: Based on the deviation of the radial distance from each edge point to the best-fit circle, obtain the sum of the deviations of the radial distances from all edge points to the best-fit circle, thus obtaining the deviation distance between the edge point and the best-fit circle.

[0090] Step 3-3: If the deviation distance is less than the preset minimum tolerance distance, and the number of edge points whose radial distance deviation to the best-fit circle is less than the preset minimum tolerance distance is greater than the preset minimum tolerance number, then the reference point position of the electronic device is obtained based on the best-fit circle.

[0091] Specifically, steps 3-1 to 3-3 above determine whether the best-fit circle is the region where the true reference point is located using the following formula. If the following formula is satisfied, it indicates that the best-fit circle is the region where the true reference point is located: , s , in, This indicates that the best-fit circle was obtained by fitting the circle. The coordinates of the edge points; This represents the number of edge points of the best-fit circle obtained through fitting; Represents the coordinates of the center of the best-fit circle; Represents the radius of the best-fit circle; This represents the preset minimum tolerance distance; s represents the number of edge points whose radial distance deviation from the best-fit circle is less than the preset minimum tolerance distance. This indicates the preset minimum tolerance quantity.

[0092] When the position of the coarsely located circular region is abnormal, the extracted edge points are not the actual location of the reference point. In this case, although the shape obtained by fitting the edge points is not circular, the radius of the near-circular shape obtained by fitting with the least squares method may be very close to the radius of the standard reference point, thus causing the incorrect fitted circle to be regarded as the reference point region. Therefore, after obtaining the best fitted circle, this application uses the above steps to judge whether the positioning is accurate again, thereby avoiding the situation of locating non-reference point regions.

[0093] like Figure 7 The diagram shown is a flowchart of the reference point testing operation for the electronic device provided in this application. Figure 8 The illustration shown is based on an embodiment of this application. Figure 7 The diagram illustrates the detection results obtained through the process shown; where... Figure 8 Image (a) in this embodiment is an image of the area to be located for a reference point. Figure 8 (b) in the text is a pair Figure 8 (a) is a schematic diagram of sub-pixel edge points obtained by detection. Figure 8 (c) in the text is a pair Figure 8 (a) is a schematic diagram of the benchmark positioning results obtained through detection. Figure 8 In this embodiment, (d) represents the second type of reference point image to be located. Figure 8 (e) in the text is a pair Figure 8 A schematic diagram of sub-pixel edge points obtained by detecting (d) in the diagram. Figure 8 (f) in the middle is a pair Figure 8 A schematic diagram of the benchmark positioning results obtained by detecting (d) in the diagram. Figure 8 In this embodiment, (g) represents the third type of reference point image to be located. Figure 8 (h) in the text represents the pair Figure 8 A schematic diagram of sub-pixel edge points obtained by detecting (g) in the diagram. Figure 8 (i) in the text represents the pair Figure 8 A schematic diagram of the benchmark positioning results obtained by detecting (g) in the diagram. Figure 8 In this embodiment, (j) represents the fourth type of reference point image to be located. Figure 8 (k) in the figure represents the pair Figure 8 A schematic diagram of sub-pixel edge points obtained by detecting (j) in the diagram. Figure 8 (l) in the middle represents the pair Figure 8 The diagram shows the benchmark positioning results obtained by detecting (j) in the diagram.

[0094] Figure 9 The diagram shown is a schematic representation of the benchmark detection and positioning effect provided in an embodiment of this application.

[0095] This application first uses a high-resolution 2D camera to acquire images of the area to be located, and then preliminarily extracts the location of the reference point by finding circular features in the whole image, analyzing spots, extracting color intervals, and combining standard reference point radius filtering and detection range limitation in industrial scenarios. Secondly, it uses the coarsely located reference point location and gradient information to extract sub-pixel edge points of the reference point; then, it uses the least squares method to fit the location of the reference point; finally, it judges the positional relationship between the sub-pixel points and the fitted circle to determine the authenticity of the fitted circle, thereby reducing the occurrence of positioning anomalies. This method of sub-pixel detection of reference point locations in electronic devices based on shape and edge gradient information is suitable for general reference point sub-pixel positioning under conditions of illumination changes, topographic changes, and complex positioning backgrounds, reducing false detections and accelerating production efficiency. Figure 8 and Figure 9 As can be seen, this method can effectively solve the problems of insufficient positioning accuracy and low robustness caused by the diversity, randomness, and low contrast of the morphology in the benchmark positioning process, thus meeting the accuracy requirements in the detection process. In addition, in 3500 actual collected test data, the benchmark positioning accuracy of the method provided in this application is less than 1 pixel. In the actual detection process, the probability of benchmark positioning anomalies is 0.00%, the probability of failure to locate is 0.01%, and the average detection time per image is 75 ms. This not only improves the detection accuracy of benchmarks but also effectively enhances the detection efficiency, providing a reference for the detection of other similar products.

[0096] Based on the reference point detection method for electronic devices provided in the above embodiments, this application also provides a reference point detection device for electronic devices, which specifically includes: The coarse positioning module 10 is used to acquire images of the area to be positioned of the reference point, detect the images of the area to be positioned of the reference point, and obtain at least one coarse positioning circular area where the reference point is located.

[0097] The coarse positioning circular region filtering module 20 is used to calculate the positioning score of each coarse positioning circular region based on the radius of each coarse positioning circular region and the radius of the standard reference point; and to select the coarse positioning circular regions whose positioning scores are greater than the preset positioning score threshold as target coarse positioning circular regions.

[0098] The edge extraction and fitting module 30 is used to extract the edge points of the target coarse positioning circular region multiple times, and to fit the edge points extracted each time to obtain multiple fitted circles corresponding to the target coarse positioning circular region.

[0099] The reference point acquisition module 40 is used to calculate the difference between the radius of each fitted circle and the radius of the standard reference point, and selects the fitted circle with the smallest difference as the best fitted circle, thereby obtaining the reference point of the electronic device.

[0100] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described reference point detection method for electronic devices.

[0101] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0102] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0103] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0104] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0105] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.

Claims

1. A method for detecting reference points in an electronic device, characterized in that, include: Acquire images of the area to be located for the reference point, detect the image of the area to be located for the reference point, and obtain at least one coarse circular region where the reference point is located. Specifically, it includes: The spot detection algorithm is used to detect the image of the reference point to be located, and it is determined whether there is at least one coarse circular region with a positioning score greater than a preset positioning score threshold in the detection result. If it exists, the coarse positioning circular region with a positioning score greater than the preset positioning score threshold is taken as the target coarse positioning circular region; if it does not exist, the whole image circle finding algorithm is used to perform secondary detection on the image of the reference point to be located, and it is determined whether there is at least one coarse positioning circular region with a positioning score greater than the preset positioning score threshold in the secondary detection result. If it exists, the coarse positioning circular region with a positioning score greater than the preset positioning score threshold is taken as the target coarse positioning circular region; if it does not exist, the color contour analysis algorithm is used to perform three detections on the image of the reference point to be located region, which specifically includes: Based on all pixels in the image of the reference point to be located, the coefficient of variation and outliers of the image of the reference point to be located are calculated; the image of the reference point to be located is scaled to obtain the image of the reference point to be located at different scales. When the coefficient of variation is less than the preset coefficient of variation threshold, the standard deviation statistical method is used to extract the color range of the region where the reference point is located in the image of the reference point to be located at different scales. When the outlier exceeds the preset outlier threshold, the median absolute deviation statistical method is used to extract the color range of the region where the reference point is located in the image of the reference point to be located at different scales. When the coefficient of variation is greater than or equal to the preset coefficient of variation threshold and the outlier is less than or equal to the preset outlier threshold, the interquartile range statistical method is used to extract the color range of the region where the reference point is located in the image of the reference point to be located at different scales. Calculate the confidence score of the color interval of the region where the reference point is located at different scales, and take the color interval of the region where the reference point has the highest confidence score as the standard interval; The image of the region to be located is segmented based on the standard interval to obtain the reference point region and the non-reference point region; the contour is extracted from the reference point region, and the geometric features of the extracted contour are compared with the radius of the standard reference point to obtain the coarse positioning circular region. Based on the radius of each coarse positioning circular region and the radius of the standard reference point, the positioning score of each coarse positioning circular region is calculated; the coarse positioning circular regions with positioning scores greater than the preset positioning score threshold are taken as the target coarse positioning circular regions. The edge points of the target coarsely localized circular region are extracted multiple times, and the edge points extracted each time are fitted to obtain multiple fitted circles corresponding to the target coarsely localized circular region. Calculate the difference between the radius of each fitted circle and the radius of the standard reference point, and take the fitted circle with the smallest difference as the best fitted circle, thus obtaining the reference point of the electronic device.

2. The reference point detection method for electronic devices according to claim 1, characterized in that, The image of the region to be located using a blob detection algorithm is analyzed, including: The image of the reference point to be located is preprocessed, and a Gaussian-scale spatial pyramid is constructed on the preprocessed image of the reference point to be located. Different thresholds are used to binarize the reference point to be located images at different scales to obtain binarized images at different scales, and the geometric features of the connected components in each binarized image are obtained. The position stability coefficient of each connected component is calculated based on the ratio of the variance of the position coordinates of each connected component in different binarized images to the preset position stability reference variance. The scale stability coefficient of each connected component is calculated based on the proportion of the scale span of each connected component to the difference between the maximum and minimum scales of the Gaussian scale space pyramid. The response stability coefficient of each connected component is calculated based on the ratio of the average response intensity to the standard deviation of the response intensity in different binarized images. The stability score of each connected component is obtained by weighting the position stability coefficient, scale stability coefficient, and response stability coefficient of each connected component. Connected components with stability scores greater than a preset score threshold are selected as coarsely located circular regions.

3. The reference point detection method for electronic devices according to claim 1, characterized in that, The image of the region to be located using the whole-image circle-finding algorithm is used for secondary detection, including: Calculate the gradient magnitude of each pixel in the image of the region to be located, and obtain the gradient magnitude image; In the gradient magnitude image, select pixels whose gradient magnitude is greater than the gradient magnitude of each pixel in the surrounding preset neighborhood as initial anchor points; Using each initial anchor point as the anchor point to be linked, a search is performed along the vertical direction of the gradient to obtain pixels that meet the search conditions. These pixels are then used as link points to connect to the anchor points to be linked, forming continuous edge segments. The newly added link points in each continuous edge segment are used as anchor points to be linked. The search is performed along the vertical direction of the gradient to update each continuous edge segment until there are no link points that meet the search conditions, thus obtaining the continuous edge segments of each initial anchor point. Fit each continuous edge segment to obtain the corresponding circular shape; verify the false alarm count of all circular shapes. If the verification is successful, each circular shape is used as the coarsely located circular region. The search conditions include a gradient magnitude greater than the gradient magnitude of the anchor to be linked, a gradient direction difference between the gradient and the anchor to be linked less than a preset difference, and a distance between the gradient and the anchor to be linked less than a preset distance threshold.

4. The reference point detection method for electronic devices according to claim 1, characterized in that, Extracting the edge points of the coarsely localized circular region of the target multiple times includes: Centered on the center of the target coarse positioning circular area, set multiple rotating rectangles with preset widths and heights at preset angle intervals along the edge of the target coarse positioning circular area, which serve as edge point extraction calipers; Using calipers extracted from each edge point, pixels on the edge of the target coarsely located circular area are sampled at a fixed step size; the gray values ​​of pixels at the same sampling position of all edge point calipers are averaged to obtain the projected average gray value sequence. The gradient sequence is obtained by performing differential calculation on the projected average grayscale sequence; multiple gradient thresholds are set, and effective gradient points in the gradient sequence are filtered based on each gradient threshold. If there are multiple consecutive effective gradient points for a gradient threshold, the pixel point corresponding to the gradient peak among the multiple effective gradient points is taken as the target pixel point, and the position of the target pixel point is taken as the edge sampling position of the edge point extraction caliper under the gradient threshold. Based on the pixel points collected at the edge sampling positions of the calipers at each edge point under each gradient threshold, a set of edge points for the coarsely localized circular region of the target is obtained.

5. The reference point detection method for electronic devices according to claim 4, characterized in that, After taking the pixel corresponding to the gradient peak among multiple valid gradient points as the target pixel, the following is also included: Select multiple pixels within a preset distance from the target pixel, perform quadratic function fitting on the multiple pixels, and calculate the extreme points of the fitted quadratic function. Calculate the sum of the gradient peak and extreme points among multiple effective gradient points, and calculate the set angle of the caliper, the cosine value of the set angle, and the product of the sum for each edge point; The sum of the x-coordinate of the center point of the caliper extracted from each edge point and the product of the product yields the x-coordinate of the edge sampling position of the caliper extracted from each edge point. The sum of the ordinate of the center point of the caliper extracted from each edge point and the product of the ordinate is used to obtain the ordinate of the edge sampling position of the caliper extracted from each edge point.

6. The reference point detection method for electronic devices according to claim 1, characterized in that, After selecting the fitted circle with the smallest difference as the best fitted circle, the following steps are also taken: Based on the fitted circle, the radius and center of the best-fit circle, calculate the deviation of the radial distance from each edge point to the best-fit circle. Based on the deviation of the radial distance from each edge point to the best-fit circle, the sum of the deviations of the radial distances from all edge points to the best-fit circle is obtained, thus obtaining the deviation distance between the edge point and the best-fit circle. If the deviation distance is less than the preset minimum tolerance distance, and the number of edge points whose radial distance to the best-fit circle is less than the preset minimum tolerance distance is greater than the preset minimum tolerance number, then the reference point position of the electronic device is obtained based on the best-fit circle.

7. A reference point detection device for an electronic device, characterized in that, include: The coarse positioning module is used to acquire images of the area to be positioned of the reference point, detect the images of the area to be positioned of the reference point, and obtain at least one coarse positioning circular area where the reference point is located. Specifically, it includes: The spot detection algorithm is used to detect the image of the reference point to be located, and it is determined whether there is at least one coarse circular region with a positioning score greater than a preset positioning score threshold in the detection result. If it exists, the coarse positioning circular region with a positioning score greater than the preset positioning score threshold is taken as the target coarse positioning circular region; if it does not exist, the whole image circle finding algorithm is used to perform secondary detection on the image of the reference point to be located, and it is determined whether there is at least one coarse positioning circular region with a positioning score greater than the preset positioning score threshold in the secondary detection result. If it exists, the coarse positioning circular region with a positioning score greater than the preset positioning score threshold is taken as the target coarse positioning circular region; if it does not exist, the color contour analysis algorithm is used to perform three detections on the image of the reference point to be located region, which specifically includes: Based on all pixels in the image of the reference point to be located, the coefficient of variation and outliers of the image of the reference point to be located are calculated; the image of the reference point to be located is scaled to obtain the image of the reference point to be located at different scales. When the coefficient of variation is less than the preset coefficient of variation threshold, the standard deviation statistical method is used to extract the color range of the region where the reference point is located in the image of the reference point to be located at different scales. When the outlier exceeds the preset outlier threshold, the median absolute deviation statistical method is used to extract the color range of the region where the reference point is located in the image of the reference point to be located at different scales. When the coefficient of variation is greater than or equal to the preset coefficient of variation threshold and the outlier is less than or equal to the preset outlier threshold, the interquartile range statistical method is used to extract the color range of the region where the reference point is located in the image of the reference point to be located at different scales. Calculate the confidence score of the color interval of the region where the reference point is located at different scales, and take the color interval of the region where the reference point has the highest confidence score as the standard interval; The image of the region to be located is segmented based on the standard interval to obtain the reference point region and the non-reference point region; the contour is extracted from the reference point region, and the geometric features of the extracted contour are compared with the radius of the standard reference point to obtain the coarse positioning circular region. The coarse positioning circular region filtering module is used to calculate the positioning score of each coarse positioning circular region based on the radius of each coarse positioning circular region and the radius of the standard reference point; and to select coarse positioning circular regions whose positioning scores are greater than the preset positioning score threshold as target coarse positioning circular regions. The edge extraction and fitting module is used to extract the edge points of the target coarsely located circular region multiple times, and fit the edge points extracted each time to obtain multiple fitted circles corresponding to the target coarsely located circular region. The reference point acquisition module is used to calculate the difference between the radius of each fitted circle and the radius of the standard reference point, and selects the fitted circle with the smallest difference as the best fitted circle, thereby obtaining the reference point of the electronic device.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the reference point detection method for the electronic device according to any one of claims 1 to 6.

Images