Method, apparatus, and electronic device for identifying a probe
By performing binary image processing and threshold segmentation on the probe image and filtering the connected component contours, the positioning deviation problem caused by probe wear and dirt during the needle insertion process was solved, and the accurate identification of the probe center and the accuracy of the needle insertion position were achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- HANGZHOU CHANGCHUAN TECH CO LTD
- Filing Date
- 2023-03-08
- Publication Date
- 2026-07-10
AI Technical Summary
The probe is prone to wear and dirt during the needle insertion process, which can cause deviation in the center of the probe and affect the accuracy of the needle insertion position.
By acquiring binary images of the probe, the contours of connected components are identified and filtered. Filtering is performed using area, radius, and roundness conditions. Secondary search and threshold segmentation of local regions are conducted to determine the main probe tip radius and center.
Accurately identify the probe center, eliminate interference from dirt and wear, and improve the accuracy of needle placement.
Smart Images
Figure CN116452503B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the semiconductor field, and more specifically, to a method, apparatus, and electronic device for identifying probes. Background Technology
[0002] Probe stations are primarily used in the semiconductor, optoelectronic, integrated circuit, and packaging industries for testing. Using probe cards as the testing interface, they mainly test bare dies on wafers. By connecting the tester and the chip, signals are transmitted to test chip parameters. One of the most crucial steps in the testing process is probe alignment. Probe alignment involves calculating the pixel coordinates of the probe center in the image, converting them to physical coordinates in the world coordinate system, and inputting these coordinates into the probe station system. This system then controls the actuator to insert the probe into the corresponding test pad on the die. Deviations in probe insertion directly affect the test quality; therefore, precise probe identification and positioning are essential.
[0003] The most commonly used probe type on probe stations is the cantilever needle. Cantilever needles are noticeably shiny, so the conventional method for probe identification is to calculate the centroid of the shiny tip and use it as the positioning result. However, cantilever needles are easily worn or affected by dirt during the insertion process, leading to positioning deviations when identifying the probe center and thus affecting the insertion position and test results.
[0004] There is currently no effective solution to the above problems. Summary of the Invention
[0005] This application provides a method, apparatus, and electronic device for identifying probes, to at least solve the technical problem that the center of the identification probe deviates due to wear and dirt during the needle insertion process, resulting in a deviation in the needle insertion position.
[0006] According to one aspect of the embodiments of this application, a method for identifying a probe is provided, comprising: acquiring a binary image corresponding to a probe image, and identifying all connected component contours in the binary image; filtering all connected component contours to obtain a target connected component contour; determining the average value of pixels within the target connected component contour and the radius corresponding to the target connected component contour; determining the target connected component contour as the contour corresponding to the main body tip of the probe when the average value is not less than a set value and the radius is not less than a first preset multiple of the set radius; performing threshold segmentation on the contour corresponding to the main body tip according to a preset step size, starting from an initial brightness threshold when the radius is greater than a second preset multiple of the set radius; and determining the radius of the main body tip and the center of the probe based on the threshold segmentation result.
[0007] Optionally, if the radius is not greater than a second preset multiple of the set radius, the probe image is identified to obtain the radius and center of the main needle tip.
[0008] Optionally, determining the radius of the main tip and the center of the probe based on the threshold segmentation result includes: during the threshold segmentation process, if, after at least two consecutive threshold segments, the absolute value of the difference between the radii of the main tip corresponding to the results of the at least two consecutive threshold segments is less than a preset value, obtaining each brightness threshold used in the at least two consecutive threshold segments; and determining the radius of the main tip and the center of the probe based on each brightness threshold.
[0009] Optionally, determining the radius of the main needle tip and the center of the probe based on each brightness threshold includes: determining the median of each brightness threshold as the first target brightness threshold; determining the radius of the connected region segmented under the first target brightness threshold as the radius of the main needle body, and the centroid of the connected region segmented under the target brightness threshold as the center of the probe.
[0010] Optionally, determining the radius of the main needle tip and the center of the probe based on each brightness threshold includes: determining the average value of each brightness threshold as the second target brightness threshold; determining the radius of the connected region segmented under the second target brightness threshold as the radius of the main needle body, and the centroid of the connected region segmented under the target brightness threshold as the center of the probe.
[0011] Optionally, the target connected component contour is obtained by filtering all connected component contours, including: filtering all connected component contours to obtain a first connected component contour set, obtaining the first centroid corresponding to each connected component contour in the first connected component contour set, and determining the connected component contour corresponding to the first centroid closest to the center position of the probe image as the target connected component contour.
[0012] Optionally, filtering all connected component contours to obtain a first set of connected component contours includes: sequentially traversing all connected component contours, and during the traversal, determining whether the area of the currently detected connected component contour is greater than a set area; if the area is greater than the set area, determining whether the radius of the contour is within a preset range; if the radius of the contour falls within the preset range, determining whether the roundness of the contour is greater than a set roundness; if the roundness of the contour is greater than the set roundness, determining that the currently detected connected component contour is an element in the first set of connected component contours.
[0013] Optionally, after determining the average value of pixels within the target connected region contour and the radius corresponding to the target connected region contour, the method further includes: if the average value is less than a set value or the radius of the target connected region contour is less than a first preset multiple of the set radius, performing a secondary screening of the target connected region contour to obtain the contour corresponding to the main body tip of the probe.
[0014] Optionally, a secondary screening is performed on the target connected component contour to obtain the contour corresponding to the main body tip of the probe, including: taking the location of the first centroid closest to the center of the probe image as the center; determining the original size of the target connected component contour; cropping a local region image from the probe image, wherein the size of the local region image is a preset multiple of the original size; performing binarization processing on the local region image to obtain a binary image corresponding to the local region image; retrieving connected components in the binary image to obtain a second connected component contour set; and determining the contour corresponding to the main body tip of the probe based on the second connected component contour set.
[0015] Optionally, determining the contour corresponding to the main tip of the probe based on the second connected component contour set includes: mapping each second connected component contour in the second connected component contour set to a local region image as a positioning mask, wherein the positioning mask corresponds one-to-one with the coordinates of the main tip in the local region image; obtaining the pixel values of each point located within the positioning mask in the local region image and the number of pixel values greater than a set value; determining the region with the most pixel values greater than the set value as the main region; obtaining the coordinates of any pixel in the main region, selecting a point with the same coordinates as any pixel in the binary image corresponding to the probe image as a seed point, determining a third connected component containing the seed point, and using the third connected component as the contour corresponding to the main tip of the probe.
[0016] According to another aspect of the embodiments of this application, an apparatus for identifying a probe is also provided, comprising: an identification module, configured to acquire a binary image corresponding to a probe image, identify all connected component contours in the binary image, and filter all connected component contours to obtain a target connected component contour; a first determination module, configured to determine the average value of pixels within the target connected component contour and the radius corresponding to the target connected component contour, and determine the target connected component contour as the contour corresponding to the main body tip of the probe when the average value is not less than a set value and the radius is not less than a first preset multiple of the set radius; a segmentation module, configured to perform threshold segmentation on the contour corresponding to the main body tip, starting from an initial brightness threshold and according to a preset step size, when the radius is greater than a second preset multiple of the set radius, wherein the second preset multiple is greater than the first preset multiple; and a second determination module, configured to determine the radius of the main body tip and the center of the probe based on the threshold segmentation result.
[0017] According to another aspect of the embodiments of this application, a non-volatile storage medium is also provided, comprising: the storage medium including a stored program, wherein, when the program is running, it controls the device where the storage medium is located to execute a method for identifying a probe.
[0018] According to another aspect of the embodiments of this application, an electronic device is also provided, including: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to execute instructions to implement any method for identifying probes.
[0019] In this embodiment, the probe binary image is filtered based on area, radius, and roundness conditions. After initial positioning, a secondary search is performed on the local area. By acquiring the binary image corresponding to the probe image and identifying all connected component contours in the binary image, the target connected component contour is obtained by filtering all connected component contours. The average value of pixels within the target connected component contour and the radius corresponding to the target connected component contour are determined. If the average value is greater than a set value and the radius is greater than a first preset multiple of the set radius, the target connected component contour is determined to be the contour corresponding to the main body tip of the probe. If the radius is greater than a second preset multiple of the set radius, the contour corresponding to the main body tip is segmented according to a preset step size, starting from the initial brightness threshold. Based on the threshold segmentation result, the radius of the main body tip and the center of the probe are determined, achieving the purpose of accurately identifying and locating the probe center. This achieves the technical effect of eliminating interference from probe dirt and probe wear and splitting when identifying and locating the probe center, thereby solving the technical problem of deviation in probe center identification caused by easy wear and dirt during the needle insertion process, resulting in deviation in needle insertion position. Attached Figure Description
[0020] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:
[0021] Figure 1 This is a flowchart illustrating a probe identification method according to an embodiment of this application;
[0022] Figure 2 This is a schematic diagram of a secondary screening process according to an embodiment of this application;
[0023] Figure 3 This is a schematic diagram of a threshold dynamic adjustment process according to an embodiment of this application;
[0024] Figure 4 This is a schematic flowchart of an optional identification probe according to an embodiment of this application;
[0025] Figure 5 This is a schematic diagram of a cantilever needle under a high-magnification camera according to an embodiment of this application;
[0026] Figure 6 This is a schematic diagram of a binary image of a cantilever needle according to an embodiment of this application;
[0027] Figure 7 This is a schematic diagram of a connected component contour image according to an embodiment of this application;
[0028] Figure 8 This is a schematic diagram of a dirty interference image according to an embodiment of this application;
[0029] Figure 9 This is a schematic diagram of a needle tip abrasion splitting image according to an embodiment of this application;
[0030] Figure 10 This is a schematic diagram of a local area image according to an embodiment of this application;
[0031] Figure 11 This is a schematic diagram of a binary image of a local region according to an embodiment of this application;
[0032] Figure 12 This is a schematic diagram of an image showing a needle tip connected to dirt according to an embodiment of this application;
[0033] Figure 13 This is a schematic diagram of a binary image with a threshold of 90 and a radius of 53 according to an embodiment of this application;
[0034] Figure 14 This is a schematic diagram of a binary image with a threshold of 100 and a radius of 50 according to an embodiment of this application;
[0035] Figure 15 This is a schematic diagram of a binary image with a threshold of 110 and a radius of 42 according to an embodiment of this application;
[0036] Figure 16 This is a schematic diagram of a binary image with a threshold of 120 and a radius of 38 according to an embodiment of this application;
[0037] Figure 17 This is a schematic diagram of a binary image with a threshold of 130 and a radius of 37 according to an embodiment of this application;
[0038] Figure 18 This is a schematic diagram of a binary image with a threshold of 140 and a radius of 36 according to an embodiment of this application;
[0039] Figure 19 This is a schematic diagram of a cantilever needle recognition result according to an embodiment of this application;
[0040] Figure 20 This is a schematic diagram of a device structure for identifying a probe according to an embodiment of this application;
[0041] Figure 21 This is a schematic block diagram of an example electronic device 2100 according to an embodiment of this application. Detailed Implementation
[0042] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.
[0043] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.
[0044] According to an embodiment of this application, a method embodiment for identifying probes is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.
[0045] Figure 1 This is a method for identifying probes according to an embodiment of this application, such as... Figure 1 As shown, the method includes the following steps:
[0046] Step S102: Obtain the binary image corresponding to the probe image, identify all connected component contours in the binary image, and filter all connected component contours to obtain the target connected component contour.
[0047] Step S104: Determine the average value of pixels within the target connected region contour and the radius corresponding to the target connected region contour. If the average value is not less than a set value and the radius is not less than a first preset multiple of the set radius, determine the target connected region contour as the contour corresponding to the main body tip of the probe.
[0048] For example, assuming the set value is 150, if the average value of pixels within the target connected region contour is less than the set value, the contour closest to the image center may be dirty, and a second screening is required. If the average value of pixels within the target connected region contour is greater than the set value, the target connected region contour can be determined to be the contour corresponding to the probe's main tip.
[0049] For example, assuming the first preset multiple is 0.8, if the radius corresponding to the target connected component contour is less than 0.8 times the set radius, the contour closest to the image center may be the tip wear and split part. Then, a second screening is performed. If the radius corresponding to the target connected component contour is greater than 0.8 times the set radius, then the target connected component contour can be determined to be the contour corresponding to the main body tip of the probe.
[0050] Step S106: When the radius is greater than a second preset multiple of the set radius, the contour corresponding to the main body needle tip is segmented according to a preset step size, starting from the initial brightness threshold. The second preset multiple is greater than the first preset multiple.
[0051] For example, assuming the second preset multiple is 1.2, if the radius corresponding to the target connected region contour is greater than 1.2 times the set radius, the main body of the needle tip may be dirty and needs to be cleaned. Starting from the initial brightness threshold, the contour corresponding to the main body needle tip is segmented according to the preset step size. Based on the threshold segmentation result, the radius of the main body needle tip and the center of the probe are determined. If the radius corresponding to the target connected region contour is less than 1.2 times the set radius, and the radius corresponding to the target connected region contour is greater than 0.8 times the set radius, then the needle tip radius and the center of the probe can be determined.
[0052] Step S108: Determine the radius of the main needle tip and the center of the probe based on the threshold segmentation results.
[0053] In this embodiment, the probe binary image is filtered based on area, radius, and roundness conditions. After initial positioning, a secondary search is performed on the local area. By acquiring the binary image corresponding to the probe image and identifying all connected component contours in the binary image, the target connected component contour is obtained by filtering all connected component contours. The average value of pixels within the target connected component contour and the radius corresponding to the target connected component contour are determined. If the average value is greater than a set value and the radius is greater than a first preset multiple of the set radius, the target connected component contour is determined to be the contour corresponding to the main body tip of the probe. If the radius is greater than a second preset multiple of the set radius, the contour corresponding to the main body tip is segmented according to a preset step size, starting from the initial brightness threshold. Based on the threshold segmentation result, the radius of the main body tip and the center of the probe are determined, achieving the purpose of accurately identifying and locating the probe center. This achieves the technical effect of eliminating interference from probe dirt and probe wear and splitting when identifying and locating the probe center, thereby solving the technical problem of deviation in probe center identification caused by easy wear and dirt during the needle insertion process, resulting in deviation in needle insertion position.
[0054] In some optional embodiments of this application, when the radius corresponding to the contour of the target connected region is not greater than a second preset multiple of the set radius, the probe image is identified to obtain the radius and center of the main probe tip.
[0055] It is understandable that if the radius corresponding to the contour of the target connected region is less than 1.2 times the set radius, and the radius corresponding to the contour of the target connected region is greater than 0.8 times the set radius, then the tip radius and the center of the probe can be determined.
[0056] In an exemplary embodiment of this application, determining the radius of the main needle tip and the center of the probe based on the threshold segmentation result can be achieved through the following steps: Specifically, during the threshold segmentation process, if the absolute value of the difference between the radii of the main needle tip corresponding to the at least two consecutive threshold segmentations is less than a preset value, then each brightness threshold used in the at least two consecutive threshold segmentations can be obtained; and the radius of the main needle tip and the center of the probe can be determined based on each brightness threshold.
[0057] For example, assuming the preset value is 3, after three consecutive brightness thresholds (120, 130, 140), the tip radii are 38, 37, and 36 respectively. That is, the absolute difference between the tip radii is less than 3. Therefore, the radius of the main tip and the center of the probe can be determined based on the tip radii corresponding to the brightness thresholds of 120, 130, and 140.
[0058] Understandably, the number of threshold segmentations, preset values, initial thresholds, and preset step sizes are set according to the actual application scenario.
[0059] Optionally, determining the radius of the main needle tip and the center of the probe based on each brightness threshold includes: determining the median of each brightness threshold as the first target brightness threshold; determining the radius of the connected region segmented under the first target brightness threshold as the radius of the main needle body, and the centroid of the connected region segmented under the target brightness threshold as the center of the probe.
[0060] For example, suppose the preset value is 3, the initial threshold is 90, the traversal end value is 200, and the preset step size is 10. After three consecutive threshold divisions, the absolute difference between the tip radii is less than 3. Starting from the initial threshold, the contour corresponding to the main tip is thresholded according to the preset step size. The radius is 38 when the threshold is 120, 37 when the threshold is 130, and 36 when the threshold is 140. The absolute difference between the radii corresponding to different thresholds is less than 3. At this time, the radius has become stable. The intermediate value of different threshold divisions can be used as the final result. Then, the radius of the connected component segmented when the threshold is 130 can be the tip radius, and its centroid can be the probe center.
[0061] In an exemplary embodiment of this application, the target connected component contour is obtained by filtering all connected component contours. This can be achieved as follows: a first connected component contour set is obtained by filtering all connected component contours, the first centroid corresponding to each connected component contour in the first connected component contour set is obtained, and the connected component contour corresponding to the first centroid closest to the center position of the probe image is determined as the target connected component contour.
[0062] As an optional implementation, filtering all connected component contours to obtain a first set of connected component contours includes: sequentially traversing all connected component contours, and during the traversal, determining whether the area of the currently detected connected component contour is greater than a set area; if the area is greater than the set area, determining whether the radius of the contour is within a preset range; if the radius of the contour falls within the preset range, determining whether the roundness of the contour is greater than a set roundness; if the roundness of the contour is greater than the set roundness, determining that the currently detected connected component contour is an element in the first set of connected component contours.
[0063] It should be noted that the preset range is defined by an upper and lower radius limit. The upper radius limit = set radius × (1 + radius upper limit deviation percentage); the lower radius limit = set radius × (1 - radius lower limit deviation percentage). Both the upper and lower radius deviation percentages can be set to 5%, depending on the specific application scenario. For example, if the set radius is 40, then the upper radius limit = 40 × (1 + 5%) = 42, and the lower radius limit = 40 × (1 - 5%) = 38. Therefore, the preset range is [38, 42].
[0064] It should be noted that the roundness of the outline refers to the ratio of the area enclosed by the outline to the area of the smallest enclosing circle. The closer the ratio is to 1, the rounder the outline is. The smallest enclosing circle refers to the smallest enclosing circle of the outline.
[0065] As an optional implementation, after determining the average value of pixels within the target connected region contour and the radius corresponding to the target connected region contour, a secondary screening of the target connected region contour can be performed if the average value is less than a set value or the radius of the target connected region contour is less than a first preset multiple of the set radius, to obtain the contour corresponding to the main body tip of the probe.
[0066] For example, assuming the set value is 150, if the average value of pixels within the target connected region contour is less than 150, then the contour closest to the image center may be dirty, and a second screening is required.
[0067] For example, assuming the first preset multiple is 0.8, if the radius corresponding to the contour of the target connected region is less than 0.8 times the set radius, then the contour closest to the center of the image may be the part of the needle tip wear and split, and a second screening is required.
[0068] Figure 2 This is a schematic diagram of a secondary screening process according to an embodiment of this application, such as... Figure 2 As shown, the method includes the following steps:
[0069] Step S202: Take the location of the first centroid, which is closest to the center of the probe image, as the center;
[0070] Step S204: Determine the original size of the target connected component contour, and crop a local region image from the probe image, wherein the size of the local region image is a preset multiple of the original size;
[0071] Step S206: Binarize the local region image to obtain a binary map corresponding to the local region image, and retrieve connected components in the binary map to obtain a set of second connected component contours;
[0072] Step S208: Determine the contour corresponding to the main body tip of the probe based on the second connected component contour set.
[0073] For example, assuming the preset multiple is 3, after determining the original size of the target connected component contour, 3 times the original size of the target connected component contour is the local region image. Connected components are retrieved in the binary image corresponding to the local region image to obtain the second connected component contour set. The contour corresponding to the main body tip of the probe is determined based on the second connected component contour set.
[0074] Figure 3 This is a schematic diagram of a threshold dynamic adjustment process according to an embodiment of this application, such as... Figure 3As shown, the method includes the following steps:
[0075] Step S302: Map each second connected component contour in the second connected component contour set to the local region image as a positioning mask, wherein the positioning mask corresponds one-to-one with the coordinates of the main needle tip in the local region image;
[0076] Step S304: Obtain the pixel values of each point located within the positioning mask in the local region image and the number of pixel values greater than a set value;
[0077] Step S306: Determine the area with the most pixel values greater than a set value as the main body area;
[0078] Step S308: Obtain the coordinates of any pixel in the main region, select a point with the same coordinates as any pixel in the binary image corresponding to the probe image as a seed point, determine the third connected region containing the seed point, and use the third connected region as the contour corresponding to the main tip of the probe.
[0079] For example, assuming the set value is 120 and the probe type is a cantilever needle, the connected components of the cantilever needle can be mapped to a local area image as a positioning mask. Then, the number of pixels with values greater than the set value of 120 in the area image corresponding to each connected component can be counted. The part with the most pixels greater than the set value is then determined to be the main body of the cantilever needle tip. Finally, any point on the connected component of the main body is selected as a seed point, and the connected component where the seed point is located is determined to be the contour corresponding to the cantilever needle tip.
[0080] To facilitate a better understanding of the technical solutions of this application by those skilled in the art, a specific embodiment will now be described.
[0081] Taking the cantilever needle as an example, Figure 4 This is a schematic diagram of an optional process for identifying a cantilever needle according to an embodiment of this application, as shown below. Figure 4 As shown, the process mainly includes the following steps:
[0082] (1) Input the image of the cantilever needle acquired by the high-magnification camera on the probe station, such as Figure 5 The image shown is of a cantilever needle captured by a high-magnification camera in this embodiment;
[0083] (2) Obtain the binary image of the cantilever needle, such as Figure 6 The image shown is a binary image of the cantilever needle obtained by converting an image captured by a high-magnification camera to grayscale in this embodiment.
[0084] (3) Retrieve the contours of all connected components in the field of view, such as Figure 7 As shown, the outline of a connected component is a white connected region in a binary image;
[0085] (4) Traverse all connected component outlines in sequence;
[0086] (5) During the traversal, determine whether the contour area is greater than the set contour area. If the contour area is greater than the set contour area, proceed to the next step; otherwise, return to step (4) to traverse the next connected component contour. The contour area refers to the pixel area within the contour line, which is composed of multiple white pixels.
[0087] (6) If the contour area is greater than the set contour area, determine whether the radius of the contour is within the preset range. If the radius of the contour is within the preset range, it means that the radius meets the requirements, and then proceed to the next step. Otherwise, return to step (4) to traverse the next connected component contour.
[0088] It should be noted that the preset range is limited by the upper and lower radius limits. The upper radius limit = set radius × (1 + radius upper limit deviation percentage); the lower radius limit = set radius × (1 - radius lower limit deviation percentage). The radius upper limit deviation percentage and the radius lower limit deviation percentage can be set to 5%, which can be set according to the specific application scenario.
[0089] (7) If the radius of the contour falls within the preset range, then determine whether the roundness of the contour is greater than the set roundness. If the roundness of the contour is greater than the set roundness, then proceed to the next step; otherwise, return to step (4) to traverse the next connected region contour.
[0090] It should be noted that the roundness of the outline refers to the quotient of the area of the outline and the area of the smallest enclosing circle. The closer the quotient is to 1, the rounder the outline is. The smallest enclosing circle refers to the smallest enclosing circle in the outline.
[0091] (8) Calculate the centroid of the profile;
[0092] (9) Filter out the centroid of the contour closest to the center of the image;
[0093] (10) Calculate the average value of the pixels inside the initially retrieved target contour. If the average value of the pixels inside the target connected region contour is less than the set value, or if the radius corresponding to the target connected region contour is less than the first preset multiple of the set radius, perform a second screening on the target connected region contour to obtain the contour corresponding to the main body tip of the probe. If the average value of the pixels inside the target connected region contour is greater than the set value, or if the radius corresponding to the target connected region contour is greater than the first preset multiple of the set radius, proceed to step (16).
[0094] Figure 8 These are schematic diagrams of dirt-interference images in some embodiments, such as... Figure 8 As shown, the contours closest to the center of the image are dirty and require secondary filtering. Figure 9These are schematic diagrams of needle tip wear splitting images in some embodiments, such as Figure 9 As shown, the contour closest to the center of the image is the part of the needle tip wear and split. In this case, a secondary screening is also required.
[0095] For example, assuming the first preset multiple is 0.8, if the radius corresponding to the target connected component contour is less than 0.8 times the set radius, the contour closest to the image center may be the tip wear and split part. Then, a second screening is performed. If the radius corresponding to the target connected component contour is greater than 0.8 times the set radius, then the target connected component contour can be determined to be the contour corresponding to the main body tip of the probe.
[0096] (11) Taking the location of the first centroid closest to the center of the probe image as the center, determine the original size of the target connected region contour, and extract a preset multiple of the original size of the target connected region contour in the probe image to obtain a local region image; it should be noted that needle tip dirt or needle tip splitting both occur in the local region of the main needle tip, so only a secondary search needs to be performed in the local region.
[0097] Figure 10 This is a schematic diagram of a local area image according to an embodiment of this application, such as... Figure 10 As shown, by cropping a target connected region contour out of a preset multiple of its original size from the probe image, a local region image can be obtained.
[0098] (12) Binarize the local region image to obtain the corresponding binary map, and search for connected components in the binary map;
[0099] It should be noted that a connected component in a binary image is a white connected region, which is easily observed to be composed of multiple white pixels.
[0100] For example, assuming the preset multiple is 3, after determining the original size of the target connected component outline, 3 times the original size of the target connected component outline is the local region image, and the connected component is retrieved in the binary image corresponding to the local region image.
[0101] Figure 11 This is a schematic diagram of a binary image of a local region according to an embodiment of this application, such as... Figure 11 As shown, a second connected component contour set is obtained from the local region image, and the contour corresponding to the main probe tip is determined based on the second connected component contour set.
[0102] (13) Map each second connected component contour in the second connected component contour set to the local region image as a positioning mask, obtain the pixel value of each point in the local region image located in the positioning mask and the number of pixel values greater than the set value, and determine the region with the most pixel values greater than the set value as the main region.
[0103] (14) Obtain the coordinates of any pixel in the main region, select the point with the same coordinates as any pixel in the binary image corresponding to the probe image as the seed point, determine the third connected region containing the seed point, and determine the third connected region as the contour corresponding to the main body tip of the probe.
[0104] It should be noted that mapping back to the original probe image for retrieval is to prevent the probe tip from being too large, and the local area in step (11) above may not include the entire probe tip body area.
[0105] (15) Calculate the radius and centroid of the main needle tip after the second retrieval;
[0106] (16) Determine whether the radius is greater than the second preset multiple of the set radius. If the radius is greater than the second preset multiple of the set radius, the main body of the needle tip may be connected to dirt. At this time, the centroid of the positioning is inaccurate. The next step is to remove the dirt from the needle tip. If the radius corresponding to the contour of the target connected region is less than the second preset multiple of the set radius, then directly proceed to step (18) to output the radius and center of the cantilever needle.
[0107] Figure 12 This is a schematic diagram of an image showing a needle tip connected to dirt according to an embodiment of this application, such as... Figure 12 As shown, when the radius is greater than a second preset multiple of the set radius, that is, the main body of the needle tip may be connected to dirt, the contour corresponding to the main body needle tip is segmented according to the preset step size, starting from the initial brightness threshold.
[0108] For example, assuming the second preset multiple is 1.2, if the radius corresponding to the target connected region contour is greater than 1.2 times the set radius, the main body of the needle tip may be dirty and needs to be cleaned. Starting from the initial brightness threshold, the contour corresponding to the main body needle tip is segmented according to the preset step size. Based on the threshold segmentation result, the radius of the main body needle tip and the center of the probe are determined. If the radius corresponding to the target connected region contour is less than 1.2 times the set radius, and the radius corresponding to the target connected region contour is greater than 0.8 times the set radius, then the needle tip radius and the center of the probe can be determined.
[0109] (17) During the threshold segmentation process, after two consecutive threshold segments, if the absolute value of the difference between the radii of the main body tip corresponding to the two consecutive threshold segmentation results is less than the preset value, the threshold segmentation is stopped, and the brightness thresholds that have participated in the threshold segmentation process are obtained. Based on the brightness thresholds, the radius of the main body tip and the center of the probe are determined. It can be understood that the number of threshold segments, the preset value, the initial threshold and the preset step size are set according to the actual application scenario.
[0110] For example, assuming the preset value is 3, the initial threshold is 90, the traversal end value is 200, and the preset step size is 10, starting from the initial threshold, the contour corresponding to the main needle tip is segmented according to the preset step size. The radius is 38 when the threshold is 120, 37 when the threshold is 130, and 36 when the threshold is 140. The absolute difference between the radii corresponding to the three consecutive thresholds is less than 3. At this time, the radius has become stable. The radius corresponding to the median of different thresholds can be taken as the final result. Then, the radius of the connected component segmented when the threshold is 130 can be the needle tip radius, and its centroid can be the probe center.
[0111] For example, assuming the preset value is 2, the initial threshold is 90, the traversal end value is 200, and the preset step size is 10, starting from the initial threshold, the contour corresponding to the main needle tip is segmented according to the preset step size. The radius is 38 when the threshold is 120 and 37 when the threshold is 130. The absolute difference between the radii corresponding to different thresholds is less than 2. At this time, the radius has become stable. The average value of the radii corresponding to different thresholds can be used as the final result. Then, the average value of the connected component radii segmented when the threshold is 120 and the threshold is 130 can be the needle tip radius, and its centroid is the probe center.
[0112] Figure 13 , Figure 14 , Figure 15 The image is a binary image after multiple thresholding operations with an initial threshold of 90 and a preset step size of 10.
[0113] Specifically, Figure 13 This is a binary image with a radius of 53 corresponding to a threshold of 90. Figure 14 This is a binary image with a radius of 50 corresponding to a threshold of 100. Figure 15 This is a binary image with a radius of 42 corresponding to a threshold of 110. Figure 16 This is a binary image with a radius of 38 corresponding to a threshold of 120. Figure 17 This is a binary image with a radius of 37 corresponding to a threshold of 130. Figure 18 It is a binary image with a radius of 36 corresponding to a threshold of 140.
[0114] (18) Output the cantilever needle recognition results: the radius and center of the cantilever needle, such as Figure 19 The image shown is the final cantilever needle recognition result obtained in this embodiment.
[0115] It is noteworthy that this application employs a method of filtering the probe binary image based on area, radius, and roundness conditions, and performing a secondary search of the local region after initial localization, which has the following beneficial effects:
[0116] (1) This application adopts a dynamic threshold adjustment method, which iterates through the adjustment threshold until the segmented cantilever needle radius area is stable. Compared with the fixed threshold segmentation method, this method can effectively remove the interference of connected dirt, making the probe positioning more accurate.
[0117] (2) After initial positioning, this application adopts a local area secondary search cantilever needle method. If the parameters are not set properly, the initial positioning result may be dirt or needle tip fragments when the needle tip is interfered with by dirt or wear. The secondary search is used to search the main body area of the needle tip within the range of the initial search, thereby making the positioning of the needle tip more accurate, preventing the probe from being misidentified under the condition of dirt interference and needle tip wear, and enhancing the stability of probe positioning.
[0118] (3) This application uses area, radius and roundness conditions to filter the contour, which can effectively remove bright spot interference and prevent the probe from misidentifying under the condition of dirt interference and tip wear.
[0119] Figure 20 This is a schematic diagram of a device structure for identifying a probe according to an embodiment of this application, as shown below. Figure 20 As shown, the device includes:
[0120] The recognition module 200 is used to acquire the binary image corresponding to the probe image, and to identify all connected component contours in the binary image, and to filter all connected component contours to obtain the target connected component contour.
[0121] The first determining module 202 is used to determine the average value of pixels within the target connected region contour and the radius corresponding to the target connected region contour. When the average value is not less than a set value and the radius is not less than a first preset multiple of the set radius, the target connected region contour is determined to be the contour corresponding to the main body tip of the probe.
[0122] The segmentation module 204 is used to perform threshold segmentation on the contour corresponding to the main body needle tip, starting from the initial brightness threshold and according to a preset step size, when the radius is greater than a second preset multiple of the set radius.
[0123] The second determining module 206 is used to determine the radius of the main needle tip and the center of the probe based on the threshold segmentation result.
[0124] In this device, the recognition module 200 is used to acquire the binary image corresponding to the probe image and recognize all connected component contours in the binary image, and filter all connected component contours to obtain the target connected component contour; the first determination module 202 is used to determine the average value of pixels in the target connected component contour and the radius corresponding to the target connected component contour, and determine the target connected component contour as the contour corresponding to the main body tip of the probe when the average value is not less than a set value and the radius is not less than a first preset multiple of the set radius; the segmentation module 204 is used to perform threshold segmentation on the contour corresponding to the main body tip according to a preset step size, starting from an initial brightness threshold when the radius is greater than a second preset multiple of the set radius, wherein the second preset multiple is greater than the first preset multiple; the second determination module 206 is used to determine the radius of the main body tip and the center of the probe based on the threshold segmentation result, thereby achieving the purpose of accurately identifying and locating the probe center, thus realizing the technical effect of eliminating the interference of probe dirt and probe wear and splitting interference when identifying and locating the probe center, and thus solving the technical problem of deviation in the identification of the probe center caused by the easy wear and dirt of the probe during the needle insertion process, resulting in deviation in the needle insertion position.
[0125] According to another aspect of the embodiments of this application, a non-volatile storage medium is also provided, the non-volatile storage medium including a stored program, wherein, when the program is running, a method for controlling the device where the non-volatile storage medium is located to execute any identification probe is provided.
[0126] Specifically, the aforementioned storage medium is used to store program instructions for the following functions, thereby implementing the following functions:
[0127] Obtain the binary image corresponding to the probe image and identify all connected component contours in the binary image. Filter all connected component contours to obtain the target connected component contour. Determine the average value of pixels within the target connected component contour and the radius corresponding to the target connected component contour. If the average value is not less than a set value and the radius is not less than a first preset multiple of the set radius, the target connected component contour is determined to be the contour corresponding to the main body tip of the probe. If the radius is greater than a second preset multiple of the set radius, threshold segmentation is performed on the contour corresponding to the main body tip, starting from the initial brightness threshold and following a preset step size, where the second preset multiple is greater than the first preset multiple. Based on the threshold segmentation results, determine the radius of the main body tip and the center of the probe.
[0128] Optionally, in this embodiment, the storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. More specific examples of the storage medium include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.
[0129] In an exemplary embodiment of this application, a computer program product is also provided, including a computer program that, when executed by a processor, implements the method for any of the aforementioned identification probes.
[0130] Optionally, when executed by a processor, the computer program may perform the following steps:
[0131] Obtain the binary image corresponding to the probe image and identify all connected component contours in the binary image. Filter all connected component contours to obtain the target connected component contour. Determine the average value of pixels within the target connected component contour and the radius corresponding to the target connected component contour. If the average value is not less than a set value and the radius is not less than a first preset multiple of the set radius, the target connected component contour is determined to be the contour corresponding to the main body tip of the probe. If the radius is greater than a second preset multiple of the set radius, threshold segmentation is performed on the contour corresponding to the main body tip, starting from the initial brightness threshold and following a preset step size, where the second preset multiple is greater than the first preset multiple. Based on the threshold segmentation results, determine the radius of the main body tip and the center of the probe.
[0132] An electronic device is provided according to an embodiment of this application, the electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform any of the above-described methods for identifying probes.
[0133] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.
[0134] Figure 21 This is a schematic block diagram of an example electronic device 2100 according to an embodiment of this application. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Figure 21As shown, device 2100 includes a computing unit 2101, which can perform various appropriate actions and processes according to a computer program stored in read-only memory (ROM) 2102 or a computer program loaded from storage unit 2108 into random access memory (RAM) 2103. The RAM 2103 may also store various programs and data required for the operation of device 2100. The computing unit 2101, ROM 2102, and RAM 2103 are interconnected via bus 2104. Input / output (I / O) interface 2105 is also connected to bus 2104.
[0135] Multiple components in device 2100 are connected to I / O interface 2105, including: input unit 2106, such as keyboard, mouse, etc.; output unit 2107, such as various types of monitors, speakers, etc.; storage unit 2108, such as disk, optical disk, etc.; and communication unit 2109, such as network card, modem, wireless transceiver, etc. Communication unit 2109 allows device 2100 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.
[0136] The computing unit 2101 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 2101 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 2101 performs the various methods and processes described above, such as the method of identifying probes. For example, in some embodiments, the method of identifying probes may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 2108. In some embodiments, part or all of the computer program may be loaded and / or installed on device 2100 via ROM 2102 and / or communication unit 2109. When the computer program is loaded into RAM 2103 and executed by the computing unit 2101, one or more steps of the method of identifying probes described above may be performed. Alternatively, in other embodiments, the computing unit 2101 may be configured to perform the method of identifying probes by any other suitable means (e.g., by means of firmware).
[0137] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.
[0138] The program code used to implement the methods of this application may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.
[0139] In the context of this application, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.
[0140] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.
[0141] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.
[0142] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.
[0143] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0144] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.
[0145] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.
[0146] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.
Claims
1. A method for identifying probes, characterized in that, include: Obtain the binary image corresponding to the probe image, identify all connected component contours in the binary image, and filter all connected component contours to obtain the target connected component contour. The average value of pixels within the target connected region contour and the radius corresponding to the target connected region contour are determined. If the average value is not less than a set value and the radius is not less than a first preset multiple of the set radius, the target connected region contour is determined to be the contour corresponding to the main body tip of the probe. When the radius is greater than a second preset multiple of the set radius, the contour corresponding to the main body needle tip is segmented according to a preset step size, starting from the initial brightness threshold, wherein the second preset multiple is greater than the first preset multiple; Determining the radius of the main needle tip and the center of the probe based on the threshold segmentation result includes: During the threshold segmentation process, if, after at least two consecutive threshold segments, the absolute value of the difference between the radii of the main body tip corresponding to the results of the at least two consecutive threshold segments is less than a preset value, then the brightness thresholds used in the at least two consecutive threshold segments are obtained; the median of the brightness thresholds is determined as the first target brightness threshold; the radius of the connected region segmented under the first target brightness threshold is determined as the radius of the main body tip, and the centroid of the connected region segmented under the target brightness threshold is determined as the center of the probe.
2. The method according to claim 1, characterized in that, If the radius is not greater than a second preset multiple of the set radius, the probe image is identified to obtain the radius of the main needle tip and the center.
3. The method according to claim 1, characterized in that, The target connected component contour is obtained by filtering all the connected component contours, including: The first connected component contour set is obtained by filtering all the connected component contours. The first centroid corresponding to each connected component contour in the first connected component contour set is obtained. The connected component contour corresponding to the first centroid closest to the center position of the probe image is determined as the target connected component contour.
4. The method according to claim 3, characterized in that, The first set of connected component contours is obtained by filtering all the connected component contours, including: Iterate through all the connected component contours in sequence, and during the traversal, determine whether the area of the currently being detected connected component contour is greater than the set area. If the area is greater than the set area, determine whether the radius of the contour is within a preset range; If the radius of the contour falls within the preset range, then it is determined whether the roundness of the contour is greater than the set roundness. If the roundness of the contour is greater than the set roundness, then the currently detected connected component contour is determined to be an element in the first connected component contour set.
5. The method according to claim 1, characterized in that, After determining the average value of the pixels within the target connected component contour and the radius corresponding to the target connected component contour, the method further includes: If the average value is less than the set value or the radius of the target connected region contour is less than a first preset multiple of the set radius, the target connected region contour is subjected to secondary screening to obtain the contour corresponding to the main body tip of the probe.
6. The method according to claim 5, characterized in that, The target connected component contour is further filtered to obtain the contour corresponding to the main body tip of the probe, including: The center is the location of the first centroid that is closest to the center of the probe image; Determine the original size of the target connected component contour, and crop a local region image from the probe image, wherein the size of the local region image is a preset multiple of the original size; The local region image is binarized to obtain a binary image corresponding to the local region image. Connected components are retrieved from the binary image to obtain a second set of connected component contours. The contour corresponding to the main tip of the probe is determined based on the second set of connected component contours.
7. The method according to claim 6, characterized in that, The contour corresponding to the main body tip of the probe is determined based on the second set of connected component contours, including: Each second connected component contour in the second connected component contour set is mapped to the local region image as a positioning mask, wherein the positioning mask corresponds one-to-one with the coordinates of the main needle tip in the local region image; Obtain the pixel values of each point located within the positioning mask in the local region image, and the number of pixel values greater than a set value; The region with the most pixel values greater than the set value is identified as the main body region. Obtain the coordinates of any pixel in the main region, select a point in the binary image corresponding to the probe image that has the same coordinates as the arbitrary pixel as a seed point, determine the third connected region containing the seed point, and use the contour of the third connected region as the contour corresponding to the main tip of the probe.
8. A device for identifying probes, characterized in that, include: The recognition module is used to acquire the binary image corresponding to the probe image, identify all connected component contours in the binary image, and filter all connected component contours to obtain the target connected component contour. The first determining module is used to determine the average value of pixels within the target connected region contour and the radius corresponding to the target connected region contour. When the average value is not less than a set value and the radius is not less than a first preset multiple of the set radius, the target connected region contour is determined to be the contour corresponding to the main body tip of the probe. The segmentation module is used to perform threshold segmentation on the contour corresponding to the main body needle tip, starting from an initial brightness threshold and according to a preset step size, when the radius is greater than a second preset multiple of the set radius; wherein the second preset multiple is greater than the first preset multiple. The second determining module is used to determine the radius of the main needle tip and the center of the probe based on the threshold segmentation result, including: during the threshold segmentation process, if, after at least two consecutive threshold segments, the absolute value of the difference between the radii of the main needle tip corresponding to the results of the at least two consecutive threshold segments is less than a preset value, obtaining each brightness threshold used in the at least two consecutive threshold segments; determining the median of each brightness threshold as a first target brightness threshold; determining the radius of the connected region segmented under the first target brightness threshold as the radius of the main needle tip, and the centroid of the connected region segmented under the target brightness threshold as the center of the probe.
9. A non-volatile storage medium, characterized in that, The storage medium includes a stored program, wherein, when the program is executed, it controls the device containing the storage medium to perform the method of the identification probe according to any one of claims 1 to 7.
10. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the instructions to implement the method of identifying probes as described in any one of claims 1 to 7.