Positioning method and device using probe identification, storage medium and electronic device

Abstract

The application discloses a probe recognition positioning method and device, a storage medium and electronic equipment. The method comprises the following steps: acquiring a binary image corresponding to a probe image, and recognizing all connected domain contours in the binary image; screening all the connected domain contours based on a predetermined contour area, a preset range and a predetermined roundness, to obtain a target connected domain contour set; determining a target radius of a minimum enclosing circle corresponding to each target connected domain in the target connected domain contour set, comparing the target radius with a predetermined radius, and determining to select a center of the minimum enclosing circle or a centroid of the target connected domain as a center of the probe. The application solves the technical problems of low accuracy of recognizing the probe center caused by the fact that a positioning method of related technologies is restricted by light, the centroid of the probe is directly determined as the center of the probe, and multiple feature similar contours may exist in a field of view of the probe image.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of semiconductors, and in particular, to a positioning method and device for probe recognition, a storage medium, and an electronic device. BACKGROUND

[0002] With the increasing requirements of modern electronic devices for miniaturization, light weight, high performance, multi-function, low power consumption and low cost, the feature size of IC chips is continuously reduced, and the integration scale is rapidly expanded, and the chip packaging technology is also continuously innovated.

[0003] Point needles are commonly used for TSV packaging testing, and can also be used for copper pillar bump packaging testing and traditional Pad testing, and have a very wide range of applications. During testing, the needle needs to be aligned, and the coordinate position of the probe needs to be determined during the alignment process. The type of probe selected for testing different types of bumps will differ, for example, flat head needles can sometimes be used for testing of copper pillar bumps, and during testing, the coordinate position of the bump needs to be determined, and the coordinate position of the probe also needs to be determined during the alignment process.

[0004] Point needles have a relatively circular shape, and there may be some bright needle tip interference inside; flat head needles also have a relatively circular shape, and edge damage has a greater impact on their shape; for point needles and flat head needles, direct use of the centroid calculation method will result in errors in positioning. Therefore, due to the differences in the shapes of point needles, flat head needles and traditional probes, the traditional probe recognition method cannot adapt to the recognition and positioning of point needles and flat head needles.

[0005] No effective solutions have been proposed to address the above problems. SUMMARY

[0006] Embodiments of the present application provide a positioning method and device for probe recognition, a storage medium, and an electronic device, to at least solve the technical problems of low accuracy in recognizing the center of the probe due to the positioning method of the related art being restricted by light, directly determining the centroid of the probe as the center of the probe, and the presence of multiple features similar to the outline in the field of view of the probe image.

[0007] According to an aspect of the embodiments of the present application, a positioning method for probe recognition is provided, including: obtaining a binary image corresponding to a probe image, and recognizing all connected domain contours in the binary image; screening all the connected domain contours based on a predetermined contour area, a preset range and a predetermined roundness, to obtain a target connected domain contour set; determining a minimum enclosing circle corresponding to each target connected domain in the target connected domain contour set; determining a target radius corresponding to the minimum enclosing circle, and comparing the target radius with a predetermined radius; and determining to select a center of the minimum enclosing circle as a center of the probe, or a centroid of the target connected domain as the center of the probe, according to a size relationship between the target radius and the predetermined radius.

[0008] Optionally, the determining to select the center of the minimum enclosing circle as the center of the probe, or the centroid of the target connected domain as the center of the probe according to the size relationship between the target radius and the predetermined radius includes: in a case where the target radius is smaller than the predetermined radius, determining a center of the minimum enclosing circle as a center of a corresponding probe; and in a case where the target radius is greater than the predetermined radius, obtaining a centroid corresponding to each target connected domain contour, and determining the centroid corresponding to each target connected domain contour as a center of each probe.

[0009] Optionally, the determining the preset range according to the predetermined radius includes: determining a sum of the predetermined radius and a radius upper limit deviation percentage, determining a product of the predetermined radius and the sum as an upper limit value of the preset range; determining a difference between the predetermined radius and the radius upper limit deviation percentage, and determining a product of the predetermined radius and the difference as a lower limit value of the preset range.

[0010] Optionally, the screening all the connected domain contours based on the predetermined contour area, the preset range and the predetermined roundness to obtain the target connected domain contour set includes: sequentially traversing all the connected domain contours, and in the traversing process, judging whether an area of a currently detected connected domain contour is greater than the predetermined contour area; in a case where the area of the currently detected connected domain contour is greater than the predetermined contour area, determining a minimum enclosing circle corresponding to the currently detected connected domain contour; in a case where the target radius falls within the preset range, judging whether a roundness of the currently detected connected domain contour is greater than a set roundness; and in a case where the roundness of the currently detected connected domain contour is greater than the set roundness, determining the currently detected connected domain contour as an element in the target connected domain contour set.

[0011] Optionally, before the judging whether the roundness of the currently detected connected domain contour is greater than the set roundness, the method further includes: determining a first area corresponding to the currently detected connected domain contour, and a second area corresponding to the minimum enclosing circle; and determining a ratio of the first area to the second area as the roundness of the currently detected connected domain contour.

[0012] Optionally, the method further comprises: establishing a coordinate system with the center of the probe image as the origin, and determining the coordinates corresponding to the center of each probe; receiving an operation instruction input by the target object, wherein the operation instruction is used to indicate the coordinates of the center probe to be determined in the coordinate system; and outputting the center corresponding to the coordinates in response to the operation instruction.

[0013] Optionally, in response to the operation instruction, the center corresponding to the coordinates is outputted, including: in the case that the operation instruction indicates that there are multiple center probes to be determined, determining the distance between each center probe to be determined and the origin, wherein the smaller the distance, the smaller the distortion degree of the probe; determining the center probe to be determined which is greater than a preset threshold as a negligible probe, determining the center probe to be determined which is less than the preset threshold as a to-be-processed probe, determining the target coordinates corresponding to the to-be-processed probe, and outputting the center corresponding to the target coordinates in response to the operation instruction.

[0014] Optionally, the binary image corresponding to the probe image is obtained, including: performing gray processing on the probe image to obtain a gray image corresponding to the probe; and calling an adaptive threshold segmentation method to perform adaptive threshold segmentation on the gray image to obtain the binary image.

[0015] Optionally, the binary image includes a first binary image and a second binary image, the adaptive threshold segmentation method is called to perform adaptive threshold segmentation on the gray image to obtain the binary image, including: obtaining the type of the probe in the probe image, wherein the type of the probe includes a first type and a second type, wherein the needle tip of the probe of the first type has a darkening feature, and the needle tip of the probe of the second type has a brightening feature; in the case that the type of the probe is the first type, calling the adaptive threshold segmentation method to segment the gray image to obtain a darkening needle body in the probe image, and performing binaryzation processing on the probe image from which the darkening needle body is segmented to obtain the first binary image; and in the case that the type of the probe is the second type, calling the adaptive threshold segmentation method to segment the gray image to obtain a brightening needle body in the probe image, and performing binaryzation processing on the probe image from which the brightening needle body is segmented to obtain the second binary image.

[0016] According to another aspect of the embodiments of the present application, a probe recognition positioning apparatus is also provided, comprising: an acquisition module configured to acquire a binary image corresponding to a probe image and identify all connected domain contours in the binary image; a screening module configured to screen all the connected domain contours based on a predetermined contour area, a preset range and a predetermined roundness to obtain a target connected domain contour set, wherein the preset range is determined according to a predetermined radius; a first determination module configured to determine a minimum enclosing circle corresponding to each target connected domain in the target connected domain contour set; a second determination module configured to determine a target radius corresponding to the minimum enclosing circle and compare the target radius with the predetermined radius; and a third determination module configured to determine, according to the size relationship between the target radius and the predetermined radius, whether to select the center of the minimum enclosing circle as the center of the probe or to select the centroid of the target connected domain as the center of the probe.

[0017] According to another aspect of the embodiments of the present application, a non-volatile storage medium is also provided, comprising: a storage medium comprising a stored program, wherein the program, when executed, controls a device in which the storage medium is located to perform any of the probe recognition positioning methods.

[0018] According to another aspect of the embodiments of the present application, an electronic device is also provided, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to execute the instructions to implement any of the probe recognition positioning methods.

[0019] In the embodiments of the present application, the area, radius and roundness conditions of the probe binary image are filtered, and the centroid or the center of the minimum enclosing circle is determined as the center of the probe after screening. Specifically, by comparing the size relationship between the radius corresponding to the minimum enclosing circle of the target connected domain and the predetermined radius, the center of the minimum enclosing circle or the centroid of the target connected domain is selected as the center of the probe, which achieves the purpose of accurately recognizing and positioning the center of the probe, thereby realizing the technical effect of accurately positioning the center of the probe according to the characteristics of different probes, and further solving the technical problem of low accuracy in recognizing the center of the probe due to the positioning method of the related art being restricted by light, the centroid being used to calculate the center of the probe, and multiple similar contours possibly existing in the field of view of the probe image. BRIEF DESCRIPTION OF DRAWINGS

[0020] The accompanying drawings, which are included to provide a further understanding of the present application, form a part of the present application and illustrate the illustrative embodiments of the present application and its description, which are used to explain the present application, and do not constitute improper limitations on the present application. In the drawings:

[0021] Figure 1 is a flowchart of a probe recognition positioning method according to an embodiment of the present application;

[0022] Figure 2is a flowchart of a process of determining a target connected component contour set according to an embodiment of the present application;

[0023] Figure 3 is a flowchart of a threshold segmentation process according to an embodiment of the present application;

[0024] Figure 4 is a flowchart of a point needle recognition process according to an embodiment of the present application;

[0025] Figure 5 is a schematic diagram of a point needle image under a high-magnification camera according to an embodiment of the present application;

[0026] Figure 6 is a schematic diagram of a point needle binary image according to an embodiment of the present application;

[0027] Figure 7 is a schematic diagram of a connected component contour image according to an embodiment of the present application;

[0028] Figure 8 is a schematic diagram of a point needle recognition result image according to an embodiment of the present application;

[0029] Figure 9 is a flowchart of a flat needle recognition process according to an embodiment of the present application;

[0030] Figure 10 is a schematic diagram of a flat needle image under a high-magnification camera according to an embodiment of the present application;

[0031] Figure 11 is a schematic diagram of a flat needle binary image according to an embodiment of the present application;

[0032] Figure 12 is a schematic diagram of a connected component contour image according to an embodiment of the present application;

[0033] Figure 13 is a schematic diagram of a flat needle recognition result image according to an embodiment of the present application;

[0034] Figure 14 is a schematic diagram of a positioning device for probe recognition according to an embodiment of the present application;

[0035] Figure 15 is a schematic block diagram of an example electronic device 1500 according to an embodiment of the present application. DETAILED DESCRIPTION

[0036] In order to make the personnel in the technical field better understand the scheme of the present application, the technical scheme in the embodiments of the present application will be clearly and completely described below in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by the person skilled in the art without creative labor should belong to the scope of protection of the present application.

[0037] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-described drawings are used to distinguish similar objects, and do not necessarily have to be used to describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not have to be limited to only those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0038] According to the embodiments of the present application, a method embodiment for positioning probe recognition is provided. It should be noted that the steps shown in the flowchart of the drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described herein can be executed in an order different from that shown herein.

[0039] Figure 1 A method for positioning probe recognition according to the embodiments of the present application, as shown in Figure 1 The method comprises the following steps:

[0040] Step S102, acquiring a binary image corresponding to a probe image, and identifying all connected domain contours in the binary image;

[0041] Step S104, screening all connected domain contours based on a predetermined contour area, a preset range and a predetermined roundness, to obtain a target connected domain contour set, wherein the preset range is determined according to a predetermined radius;

[0042] Step S106, determining the minimum enclosing circle corresponding to each target connected domain in the target connected domain contour set;

[0043] Step S108, determining a target radius corresponding to the minimum enclosing circle, and comparing the target radius with the predetermined radius;

[0044] In step S110, according to the size relationship between the target radius and the predetermined radius, the center of the minimum enclosing circle is determined as the center of the probe, or the centroid of the target connected domain is determined as the center of the probe.

[0045] In the embodiments of the present application, the area, radius and roundness conditions of the probe binary image are filtered, and the centroid or the center of the minimum enclosing circle of the probe is determined as the probe center. The binary image corresponding to the probe image is obtained, and all connected domain contours in the binary image are recognized. All connected domain contours are filtered based on the predetermined contour area, the preset range and the predetermined roundness, and a target connected domain contour set is obtained. The target radius corresponding to the minimum enclosing circle of each target connected domain in the target connected domain contour set is determined, and the size of the target radius and the predetermined radius is compared. According to the comparison result, the center of the minimum enclosing circle is determined as the center of the probe, or the centroid of the target connected domain is determined as the center of the probe, which achieves the purpose of accurately identifying and positioning the probe center, thereby realizing the technical effect of accurately positioning the probe center according to the characteristics of different probes, and further solving the technical problems of low accuracy of identifying the probe center due to the positioning method of the related art being restricted by light, directly determining the centroid of the probe as the center of the probe, and multiple contours with similar characteristics in the field of view of the probe image.

[0046] In some optional embodiments of the present application, according to the size relationship between the target radius and the predetermined radius, the center of the target connected domain is determined as the center of the probe, or the centroid of the target connected domain is determined as the center of the probe. Specifically, in the case that the target radius is smaller than the predetermined radius, the center of the minimum enclosing circle corresponding to the target radius is determined as the center of the corresponding probe. In the case that the target radius is greater than the predetermined radius, the centroid corresponding to each target connected domain contour is obtained, and the centroid corresponding to each target connected domain contour is determined as the center of each probe. It is easy to note that by comparing the target radius of the minimum enclosing circle with the predetermined radius, the center of the probe can be quickly and accurately determined. It should be noted that when the target radius is normal or small, it indicates that the edge of the probe is damaged, and the result of using the center positioning will be more accurate. When the target radius is large, it indicates that the outside of the probe contour is disturbed, and the result of using the centroid positioning will be more accurate. The above-mentioned probe includes but is not limited to a point needle, a cantilever needle and other types of probes.

[0047] Optionally, the preset range is determined according to the predetermined radius, including: determining a sum of the predetermined radius and a radius upper limit deviation percentage, determining a product of the predetermined radius and the sum as an upper limit value of the preset range; determining a difference between the predetermined percentage and the radius upper limit deviation percentage, determining a product of the predetermined radius and the difference as a lower limit value of the preset range. For example, assuming that the radius upper limit deviation percentage and the radius lower limit deviation percentage are 5%, and the radius is set to 40, then the radius upper limit = 40*(1+5%) = 42, and the radius lower limit = 40*(1-5%) = 38, so it can be confirmed that the preset range is [38, 42], and it should be noted that the radius upper limit deviation percentage and the radius lower limit deviation percentage can be flexibly set according to specific application scenarios.

[0048] As an optional implementation, the radius upper limit deviation percentage and the radius lower limit deviation percentage can be determined by: obtaining a size of the probe, wherein the size at least includes a diameter of the probe, and then the radius upper limit deviation percentage and the radius lower limit deviation percentage can be determined according to the diameter, wherein the smaller the diameter, the smaller the radius upper limit deviation percentage and the radius lower limit deviation percentage. It can be understood that when the size of the probe is small, the requirement for the accuracy of positioning the center of the probe is higher, so the filtering degree of the preliminary screening can be improved by appropriately adjusting the size of the radius upper limit deviation percentage and the radius lower limit deviation percentage, in addition, when the radius of the probe itself is small, the error range it can allow will be naturally smaller, therefore, based on the above two points, in the embodiments of the present application, the radius upper limit deviation percentage and the radius lower limit deviation percentage can be determined according to the diameter, wherein the smaller the diameter, the smaller the radius upper limit deviation percentage and the radius lower limit deviation percentage.

[0049] Figure 2 is a flowchart of determining a target connected contour set according to an embodiment of the present application, as shown in Figure 2 , the method includes the following steps:

[0050] Step S202, all connected contours are traversed in turn, and in the traversal process, it is judged whether the area of the connected contour currently being detected is greater than a predetermined contour area;

[0051] Step S204, in the case that the area of the connected contour currently being detected is greater than the predetermined contour area, the minimum enclosing circle corresponding to the connected contour currently being detected is determined;

[0052] Step S206, in the case that the target radius falls within the preset range, it is judged whether the circularity of the connected contour currently being detected is greater than a set circularity;

[0053] Step S208, in the case that the roundness of the current being detected connected domain contour is greater than the set roundness, then the current being detected connected domain contour is determined as an element in the target connected domain contour set.

[0054] In some optional embodiments of the present application, before determining whether the roundness of the current being detected connected domain contour is greater than the set roundness, the first area corresponding to the current being detected connected domain contour and the second area corresponding to the minimum enclosing circle can be determined; and the ratio of the first area to the second area is the roundness of the current being detected connected domain contour. The closer the above ratio is to 1, the more round the contour is.

[0055] Since there can be multiple probe images in the same image, as an optional implementation, a coordinate system can be established with the center of the probe image as the origin, and the coordinates corresponding to the center of each probe can be determined; an operation instruction input by the target object can be received, wherein the operation instruction is used to indicate the coordinates of the center of the probe to be determined in the coordinate system; and the center corresponding to the target coordinates can be output in response to the operation instruction. Through the above technical solution, the center of any one probe in the same probe image can be displayed according to the operation instruction input by the target object.

[0056] Generally, when the probe is far away from the center position in the probe image, the possibility of the distortion degree of the probe is greater, and the corresponding distortion degree is greater. Even if the center of the probe is obtained through the above implementation, the accuracy is often poor. Therefore, as an optional implementation, in the case that there are multiple probes to be determined in the operation instruction, the probe to be determined whose distance from the origin is greater than a preset threshold can be determined as a negligible probe, the probe to be determined whose distance from the origin is less than the preset threshold can be determined as a probe to be processed, the target coordinates corresponding to the probe to be processed can be determined, the center corresponding to the target coordinates can be output in response to the operation instruction. As known from the above, the smaller the distance from the center of the probe image, the smaller the distortion degree of the probe. Therefore, when the probes far away from the origin (i.e. the center of the probe image) are filtered out through the distance from the origin, the processing resources are saved, invalid processing is avoided, and the processing efficiency of the probe to be processed (i.e. the probe close to the origin) is accelerated. It should be noted that the above preset threshold can be set according to the actual application scenario. For example, the critical distance of distortion can be obtained through multiple experiments, and the critical distance can be used as the above preset threshold.

[0057] Of course, it can be understood that the critical distance can be appropriately adjusted, for example, the above preset threshold can be obtained by adding an allowable error range to the critical distance.

[0058] In some optional embodiments of the present application, the binary image corresponding to the probe image can be determined by the following steps. Specifically, the probe image can be subjected to grayscale processing to obtain a grayscale image corresponding to the probe; and an adaptive threshold segmentation method can be called to perform adaptive threshold segmentation on the grayscale image to obtain the binary image.

[0059] It should be noted that the optional threshold segmentation method includes but is not limited to local adaptive threshold segmentation, maximum entropy threshold segmentation, Otsu method, iterative threshold segmentation and fixed threshold segmentation.

[0060] Since the needle tip of some probes has a bright feature and the needle tip of some probes has a dark feature, in an example embodiment of the present application, the type of the probe can be identified in the process of calling the adaptive threshold segmentation method to perform adaptive threshold segmentation on the grayscale image to obtain the binary image. Specifically, as shown in Figure 3 The adaptive threshold segmentation includes the following steps:

[0061] Step S302: The type of the probe in the probe image is obtained, wherein the type of the probe includes a first type and a second type, wherein the needle tip of the probe of the first type has a dark feature and the needle tip of the probe of the second type has a bright feature.

[0062] Step S304: In the case where the type of the probe is the first type, the adaptive threshold segmentation method is called to segment the grayscale image to obtain a dark needle body in the probe image, and the probe image from which the dark needle body is segmented is subjected to binaryzation processing to obtain a first binary image.

[0063] Step S306: In the case where the type of the probe is the second type, the adaptive threshold segmentation method is called to segment the grayscale image to obtain a bright needle body in the probe image, and the probe image from which the bright needle body is segmented is subjected to binaryzation processing to obtain a second binary image.

[0064] It can be understood that the above technical solution of the present application can realize the identification of two probes with different needle tip features based on a set of identification algorithm, thereby improving the universality of the algorithm.

[0065] For example, the probe of the first type can be a point needle and the probe of the second type can be a flat needle, wherein the needle tip of the point needle has a dark feature and the needle tip of the flat needle has a bright feature.

[0066] It can be understood that the present application is described with respect to a point needle and a flat needle for related embodiments, which is applicable to any probe with a bright needle tip feature or a dark needle tip feature. Specifically, taking the point needle as an example, Figure 4 is a flowchart of a point needle identification process according to an embodiment of the present application, as shown in Figure 4 The flowchart mainly includes the following steps:

[0067] (1) inputting the point needle image collected by the high-magnification camera of the probe station, as shown in FIG. 2; Figure 5 FIG. 2 shows the point needle image collected by the high-magnification camera in this embodiment;

[0068] (2) obtaining the binary image of the point needle, as shown in FIG. 3; Figure 6 FIG. 3 shows that the collected image is grayed, and after the threshold segmentation, the dark needle body is segmented out to obtain the binary image of the point needle;

[0069] It should be noted that the threshold segmentation method that can be selected includes but is not limited to local adaptive threshold segmentation method, maximum entropy threshold segmentation method, Otsu method, iterative threshold segmentation method and fixed threshold segmentation method, and the local adaptive threshold segmentation method can be preferred.

[0070] (3) identifying the contour of all connected domains in the binary image, as shown in FIG. 4; Figure 7 FIG. 4 shows that the contour of the connected domain is a white connected region in the binary image;

[0071] (4) traversing all the contours of the connected domains in turn;

[0072] (5) in the traversal process, it is judged whether the area of the contour of the connected domain is greater than the predetermined contour area, if the area of the contour is greater than the predetermined contour area, the next step is entered, otherwise the next connected domain contour is returned to step (4) for traversal, and it should be noted that the contour area is the pixel area within the contour line;

[0073] (6) in the case where the area of the contour of the connected domain is greater than the predetermined contour area, the minimum enclosing circle corresponding to the currently detected contour of the connected domain is determined, the minimum enclosing circle of the contour of the connected domain is calculated, and the center and radius of the minimum enclosing circle are calculated;

[0074] (7) it is judged whether the radius of the minimum enclosing circle falls within the preset range, if the radius of the minimum enclosing circle falls within the preset range, it means that the radius meets the requirements, then the next step is entered, otherwise the next connected domain contour is returned to step (4) for traversal;

[0075] It should be noted that the above-mentioned preset range is limited by the upper limit of the radius and the lower limit of the radius, wherein the upper limit of the radius = the predetermined radius x (1+the upper limit deviation percentage of the radius); the lower limit of the radius = the predetermined radius x (1-the lower limit deviation percentage of the radius), wherein the upper limit deviation percentage of the radius and the lower limit deviation percentage of the radius can be set to 5%.

[0076] (8) it is judged whether the roundness of the contour is greater than the set roundness, if yes, the next step is entered, otherwise the next connected domain contour is returned to step (4) for traversal;

[0077] It is easy to note that through the above steps (4) to step (8), the screening of all connected domain contours can be completed based on the predetermined contour area, the preset range and the predetermined roundness, and the target connected domain contour set meeting the requirements is obtained.

[0078] (9) Determine whether the radius of each target contour in the target connected domain contour set is greater than the predetermined radius. If the contour radius is greater than the predetermined radius, the centroid of the target connected domain is taken as the center of the point needle, otherwise the center of the minimum enclosing circle obtained in step (6) is directly taken as the center of the point needle.

[0079] (10) Screen out the target connected domain contour closest to the image center, and determine the target connected domain contour as the connected domain contour of the point needle center to be positioned;

[0080] (11) Output the point needle recognition result, which includes the point needle radius and center, as shown in Figure 8 , which is the point needle recognition result diagram obtained by the final recognition and positioning of this embodiment.

[0081] Taking a flat needle as an example, Figure 9 is a flowchart of a flat needle recognition process according to an embodiment of the present application, as shown in Figure 9 , the process mainly includes the following steps:

[0082] (1) Input the flat needle image collected by the probe station high-power camera, as shown in Figure 10 , which is the flat needle image collected by the high-power camera in this embodiment;

[0083] (2) Obtain the binary image of the flat needle, as shown in Figure 11 , which is the image collected in this embodiment, and the image is grayed, and after grayed, threshold segmentation is performed to segment out the darkened needle body, and the binary image of the flat needle is obtained;

[0084] It should be noted that the threshold segmentation method that can be selected includes but is not limited to maximum entropy threshold segmentation method, local adaptive threshold segmentation method, Otsu method, iterative threshold segmentation method and fixed threshold segmentation method, and the maximum entropy threshold segmentation method can be preferred.

[0085] (3) Identify the contour of all connected domains in the binary image, as shown in Figure 12 , the contour of the connected domain is a white connected region in the binary image;

[0086] (4) Traverse all connected domain contours in turn;

[0087] (5) In the traversal process, it is judged whether the contour area of the connected domain is greater than a predetermined contour area. If the contour area is greater than the predetermined contour area, the next step is entered, otherwise the next connected domain contour is traversed and returned to step (4);

[0088] (6) In the case where the contour area of the connected domain is greater than the predetermined contour area, the minimum enclosing circle corresponding to the connected domain contour being currently detected is determined, the minimum enclosing circle of the connected domain contour is calculated, and the center and radius of the minimum enclosing circle are calculated;

[0089] (7) It is judged whether the radius of the minimum enclosing circle is within a preset range. If the radius of the minimum enclosing circle falls within the preset range, it means that the radius meets the requirements, and the next step is entered, otherwise the next connected domain contour is traversed and returned to step (4);

[0090] (8) It is judged whether the roundness of the contour is greater than a set roundness. If it is true, the next step is entered, otherwise the next connected domain contour is traversed and returned to step (4);

[0091] It is easy to note that through the above steps (4) to (8), the screening of all connected domain contours can be completed based on the predetermined contour area, the predetermined radius and the predetermined roundness, and a target connected domain contour set meeting the requirements is obtained.

[0092] (9) It is judged whether the radius of each target contour in the target connected domain contour set is greater than a predetermined radius. If the contour radius is greater than the predetermined radius, the center of mass of the flat-headed needle is taken as the center of the flat-headed needle, otherwise the center of the circle obtained in step (8) is directly taken as the center of the flat-headed needle.

[0093] (10) The target connected domain contour closest to the center of the image is screened out, and the target connected domain contour is determined as the connected domain contour of the flat-headed needle center to be positioned;

[0094] (11) The flat-headed needle recognition result is outputted, which includes the flat-headed needle radius and center, as shown in FIG. 6, which is a schematic diagram of the flat-headed needle recognition result image finally recognized and positioned by the embodiment. Figure 13

[0095] It is easy to note that the application adopts the filtering of the area, radius and roundness conditions of the probe binary image, and the center of mass or the center of the minimum enclosing circle is determined as the probe center after screening, which has the following beneficial effects:

[0096] ​(1) According to the characteristics of the point needle, a local adaptive threshold segmentation method is selected to enhance the adaptability of the recognition algorithm to light, so that the stability of point needle recognition is stronger; according to the characteristics of the flat needle, a maximum entropy threshold segmentation method is selected, and after calculating the maximum entropy threshold of the flat needle image, the maximum entropy threshold is used for threshold segmentation, which enhances the adaptability of the recognition algorithm to light, so that the stability of flat needle recognition is stronger.

[0097] (2) The center of the centroid combined with the center of the circle is used to adapt to the shape of the probe, which can effectively resist dirt and wear interference, and the positioning is more accurate.

[0098] (3) The area, radius and roundness conditions are used to filter the outline, which can effectively remove some bright spot interference that does not meet the conditions, prevent misidentification, and make the probe positioning more accurate.

[0099] Figure 14 Figure 1 is a structural schematic diagram of a probe recognition positioning device according to an embodiment of the present application, as shown in the figure, the device comprises: Figure 14

[0100] The acquisition module 140 is configured to acquire a binary image corresponding to the probe image, and identify all connected domain outlines in the binary image.

[0101] The screening module 142 is configured to screen all connected domain outlines based on a predetermined outline area, a preset range and a predetermined roundness, to obtain a target connected domain outline set, wherein the preset range is determined according to a predetermined radius.

[0102] The first determination module 144 is configured to determine the minimum enclosing circle corresponding to each target connected domain in the target connected domain outline set.

[0103] The second determination module 146 is configured to determine the target radius corresponding to the minimum enclosing circle, and compare the target radius with the predetermined radius.

[0104] The third determination module 148 is configured to determine whether to use the center of the minimum enclosing circle as the center of the probe or the centroid of the target connected domain as the center of the probe according to the size relationship between the target radius and the preset radius.

[0105] ​In the device, the area, radius and roundness conditions of the probe binary image are filtered, and the center of the minimum enclosing circle or the center of the mass is determined as the probe center after screening. By comparing the size relationship between the radius corresponding to the minimum enclosing circle of the target connected domain and the predetermined radius, the center of the minimum enclosing circle is determined as the probe center, or the center of the mass of the target connected domain is determined as the probe center, so that the technical effect of accurately positioning the probe center according to the characteristics of different probes is realized, and the technical problem of low accuracy of identifying the probe center caused by the fact that the positioning method of the related art is restricted by light, the center of the probe is calculated by the center of the mass, and there may be multiple similar contours in the field of view of the probe image is solved.

[0106] According to another aspect of the embodiments of the present application, a non-volatile storage medium is also provided, which includes a stored program, wherein the program controls the device in which the non-volatile storage medium is located to perform any one of the positioning methods for probe identification when the program is running.

[0107] Specifically, the storage medium is used to store program instructions of the following functions to realize the following functions:

[0108] Obtain the binary image corresponding to the probe image, and identify all connected domain contours in the binary image; screen all connected domain contours based on a predetermined contour area, a predetermined range and a predetermined roundness to obtain a target connected domain contour set; determine the minimum enclosing circle corresponding to each target connected domain in the target connected domain contour set; determine the target radius corresponding to the minimum enclosing circle, and compare the size of the target radius with a predetermined radius; and according to the size relationship between the target radius and the predetermined radius, determine to use the center of the minimum enclosing circle as the center of the probe, or use the center of the mass of the target connected domain as the center of the probe.

[0109] Alternatively, in the present embodiment, the storage medium can include but is not limited to electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or apparatus, or any suitable combination of the above. More specific examples of the storage medium include one or more electrical connections based on one or more wires, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above.

[0110] In an exemplary embodiment of the present application, a computer program product is also provided, which includes a computer program that implements any one of the positioning methods for probe identification described above when executed by a processor.

[0111] Alternatively, the computer program can implement the following steps when executed by a processor:

[0112] Obtaining a binary image corresponding to the probe image, and identifying all connected domain contours in the binary image; performing screening on all the connected domain contours based on a predetermined contour area, a preset range, and a predetermined roundness, to obtain a target connected domain contour set, wherein the preset range is determined according to a preset radius; determining a minimum enclosing circle corresponding to each target connected domain in the target connected domain contour set, determining a target radius corresponding to the minimum enclosing circle, and comparing the target radius with the predetermined radius; and determining, according to a size relationship between the target radius and the predetermined radius, whether to use a center of the minimum enclosing circle as a center of the probe or to use a centroid of the target connected domain as the center of the probe.

[0113] According to an embodiment of the present application, an electronic device is provided, which includes at least one processor, and a memory connected with the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the positioning method of any one of the probe identification.

[0114] Optionally, the electronic device can further include a transmission device and an input / output device, wherein the transmission device is connected with the processor, and the input / output device is connected with the processor.

[0115] Figure 15 is a schematic block diagram of an example electronic device 1500 according to an embodiment of the present application. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown here, their connections and relationships, and their functions, are meant to be examples only, and are not intended to limit the implementations of the present application described and / or claimed in this document.

[0116] As shown in Figure 15 , the device 1500 includes a computing unit 1501, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1502 or a computer program loaded from a storage unit 1508 into a random access memory (RAM) 1503. In the RAM 1503, various programs and data required for the operation of the device 1500 can also be stored. The computing unit 1501, the ROM 1502, and the RAM 1503 are connected to each other through a bus 1504. An input / output (I / O) interface 1505 is also connected to the bus 1504.

[0117] A number of components in the device 1500 are connected to the I / O interface 1505, including: an input unit 1506, such as a keyboard, a mouse, etc.; an output unit 1507, such as various types of displays, speakers, etc.; a storage unit 1508, such as a magnetic disk, a magneto-optical disk, etc.; and a communication unit 1509, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 1509 allows the device 1500 to exchange information / data with other devices over a computer network, such as the Internet, and / or various telecommunication networks.

[0118] The computing unit 1501 can be various general and / or special purpose processing components with processing and computing capabilities. Some examples of the computing unit 1501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 1501 performs various methods and processes described above, such as the probe identification positioning method. For example, in some embodiments, the probe identification positioning method can be implemented as a computer software program, which is tangibly embodied in a machine-readable medium, such as the storage unit 1508. In some embodiments, part or all of the computer program can be loaded and / or installed onto the device 1500 via the ROM 1502 and / or the communication unit 1509. When the computer program is loaded onto the RAM 1503 and executed by the computing unit 1501, one or more steps of the probe identification positioning method described above can be performed. Alternatively, in other embodiments, the computing unit 1501 can be configured to perform the probe identification positioning method by any other appropriate means, such as by means of firmware.

[0119] The various implementations of the systems and techniques described above can be realized in a digital electronic circuit system, an integrated circuit system, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), an application specific standard product (ASSP), a system on a chip system (SOC), a programmable logic device (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0120] Program code for carrying out methods of the present application can be written in any combination of one or more programming languages. This program code can be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the program code, when executed by the processor or controller, produces a means for implementing the functions / acts specified in the flowcharts and / or block diagrams. The program code can be retrieved from a machine-readable medium or device and executed by a processor to produce a machine for implementing the functions / acts specified in the flowcharts and / or block diagrams. The program code can be executed as a stand-alone program, or in combination with other program codes, on the machine to produce a machine that implements the functions / acts specified in the flowcharts and / or block diagrams.

[0121] In the context of the present application, a machine-readable medium can be a tangible medium that can contain or store program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable storage media can include, without limitation, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media can include, but are not limited to, an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0122] To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0123] The systems and techniques described herein can be implemented in a computing system that includes a back end component, e.g., as a data server, or that includes a middleware component, e.g., an application server, or that includes a front end component, e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described herein, or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication, e.g., a communication network. Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0124] The computer system can include clients and servers. The clients and the servers are generally remote from each other and typically interact through a communication network. The relationship of client and server can arise by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The servers can be cloud servers, servers of a distributed system, or servers combined with a blockchain.

[0125] The above-mentioned sequence numbers of the embodiments of the present application are only for description, and do not represent the advantages and disadvantages of the embodiments.

[0126] In the above-described embodiments of the present application, the description of each embodiment has its own focus, and the parts not described in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0127] In the several embodiments provided by the present application, it should be understood that the disclosed technology can be implemented in other ways. Of course, the device embodiment described above is only illustrative, and for example, the division of the units can be a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the shown or discussed units can be indirect coupling or communication connection through some interfaces, units or modules, which can be electrical or other forms.

[0128] The units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, i.e., they can be located in one place or distributed on multiple units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiments.

[0129] In addition, each function unit in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software function unit.

[0130] When the integrated unit is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application, essentially or in the form of a contribution to the prior art, or all or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in each embodiment of the present application. The foregoing storage medium includes various media that can store program codes, such as a U disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a mobile hard disk, a magnetic disk or an optical disk, etc.

[0131] The above is only the preferred embodiment of the present application, and it should be pointed out that, for those skilled in the art, without departing from the principles of the present application, a number of improvements and refinements can be made, and these improvements and refinements should be considered as the protection scope of the present application.

Claims

1. A positioning method for probe identification, characterized in that: include: Obtaining a binary image corresponding to the probe image, and identifying contours of all connected domains in the binary image; Screening all connected domain contours based on a predetermined contour area, a preset range, and a predetermined roundness to obtain a target connected domain contour set, wherein the preset range is determined according to a predetermined radius; Determine the minimum enclosing circle corresponding to each target connected domain of the target connected domain outline set; Determine a target radius corresponding to the minimum enclosing circle, and compare the target radius with the predetermined radius; According to the size relationship between the target radius and the predetermined radius, it is determined to select the center of the minimum enclosing circle as the center of the probe, or the centroid of the target connected domain as the center of the probe.

2. The method according to claim 1, characterized in that Determining, based on a relationship between the target radius and the predetermined radius, to select the center of the target connected domain as the center of the probe, or the centroid of the target connected domain as the center of the probe, includes: When the target radius is smaller than the predetermined radius, determining the center of the minimum enclosing circle to be the center of the corresponding probe; In the case that the target radius is greater than the predetermined radius, the centroid corresponding to the contours of each target connected domain is obtained, and the centroid corresponding to the contours of each target connected domain is determined as the center of each probe.

3. The method according to claim 1, characterized in that Determining the preset range according to the predetermined radius includes: Determine a sum of 1 and a radius upper limit deviation percentage, and determine the product of the predetermined radius and the sum as the upper limit of the preset range; Determine the difference between 1 and the radius lower limit deviation percentage, and determine the product of the predetermined radius and the difference as the lower limit value of the preset range.

4. The positioning method according to claim 1, wherein: All connected domain contours are screened based on a predetermined contour area, a preset range, and a predetermined circularity to obtain a target connected domain contour set, including: Traversing all the connected domain contours in sequence, and during the traversal process, determining whether the area of ​​the connected domain contour currently being detected is greater than the predetermined contour area; In a case where the area of ​​the connected domain contour currently being detected is greater than the predetermined contour area, determining a minimum enclosing circle corresponding to the connected domain contour currently being detected; If the target radius falls within a preset range, determining whether the circularity of the connected domain contour currently being detected is greater than a set circularity; In a case where the circularity of the connected domain contour currently being detected is greater than the set circularity, the connected domain contour currently being detected is determined to be an element in the target connected domain contour set.

5. The positioning method according to claim 4, characterized in that: Before determining whether the circularity of the connected component contour currently being detected is greater than a set circularity, the method further includes: Determine a first area corresponding to the connected domain outline currently being detected and a second area corresponding to the minimum enclosing circle; The ratio of the first area to the second area is determined as the roundness of the contour of the connected region currently being detected.

6. The method according to claim 1, characterized in that The method further comprises: Establishing a coordinate system with the center of the probe image as the origin, and determining the coordinates corresponding to the center of each probe; receiving an operation instruction input by a target object, wherein the operation instruction is used to indicate the coordinates of the center probe to be determined in the coordinate system; The center corresponding to the coordinates is output in response to the operation instruction.

7. The method according to claim 6, characterized in that Outputting the center corresponding to the coordinates in response to the operation instruction includes: In the case that there are multiple center probes to be determined indicated by the operation instruction, based on the distance between each center probe to be determined and the origin, wherein the smaller the distance, the smaller the degree of distortion of the probe; Determine the center probe to be determined whose distance from the origin is greater than a preset threshold as an ignorable probe, determine the center probe to be determined whose distance from the origin is less than the preset threshold as a probe to be processed, determine the target coordinates corresponding to the probe to be processed, and output the center corresponding to the target coordinates in response to the operation instruction.

8. The method according to claim 1, characterized in that Get the binary image corresponding to the probe image, including: grayscale processing is performed on the probe image to obtain a grayscale image corresponding to the probe; An adaptive threshold segmentation method is called to perform adaptive threshold segmentation on the grayscale image to obtain the binary image.

9. The method according to claim 8, characterized in that The binary image includes: a first binary image and a second binary image, and calling an adaptive threshold segmentation method to perform adaptive threshold segmentation on the grayscale image to obtain the binary image includes: Acquiring a type of a probe in the probe image, wherein the type of the probe includes: a first type and a second type, wherein the tip of the probe of the first type has a dark feature, and the tip of the probe of the second type has a bright feature; When the probe is of the first type, an adaptive threshold segmentation method is used to segment the grayscale image to obtain a dark needle body in the probe image, and a binarization process is performed on the segmented probe image to obtain the first binary image. When the probe is of the second type, an adaptive threshold segmentation method is called to segment the grayscale image to obtain the shiny needle body in the probe image, and the segmented probe image of the shiny needle body is binarized to obtain the second binary image.

10. A positioning device for probe identification, characterized in that: include: An acquisition module is used to acquire a binary image corresponding to the probe image and identify contours of all connected domains in the binary image; a screening module, configured to screen all connected domain contours based on a predetermined contour area, a preset range, and a predetermined roundness to obtain a target connected domain contour set, wherein the preset range is determined according to a predetermined radius; A first determining module is used to determine the minimum enclosing circle corresponding to each target connected domain in the target connected domain outline set; A second determining module is configured to determine a target radius corresponding to the minimum enclosing circle and compare the target radius with the predetermined radius; The third determining module is used to determine whether to select the center of the minimum enclosing circle as the center of the probe or the centroid of the target connected domain as the center of the probe according to the size relationship between the target radius and the predetermined radius.

11. A non-volatile storage medium, characterized in that: The storage medium includes a stored program, wherein when the program is run, the device where the storage medium is located is controlled to execute the probe identification positioning method according to any one of claims 1 to 9.

12. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the probe identification and positioning method according to any one of claims 1 to 9.

Images