Sub-pixel precision robust pin perpendicularity detection method and device and storage medium
By using subpixel network segmentation and global iterative point set registration algorithms, the problems of low accuracy and insufficient robustness in PIN pin alignment detection are solved, achieving efficient and low-cost subpixel-level PIN pin detection.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- 四川启睿克科技有限公司
- Filing Date
- 2023-11-21
- Publication Date
- 2026-07-21
Smart Images
Figure CN117557531B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of industrial visual inspection technology, and in particular to a robust PIN alignment detection method, apparatus and storage medium with sub-pixel accuracy. Background Technology
[0002] PIN alignment inspection, which checks the discrepancy between the PIN pin position and the standard template, is a crucial and common step in connector manufacturing and the final stage of product quality control. Current alignment inspection methods primarily rely on manual inspection using grid boards, which suffers from high costs, low accuracy, and slow speed. Therefore, there is an urgent need to implement vision-based alignment inspection to improve accuracy and efficiency, thereby enhancing companies' ability to control the quality of their output products.
[0003] Existing pin positioning algorithms are mostly based on traditional binarization, edge detection, and template matching, which lack robustness and perform poorly in situations such as non-luminous pin tips, irregular pin tips, and interference from the pin's side. In actual production processes, the repeatability accuracy of these algorithms is insufficient, and traditional sub-pixel edge extraction algorithms have poor anti-interference and complex extraction capabilities. Therefore, researching robust sub-pixel pin positioning algorithms is a problem that needs to be solved.
[0004] If a coordinate system is established using the center coordinates of the needle, a single matching may not be robust. Existing point set matching algorithms mostly use least squares estimation to calculate the optimal transformation matrix. The transformation matrix contains rotation parameters and scaling factors, which are not suitable for the orthogonality detection scenario. Therefore, it is necessary to study a fast point set registration algorithm with only translation to meet the orthogonality detection requirements. Summary of the Invention
[0005] This invention provides a robust PIN pin alignment detection method, apparatus, and storage medium with subpixel accuracy to solve the technical problems in the prior art.
[0006] The technical solution adopted by this invention is as follows: Firstly, this invention provides a robust PIN alignment detection method with sub-pixel precision, comprising:
[0007] Acquire images of the PIN pins of the product to be tested, and the outline information of the pin body / tip can be presented in the image;
[0008] Locate the pin area to obtain the region of interest for each pin;
[0009] Subpixel segmentation of the PIN pin is performed in the region of interest of each PIN pin to obtain subpixel-level precision pin body / tip segmentation results.
[0010] Based on the pin alignment segmentation results, obtain the center coordinates of the pin body / pin tip;
[0011] If the number of needle body / needle tip centers obtained is not equal to the number of template centers, it is determined that there are missing needles or crooked needles, and the product is then determined to be a defective product; if the number of needle body / needle tip centers obtained is equal to the number of template centers, coordinate system O is established based on the obtained needle body / needle tip center coordinates.
[0012] Perform point set registration between coordinate system O and template center coordinate system, and determine whether the alignment meets the detection requirements based on the point set registration results.
[0013] Furthermore, the segmentation network used for sub-pixel segmentation of the PIN pins is: a sub-pixel feature extraction module, a feature fusion module, and a downsampling learnable module are added to the segmentation network.
[0014] Furthermore, the feature extraction in the sub-pixel feature extraction module uses a backbone network to extract pixel-level features from the region of interest of each PIN pin; a super-resolution network is used to upsample the pixel-level features to obtain sub-pixel features.
[0015] Furthermore, the sub-pixel features obtained from the sub-pixel feature extraction module are combined with the features from the segmentation network encoding stage using multi-scale feature concatenation and fusion to obtain the fused features.
[0016] Furthermore, the fused features are downsampled using a downsampling learnable module to obtain a segmented image of the same size as the input; the fused features are then input into the decoding stage of the segmentation network to obtain an input size n. 2 The image is segmented by a factor of 1, and the segmentation accuracy is 1 / n. 2 There are 1 pixel, where n is the upsampling factor, and n>1 is an integer.
[0017] Furthermore, it also includes: taking the input size n 2 The segmented image, which is 10 times larger than the input, is downsampled using a downsampling learnable module to obtain a segmented image of the same size as the input.
[0018] Furthermore, the method for registering the coordinate system O with the template center coordinate system and determining whether the alignment meets the detection requirements based on the registration result includes:
[0019] In coordinate system O, align the origin of the needle body / needle tip center with the origin of the template center, and calculate the offset (dx0, dy0) and distance d0 of the remaining needles relative to the template center.
[0020] Determine whether the offsets (dx0, dy0) and distances d0 between all needles and the template center meet the alignment test standard. If they do, the product is considered a qualified product; otherwise, calculate the mean of the absolute values of the offsets between the center of all needles / needle tips and the template center, excluding the origin.
[0021] With the template center as the center, the search range in the x-direction is The search range in the y-direction is Point set registration is performed on a rectangular region with a search step size of s, and the total number of searches is . Where k1 and k2 are positive numbers;
[0022] In each loop, the center coordinates of the PIN pins are translated by step size and direction, and the offset (dx) of all pins relative to the template center is recalculated. i dy i and distance d i If the product meets the alignment test standard, it is considered a qualified product; otherwise, the cycle continues.
[0023] If the loop ends and none of the results meet the orthogonality detection standard, then select the (dx) value from the loop. i dy i The smallest match is displayed as the optimal match result.
[0024] Secondly, the present invention provides a robust PIN alignment detection device with sub-pixel accuracy, comprising:
[0025] The PIN image acquisition module is used to acquire images of the PINs of the product to be tested, and can present the outline information of the needle body / tip in the image;
[0026] The PIN pin region positioning module is used to locate the PIN pin region and obtain the region of interest for each PIN pin.
[0027] The PIN subpixel segmentation module is used to perform subpixel segmentation of the PIN in the region of interest of each PIN, so as to obtain the pin body / tip segmentation result with subpixel precision.
[0028] The PIN needle center coordinate positioning module is used to obtain the center coordinates of the needle body / needle tip based on the PIN needle alignment segmentation results.
[0029] The PIN pin count determination module is used to determine whether the number of pin body / tip center points obtained is equal to the number of template center points. If they are not equal, it is determined that there are missing pins or crooked pins, and the product is then determined to be a defective product. If the number of pin body / tip center points obtained is equal to the number of template center points, the PIN pin center coordinate system transformation module establishes coordinate system O based on the obtained pin body / tip center coordinates.
[0030] The point set registration module is used to register the coordinate system O with the template center coordinate system, and to determine whether the alignment meets the detection requirements based on the point set registration results.
[0031] Thirdly, the present invention provides an electronic device, the electronic device comprising:
[0032] One or more processors;
[0033] Storage device for storing one or more programs;
[0034] When the one or more programs are executed by the one or more processors, the one or more processors implement the robust PIN alignment detection method with subpixel precision as described in the first aspect.
[0035] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the robust PIN alignment detection method with sub-pixel precision as described in the first aspect.
[0036] The beneficial effects of this invention are: This invention, through a "sub-pixel network segmentation" strategy, by adding a sub-pixel feature extraction module, a feature fusion module, and a downsampling learnable module to the existing segmentation network, achieves 1 / n... 2 The PIN needle segmentation results with an accuracy of (n>1, an integer) pixels reduce the hardware cost of high-precision imaging schemes. A learnable approach improves the segmentation accuracy of PIN needles and further enhances the algorithm's repeatability. The deep learning algorithm can adapt to situations such as non-luminous needle tips, irregular needle tips, and interference from the needle's side. Through a "global iterative point set registration" strategy, using the initial matching offset as the point set search range, iterative point set registration of the PIN needle center coordinates and the template center coordinates can be performed. Compared to existing orthogonality detection methods, sub-pixel segmentation ensures high segmentation accuracy and reduces the hardware cost of high-precision imaging schemes. The point set registration method used is scale-free and rotation-free, highly efficient, controllable, and robust, meeting the orthogonality detection requirements. Attached Figure Description
[0037] Figure 1 This is a flowchart illustrating the robust PIN alignment detection method with sub-pixel precision disclosed in this invention.
[0038] Figure 2 This is a schematic diagram of PIN pin subpixel segmentation disclosed in this invention;
[0039] Figure 3 This is a schematic diagram of PIN pin dot set registration disclosed in this invention;
[0040] Figure 4 This is a structural block diagram of the robust PIN alignment detection device with sub-pixel precision disclosed in this invention.
[0041] Figure 5 This is a schematic diagram of the structure of the electronic device disclosed in this invention. Detailed Implementation
[0042] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.
[0043] It should be understood that, when used in this specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.
[0044] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of the application. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.
[0045] As used in this specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrases "if determined" or "if [described condition or event] is detected" may be interpreted, depending on the context, as "once determined," "in response to determination," "once [described condition or event] is detected," or "in response to detection of [described condition or event]."
[0046] Example 1:
[0047] See Figure 1 This embodiment discloses a robust PIN alignment detection method with sub-pixel accuracy, comprising the following steps:
[0048] S1: Obtain an image of the PIN needle of the product to be tested, which can display the outline information of the needle body / tip in the image.
[0049] Specifically, the PIN pin image acquisition described in step S1 can be achieved using methods including, but not limited to, 2D and 3D cameras. In this embodiment, it is specifically implemented as follows: a color industrial camera + telecentric lens + ring light is used. The camera resolution and lens are selected according to the workpiece size and precision. For example, if the workpiece size is approximately 20mm, a Basler 20-megapixel color camera is selected, with a field of view of 25.7mm x 17.1mm and a single pixel precision of 0.0047mm. During sampling, the image is taken from a top-down view. The "pin tip" refers to the bright part of the PIN pin tip; the "pin body" refers to the entire tip of the PIN pin, not the pin shank.
[0050] S2: Locate the PIN pin area to obtain the region of interest for each PIN pin.
[0051] Specifically, the methods used in step S2 include, but are not limited to, prior information, template matching, and object detection.
[0052] In this embodiment, the specific implementation is as follows: The Unet segmentation model is used to segment the ROI (Region of Interest) of the workpiece area to obtain the outer contour of the workpiece and the minimum bounding rectangle. The rotation angle of the workpiece is obtained based on the minimum bounding rectangle, and the workpiece is turned upright. The coordinates of the template center are transformed to the image coordinate system. The offset coordinates between the first needle in the upper left corner and the upper left corner of the workpiece are calculated. The coordinates of all needles in the image coordinate system are added to the offset coordinates to obtain the prior coordinates of the needle center. Using the prior coordinates as the center, the ROI that can contain the needle is extracted, such as 128x128.
[0053] S3: Perform subpixel segmentation on the region of interest of each PIN to obtain subpixel-level precision needle body / tip segmentation results.
[0054] For details, see Figure 2 The segmentation network used for sub-pixel segmentation of the PIN pins in step S3 is an existing segmentation network with added sub-pixel feature extraction, feature fusion, and downsampling learnable modules. Ultimately, two outputs are obtained: a segmented image of the same size as the input and an input image of size n. 2 A segmented image with a magnification factor of 1x (where n is the magnification factor, n>1 is an integer) and a segmentation precision of 1 / n. 2 Sub-pixels.
[0055] Specifically, in the subpixel feature extraction module of PIN pin subpixel segmentation, common backbone networks such as ResNet can be used to extract pixel-level features from the region of interest of each PIN pin. The depth-to-space module in the super-resolution network is used to upsample the pixel-level features to obtain N*M*nW*nH (* is the multiplication sign, W and H are the width and height of the input image, N is the batch, M is the number of feature channels, n is the upsampling factor, n>1, and is an integer) subpixel features.
[0056] Specifically, the feature fusion module in PIN needle subpixel segmentation performs multi-scale concat feature fusion (i.e. feature splicing fusion) with the subpixel features extracted from the subpixel feature extraction module and the features from the segmentation network encoding stage to obtain fused features; multiple fusion resolutions can be selected, such as features with the same resolution as the input, features with a resolution of 1 / 2 the input size, etc.
[0057] Specifically, the fused features are input into the decoding stage of the segmentation network to obtain an input size n. 2 The image is segmented by a factor of 1, and the segmentation accuracy is 1 / n. 2 Let n be the upsampling factor, where n > 1 and is an integer. The input size n... 2 The segmented image and fused features are downsampled by the space-to-depth module in the downsampling learnable module of the PIN needle subpixel segmentation, and the N*M*nW*nH output and features are mapped back to the size WxH to obtain a segmented image of the same size as the input.
[0058] It should be noted that the decoded input size n 2 An upsampled image is a segmented image that has been processed by a downsampling learnable module, while a downsampled image is a segmented image. Both are subpixel-level precision images, and in practical applications, one of them should be selected based on actual needs. Furthermore, the decoded segmented image can be input into the downsampling learnable module for processing to obtain a downsampled image, which is superior to the segmented image that has only been decoded, specifically in terms of fewer jagged edges and better segmentation results.
[0059] like Figure 2 As shown, in this embodiment, the specific implementation is as follows: The existing segmentation network adopts the classic encoder-decoder network Unet. Pixel-level feature extraction in the sub-pixel feature extraction module can use the common ResNet18, and the extracted features are 32*H*W. Sub-pixel feature extraction uses the depth-to-space module in the super-resolution network to magnify the features by a factor of 2, rearranging the features from the channel dimension to the width and height dimensions, resulting in a final feature dimension of 8*2H*2W. The features in the sub-pixel feature extraction module are concat-fused with the features of the Unet encoder network. The fused feature size is selected from multiple scales (H*W, 1 / 2H*W, 1 / 4H*W, 1 / 8H*W), and the fused feature is input into the Unet decoder network. The downsampling learnable module uses the space-to-depth module to downsample the 2H*2W feature map and the 2H*2W segmentation image to H*W size. Finally, two output segmentation images are obtained, one of size H*W and the other of size 2H*2W, with a segmentation accuracy of 1 / 4 pixel.
[0060] S4: Obtain the center coordinates of the needle body / needle tip based on the PIN needle alignment segmentation results.
[0061] Specifically, the PIN pin center coordinate positioning described in step S4 employs methods including, but not limited to, the centroid calculated from the second moment of the profile, the center of the smallest circumscribed rectangle, and the center of the smallest circumscribed circle. The pin body / tip center should be interpreted as either the pin body center or the pin tip center. The pin tip center is the center of the illuminated area at the tip of the PIN pin, while the pin body center refers to the center of the entire PIN pin tip.
[0062] In this embodiment, the specific implementation is as follows: the output segmented image, which is the same size as the input after downsampling, is binarized with a threshold of 0.5, and the minimum bounding rectangle of the outer contour is obtained. The center of the minimum bounding rectangle is used as the coordinate of the needle center.
[0063] S5: If the number of needle body / needle tip centers obtained is not equal to the number of template centers, it is determined that there are missing needles or crooked needles, and the product is then determined to be a defective product; if the number of needle body / needle tip centers obtained is equal to the number of template centers, coordinate system O is established based on the obtained needle body / needle tip center coordinates.
[0064] Specifically, if the number of needle centers obtained is not equal to the number of template centers, it is determined that there are missing needles, crooked needles, etc., and the product can be judged as unqualified without point set registration in S6.
[0065] Specifically, in step S5, coordinate system O is established by selecting two needles according to the detection requirements to establish the x-axis of the coordinate system, with the y-axis perpendicular to the x-axis, and the center coordinates of the PIN needles are transformed to the established coordinate system according to geometric relationships.
[0066] In this embodiment, the specific implementation is as follows: A line is drawn connecting the first needle in the lower left corner and the last needle in the lower right corner to obtain the angle θ in the current coordinate system. The rotation center is (centerX, centerY), the original needle center coordinates are (x, y), and the coordinates in the new coordinate system are (dstX, dstY).
[0067] srcX=x-centerX, srcY=y-centerY
[0068] dstX=srcX*cos(θ)+srcY*sin(θ)+centerX
[0069] dstY=-srcX*sin(θ)+srcY*cos(θ)+centerY
[0070] S6: Perform point set registration between coordinate system O and template center coordinate system, and determine whether the alignment meets the detection requirements based on the point set registration results.
[0071] For details, see Figure 3S6 includes: S61: Under coordinate system O, the origin of the needle body / needle tip center coincides with the origin of the template center, and the offset (dx0, dy0) and distance d0 of the remaining needles relative to the template center are calculated;
[0072] S62: Determine whether the offset (dx0, dy0) and distance d0 of all needles from the center of the template meet the alignment test standard. If they meet the standard, the product is considered a qualified product; otherwise, proceed to S63.
[0073] S63: Calculate the mean of the absolute values of the offsets between the center of all needle bodies / needle tips and the center of the template, excluding the origin.
[0074] S64: Consider The directionality, centered on the template center, extends the search range in the x-direction. The search range in the y-direction is Point set registration is performed on a rectangular region with a search step size of s, and the total number of searches is . Where k1 and k2 are positive numbers, and s is the step size;
[0075] S65: In each loop, the center coordinates of the PIN pins are translated by step size and direction, and the offset (dx) of all pins relative to the template center is recalculated. i dy i and distance d i If the product meets the alignment test standard, it is considered a qualified product; otherwise, the cycle continues.
[0076] S66: If the loop ends and none of the results meet the alignment test criteria, then select the result from the loop (dx). i dy i The smallest match is displayed as the optimal match result.
[0077] In this embodiment, the specific implementation is as follows: There are 100 needles. The first needle in the lower left corner is taken as the origin, coinciding with the origin of the template center. The offset of the remaining 99 needles from the template center is calculated. If any needle is offset by 0.3mm, which is greater than the standard of 0.2mm, proceed to step S63. The average offset of the 99 needles from the template center is calculated and converted to pixels (8, 10), with a step size of 1. Following the above process, the total number of searches is 2x8x2x9 / 1 = 288. In each loop, the template center coordinates are translated, and the offset of all needles from the template center is calculated. For example, in the 101st loop, if the offset of all needles from the template center is less than 0.2mm, the alignment detection returns OK.
[0078] This embodiment provides a robust PIN pin alignment detection method with sub-pixel precision. Its advantages are: by employing a "sub-pixel network segmentation" strategy, and by adding a sub-pixel feature extraction module, a feature fusion module, and a downsampling learnable module to the existing segmentation network, a 1 / n accuracy is obtained. 2 The PIN needle segmentation result with an accuracy of (n>1, an integer) pixels reduces the hardware cost of high-precision imaging schemes. It improves the segmentation accuracy of PIN needles by adopting a learnable approach and can further improve the repeatability accuracy of the algorithm. The deep learning algorithm can adapt to situations such as non-luminous needle tips, irregular needle tips, and interference from the needle sides. Through the "global iterative point set registration" strategy, the offset at the initial matching is used as the point set search range. Iterative point set registration of the PIN needle center coordinates and the template center coordinates can be performed. Compared with existing point set registration methods, there is no scaling or rotation, which is efficient, controllable, and robust, meeting the requirements of orthogonality detection.
[0079] Example 2:
[0080] See Figure 4 This embodiment discloses a robust PIN alignment detection device with sub-pixel accuracy, comprising:
[0081] The PIN image acquisition module 410 is used to acquire images of the PIN pins of the product to be tested, and the outline information of the pin body / tip can be presented in the image;
[0082] The PIN pin region positioning module 420 is used to locate the PIN pin region and obtain the region of interest for each PIN pin.
[0083] The PIN pin subpixel segmentation module 430 is used to perform subpixel segmentation of the PIN pin in the region of interest of each PIN pin to obtain the pin body / tip segmentation result with subpixel precision.
[0084] The PIN needle center coordinate positioning module 440 is used to obtain the center coordinates of the needle body / needle tip based on the PIN needle alignment segmentation results.
[0085] The PIN pin count determination module 450 is used to determine whether the number of pin body / tip center points obtained is equal to the number of template center points. If they are not equal, it is determined that there are missing pins or crooked pins, and the product is then determined to be a defective product. If the number of pin body / tip center points obtained is equal to the number of template center points, the PIN pin center coordinate system transformation module 460 establishes a coordinate system O based on the obtained pin body / tip center coordinates.
[0086] The point set registration module 470 is used to register the coordinate system O with the template center coordinate system, and to determine whether the alignment meets the detection requirements based on the point set registration result.
[0087] The above-described device can execute the subpixel precision robust PIN pin alignment detection method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of executing the subpixel precision robust PIN pin alignment detection method.
[0088] Example 3:
[0089] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Figure 5 A block diagram is shown of an exemplary electronic device 50 suitable for implementing embodiments of the present invention. Figure 5 The electronic device 50 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of the present invention.
[0090] like Figure 5 As shown, the electronic device 50 is represented in the form of a general-purpose computing device. The components of the electronic device 50 may include, but are not limited to: one or more processors or processing units 501, system memory 502, and bus 503 connecting different system components (including system memory 502 and processing unit 501).
[0091] Bus 503 represents one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. Examples of these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.
[0092] Electronic device 50 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by electronic device 50, including volatile and non-volatile media, removable and non-removable media.
[0093] System memory 502 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 504 and / or cache memory 505. Electronic device 50 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 506 may be used to read and write non-removable, non-volatile magnetic media (… Figure 5 Not shown; usually referred to as a "hard drive"). Although Figure 5Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk") and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 503 via one or more data media interfaces. Memory 502 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of the present invention.
[0094] A program / utility 508 having a set (at least one) of program modules 507 may be stored, for example, in memory 502. Such program modules 507 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 507 typically perform the functions and / or methods described in the embodiments of the present invention.
[0095] Electronic device 50 can also communicate with one or more external devices 509 (e.g., keyboard, pointing device, display 510, etc.), and with one or more devices that enable a user to interact with the electronic device 50, and / or with any device that enables the electronic device 50 to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication can be performed via input / output (I / O) interface 511. Furthermore, electronic device 50 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 512. As shown, network adapter 512 communicates with other modules of electronic device 50 via bus 503. It should be understood that, although... Figure 5 As not shown, other hardware and / or software modules may be used in conjunction with electronic device 50, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.
[0096] The processing unit 501 executes various functional applications and data processing by running programs stored in the system memory 502, such as implementing the robust PIN pin alignment detection method with sub-pixel accuracy provided in the embodiments of the present invention.
[0097] Example 4:
[0098] This embodiment provides a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform a robust PIN alignment detection method with subpixel precision.
[0099] The computer storage medium of this invention can be any combination of one or more computer-readable media. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.
[0100] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.
[0101] Program code contained on a computer-readable medium may be transmitted using any suitable medium, including—but not limited to—wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.
[0102] Computer program code for performing the operations of embodiments of the present invention can be written in one or more programming languages or a combination thereof, including object-oriented programming languages such as Java, Smalltalk, and C++, and conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0103] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A robust PIN alignment detection method with sub-pixel precision, characterized in that, include: Acquire images of the PIN pins of the product to be tested, and the outline information of the pin body / tip can be presented in the image; Locate the pin area to obtain the region of interest for each pin; Subpixel segmentation of the PIN pin is performed in the region of interest of each PIN pin to obtain subpixel-level precision pin body / tip segmentation results. Based on the pin alignment segmentation results, obtain the center coordinates of the pin body / pin tip; If the number of needle body / needle tip centers obtained is not equal to the number of template centers, it is determined that there are missing needles or crooked needles, and the product is then determined to be a defective product; if the number of needle body / needle tip centers obtained is equal to the number of template centers, coordinate system O is established based on the obtained needle body / needle tip center coordinates. Perform point set registration on the coordinate system O and the template center coordinate system, and determine whether the orthotopicity meets the detection requirements according to the point set registration results; the specific implementation steps are as follows: Coincide the origin of the needle body / tip center with the origin of the template center under the coordinate system O, and calculate the offsets (dx0, dy0) and distance d0 of the remaining needles from the template center; determine whether the offsets (dx0, dy0) and distance d0 of all needles from the template center meet the orthotopicity detection standard. If they meet, determine that the product is a qualified product; if not, calculate the average value of the absolute values of the offsets of all needle body / tip centers from the template center except the origin. , ); Centered on the template center, perform point set registration in a rectangular area with a search range of (- ) in the x direction and (- ) in the y direction. The search step size is s, and the total number of searches is (2 * ) / s, where is a positive number; in each loop, translate the center coordinates of the PIN needles according to the step size and direction, and recalculate the offsets (dx i , dy i ) and distance d i of all needles from the template center. If the orthotopicity detection standard is met, determine it as a qualified product; otherwise, continue the loop; if all do not meet the orthotopicity detection standard after the loop ends, select the one with the smallest (dx i , dy i ) in the loop as the optimal matching result for display.
2. The robust PIN alignment detection method with sub-pixel precision according to claim 1, characterized in that, The segmentation network used for subpixel segmentation of the PIN pins is as follows: a subpixel feature extraction module, a feature fusion module, and a downsampling learnable module are added to the segmentation network.
3. The robust PIN alignment detection method with sub-pixel precision according to claim 2, characterized in that, The sub-pixel feature extraction module uses a backbone network to extract pixel-level features from the region of interest of each PIN pin; a super-resolution network is then used to upsample the pixel-level features to obtain sub-pixel features.
4. The robust PIN alignment detection method with sub-pixel precision according to claim 3, characterized in that, The sub-pixel features obtained by the sub-pixel feature extraction module are combined with the features from the encoding stage of the segmentation network through multi-scale feature concatenation and fusion to obtain the fused features.
5. The robust PIN alignment detection method with sub-pixel precision according to claim 4, characterized in that, The fused features are downsampled using a downsampling learnable module to obtain a segmented image of the same size as the input; the fused features are then input into the decoding stage of the segmentation network to obtain an input size n. 2 The image is segmented by a factor of 1, with a segmentation accuracy of 1 / n. 2 There are 1 pixel, where n is the upsampling factor, and n>1 is an integer.
6. The robust PIN alignment detection method with sub-pixel precision according to claim 5, characterized in that, Also includes: Input size n 2 The segmented image, which is 10 times larger than the input, is downsampled using a downsampling learnable module to obtain a segmented image of the same size as the input.
7. A robust PIN alignment detection device with sub-pixel precision, characterized in that, include: The PIN image acquisition module is used to acquire images of the PINs of the product to be tested, and can present the outline information of the needle body / tip in the image; The PIN pin region positioning module is used to locate the PIN pin region and obtain the region of interest for each PIN pin. The PIN subpixel segmentation module is used to perform subpixel segmentation of the PIN in the region of interest of each PIN, so as to obtain the pin body / tip segmentation result with subpixel precision. The PIN needle center coordinate positioning module is used to obtain the center coordinates of the needle body / needle tip based on the PIN needle alignment segmentation results. The PIN pin count determination module is used to determine whether the number of pin body / tip center points obtained is equal to the number of template center points. If they are not equal, it is determined that there are missing pins or crooked pins, and the product is then determined to be a defective product. If the number of pin body / tip center points obtained is equal to the number of template center points, the PIN pin center coordinate system transformation module establishes coordinate system O based on the obtained pin body / tip center coordinates. The point set registration module is used to register the point set between coordinate system O and the template center coordinate system, and to determine whether the alignment meets the detection requirements based on the point set registration result. The specific implementation steps of the point set registration module are as follows: Under coordinate system O, align the origin of the needle body / needle tip center with the origin of the template center, and calculate the offset (dx0, dy0) and distance d0 of the remaining needles relative to the template center; determine whether the offset (dx0, dy0) and distance d0 of all needles relative to the template center meet the alignment test standard. If they do, the product is considered a qualified product; if not, calculate the mean of the absolute values of the offsets of all needle body / needle tip centers relative to the template center, excluding the origin. , With the template center as the center, the search range in the x-direction is (- The search range in the y direction is (- The point set is registered within a rectangular region of ( ), with a search step size of s, and the total number of searches is (2 ) * ) / s, where It is a positive number; in each loop, the center coordinates of the PIN pins are translated by step size and direction, and the offset of all pins from the center of the template is recalculated (dx). i dy i and distance d i If the product meets the alignment test standard, it is considered a qualified product; otherwise, the cycle continues. If, at the end of the cycle, none of the products meet the alignment test standard, then the product selected from the cycle (dx) is considered qualified. i dy i The smallest match is displayed as the optimal match result.
8. An electronic device, characterized in that, The electronic device includes: One or more processors; Storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the robust PIN alignment detection method with sub-pixel precision as described in any one of claims 1-6.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by the processor, the program implements a robust PIN alignment detection method with subpixel precision as described in any one of claims 1-6.