Pin defect detection method, device, equipment and storage medium

Abstract

The application discloses a PIN needle defect detection method, device and equipment and a storage medium, relates to the technical field of defect detection, and the PIN needle defect detection method comprises the following steps: acquiring a single-frame image of a PIN needle to be tested; inputting the single-frame image into a preset image segmentation model to obtain an instance segmentation mask of each PIN needle to be tested; extracting geometric feature parameters of each PIN needle to be tested according to the instance segmentation mask; and performing abnormality discrimination on the geometric feature parameters to obtain a defect discrimination result of the PIN needle to be tested. The application can reduce the cost of PIN needle defect detection.

Description

Technical Field

[0001] This application relates to the field of defect detection technology, and in particular to a method, apparatus, device, and storage medium for detecting defects in PIN pins. Background Technology

[0002] In connector manufacturing, pins are critical conductive components, and defects such as misalignment, missing pins, and breaks directly impact product quality. Currently, defect detection is primarily performed using 3D scanning cameras or multi-camera scanning methods, which are costly in terms of hardware. Therefore, reducing the cost of pin defect detection remains a problem that needs to be addressed.

[0003] The above content is only used to help understand the technical solution of this application and does not represent an admission that the above content is prior art. Summary of the Invention

[0004] The main objective of this application is to provide a method, apparatus, device, and storage medium for detecting defects in PIN pins, aiming to solve the technical problem of how to reduce the cost of PIN pin defect detection.

[0005] To achieve the above objectives, this application proposes a method for detecting defects in PIN pins, the method comprising: Acquire a single-frame image of the PIN pin to be tested; The single-frame image is input into a preset image segmentation model to obtain the instance segmentation mask for each of the PIN pins to be tested; Geometric feature parameters of each of the PIN pins to be tested are extracted based on the instance segmentation mask. Anomaly detection is performed on the geometric feature parameters to obtain the defect detection result of the PIN pin under test.

[0006] In one embodiment, the preset image segmentation model includes a U-Net network model, and before the step of acquiring a single-frame image of the PIN pin to be tested, the method further includes: Acquire images of normal PIN pins and defective PIN pins; The normal PIN pin image is synthesized to generate a synthetic defect image; The normal PIN image, the defective PIN image, and the synthetic defective image are used as training data to train the initial U-Net network model, resulting in the trained U-Net network model.

[0007] In one embodiment, the step of synthesizing the normal PIN pin image to generate a synthesized defective image includes: Obtain the PIN pin region from the normal PIN pin image; The PIN pin area is subjected to pixel displacement or skeleton bending processing to generate a synthetic defect image with a curved shape.

[0008] In one embodiment, the step of extracting the geometric feature parameters of each of the PIN pins to be tested based on the instance segmentation mask includes: Contour extraction is performed on the instance segmentation mask to obtain contour geometric features; Based on the relative direction between the camera optical axis and the PIN pin to be tested, the center point or centerline skeleton of each PIN pin is extracted from the contour geometric features, and the center point or the centerline skeleton is determined as the geometric feature parameter.

[0009] In one embodiment, the step of performing anomaly detection on the geometric feature parameters to obtain the defect detection result of the PIN pin under test includes: When the geometric feature parameter is the center point of each of the PIN pins to be tested, anomaly discrimination is performed based on the center point to obtain the defect discrimination result of the PIN pin to be tested; When the geometric feature parameters include the centerline skeleton of each of the PIN pins to be tested, anomaly detection is performed based on the centerline skeleton to obtain the defect detection result of the PIN pin to be tested.

[0010] In one embodiment, the step of performing anomaly detection based on the center point to obtain the defect detection result of the PIN pin under test includes: The center points are sorted according to a preset direction, and the number of the sorted center points is counted to obtain the quantity results. Calculate the contour area of ​​the needle tip region corresponding to the center point; Calculate the aspect ratio of the minimum bounding rectangle of the needle tip region corresponding to the center point; Calculate the distance and direction of the line connecting adjacent center points; Based on the quantity results, the outline area, the aspect ratio of the minimum circumscribed rectangle, the spacing, and the direction of the connecting lines, it is determined whether the PIN pin under test has a defect, and the defect discrimination result of the PIN pin under test is obtained.

[0011] In one embodiment, the step of performing anomaly detection based on the centerline skeleton to obtain the defect detection result of the PIN pin under test includes: Calculate the length, straightness, and principal direction vector of the centerline skeleton respectively; Based on the length, straightness, and principal direction vector, it is determined whether the PIN under test has a defect, and the defect identification result of the PIN under test is obtained.

[0012] Furthermore, to achieve the above objectives, this application also proposes a defect detection device for PIN pins, the defect detection device for PIN pins comprising: The acquisition module is used to acquire a single-frame image of the PIN pin under test; The segmentation module is used to input the single-frame image into a preset image segmentation model to obtain the instance segmentation mask for each of the PIN pins to be tested; The extraction module is used to extract the geometric feature parameters of each of the PIN pins to be tested based on the instance segmentation mask; The discrimination module is used to discriminate anomalies in the geometric feature parameters and obtain the defect discrimination result of the PIN pin under test.

[0013] In addition, to achieve the above objectives, this application also proposes a PIN pin defect detection device, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the PIN pin defect detection method described above.

[0014] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the PIN pin defect detection method described above.

[0015] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the PIN pin defect detection method described above.

[0016] This application provides a method for detecting defects in PIN pins. The method involves acquiring a single-frame image of the PIN pin to be tested; inputting the single-frame image into a preset image segmentation model to obtain instance segmentation masks for each PIN pin; extracting geometric feature parameters of each PIN pin based on the instance segmentation masks; and performing anomaly detection on the geometric feature parameters to obtain a defect detection result for the PIN pin. This method, by acquiring a single-frame image of the PIN pin to be tested and using an image segmentation model to extract geometric feature parameters for anomaly detection, eliminates the need for multiple cameras or 3D scanning equipment during the defect detection process. Only a single camera is required to complete the defect identification, thus reducing the cost of PIN pin defect detection. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 A flowchart illustrating an embodiment of the defect detection method for PIN pins in this application; Figure 2 A schematic diagram of the acquisition device provided in Embodiment 1 of the defect detection method for PIN pins of this application; Figure 3 A schematic diagram of a single frame image of the PIN pin under test provided in Embodiment 1 of the PIN pin defect detection method of this application; Figure 4 A schematic diagram of a preprocessed single-frame image provided for Embodiment 1 of the PIN pin defect detection method of this application; Figure 5 A schematic diagram of a PIN mask provided for Embodiment 1 of the PIN pin defect detection method of this application; Figure 6 A schematic diagram of the PIN pin outline provided in Embodiment 1 of the PIN pin defect detection method of this application; Figure 7 A flowchart illustrating Embodiment 2 of the PIN pin defect detection method of this application; Figure 8 This is a schematic diagram of the center point connection of vertically arranged PIN pins provided in Embodiment 2 of the PIN pin defect detection method of this application; Figure 9 A schematic diagram of the center point connection of horizontally arranged PIN pins provided in Embodiment 2 of the PIN pin defect detection method of this application; Figure 10 This is a schematic diagram of the module structure of the PIN pin defect detection device according to an embodiment of this application; Figure 11 This is a schematic diagram of the hardware operating environment involved in the defect detection method for PIN pins in this application embodiment.

[0020] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0021] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0022] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0023] This application acquires a single-frame image of the PIN pin to be tested; inputs the single-frame image into a preset image segmentation model to obtain an instance segmentation mask for each of the PIN pins to be tested; extracts the geometric feature parameters of each of the PIN pins to be tested based on the instance segmentation mask; and performs anomaly detection on the geometric feature parameters to obtain the defect detection result of the PIN pin to be tested.

[0024] In connector manufacturing, pins are critical conductive components, and defects such as misalignment, missing pins, and breaks directly impact product quality. Currently, defect detection is primarily performed using 3D scanning cameras or multi-camera scanning methods, which are costly in terms of hardware. Therefore, reducing the cost of pin defect detection remains a problem that needs to be addressed.

[0025] This application acquires a single-frame image of the PIN pin to be tested and uses an image segmentation model to extract geometric feature parameters for anomaly detection. During the detection process, multiple cameras or 3D scanning equipment are not required; only a single camera is needed to complete the defect identification, thus reducing the cost of PIN pin defect detection.

[0026] All user-related data involved in this application were obtained with the user's permission or consent; that is, when this application is applied to specific products or technologies, user permission is required to obtain and process the relevant data, and the processing of the relevant data must comply with the relevant laws, regulations and regulatory standards of the relevant countries and regions.

[0027] Based on this, embodiments of this application provide a method for detecting defects in PIN pins, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the PIN pin defect detection method of this application.

[0028] In this embodiment, the defect detection method for the PIN pin includes steps S10 to S40: Step S10: Obtain a single-frame image of the PIN pin to be tested; It should be noted that the executing entity in this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or an electronic device capable of performing the above functions, a PIN pin defect detection device, etc. The following description uses a PIN pin defect detection device as an example to illustrate this embodiment and the subsequent embodiments.

[0029] It should be noted that the schematic diagram of the acquisition device for obtaining a single frame image of the PIN pin under test in this embodiment can be found in the following reference. Figure 2 , Figure 2Number 1 is a single industrial camera, the lens of which can be fitted with a linear polarizer to further suppress reflections. Number 2 is a single illumination source fixedly installed according to the pin arrangement characteristics, such as a white ring LED or a strip LED. Numbers 3 and 4 are the carrier and the workpiece with pins to be inspected, respectively. Number 5 is an image processing unit connected to the camera, which can be deployed with a dedicated neural network and geometric analysis module.

[0030] It should be noted that before the test begins, the focal length, exposure value, and illumination of the ring LED light source of the industrial camera lens can be adjusted to ensure that the camera's optical axis is perpendicular to the surface of the PIN pins, so that each PIN pin is clearly and normally imaged. Then, a grayscale image of the PIN pin under test is acquired using a single industrial camera, and this image serves as the input for subsequent processing. In this embodiment, the image resolution can be set to 1920×1080 pixels.

[0031] The obtained single-frame image can be used as a reference. Figure 3 , Figure 3 This is a schematic diagram of a single frame image of the PIN pin under test.

[0032] In one feasible approach, the preset image segmentation model includes a U-Net network model. Before the step of acquiring a single-frame image of the PIN pin to be tested, the approach further includes: acquiring normal PIN pin images and defective PIN pin images; performing synthesis processing on the normal PIN pin images to generate a synthesized defective image; using the normal PIN pin images, the defective PIN pin images, and the synthesized defective image as training data to train the initial U-Net network model, thereby obtaining the trained U-Net network model.

[0033] It should be noted that this embodiment can use the U-Net network model as the preset image segmentation model. The U-Net network model is a convolutional neural network specifically designed for image segmentation. During the model training phase, the same imaging system as in the detection phase can be used to acquire a large number of real PIN images. These images are divided into two categories: normal PIN images (images where all PINs are intact and neatly arranged) and defective PIN images (images containing actual defects such as skewness, missing pins, bends, and stains). Both types of images require pixel-level annotation, meaning the outline of each PIN in each image needs to be manually or semi-automatically labeled. Furthermore, to address the scarcity of real defect samples, artificial defect synthesis can be performed on the acquired normal PIN images. For example, image processing algorithms can be used to simulate defects such as bending, missing pins, and stains on normal PIN images, generating a large number of synthetic defect images. The three types of images—real normal images, real defect images, and synthetic defect images—are merged into a training dataset. An improved U-Net network will be used as the initial model. During training, the input image is forward-propagated through the network to output a predicted mask, which is then compared with the labeled ground truth mask to calculate the loss function. The network parameters are updated through backpropagation. After multiple rounds of iterative training, when the accuracy on the validation set meets the requirements, the model parameters are saved, resulting in the trained U-Net network model.

[0034] In one feasible approach, the step of synthesizing the normal PIN image to generate a synthetic defect image includes: obtaining the PIN region in the normal PIN image; performing pixel displacement or skeleton bending processing on the PIN region to generate a synthetic defect image with a curved shape.

[0035] It should be noted that during the compositing process, the local region containing each PIN pin is first extracted from the acquired normal PIN pin images based on the pre-labeled instance segmentation mask. Specifically, for each instance segmentation mask, its bounding rectangle is first calculated, and then extended by 10 to 20 pixels in each direction on the outermost side of the bounding rectangle. This rectangular area is used to crop a sub-image from the original grayscale image. This sub-image is the PIN pin region, containing a single complete normal PIN pin and its surrounding local background. After cropping, the sub-image size is uniformly scaled to a preset size for subsequent processing.

[0036] Then, pixel displacement or skeleton bending processing is performed on the PIN area. Pixel displacement refers to moving each pixel in the image to a new position by defining a displacement field, thereby bending the originally straight PIN. Specifically, an image coordinate system can be established with the original centerline of the PIN as a reference, and the centerline along the length of the PIN. Then, a bending function is defined, which describes the lateral offset at each position along the length of the PIN. The offset varies with the position along the length direction, usually with the largest offset in the middle of the PIN and zero offset at both ends. The magnitude of the offset determines the bending amplitude, which can be set to a factor of zero to several times the width of the PIN as needed. Next, for each pixel in the PIN area image, the corresponding lateral offset is calculated based on its position along the length direction to obtain the target coordinates of that pixel. Finally, pixel resampling is performed on the target coordinates using an interpolation method to generate a synthetic defect image. The PIN in the synthetic defect image exhibits a smooth arc or wavy bend.

[0037] The skeleton bending method first extracts the skeleton of the PIN, then performs a geometric transformation on the skeleton, and finally reconstructs the PIN image based on the deformed skeleton. Specifically, the skeleton is extracted from the PIN region image. During skeleton extraction, the PIN region is binarized, and then a morphological thinning algorithm is used to iteratively peel away edge pixels until a connected curve with a single pixel width is obtained. This curve is the skeleton, representing the centerline of the PIN. Then, a bending transformation is applied to the skeleton. For example, each point on the original skeleton is moved a certain distance laterally, with the midpoint moving the largest distance and the ends moving zero distance, thus forming a parabolic curved skeleton. Next, a deformation field is constructed based on the correspondence between the original skeleton and the bent skeleton. The deformation mapping of the entire image can be calculated using methods such as thin-plate spline interpolation or moving least squares. Finally, each pixel of the original PIN region image is mapped to the new position according to the deformation field and resampled to generate a synthetic defect image with a bent shape.

[0038] Step S20: Input the single-frame image into a preset image segmentation model to obtain the instance segmentation mask for each of the PIN pins to be tested; It should be noted that before inputting the single-frame image into the preset image segmentation model, the single-frame image can be preprocessed. Specifically, this can involve cropping the region of interest, using adaptive histogram equalization to enhance contrast, and performing filtering to suppress background unevenness and reduce noise. (See reference...) Figure 4 , Figure 4 This is a schematic diagram of a preprocessed single-frame image. The preprocessed single-frame image is then input into a preset image segmentation model to obtain the instance segmentation mask for each of the PIN pins to be tested. (See reference...) Figure 5 , Figure 5 This is a schematic diagram of a PIN mask.

[0039] Step S30: Extract the geometric feature parameters of each of the PIN pins to be tested based on the instance segmentation mask; It should be noted that the geometric feature parameters can be the center point or the center line skeleton, which can be determined according to the direction of the PIN pin.

[0040] In one feasible approach, the step of extracting the geometric feature parameters of each of the PIN pins to be tested based on the instance segmentation mask includes: extracting the contour of the instance segmentation mask to obtain contour geometric features; extracting the center point or centerline skeleton of each PIN pin from the contour geometric features according to the relative direction between the camera optical axis and the PIN pin to be tested, and determining the center point or the centerline skeleton as geometric feature parameters.

[0041] It should be noted that, specifically, the center point or centerline skeleton of each filtered pin can be extracted based on the relative direction of the camera's optical axis and the pin itself. For cases where the camera lens's optical axis is perpendicular to the pin surface, the center point coordinates of the pin tip region are extracted from the contour and used as the geometric feature parameters of that pin, which can then be used for... Figure 6 , Figure 6 This is a schematic diagram of the PIN pin outline. If the camera optical axis is at a different angle to the PIN pin, the centerline skeleton is extracted as the geometric feature parameter of the PIN pin.

[0042] Step S40: Perform anomaly detection on the geometric feature parameters to obtain the defect detection result of the PIN pin to be tested.

[0043] It should be noted that the extracted center point coordinates of each PIN are used as input for multi-dimensional geometric analysis. The analysis dimensions include counting the number of center points, calculating the pin tip outline area, calculating the aspect ratio of the minimum bounding rectangle, calculating the distance between adjacent center points and the direction of the connecting lines, and comparing these calculated values ​​with preset standard values ​​and allowable thresholds. Based on the comparison results, it is determined whether each PIN has defects such as missing pins, short pins, bends, or skewness, and the final output is a normal or abnormal result.

[0044] This embodiment acquires a single-frame image of the PIN pin to be tested; inputs the single-frame image into a preset image segmentation model to obtain instance segmentation masks for each of the PIN pins to be tested; extracts geometric feature parameters for each of the PIN pins to be tested based on the instance segmentation masks; and performs anomaly detection on the geometric feature parameters to obtain the defect detection result of the PIN pin to be tested. This embodiment acquires a single-frame image of the PIN pin to be tested and uses an image segmentation model to extract geometric feature parameters for anomaly detection. During the detection process, multiple cameras or 3D scanning equipment are not required; only a single camera is needed to complete defect identification, reducing the cost of PIN pin defect detection.

[0045] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to that in the first embodiment described above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 7 Step S40 also includes steps S401 to S402: Step S401: When the geometric feature parameter is the center point of each of the PIN pins to be tested, anomaly detection is performed based on the center point to obtain the defect detection result of the PIN pin to be tested; It should be noted that if the extracted geometric feature parameters are the center points of each of the PIN pins to be tested (corresponding to the imaging posture of the pin head facing the camera), then the center points are used for anomaly detection.

[0046] In one feasible approach, the step of determining the defect of the PIN pin under test based on the center point includes: sorting the center points according to a preset direction and counting the number of sorted center points to obtain a quantity result; calculating the contour area of ​​the pin tip region corresponding to the center point; calculating the aspect ratio of the minimum bounding rectangle of the pin tip region corresponding to the center point; calculating the distance and connecting direction between adjacent center points; and determining whether the PIN pin under test has a defect based on the quantity result, the contour area, the aspect ratio of the minimum bounding rectangle, the distance, and the connecting direction to obtain the defect determination result of the PIN pin under test.

[0047] It should be noted that for the center point, defect detection dimensions include the pin tip reflective area, pin tip shape, and contextual constraint verification. The pin tip reflective area is evaluated by calculating the area of ​​the pin tip outline; if the area is smaller than that of a normal pin, it indicates a short pin / missing pin. The pin tip shape is evaluated by calculating the geometric parameters of the reflective surface (e.g., the aspect ratio of the minimum bounding rectangle); if these parameters differ significantly from those of a normal pin, it indicates a bent pin / outlined pin. Contextual constraint verification involves analyzing and verifying the overall arrangement rationality based on the periodic structure of the pin array, such as equal spacing and row / column alignment. This primarily verifies the number of pins, the spacing between adjacent pins, and the angle between adjacent pins.

[0048] Specifically, the pins can be sorted from smallest to largest by the x-value of their center point pixel coordinates. Then, the following checks are performed sequentially: PIN count check (whether the number of center points matches the normal number of pins); and pin vertical alignment check (if the count check passes, group the pin center points in pairs sequentially and calculate the distance and direction of the line connecting the center points of each pair). (Refer to [reference needed]). Figure 8 , Figure 8This diagram illustrates the connection between the center points of vertically aligned pins. A significant deviation from the normal value indicates pin misalignment or other defects. For horizontal pin alignment testing, if the vertical alignment test is passed, calculate the distance and direction of each pin's center to the centers of the other pins. (Refer to the diagram for reference.) Figure 9 , Figure 9 This is a schematic diagram of the center points of horizontally arranged pins. The number of adjacent points in the horizontal direction is counted. If there is a pin center that has no adjacent points in the horizontal direction, or if there are more than 2 adjacent points in the horizontal direction, it indicates that the pin has defects such as distortion or contamination.

[0049] For PINs that pass all the above tests, the output is normal; otherwise, it is abnormal.

[0050] Step S402: When the geometric feature parameters include the centerline skeleton of each of the PIN pins to be tested, anomaly detection is performed based on the centerline skeleton to obtain the defect detection result of the PIN pin to be tested.

[0051] It should be noted that if the geometric feature parameter is the centerline skeleton of each of the PINs to be tested (corresponding to the imaging posture of the needle body facing the camera), then anomaly detection is performed based on the centerline skeleton.

[0052] In one feasible approach, the step of performing anomaly detection based on the centerline skeleton to obtain the defect detection result of the PIN pin under test includes: calculating the length, straightness, and principal direction vector of the centerline skeleton respectively; determining whether the PIN pin under test has a defect based on the length, straightness, and principal direction vector, and obtaining the defect detection result of the PIN pin under test.

[0053] It should be noted that the following geometric analyses are performed on the PINs from which the center line is extracted: length consistency (calculating the skeleton length and comparing it with the average of neighboring PINs; if the deviation is greater than a preset value, such as 5%), it indicates a short or missing PIN); straightness evaluation (fitting the skeleton to a segmented straight line and calculating the maximum deviation distance; if the ratio of the maximum deviation distance to the length of the center line skeleton is greater than a preset value (e.g., 3%), it is determined to be bent); direction consistency (calculating the principal direction vector of each PIN; if the angle between the principal direction vector and the overall arrangement direction is greater than a preset angle value (e.g., 3°), it indicates skewness); and surface anomaly evaluation (calculating local texture features within the mask area and combining them with a classification head to determine oxidation or contamination). Thus, it is possible to determine whether the PIN under test has defects based on the length, straightness, and principal direction vector.

[0054] In this embodiment, when the geometric feature parameter is the center point of each of the PINs under test, anomaly detection is performed based on the center point to obtain the defect detection result of the PIN. When the geometric feature parameter includes the centerline skeleton of each of the PINs under test, anomaly detection is performed based on the centerline skeleton to obtain the defect detection result of the PIN. This embodiment adaptively selects the center point or centerline skeleton discrimination mode according to the imaging posture, which can be compatible with both needle tip facing and needle body facing detection scenarios without hardware adjustment, thus improving the versatility and flexibility of the method.

[0055] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the defect detection method of the PIN pin of this application. Any simple modifications based on this technical concept are within the protection scope of this application.

[0056] This application also provides a defect detection device for PIN pins, please refer to... Figure 10 The defect detection device for the PIN pin includes: The acquisition module 10 is used to acquire a single-frame image of the PIN pin under test; The segmentation module 20 is used to input the single-frame image into a preset image segmentation model to obtain the instance segmentation mask of each of the PIN pins to be tested; Extraction module 30 is used to extract the geometric feature parameters of each of the PIN pins to be tested based on the instance segmentation mask; The discrimination module 40 is used to perform anomaly discrimination on the geometric feature parameters to obtain the defect discrimination result of the PIN pin to be tested.

[0057] This application acquires a single-frame image of the PIN pin to be tested; inputs the single-frame image into a preset image segmentation model to obtain an instance segmentation mask for each of the PIN pins to be tested; extracts the geometric feature parameters of each of the PIN pins to be tested based on the instance segmentation mask; and performs anomaly detection on the geometric feature parameters to obtain the defect detection result of the PIN pin to be tested. This application acquires a single-frame image of the PIN pin to be tested and uses an image segmentation model to extract geometric feature parameters for anomaly detection. During the detection process, multiple cameras or 3D scanning equipment are not required; only a single camera is needed to complete defect identification, reducing the cost of PIN pin defect detection.

[0058] In one embodiment, the acquisition module 10 is further configured to acquire normal PIN images and defective PIN images; perform synthesis processing on the normal PIN images to generate a synthesized defective image; and use the normal PIN images, the defective PIN images, and the synthesized defective image as training data to train the initial U-Net network model to obtain the trained U-Net network model.

[0059] In one embodiment, the acquisition module 10 is further configured to acquire the PIN pin region in the normal PIN pin image; and to perform pixel displacement or skeleton bending processing on the PIN pin region to generate a synthetic defect image with a curved shape.

[0060] In one embodiment, the extraction module 30 is further configured to extract the contour of the instance segmentation mask to obtain contour geometric features; and extract the center point or center line skeleton of each PIN from the contour geometric features according to the relative direction between the camera optical axis and the PIN to be tested, and determine the center point or the center line skeleton as geometric feature parameters.

[0061] In one embodiment, the discrimination module 40 is further configured to perform anomaly discrimination based on the center point when the geometric feature parameter is the center point of each of the PIN pins to be tested, and obtain the defect discrimination result of the PIN pin to be tested; and to perform anomaly discrimination based on the center line skeleton when the geometric feature parameter includes the center line skeleton of each of the PIN pins to be tested, and obtain the defect discrimination result of the PIN pin to be tested.

[0062] In one embodiment, the discrimination module 40 is further configured to sort the center points according to a preset direction, and count the number of sorted center points to obtain a quantity result; calculate the contour area of ​​the pin tip region corresponding to the center point; calculate the aspect ratio of the minimum bounding rectangle of the pin tip region corresponding to the center point; calculate the distance and connecting direction between adjacent center points; and determine whether the PIN pin under test has a defect based on the quantity result, the contour area, the aspect ratio of the minimum bounding rectangle, the distance, and the connecting direction, thereby obtaining a defect discrimination result for the PIN pin under test.

[0063] In one embodiment, the discrimination module 40 is further configured to calculate the length, straightness, and principal direction vector of the centerline skeleton respectively; determine whether the PIN pin under test has a defect based on the length, straightness, and principal direction vector, and obtain the defect discrimination result of the PIN pin under test.

[0064] The PIN pin defect detection device provided in this application, employing the PIN pin defect detection method described in the above embodiments, can solve the technical problem of how to reduce the cost of PIN pin defect detection. Compared with the prior art, the beneficial effects of the PIN pin defect detection device provided in this application are the same as those of the PIN pin defect detection method provided in the above embodiments, and other technical features in the PIN pin defect detection device are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0065] This application provides a PIN pin defect detection device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, which are executed by the at least one processor to enable the at least one processor to perform the PIN pin defect detection method in the first embodiment described above.

[0066] The following is for reference. Figure 11 The diagram illustrates a structural schematic of a PIN pin defect detection device suitable for implementing embodiments of this application. The PIN pin defect detection device in this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 11 The PIN pin defect detection device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments of this application.

[0067] like Figure 11As shown, the PIN pin defect detection device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in ROM (Read Only Memory) 1002 or a program loaded from storage device 1003 into RAM (Random Access Memory) 1004. RAM 1004 also stores various programs and data required for the operation of the PIN pin defect detection device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via bus 1005. Input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to I / O interface 1006: input devices 1007 including, for example, touch screens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. Communication device 1009 allows the PIN defect detection device to communicate wirelessly or wiredly with other devices to exchange data. Although the figures show PIN defect detection devices with various systems, it should be understood that implementation or possession of all the systems shown is not required. More or fewer systems may be implemented alternatively.

[0068] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0069] The PIN pin defect detection device provided in this application, employing the PIN pin defect detection method described in the above embodiments, can solve the technical problem of how to reduce the cost of PIN pin defect detection. Compared with the prior art, the beneficial effects of the PIN pin defect detection device provided in this application are the same as those of the PIN pin defect detection method provided in the above embodiments, and other technical features of this PIN pin defect detection device are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0070] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0071] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0072] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the PIN pin defect detection method in the above embodiments.

[0073] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0074] The aforementioned computer-readable storage medium may be included in a PIN pin defect detection device; or it may exist independently and not assembled into a PIN pin defect detection device.

[0075] The aforementioned computer-readable storage medium carries one or more programs. When the aforementioned one or more programs are executed by the PIN pin defect detection device, the PIN pin defect detection device causes the following: acquiring a single-frame image of the PIN pin to be tested; inputting the single-frame image into a preset image segmentation model to obtain an instance segmentation mask for each of the PIN pins to be tested; extracting geometric feature parameters of each of the PIN pins to be tested based on the instance segmentation mask; and performing anomaly detection on the geometric feature parameters to obtain a defect detection result for the PIN pin to be tested.

[0076] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0077] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0078] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0079] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described PIN pin defect detection method, thereby solving the technical problem of how to reduce the cost of PIN pin defect detection. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the PIN pin defect detection method provided in the above embodiments, and will not be repeated here.

[0080] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the PIN pin defect detection method described above.

[0081] The computer program product provided in this application solves the technical problem of how to reduce the cost of PIN pin defect detection. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the PIN pin defect detection method provided in the above embodiments, and will not be repeated here.

[0082] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A method for detecting defects in PIN pins, characterized in that, The method includes: Acquire a single-frame image of the PIN pin to be tested; The single-frame image is input into a preset image segmentation model to obtain the instance segmentation mask for each of the PIN pins to be tested; Geometric feature parameters of each of the PIN pins to be tested are extracted based on the instance segmentation mask. Anomaly detection is performed on the geometric feature parameters to obtain the defect detection result of the PIN pin under test.

2. The method as described in claim 1, characterized in that, The preset image segmentation model includes a U-Net network model. Before the step of acquiring a single-frame image of the PIN pin to be tested, the method further includes: Acquire images of normal PIN pins and defective PIN pins; The normal PIN pin image is synthesized to generate a synthetic defect image; The normal PIN image, the defective PIN image, and the synthetic defective image are used as training data to train the initial U-Net network model, resulting in the trained U-Net network model.

3. The method as described in claim 2, characterized in that, The step of synthesizing the normal PIN pin image to generate a synthesized defective image includes: Obtain the PIN pin region from the normal PIN pin image; The PIN pin area is subjected to pixel displacement or skeleton bending processing to generate a synthetic defect image with a curved shape.

4. The method as described in claim 1, characterized in that, The step of extracting the geometric feature parameters of each of the PIN pins to be tested based on the instance segmentation mask includes: Contour extraction is performed on the instance segmentation mask to obtain contour geometric features; Based on the relative direction between the camera optical axis and the PIN pin to be tested, the center point or centerline skeleton of each PIN pin is extracted from the contour geometric features, and the center point or the centerline skeleton is determined as the geometric feature parameter.

5. The method as described in claim 1, characterized in that, The step of performing anomaly detection on the geometric feature parameters to obtain the defect detection result of the PIN pin under test includes: When the geometric feature parameter is the center point of each of the PIN pins to be tested, anomaly discrimination is performed based on the center point to obtain the defect discrimination result of the PIN pin to be tested; When the geometric feature parameters include the centerline skeleton of each of the PIN pins to be tested, anomaly detection is performed based on the centerline skeleton to obtain the defect detection result of the PIN pin to be tested.

6. The method as described in claim 5, characterized in that, The step of performing anomaly detection based on the center point to obtain the defect detection result of the PIN pin under test includes: The center points are sorted according to a preset direction, and the number of the sorted center points is counted to obtain the quantity results. Calculate the contour area of ​​the needle tip region corresponding to the center point; Calculate the aspect ratio of the minimum bounding rectangle of the needle tip region corresponding to the center point; Calculate the distance and direction of the line connecting adjacent center points; Based on the quantity results, the outline area, the aspect ratio of the minimum circumscribed rectangle, the spacing, and the direction of the connecting lines, it is determined whether the PIN pin under test has a defect, and the defect discrimination result of the PIN pin under test is obtained.

7. The method as described in claim 5, characterized in that, The step of performing anomaly detection based on the centerline skeleton to obtain the defect detection result of the PIN pin under test includes: Calculate the length, straightness, and principal direction vector of the centerline skeleton respectively; Based on the length, straightness, and principal direction vector, determine whether the PIN under test has a defect, and obtain the defect identification result of the PIN under test.

8. A defect detection device for PIN pins, characterized in that, The device includes: The acquisition module is used to acquire a single-frame image of the PIN pin under test; The segmentation module is used to input the single-frame image into a preset image segmentation model to obtain the instance segmentation mask for each of the PIN pins to be tested; The extraction module is used to extract the geometric feature parameters of each of the PIN pins to be tested based on the instance segmentation mask; The discrimination module is used to discriminate anomalies in the geometric feature parameters and obtain the defect discrimination result of the PIN pin under test.

9. A defect detection device for PIN pins, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the method for detecting defects in a PIN pin as described in any one of claims 1 to 7.

10. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the defect detection method for PIN pins as described in any one of claims 1 to 7.

Images