Carrier tape detection method based on machine vision

Abstract

The present invention discloses a carrier tape detection method based on machine vision, which belongs to the technical field of punched paper tape. The carrier tape detection method based on machine vision of the present invention first builds a hardware platform, and respectively collects images of qualified punched paper tapes and defective punched paper tapes, uses the images collected from a large number of qualified paper tapes as detection templates, and matches and compares the paper tape to be tested with the detection templates, thereby determining whether the paper tape to be tested meets production requirements. Among them, according to the neat arrangement characteristics of the hole shapes of the punched paper tape, hole shape positioning and area segmentation for the hole-shaped paper tape are proposed. Then, based on the hole shape image after area segmentation, an 8-neighborhood bidirectional template matching method is proposed, which significantly improves the efficiency of hole shape qualified detection. Therefore, the carrier tape detection method based on machine vision of the present invention solves the technical problems of slow detection speed, large human error and low efficiency existing in punched paper tape.

Description

Technical Field

[0001] The present invention relates to the technical field of punched paper tapes, and in particular to a carrier tape detection method based on machine vision. Background Art

[0002] With the rapid development of the electronics industry, electronic components in integrated circuits and chip-type electronic devices are becoming smaller, more numerous, more densely packed, and have longer lifespans. To increase the lifespan and environmental resistance of resistors in electronic devices, chip fixed resistors have emerged. Chip fixed resistors, also known as surface mount resistors, are widely used in various products in the electronics industry due to their small size, light weight, high mounting density, low assembly cost, and compatibility with automated placement equipment. Furthermore, surface mount technology, often referred to as SMT, is a recently developed and widely used technology for the automated assembly of electronic components.

[0003] Today, medium and large enterprises are using surface mount technology (SMT) for large-scale chip resistor assembly. The expanded application of SMT for chip resistors has placed high demands on the dimensional accuracy of the substrate used to package the resistors. Punched paper tape, for example, utilizes a precision singeing process, effectively controlling the generation of fuzz from the holes; therefore, it is often used for chip resistor packaging. Conventional chip resistors come in seven sizes, ranging from 0201 to 2512 inches, with 0603 being the most widely used. The high dimensional accuracy requirements for chip resistors also necessitate even higher precision specifications for punched paper tape. Furthermore, the quality of the punched paper tape holes directly impacts the chip resistor packaging efficiency, which in turn affects SMT technology and, in severe cases, can even lead to solder joint defects. Therefore, hole defect detection in punched paper tape is a primary factor impacting the efficiency of SMT chip resistor SMT. However, during the punched paper tape production process, hole defect detection is performed visually. During the long and focused process of inspecting tiny punched paper tapes, inspectors inevitably experience loss of concentration and eye fatigue, resulting in slow hole shape inspection, high error rates, and high labor costs. Furthermore, defective paper tapes can negatively impact customers and hinder the development of punched paper tapes.

[0004] Based on this, Chinese patent CN105203557B discloses an electronic paper tape papermaking quality online detection system, which includes a horizontal paper reel. The horizontal paper reel is provided with a first guide roller, a cold water cylinder, a scanner, a second guide roller, a paper reel cylinder and a paper roll in sequence in front of the forward direction of the paper web. The scanner is installed above the paper web and scans back and forth longitudinally along the two ends of the paper web banner. The papermaking machine paper reeling system also includes a surface defect online detection system and an insect repellent device arranged between the scanner and the second guide roller. The surface defect online detection system is connected to the vehicle speed signal, paper break signal, and roll change signal of the horizontal paper reel and is displayed and controlled by the detection workstation; the insect repellent device includes a mosquito repellent lamp and a blower. The mosquito repellent lamp and the blower are arranged near the light source of the surface defect online detection system and the air outlet of the blower faces upward. This online detection system can realize online detection of papermaking quality and can detect external defects or internal defects of paper tape, such as stains, black blocks, black dust, color spots on the paper surface and color spots, dirt blocks and other stains and foreign objects in the interlayer; at the same time, it is also equipped with an insect repellent device to prevent mosquitoes, moths, etc. from interfering with the CCD camera group's photography.

[0005] However, if the electronic paper tape online quality detection system disclosed in the prior art is directly applied to the field of electronic carrier tape processing, there is still the technical problem of being unable to accurately detect punched paper tape. Specifically, for small-sized, high-precision objects such as punched paper tape, traditional detection relies on the human eye under a microscope. Therefore, while the electronic paper tape online quality detection system disclosed in the prior art can monitor defects such as stains on the paper tape, it is difficult to monitor factors such as the shape and size of the neatly arranged holes on the punched paper tape. In addition, the holes on the punched paper tape are neatly arranged and closely spaced, so the paper tape cannot be inspected at high speeds. Otherwise, it is easy to cause smearing in the detection system, which can easily lead to incorrect judgments. Although the paper tape can maintain a certain accuracy rate at low speeds, the detection efficiency is too low. Therefore, the speed of punched paper tape inspection becomes a limiting factor in detection efficiency. Summary of the Invention

[0006] Based on this, it is necessary to provide a carrier tape detection method based on machine vision to solve the technical problems of slow detection speed, large human error and low efficiency of punched paper tape.

[0007] A method for detecting a carrier tape based on machine vision comprises the following steps:

[0008] S1: First, a hardware platform is built, and then the hardware platform is used to collect images of qualified hole shapes of a preset number of punched paper tapes, and the collected qualified hole shape images are pre-processed; after analysis and calculation, a standard hole shape image is obtained as a standard template, and the characteristics of the hole shape in the standard template are obtained;

[0009] S2: Then, the hole shape positioning region of the punched paper tape hole shape image is segmented according to the obtained hole shape features to obtain the position coordinates of each hole shape;

[0010] S3: Then, the hardware platform is used to collect hole shape images of the paper tape to be tested, and the images are pre-processed; the obtained hole shape images are matched with the standard template, and the matching degree of each hole shape is obtained;

[0011] S4: Finally, the matching degree of each hole shape is judged; if the matching degrees of all hole shapes are greater than a preset threshold, the paper tape segment is determined to be a qualified product; otherwise, the paper tape segment is determined to be defective.

[0012] Specifically, the image preprocessing process includes: image grayscale conversion, image filtering, image segmentation, morphological transformation and edge detection.

[0013] Specifically, during the image grayscale processing, the original hole-shaped image I(i, j) is first grayscaled according to the following formula to obtain the grayscale image G(i, j) of the hole-shaped image:

[0014] Formula 1: G(i,j)=0.30I R (i,j)+0.59I G (i,j)+0.11I B (I,j);

[0015] Where I R (i,j),I G (i,j),I B (I, j) represent the R, G, and B components in the hole image respectively.

[0016] Specifically, in the image filtering step, a 3x3 template is used to remove the two pixels with the smallest grayscale values in the filtering window and take the median. The obtained grayscale value is then compared with the original grayscale value, and whether to replace the grayscale value is determined based on the size of the relative difference.

[0017] Furthermore, the template matching process is as follows:

[0018] a. Get template: perform edge detection on a preset amount of qualified hole-shaped images, analyze their edge features one by one, and obtain a standard qualified hole-shaped image edge map;

[0019] b. Combination of region segmentation and edge detection: The analysis object is the hole edge image after preprocessing, and the region segmentation is performed on it, and the edge image is divided into regions;

[0020] c. Matching: Use a 3*3 8-neighborhood window to perform bidirectional matching between the template image and the hole image.

[0021] Furthermore, the specific steps of matching are as follows:

[0022] S31: Scan the hole-shaped area based on the hole-shaped image to be detected. If the pixel is a boundary point, check whether there is a boundary point in the 8-neighborhood of the corresponding point in the template image. If there is a boundary point, the pixel is defined as a successful match. Finally, the matching failure value of the hole-shaped area is calculated, which is recorded as P1: P1=S1 / S; where P1 represents the matching failure value of the forward match, S1 represents the total number of pixels in the hole-shaped area that failed to match, and S represents the total number of pixels in the hole-shaped area in the template.

[0023] S32: Using the template image as a reference, scan the template image to find whether there is a boundary point in the 8-neighborhood of the corresponding point in the image to be detected. If so, the pixel point is defined as a successful match. Finally, the matching failure value of the hole-shaped area is calculated, which is recorded as P2: P2=S2 / S; where P2 represents the matching failure value of the reverse matching, S2 represents the total number of pixels in the hole-shaped area that failed to match, and S represents the total number of pixels in the hole-shaped area in the template.

[0024] S33: Compare P1 and P2, select the larger matching degree as the matching failure ratio of this matching, defined as P; then compare P with the threshold value p; if the matching degree is greater than the threshold value, the hole shape is judged to be unqualified; if the matching degree is less than the threshold value, the hole shape is judged to be qualified.

[0025] In summary, the present invention's carrier tape inspection method based on machine vision first builds a hardware platform and captures images of qualified punched paper tapes and defective punched paper tapes, respectively. Images captured from a large number of qualified paper tapes are used as inspection templates, and the paper tape to be inspected is matched and compared with the inspection template to determine whether the paper tape to be inspected meets production requirements. Specifically, based on the neatly arranged hole shapes of the punched paper tape, hole shape positioning and region segmentation for the hole-shaped paper tape are proposed. Then, based on the hole shape images after region segmentation, an 8-neighborhood bidirectional template matching method is proposed, which significantly improves the efficiency of qualified hole shape detection. Therefore, the present invention's carrier tape inspection method based on machine vision solves the technical problems of slow inspection speed, large human error, and low efficiency that exist in punched paper tape. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 Flowchart of the carrier tape detection method based on machine vision of the present invention;

[0027] Figure 2 An embodiment of a hardware platform for applying the carrier tape detection method based on machine vision of the present invention;

[0028] Figure 3 Flowchart of the image preprocessing steps of the carrier tape detection method based on machine vision of the present invention;

[0029] Figure 4 Schematic diagram of an embodiment of a standard template of a carrier tape detection method based on machine vision of the present invention;

[0030] Figure 5 Schematic diagram of a defect hole shape image of a carrier tape detection method based on machine vision of the present invention;

[0031] Figure 6 Schematic diagram of another defect hole shape image of the carrier tape detection method based on machine vision of the present invention;

[0032] Figure 7 Schematic diagram of a 3x3 matching window of the carrier tape detection method based on machine vision of the present invention;

[0033] Figure 8 Schematic diagram of detection of a standard template of a carrier tape detection method based on machine vision of the present invention;

[0034] Figure 9 Schematic diagram of the detection of a paper tape to be tested in the carrier tape detection method based on machine vision of the present invention;

[0035] Figure 10 Schematic diagram of hole tear detection in a carrier tape detection method based on machine vision according to the present invention;

[0036] Figure 11 Schematic diagram of detection of hole shape default in the carrier tape detection method based on machine vision of the present invention. Implementation Method

[0037] To make the above-mentioned objects, features, and advantages of the present invention more readily apparent, specific embodiments of the present invention are described in detail below with reference to the accompanying drawings. The following description sets forth numerous specific details to facilitate a full understanding of the present invention. However, the present invention can be implemented in many other ways than those described herein, and those skilled in the art may make similar modifications without departing from the scope of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0038] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", "axial", "radial", "circumferential" and the like to indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as limiting the present invention.

[0039] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referred to. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of the present invention, "plurality" means at least two, such as two, three, etc., unless otherwise specifically defined.

[0040] In the present invention, unless otherwise specified or limited, the terms "installed," "connected," "connect," "fixed," etc. should be understood in a broad sense. For example, they can refer to fixed connection, detachable connection, or integration; mechanical connection, electrical connection; direct connection, or indirect connection through an intermediate medium; internal communication between two components, or interaction between two components, unless otherwise specified. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0041] In the present invention, unless otherwise expressly specified or limited, when a first feature is "above" or "below" a second feature, it may mean that the first and second features are in direct contact, or that the first and second features are in indirect contact through an intermediary. Furthermore, when a first feature is "above," "above," or "above" a second feature, it may mean that the first feature is directly above or diagonally above the second feature, or simply means that the first feature is at a higher level than the second feature. When a first feature is "below," "below," or "below" a second feature, it may mean that the first feature is directly below or diagonally below the second feature, or simply means that the first feature is at a lower level than the second feature.

[0042] It should be noted that when an element is referred to as being "fixed to" or "disposed on" another element, it may be directly on the other element or there may be an intermediate element. When an element is considered to be "connected to" another element, it may be directly connected to the other element or there may be an intermediate element. The terms "vertical," "horizontal," "upper," "lower," "left," "right," and similar expressions used herein are for illustrative purposes only and do not represent the only implementation methods.

[0043] See also Figure 1 ,like Figure 1 As shown, the carrier tape detection method based on machine vision of the present invention includes the following steps:

[0044] S1: First, a hardware platform is built. Then, the hardware platform is used to collect images of qualified hole shapes of a preset number of punched paper tapes. The collected images of qualified hole shapes are pre-processed. After analysis and calculation, a standard hole shape image is obtained as a standard template, and the characteristics of the hole shape in the standard template are obtained.

[0045] S2: Then, the hole shape positioning region of the punched paper tape hole shape image is segmented according to the obtained hole shape features to obtain the position coordinates of each hole shape;

[0046] S3: Then, the hardware platform is used to collect hole shape images of the paper tape to be tested, and the images are pre-processed; the obtained hole shape images are matched with the standard template, and the matching degree of each hole shape is obtained;

[0047] S4: Finally, the matching degree of each hole shape is judged; if the matching degrees of all hole shapes are greater than a preset threshold, the paper tape segment is determined to be a qualified product; otherwise, the paper tape segment is determined to be defective.

[0048] Specifically, in the carrier tape detection method based on machine vision of the present invention, the hardware platform mainly realizes the image acquisition of the punched paper tape and provides the acquired image to the software system for processing and detection. Figure 2 ,like Figure 2 As shown, the hardware platform for collecting the hole shape of punched paper tape that meets the basic requirements mainly includes the following hardware: light source structure 1, camera equipment 2, lens structure 3 and computer 4.

[0049] Specifically, in the visual inspection process for punched paper tape, the acquisition of the target image is directly linked to the light source. The quality of the light source directly affects the image quality. Therefore, no matter how excellent the inspection algorithm, poor light source selection can lead to system failure. A suitable light source can enhance the feature contrast of the target image, reduce and suppress the impact of noise, and improve the acquired image quality. Therefore, selecting the optimal light source is crucial for successful hole-shaped defect detection in punched paper tape. There are various light source types, but generally, three factors must be considered for a light source: structure. Light sources can be distinguished by the structure of the light they emit, and different light structures determine different luminous characteristics. Light source characteristics. These primarily include light spectrum distribution, brightness, heat dissipation, and operating life. Inspection object. Different materials of the inspection object react differently to various light sources, causing reflection, refraction, and transmission of light. If the light color of the inspection object is similar, the inspection object will appear brighter; if it is different, the inspection object will appear darker. Therefore, light source selection must be considered in conjunction with the characteristics of the inspection object. Currently, LED, nitrogen, and high-frequency fluorescent lamps are the most commonly used in industrial inspection applications. The hardware platform's goal is to capture images of the hole patterns in punched paper tape to improve the precision of the hole-pattern parameters. Therefore, a strong contrast between the foreground and background of the target hole image is required. Furthermore, a light source with a similar color to the tape increases the tape's brightness, maximizing the color difference between the tape and the holes, making it easier to distinguish between the two. Light source structure 1 preferably uses a medium-bright white LED ring light source, and it is recommended that the light be illuminated at a 45-degree angle from the side of the punched paper tape to be tested.

[0050] Specifically, in industrial inspection processes, camera selection becomes particularly crucial due to the high-precision, high-speed inspection requirements for inspected objects. If the selected camera fails to meet these requirements, image acquisition will be directly impacted, leading to a weakening of the inspection system's functionality. As the core component of cameras, the industry places significant emphasis on the sensor. Both CCD and CMOS utilize photodiodes to convert captured light signals into electrical signals, transforming image information into digital information that can be analyzed and calculated by computers. The difference lies in the integration of light and image signals using different semiconductor materials, resulting in significant differences in circuit design and charge transfer methods. Overall, each camera has its own advantages and disadvantages in image acquisition, requiring a comprehensive consideration of the object being inspected and the inspection function. The present invention can select a camera based on the following four factors: First, when capturing images from punched paper tape, vibration can severely interfere with the inspection system. Therefore, the system requires the camera to be fixed; therefore, area array cameras are preferred. Second, due to the precise dimensions of the holes in punched paper tape, the camera requires high resolution and a low signal-to-noise ratio. Third, because the paper tape and the holes are clearly different, and the paper tape is white, a monochrome or color camera is sufficient. Fourth, because hole defect detection in punched paper tape requires speed and efficiency, the camera selected needs to have a high frame rate. By comprehensively considering the hardware platform design principles, system detection requirements, and camera performance, a CCD camera is preferably used as the camera device 2 in the present invention's machine vision-based carrier tape inspection method.

[0051] Specifically, in the aforementioned hardware platform, the lens exists as another core device, and its function is to image the target to be measured onto the sensor by gathering light. The choice of lens directly determines the digital image acquisition effect of the object to be detected and affects the accuracy of the detection system. Therefore, the lens selection of the hardware platform must also be strictly considered in all aspects. In general, the selection of the lens needs to consider various factors such as focal length, field of view, working distance, resolution, distortion, etc. Working distance: The focal length has a working range. When the target object is within the range, the distance between the lens and the target object is the working distance. The aforementioned hardware platform mainly measures the impact of the following two factors on the detection system: First, the hole shape of the punched paper tape is precise and the detection accuracy requirement is high, so the working distance must be close. Second, the influence of distortion exists in any visual detection, so the influence of distortion should be minimized. Therefore, the lens structure 3 is preferably the Computar series fixed-focus lens of the M0814-MP2 model; or a similar lens with similar parameters is also acceptable. The detailed parameters of the preferred lens are as follows Table 1:

[0052] Table 1: Parameters of fixed-focus lenses

[0053]

[0054] Furthermore, in the present invention's machine vision-based carrier tape inspection method, the software system for detecting hole-shaped defects in punched paper tape primarily consists of FlyCapture and VS. FlyCapure is used to acquire images captured by image acquisition and position them to obtain images corresponding to the punched paper tape hole shapes; VS is primarily used to preprocess and detect the acquired punched paper tape hole shape images. Both software components are pre-installed on the computer 4.

[0055] Specifically, after using the aforementioned hardware platform to capture images from the punched paper tape, the captured images must be preprocessed. Image preprocessing primarily enhances the features of the image to be processed, de-emphasizes irrelevant areas, transforms image pixels through a series of algorithms, and corrects the image to achieve the desired image quality, enabling efficient subsequent detection.

[0056] During the image acquisition process of the punched paper tape by the hardware system composed of CCD cameras, the obtained hole-shaped image of the punched paper tape cannot truly reflect all the information of the original hole-shaped image due to the influence of the hardware equipment itself and external environmental factors. Therefore, after the hole-shaped image of the punched paper tape is acquired, it must be preprocessed, otherwise it will directly affect the accuracy of subsequent operations. The main process of hole-shaped image preprocessing is as follows: Figure 3 As shown in the figure, the overall image preprocessing process includes: image grayscale conversion, image filtering, image segmentation, morphological transformation, and edge detection. Image grayscale conversion is performed because the punched tape hole image captured by the hardware system is an RGB image. Directly processing the captured RGB image would increase storage space and algorithm complexity, which would in turn affect overall algorithm processing time and reduce system efficiency. Converting to grayscale maintains the original image brightness and grayscale distribution, while significantly reducing computational complexity and improving system processing efficiency. Image filtering is performed because the punched tape hole image is subject to interference from various factors during acquisition and transmission, resulting in varying degrees of noise. Therefore, image filtering is necessary to minimize image degradation. Image segmentation is performed to separate the foreground and background of the hole image, making the hole features more distinct and facilitating subsequent processing. Morphological transformation is performed because the segmented hole image will produce jagged edges. To minimize the impact of jagged edges on the system, morphological transformations such as erosion and dilation are required. Edge detection is performed to obtain the edge features of each hole shape, and then template matching is performed on the edge features, which greatly reduces the algorithm processing time.

[0057] More specifically, the purpose of grayscale conversion is to adjust the color image, that is, the RGB image, into a grayscale image, thereby reducing the complexity of the algorithm processing. Under the RGB model, there is a special case: R=G=B, in which case the color is only displayed as a grayscale value. Therefore, each pixel in the grayscale image only needs 8 bits to store the grayscale value, ranging from 0 to 255. There are four most commonly used algorithms for grayscale conversion: component method, maximum method, average method and weighted average method. In the present invention, the image information required for the punched paper tape image is only related to the bright spot information in the acquisition system. If the RGB image is used directly for processing, the storage space will be increased and the complexity of the algorithm will be increased. In order to meet the needs of each algorithm module and maximize the efficiency of the detection system, the algorithm first grayscales the original hole-shaped image I(i,j) according to the following formula 1 to obtain the grayscale image G(i,j) of the hole-shaped image:

[0058] Formula 1: G(i,j)=0.30I R (i,j)+0.59I G (i,j)+0.11I B (I,j)

[0059] Where I R (i,j),I G (i,j),I B (I, j) represent the R, G, and B components in the hole image respectively.

[0060] Specifically, during industrial inspection, image acquisition, transmission, and conversion are subject to varying degrees of influence from the equipment itself and the external environment. The resulting images are contaminated with various unpredictable noises, varying only in type and intensity. Therefore, to reduce noise interference, filtered denoising is necessary to improve overall image quality and enhance image characterization. The denoising quality of the filtering algorithm directly impacts subsequent operations such as binarization. Frequency domain and spatial domain methods are the two main approaches for image filtering and denoising. In the frequency domain, noise is most prevalent in high-frequency bands, so various low-pass filters can be used to filter it. In the spatial domain, domain averaging is generally used to reduce noise. Specifically, image filtering primarily utilizes various filters, which can be categorized as linear or nonlinear depending on the filter selection. If linear operations are applied to pixels within a domain, this operation is called linear spatial filtering; otherwise, it is called nonlinear spatial filtering.

[0061] More specifically, median filtering is one of the most widely used nonlinear filtering techniques. It uses the theory of sorting statistics and can effectively suppress image noise. The principle of median filtering is to set a template field and replace the grayscale of the pixel points in the image with the middle value after the grayscale sorting in the corresponding field, so that the grayscale value of the target is closer to the grayscale value it actually represents. The field of median filtering generally adopts square, circular, cross or cross-shaped structures. Its principle is shown in the following formula 2:

[0062] Formula 2:

[0063]

[0064] In the above formula 2, Med is the filter function, M is the set template, and (k, l) is the pixel coordinates within the template area.

[0065] Although median filtering has advantages such as simple algorithm implementation, fast processing speed, strong ability to suppress various noises, and overcoming the detail blurring defects of linear filtering under certain conditions, it suffers from significant damage to details in images with a large number of points, lines, and sharp corners, and its processing effect on punched paper tape hole-shaped images is still less than ideal. Therefore, the present invention improves median filtering for punched paper tape hole-shaped images by using a carrier tape detection method based on machine vision as follows: using a 3x3 template, the two pixels with the smallest grayscale values in the filter window are removed and the median is taken, and then the obtained grayscale value is compared with the original grayscale value, and the decision on whether to replace the grayscale value is made based on the size of the relative difference. Through relevant experimental comparisons, it is concluded that the improved median filter has a better smoothing effect than other filters, effectively removes noise interference, and better preserves the boundary information of the hole shape in the image. At the same time, the algorithm is simple and the processing speed is fast. Therefore, the improved median filter can be used to filter punched paper tape hole-shaped images.

[0066] Specifically, image segmentation involves dividing an image into several regions of specific significance, with no overlap and varying characteristics. The main existing image segmentation methods include threshold segmentation, region segmentation, edge segmentation, and feature-based segmentation. Generally, the most frequently used method for image segmentation is threshold segmentation. This method is suitable for situations where there is a clear contrast in the grayscale values of the target and background regions. It primarily uses the grayscale difference between the target and background regions to define the image into two regions with different grayscale levels: the target region and the background region. An optimal threshold is selected, and the grayscale level of each pixel is compared with the threshold to classify the pixel as foreground or background, ultimately generating a corresponding binary image. Therefore, selecting an appropriate threshold is crucial to the effectiveness of image segmentation. Threshold selection can be broadly categorized into adaptive thresholding, global thresholding, and local thresholding.

[0067] Specifically, morphological transformation primarily involves using digital morphology to analyze and process the shape and structure of an image, refining its features to more accurately reflect the target's true appearance. In a binary image, a collection of black pixels typically represents the target, while the remaining white pixels represent the background. After median filtering and image segmentation, the hole edges of the hole-shaped image appear jagged, inconsistent with the original punched tape hole shape. Therefore, a morphological transformation is required on the binary punched tape hole image, primarily employing the fundamental techniques of erosion and dilation in digital morphology.

[0068] More specifically, the core concept of dilation is to fill in structuring elements. It uses a specific structuring element to continuously detect the target image. The dilation operation merges two closely spaced target regions into one, thus eliminating internal voids within the target region. Dilation also extends the boundaries of the target region outward. The calculation formula is shown in Equation 3 below:

[0069]

[0070] In Equation 3, A represents the image to be processed, and B is the structuring element. The hole-shaped image's boundary jagged edges are expanded outward. Some of the boundary jagged edges are more pronounced, and closely spaced jagged edges are fused regardless of their spacing. While the fused jagged edges significantly improve the edge quality, the larger, more uniform jagged edges are detrimental to subsequent processing. Therefore, the expanded hole-shaped image needs to be eroded.

[0071] More specifically, erosion operates on the opposite principle of dilation, eliminating irrelevant pixels along the target boundary, which in turn compresses the boundary inward. By performing dilation and erosion on the binary hole image, the jagged edges of the hole boundary are significantly improved, with some boundaries smoothed, and the image more closely resembling the true hole shape.

[0072] Specifically, the most important feature in an image is the edge feature, which plays a crucial role in image defect detection. The edge can roughly represent the general outline of the target. It is the dividing line from the foreground of the target to the background, marking the end of the target area and the beginning of the background area. After edge detection, the information inside the target is removed, and only all edge points are displayed on the image, which reduces the amount of data for image defect detection to a certain extent while retaining the structural properties of the target. The basic edge detection algorithms can be divided into two types: search-based and zero-crossing-based. The principle of the search-based edge detection algorithm is to find the maximum and minimum values in the first-order derivative of the image to detect the boundary. Considering that the edge detection object of the punched paper tape is a binary image after morphological transformation, the Canny operator can be used for processing.

[0073] Furthermore, the defects in the hole shape of punched paper tape are mainly reflected in the hole shape image. The main idea is to compare the current image with the standard image obtained through multiple tests. When the difference is greater than the set threshold, the current hole shape image is judged to be unqualified and defective. The abnormality of the hole shape image means that there is a problem with the current section of punched paper tape, which achieves high-precision defect detection of punched paper tape. Moreover, the hole shape of punched paper tape has the characteristics of neat arrangement, simple hole shape, and high dimensional accuracy. Qualified punched paper tape has equally spaced circular holes on the top of the tape and equally spaced square holes on the bottom, arranged in order from left to right. Defect detection mainly detects the hole shape quality of circular holes and square holes. Combining the characteristics of punched paper tape's hole pattern arrangement with the requirements for efficient defect detection, the present invention analyzes and extracts features from large amounts of hole pattern images, deriving the approximate ranges of each hole pattern within the hole pattern image. Based on the edge features of qualified hole patterns, the hole patterns within the hole pattern image are located and divided into regions. Defect detection then only requires the divided regions, significantly reducing algorithm processing time and improving detection efficiency. However, this method is highly targeted and only applies to objects with consistent features, requiring each image to be taken with the object's position fixed. For example, punched paper tape has neatly arranged holes, and the hole positions in each acquired hole pattern image are essentially fixed.

[0074] Furthermore, template matching is the most commonly used method in machine vision-based industrial defect detection. Traditional template matching relies on a specific algorithm to search through an image for objects that match a template image. Compared to the punched paper tape used in this invention, using a traditional template matching algorithm requires classifying a large number of defect images and creating templates for the most likely defects. These defect templates are then matched against the hole-shaped image. If the degree of match exceeds a threshold, the hole-shaped image is considered defective and the defect type matches the template type.

[0075] Therefore, the traditional template matching method has a simple principle and is easy to implement. However, this method is not applicable to the current production needs, mainly in terms of speed and defects. In terms of speed: there are many types of defects in the hole shape of the hole image. In order to accurately match the hole defects, the number of templates required is relatively high. As a result, the number of template matches is increased, which also causes irrelevant matching operations. Even if the image has no defects, all templates are still matched, which greatly increases the processing time of the algorithm. For the existing needs that require fast real-time performance, this algorithm is not applicable. As for defects: there are many types of defects in the hole shape of the hole image, and there is uncertainty in the location where the defects occur. The defect location and defect size are not the same, which brings difficulties to template production. For example, for a rectangular hole shape, the hole shape default defect may appear in the upper left corner or in the lower right corner.

[0076] Furthermore, the edge-based template matching algorithm for hole-shaped images mainly utilizes the regular arrangement of hole shapes in punched paper tape and the consistency of the collected hole-shaped images. The specific steps are as follows:

[0077] 1. Get template: perform edge detection on a preset number of qualified hole-shaped images, analyze their edge features one by one, and obtain a standard qualified hole-shaped image edge map, such as Figure 4 As shown. Since the environment during image acquisition remains unchanged and the image preprocessing algorithm is consistent, the basic image information of qualified hole-shaped images after edge detection is the same, with only slight differences. A large number of qualified hole-shaped images are analyzed to obtain the hole-shaped image with the highest similarity. In the repeated experiment with controlled variables, 1000 qualified hole-shaped images were edge detected, of which 960 images were consistent with the Figure 4 Therefore, we can Figure 4 The content shown is used as a matching template for template matching.

[0078] 2. Combination of region segmentation and edge detection: The template has been completed in the first step, and the next step is to analyze the image to be detected. The analysis object is the hole edge image after preprocessing, and the region segmentation is performed on it. The region segmentation effect of the common hole defect image is shown in the figure below. Figures 5 to 6 shown. Figure 5 and Figure 6 This is a diagram of the most common hole-shaped defects found in punched paper tape. Edge detection is then performed on this defect image to determine the edges. The gray boxes do not actually exist; they are provided for ease of understanding and represent the segmented areas. To reduce irrelevant matches and unnecessary detection time while ensuring accurate detection of various defects, template matching is performed within each segmented area.

[0079] III. Matching: The key concept behind the hole image matching algorithm proposed in this paper is to use a 3x3, 8-neighborhood window to perform bidirectional matching between the template image and the hole image. To reduce irrelevant detections and facilitate subsequent defect classification, the algorithm matches only one hole region at a time. A hole image has eight holes, so a one-way match requires eight region matches, while a two-way match requires 16. Because the matching area is small, multiple region matches can fully meet the time requirements. The specific steps for region matching are as follows:

[0080] S31: Taking the hole-shaped image to be detected as a reference, scan the hole-shaped area. If the pixel point is a boundary point, find out whether there is a boundary point in the 8-neighborhood of the corresponding point in the template image. If there is, define the pixel point as a successful match. Finally, calculate the matching failure value of the hole-shaped area, which is recorded as P1, as shown in Formula 4: P1=S1 / S; where P1 represents the matching failure value of the forward match, S1 represents the sum of the pixel points that failed to match the hole-shaped area, and S represents the sum of the hole-shaped pixel points in the template.

[0081] S32: Taking the template image as the reference, scan the template image to find whether there is a boundary point in the 8-neighborhood of the corresponding point in the image to be detected. If there is, the pixel point is defined as a successful match. Finally, the matching failure value of the hole-shaped area is calculated, which is recorded as P2, as shown in Formula 5: P2=S2 / S; where P2 represents the matching failure value of the reverse matching, S2 represents the sum of the pixel points that failed to match the hole-shaped area, and S represents the sum of the pixel points of the hole in the template.

[0082] S33: Compare P1 and P2, and select the larger matching degree as the matching failure ratio for this match, defined as P. P is then compared with a threshold value p. If the matching degree is greater than the threshold, the hole shape is deemed unqualified; if the matching degree is less than the threshold, the hole shape is deemed qualified. The matching degree P is shown in Equation 6, and the hole shape determination is shown in Equation 7.

[0083] Formula 6:

[0084]

[0085] Formula 7:

[0086]

[0087] Specifically, the reason for using a 3*3 8-neighborhood window is that during the production process of punched paper tape, slight deviations in the hole shape may sometimes occur. These slight deviations are within the error tolerance and will not affect the installation of chip resistors on the punched paper tape. In addition, the hole shape image will be interfered with by the external environment and the device itself during image acquisition. Under these unfavorable conditions, the edge image obtained cannot match the template image 100%. Therefore, in order to achieve the correct detection purpose, a 3*3 window is used for 8-neighborhood matching during the template matching process. The 3*3 matching window is as follows: Figure 7 shown. Figure 7 In the example, X represents the pixel to be matched, and X1 to X8 represent the pixels whose distance from X pixel is less than or equal to the row, that is, the pixels directly connected to X. 8-neighborhood matching is to search X1, X2, X3...X8. As long as there is a pixel value of 255, the pixel X is considered to be matched successfully. The matching process is as follows: Figure 8 and Figure 9 As shown. Among them, Figure 8is the template hole shape diagram; Figure 9 The hole image to be inspected. In both figures, x and y represent the scanning direction, f(x, y) represents an edge pixel with a grayscale value of 255, and S5 represents hole 5 in the hole image. The hole 5 region is scanned from left to right and from top to bottom. When f(x, y) = 255 appears in the template image, a match is performed on the corresponding pixel at (x, y) in the image to be inspected. If a grayscale value of 255 exists between x1 and x8, the pixel at (x, y) is successfully matched.

[0088] More specifically, the bidirectional matching method is used because in the case of unidirectional matching, there are cases where detection fails. Figure 10 and Figure 11 As shown. Figure 10 For the defect shown, if only the template is used as the reference, only the part inside the small circle will not be matched, and the obtained matching failure value P≈2%. The system defaults to the hole shape being qualified, but the hole shape has defects and the detection fails. Figure 11 For the defect shown in the figure, only the image to be tested is used as the reference, and the matching failure value P=0 is obtained. The system assumes that the hole shape is qualified. In fact, the hole shape is default and the detection fails. If two-way matching is used, it can correctly detect whether the hole shape is qualified. The reason for using the matching failure value to judge is: Figure 10 If the match success value is used, then after bidirectional matching, the match success value P is close to 1. The system assumes that the hole shape is qualified and the detection fails. The match success value is shown in Equation 8. S' represents the total number of successfully matched pixels, and S represents the total number of pixels with a grayscale value of 255 in the template. Equation 8: P = S' / S.

[0089] Therefore, the eight-domain bidirectional template matching concept is simple and easy to understand, and the algorithm has been implemented, which can quickly detect hole defects. Because defective holes are extremely rare, running the classification algorithm after determining whether the hole shape is qualified greatly reduces irrelevant operations and improves algorithm processing time.

[0090] Based on the above steps, the following reveals the test results of 100 qualified hole shape images and 100 hole shape defect images, as shown in Table 2:

[0091] Table 2: Hole shape defect detection test results

[0092]

[0093] As shown in Table 2, the algorithm can correctly identify qualified hole images and defective hole images. In terms of detection accuracy, the algorithm is suitable for hole shape detection in punched paper tape.

[0094] In summary, the present invention's carrier tape inspection method based on machine vision first builds a hardware platform and captures images of qualified punched paper tapes and defective punched paper tapes, respectively. Images captured from a large number of qualified paper tapes are used as inspection templates, and the paper tape to be inspected is matched and compared with the inspection template to determine whether the paper tape to be inspected meets production requirements. Specifically, based on the neatly arranged hole shapes of the punched paper tape, hole shape positioning and region segmentation for the hole-shaped paper tape are proposed. Then, based on the hole shape images after region segmentation, an 8-neighborhood bidirectional template matching method is proposed, which significantly improves the efficiency of qualified hole shape detection. Therefore, the present invention's carrier tape inspection method based on machine vision solves the technical problems of slow inspection speed, large human error, and low efficiency that exist in punched paper tape.

[0095] The technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above-mentioned embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0096] The above-described embodiments merely illustrate several implementations of the present invention, and while their descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the patent. It should be noted that a person skilled in the art would be able to make numerous variations and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present invention. Therefore, the scope of protection of the patent for this invention shall be determined by the appended claims.

Claims

1. A carrier tape detection method based on machine vision, characterized in that: It includes the following steps: S1: First, a hardware platform is built, and then the hardware platform is used to collect images of qualified hole shapes of a preset number of punched paper tapes, and the collected qualified hole shape images are pre-processed; after analysis and calculation, a standard hole shape image is obtained as a standard template, and the characteristics of the hole shape in the standard template are obtained; S2: Then, the hole shape positioning region of the punched paper tape hole shape image is segmented according to the obtained hole shape features to obtain the position coordinates of each hole shape; S3: Then, the hardware platform is used to collect hole shape images of the paper tape to be tested, and the images are pre-processed; the obtained hole shape images are matched with the standard template, and the matching degree of each hole shape is obtained; S4: Finally, the matching degree of each hole shape is judged; if the matching degree of all hole shapes is greater than a preset threshold, the paper tape is judged to be a qualified product; otherwise, the paper tape is judged to be defective; The template matching process is as follows: a. Get template: perform edge detection on a preset amount of qualified hole-shaped images, analyze their edge features one by one, and obtain a standard qualified hole-shaped image edge map; b. Combination of region segmentation and edge detection: The analysis object is the hole edge image after preprocessing, and the region segmentation is performed on it, and the edge image is divided into regions; c. Matching: Use a 3*3 8-neighborhood window to perform bidirectional matching between the template image and the hole image; Specifically, the matching steps are as follows: S31: Scan the hole-shaped area based on the hole-shaped image to be detected. If the pixel point is a boundary point, check whether there is a boundary point in the 8-neighborhood of the corresponding point in the template image. If so, the pixel point is defined as a successful match. Finally, calculate the matching failure value of the hole-shaped area, which is recorded as P1: P1=S1 / S; where P1 represents the matching failure value of the forward match, S1 represents the total number of pixels in the hole-shaped area that failed to match, and S represents the total number of pixels in the hole-shaped area in the template. S32: Using the template image as a reference, scan the template image to find whether there is a boundary point in the 8-neighborhood of the corresponding point in the image to be detected. If so, the pixel point is defined as a successful match. Finally, the matching failure value of the hole-shaped area is calculated, which is recorded as P2: P2=S2 / S; where P2 is the matching failure value of the reverse matching, S2 is the sum of the pixel points that failed to match the hole-shaped area, and S is the sum of the hole-shaped pixels in the template. S33: Compare P1 and P2, select the larger matching degree as the matching failure ratio, defined as P; then compare P with the threshold value p; if the matching degree is greater than the threshold value, the hole shape is judged to be unqualified; if the matching degree is less than the threshold value, the hole shape is judged to be qualified.

2. The method for detecting carrier tapes based on machine vision according to claim 1, wherein: The image preprocessing process includes: image grayscale, image filtering, image segmentation, morphological transformation and edge detection.

3. The method for detecting carrier tapes based on machine vision according to claim 2, wherein: During the image grayscale processing, the original hole-shaped image I(i, j) is first grayscaled according to the following formula to obtain the grayscale image G(i, j) of the hole-shaped image: Equation 1: G(i,j)=0.30I R (i,j)+0.59I G (i,j)+0.11I B (I,j); Where I R (i,j),I G (i,j),I B (I, j) represent the R, G, and B components in the hole image respectively.

4. The method for detecting carrier tapes based on machine vision according to claim 3, wherein: In the image filtering step, a 3x3 template is used to remove the two pixels with the smallest grayscale values in the filtering window and take the median. The obtained grayscale value is then compared with the original grayscale value, and the decision on whether to replace the grayscale value is made based on the size of the relative difference.

Images