Method for detecting defects in base of small pluggable light-emitting and transceiver modules

Abstract

The present invention discloses a method for detecting defects in the base of a small pluggable light-emitting / receiving module. The method comprises the following steps: acquiring an image of the base of the light-emitting module; preprocessing the image of the base of the light-emitting module; extracting at least one region of interest from the preprocessed image based on the distribution of defects in the base of the light-emitting module; performing defect detection on the extracted region of interest as a detection block to determine the defect type of the detection block; extracting a feature vector of the detection block using an algorithm corresponding to the defect type; and inputting the feature vector into a trained classification model to determine whether the detection block contains a defect corresponding to the defect type. The technical solution of the present invention improves the accuracy of defect detection in the base of a small pluggable light-emitting / receiving module.

Description

Technical Field

[0001] The present invention relates to the field of defect detection, and in particular to a method, computing device, and readable storage medium for detecting defects in a base of a small pluggable light-emitting and light-emitting module based on machine vision. Background Art

[0002] With the advent of the 5G era, the fiber optic communications market is rapidly expanding, and demand for small form-factor pluggable (SFP) optical transceiver (O / P) housings, one of the most fundamental components in this field, is also rapidly increasing. The production quality of SFP optical module housings not only directly impacts the assembly quality, lifespan, and performance of the SFP, but defects in the SFP housings can lead to improper assembly and unstable mating, impacting the stability of the SFP and potentially affecting the operation of the entire fiber optic communications system. Therefore, strict control of SFP housing production quality is essential.

[0003] Currently, SFP bases are all cast from zinc alloy and undergo processes such as sandblasting, manual polishing, and electroplating to achieve standard-compliant SFP bases. Due to issues with manufacturing processes and inadequate manual handling, the final SFP bases inevitably exhibit surface defects such as particles, excess material, burrs, stuck stones, and missing material. For most SFP base manufacturers, quality inspections typically rely on inspectors using magnifying glasses. However, this manual inspection method presents numerous challenges: SFP bases are small, making manual inspection time-consuming and labor-intensive. Long-term inspections can lead to fatigue and negatively impact the physical and mental health of inspectors. Furthermore, manual inspections require subjective judgment and are significantly affected by factors such as the state of the product and the environment, making it difficult to guarantee accurate and efficient inspections. Furthermore, with labor costs increasing annually, the production costs of SFP bases are also rising. The surface defect detection of SFP base relies solely on manual visual inspection. The detection accuracy and efficiency cannot meet the automated production requirements of SFP base. In order to promote the comprehensive development of intelligent manufacturing in the communications industry, the study of intelligent detection methods for SFP base surface defects has extremely important engineering significance and application value.

[0004] Among existing machine vision-based optical module surface defect detection solutions, Liao Jiale's proposed machine vision-based optical module base surface defect detection method uses morphological methods to eliminate tiny edge features in order to detect defects such as stuck stones, particles, and burrs on the optical module base. Defects are extracted by performing image difference between the morphologically processed image and the original image. For defects that cannot be handled by morphological methods, feature point extraction based on an improved SURF algorithm is used. This is combined with the MSAC-based bidirectional nearest neighbor method to eliminate mismatched points, complete the registration between the detected image and the standard image, and then complete defect extraction based on an improved image difference method. Essentially, this solution uses traditional image processing methods to study some surface defects of optical modules and uses manually designed thresholds to determine defects based on the extracted features. Its detection efficiency and robustness need to be improved. Xu Chenghong also proposed a research algorithm for fiber optic module detection based on stereo vision. To detect deformation defects in the fiber optic module shrapnel area, binocular stereo vision was used for image acquisition. Improved SURF feature point extraction and an improved KD-Tree were used for stereo feature matching to obtain three-dimensional point cloud information of the inspected workpiece. The point cloud difference between the standard workpiece and the inspected workpiece was used to determine whether deformation defects were present. This solution uses binocular vision for fiber optic module shrapnel edge detection, but its detection accuracy is not high and it is not effective for minor defects. Furthermore, the hardware cost required for binocular vision is high, and the calibration process is cumbersome, leaving room for improvement.

[0005] To this end, the present invention provides a solution for detecting defects in a base of a small pluggable light-emitting and light-receiving module based on machine vision to solve the problems existing in the prior art. Summary of the Invention

[0006] To this end, the present invention provides a method, a computing device, and a readable storage medium for detecting defects in a base of a small pluggable light-emitting and light-receiving module based on machine vision, so as to solve or at least alleviate the above problems.

[0007] According to a first aspect of the present invention, a method for detecting defects in the base of a small pluggable optical transceiver module is provided, the method comprising: acquiring an image of the optical module base; preprocessing the image of the optical module base; extracting at least one region of interest from the preprocessed image based on the distribution position of defects in the optical module base; performing defect detection on the extracted region of interest as a detection block to obtain the defect type of the detection block; extracting a feature vector of the detection block using an algorithm corresponding to the defect type; and inputting the feature vector into a trained classification model to determine whether a defect corresponding to the defect type exists in the detection block.

[0008] Optionally, in the method for detecting defects in the base of a small pluggable optical transceiver module according to the present invention, the image of the optical module base is preprocessed, including: segmenting the part including the optical module base as the base image from the image of the optical module base; and performing tilt correction on the base image.

[0009] Optionally, in the method for detecting defects in the base of a small pluggable light-emitting module according to the present invention, the part containing the light module base is segmented out as a base image from the image of the light module base, including: setting a grayscale segmentation threshold to divide the image of the light module base into two parts with a grayscale value greater than the grayscale segmentation threshold and a grayscale value less than or equal to the grayscale segmentation threshold; determining the variance between the part with a grayscale value greater than the grayscale segmentation threshold and the part with a grayscale value less than or equal to the grayscale segmentation threshold; traversing the grayscale segmentation threshold at different values ​​to determine the value of the grayscale segmentation threshold when the variance is maximum; using the value of the grayscale segmentation threshold when the variance is maximum to binarize the image of the light module base; and segmenting the base image from the image of the light module base based on the binarized image.

[0010] Optionally, in the method for detecting defects in the base of a small pluggable light receiving and light emitting module according to the present invention, the base image is segmented from the image of the light module base based on the binarized image, including: removing noise from the binarized image; calculating the coordinate information of the maximum circumscribed matrix of the base image in the image after noise removal; and segmenting the base image from the image of the light module base based on the coordinate information.

[0011] Optionally, in the method for detecting defects in the base of a small pluggable light-emitting module according to the present invention, a base image is segmented from the image of the optical module base based on the binarized image, including: removing the background in the binarized image to obtain the base image.

[0012] Optionally, in the method for detecting defects in the base of a small pluggable light-emitting module according to the present invention, the base image is tilt-corrected, including: binarizing the base image to obtain a binary image of the base; rotating and transforming the binary image of the base at multiple angles; determining the projection length of the binary image of the base in the vertical direction at each angle; using the angle corresponding to the minimum projection length as the correction angle of the base image; and correcting the base image using the correction angle.

[0013] Optionally, in the method for detecting defects in the base of a small pluggable light-emitting and receiving module according to the present invention, the extracted region of interest is used as a detection block for defect detection to obtain the defect type of the detection block, including: if the aspect ratio of the detection block is greater than a predetermined value, then: subdividing the detection block into multiple sub-detection blocks, and determining the sub-detection block with defects therefrom; binarizing the sub-detection block with defects; smoothing the sub-detection block after binarization; filling the holes on the sub-detection block after smoothing to generate a standard template; performing image difference between the sub-detection block after smoothing and the standard template; and judging the defect type of the detection block by the height peak of the connected area in the result of the image difference. Determine whether the defect type is a material shortage defect, and determine whether the defect type is a particle defect through the length, height and area of ​​the white connected area; if the aspect ratio of the detection block is less than or equal to the predetermined value, then: perform binarization on the detection block; determine whether the defect type is a multi-material defect through the ratio of the area of ​​white pixels in the detection block after binarization to the area of ​​its minimum circumscribed matrix; determine whether the defect type is a stone defect through the ratio of the area of ​​white pixels in the detection block after binarization to the area of ​​the entire detection block; perform local binary pattern processing on the detection block; determine whether the defect type is a burr defect through the grayscale statistical distribution of the image after local binary pattern processing.

[0014] Optionally, in the method for detecting defects in the base of a small pluggable light-emitting module according to the present invention, an algorithm corresponding to the defect type is used to extract a feature vector of the detection block, including: if the defect type is a material shortage defect, constructing a feature vector of the detection block based on the height peak of the connected area, the ratio of the height peak to the height mean of the connected area, and the area of ​​the connected area.

[0015] Optionally, in the method for detecting defects in the base of a small pluggable light-emitting module according to the present invention, an algorithm corresponding to the defect type is used to extract a feature vector of the detection block, including: if the defect type is a particle defect, constructing a feature vector of the detection block based on the height peak of the white connected area, the ratio of the height peak to the height mean of the white connected area, and the area of ​​the white connected area.

[0016] Optionally, in the method for detecting defects in the base of a small pluggable light-emitting module according to the present invention, an algorithm corresponding to the defect type is used to extract a feature vector of the detection block, including: if the defect type is a multi-material defect, constructing a feature vector of the detection block based on the area of ​​the white pixel point and the ratio of the area of ​​the white pixel point to its minimum external matrix area.

[0017] Optionally, in the method for detecting defects in the base of a small pluggable light-emitting module according to the present invention, an algorithm corresponding to the defect type is used to extract the feature vector of the detection block, including: if the defect type is a stone defect, then constructing a feature vector of the detection block based on the area of ​​white pixels and the number of connected areas composed of white pixels.

[0018] Optionally, in the method for detecting defects in the base of a small pluggable light-emitting module according to the present invention, an algorithm corresponding to the defect type is used to extract a feature vector of the detection block, including: if the defect type is a burr defect, then based on the number of pixels of each grayscale value in the image processed by the local binary pattern, constructing a feature vector of the detection block.

[0019] Optionally, the method for detecting defects in the base of a small pluggable optical transceiver module according to the present invention further includes: if the detection block has defects and is subdivided into multiple sub-detection blocks, the defective sub-detection block is displayed on the image of the optical module base; if the detection block has defects and is not subdivided into multiple sub-detection blocks, the detection block is displayed on the image of the optical module base.

[0020] Optionally, in the method for detecting defects of the base of a small pluggable light-emitting module according to the present invention, an image of the light module base is obtained, including: constructing an imaging platform, the imaging platform including an industrial camera, a ring light source, a detection tray and a guide rail, the industrial camera is equipped with a telecentric lens, and the ring light source is used to provide a light source for the industrial camera when capturing images; moving the detection tray through the guide rail so that the light module base on the detection tray is located at the collection position of the industrial camera; and collecting the image of the light module base at the collection position through the industrial camera.

[0021] Optionally, in the method for detecting defects in the base of a small pluggable light emitting and receiving module according to the present invention, the trained classification model is a support vector machine, and the kernel function of the support vector machine is a Gaussian kernel function.

[0022] According to a third aspect of the present invention, there is provided a computing device comprising: at least one processor; and a memory storing program instructions, wherein the program instructions are configured to be suitable for execution by the at least one processor, and the program instructions include instructions for executing the method described above.

[0023] According to a fourth aspect of the present invention, a readable storage medium storing program instructions is provided. When the program instructions are read and executed by a computing device, the computing device executes the method described above.

[0024] According to the technical solution of the present invention, by preprocessing the image of the optical module base, the portion of the optical module base image containing the optical module base is cropped from the background, and the tilted optical module base is corrected. Since it is difficult to achieve a completely horizontal placement of the optical module base in reality, the impact of the tilted placement of the optical module base can be avoided by correcting the image of the optical module base. The region of interest is extracted based on the defect distribution of the optical module base. Based on the unique defect distribution of the optical module base, the location where the defect may occur is used as the region of interest, which improves detection efficiency and saves detection time. By performing defect detection on the detection block in the region of interest, the defect type of the detection block can be obtained. The feature vector of the detection block is extracted using an algorithm corresponding to the defect type, which is highly targeted and has a high detection accuracy. The classification model is used to determine whether the detection block has defects, further improving the detection accuracy and effectively avoiding the occurrence of misjudgments.

[0025] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are specifically listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] To achieve the above and related purposes, certain illustrative aspects are described herein in conjunction with the following description and accompanying drawings, which indicate various ways in which the principles disclosed herein may be practiced, and all aspects and their equivalents are intended to fall within the scope of the claimed subject matter. The above and other objects, features, and advantages of the present disclosure will become more apparent by reading the following detailed description in conjunction with the accompanying drawings. Throughout this disclosure, the same reference numerals generally refer to the same parts or elements.

[0027] Figure 1 A block diagram illustrating the physical components of computing device 100 is shown;

[0028] Figure 2 A schematic diagram of a defect in the optical module base is shown;

[0029] Figure 3 A flowchart of a method 300 for detecting defects in a base of a small pluggable light emitting and receiving module based on machine vision according to an embodiment of the present invention is shown;

[0030] Figure 4 shows a schematic diagram of a detection tray according to one embodiment of the present invention;

[0031] Figure 5 A schematic diagram of a binarized image according to an embodiment of the present invention is shown;

[0032] Figure 6A schematic diagram of image segmentation of an optical module base according to an embodiment of the present invention is shown;

[0033] Figure 7 A schematic diagram showing an inclination angle and a vertical projection length thereof according to an embodiment of the present invention is shown;

[0034] Figure 8 A schematic diagram illustrating tilt correction of an optical module base image based on minimum projection length according to an embodiment of the present invention is shown;

[0035] Figure 9 FIG2 shows a schematic diagram of detection block division according to an embodiment of the present invention;

[0036] Figure 10 A schematic diagram showing feature extraction of a material shortage defect according to an embodiment of the present invention is shown;

[0037] Figure 11 A schematic diagram showing feature extraction of particle defects according to one embodiment of the present invention is shown;

[0038] Figure 12 A schematic diagram showing feature extraction of multi-material defects according to one embodiment of the present invention is shown;

[0039] Figure 13 A schematic diagram showing feature extraction of a stone defect according to an embodiment of the present invention is shown;

[0040] Figure 14 A schematic diagram showing feature extraction of burr defects according to an embodiment of the present invention is shown;

[0041] Figure 15 A schematic diagram of a two-dimensional linear support vector machine according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0042] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0043] Figure 1A block diagram of the physical components (i.e., hardware) of a computing device 100 is shown. In a basic configuration, the computing device 100 includes at least one processing unit 102 and a system memory 104. According to one aspect, depending on the configuration and type of the computing device, the processing unit 102 can be implemented as a processor. The system memory 104 includes, but is not limited to, volatile storage (e.g., random access memory), non-volatile storage (e.g., read-only memory), flash memory, or any combination of such memories. According to one aspect, the system memory 104 includes an operating system 105 and a program module 106, the program module 106 including program instructions 120 for executing the method of detecting defects in the base of a small pluggable light-emitting module of the present invention.

[0044] According to one aspect, operating system 105 is suitable for controlling the operation of computing device 100, for example. Furthermore, examples may be practiced in conjunction with graphics libraries, other operating systems, or any other application programs, and are not limited to any particular application or system. Figure 1 This basic configuration is illustrated in FIG by those components within dashed line 108. According to one aspect, computing device 100 has additional features or functionality. For example, according to one aspect, computing device 100 includes additional data storage devices (removable and / or non-removable), such as magnetic disks, optical disks, or tapes. Such additional storage Figure 1 1 is illustrated by a removable storage device 109 and a non-removable storage device 110.

[0045] As stated above, according to one aspect, a program module 106 is stored in the system memory 104. According to one aspect, the program module 106 may include one or more application programs, and the present invention is not limited to the type of application program. For example, the application program may include an email and contact application program, a word processing application program, a spreadsheet application program, a database application program, a slide presentation application program, a drawing or computer-aided application program, a web browser application program, etc.

[0046] According to one aspect, examples may be practiced on a circuit comprising discrete electronic components, a packaged or integrated electronic chip containing logic gates, a circuit utilizing a microprocessor, or a single chip containing electronic components or a microprocessor. Figure 1Each or many components shown in can be integrated into a system on a chip (SOC) on a single integrated circuit to practice examples. According to one aspect, such an SOC device may include one or more processing units, a graphics unit, a communication unit, a system virtualization unit, and various application functions, all of which are integrated (or "burned") onto a chip substrate as a single integrated circuit. When operated via SOC, the functions described in this article can be operated via dedicated logic integrated with other components of the computing device 100 on a single integrated circuit (chip). Embodiments of the present invention can also be practiced using other technologies capable of performing logical operations (such as AND, OR, and NOT), including but not limited to mechanical, optical, fluid, and quantum technologies. In addition, embodiments of the present invention can be practiced in a general-purpose computer or in any other circuit or system.

[0047] According to one aspect, the computing device 100 may also have one or more input devices 112, such as a keyboard, a mouse, a pen, a voice input device, a touch input device, etc. It may also include an output device 114, such as a display, a speaker, a printer, etc. The aforementioned devices are examples, and other devices may also be used. The computing device 100 may include one or more communication connections 116 that allow communication with other computing devices 118. Examples of suitable communication connections 116 include, but are not limited to: RF transmitter, receiver, and / or transceiver circuitry; Universal Serial Bus (USB), parallel, and / or serial ports.

[0048] As used herein, the term computer-readable medium includes computer storage media. Computer storage media can include volatile and non-volatile, removable and non-removable media implemented with any method or technology for storing information (e.g., computer-readable instructions, data structures, or program modules). System memory 104, removable storage device 109, and non-removable storage device 110 are all examples of computer storage media (i.e., memory storage). Computer storage media can include random access memory (RAM), read-only memory (ROM), electrically erasable read-only memory (EEPROM), flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical storage, cassette tape, magnetic tape, disk storage or other magnetic storage device, or any other product that can be used to store information and can be accessed by computer device 100. According to one aspect, any such computer storage medium can be a part of computing device 100. Computer storage media does not include carrier waves or other propagated data signals.

[0049] According to one aspect, communication media is implemented by computer-readable instructions, data structures, program modules, or other data in a modulated data signal (e.g., a carrier wave or other transport mechanism), and includes any information delivery media. According to one aspect, the term "modulated data signal" describes a signal that has one or more characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, radio frequency (RF), infrared, and other wireless media.

[0050] In one embodiment of the present invention, a computing device 100 includes one or more processors and one or more readable storage media storing program instructions. When the program instructions are configured to be executed by the one or more processors, the computing device executes the method for detecting defects in the base of a small form-factor pluggable light-emitting and receiving module according to an embodiment of the present invention.

[0051] The present invention relates to a small pluggable optical module base. The optical module base and SFP base referred to in the present invention both refer to a small pluggable optical module base. Common defects of the base include: missing material, particles, stuck stones, burrs, and excess material. The SFP base is different from other types of components. The five types of defects are distributed in different areas of the SFP base and have different characteristics. Among them, the missing material defect is an irreparable defect, and the entire SFP base must be recast to recover the raw materials; while the particles, excess material, burrs, and stuck stone defects are surface defects that can be subjected to secondary processing and then continue to be sent to the electroplating process.

[0052] The following is an analysis based on the causes and characteristics of these five types of defects:

[0053] (1) Material shortage: Since the SFP base is a metal casting with many sharp edges, it is very easy to scratch other SFP bases during production and transportation, resulting in material shortage defects. Material shortage defects are often located in the upper and lower edge areas of the SFP base, and the edges will be concave inward. Figure 2 The figure shows a defect diagram of the optical module base. Figure 2 As shown, 210 is a material shortage defect. The normal edge presents a high grayscale value on the image, while the area where the material shortage defect exists is close to black and has a lower grayscale value.

[0054] (2) Particles: The SFP base requires sandblasting during production. The stone particles used in sandblasting may become lodged in the SFP base. If not detected and cleaned, irreversible damage will occur after the electroplating process. 220 is a particle defect, where small particles remain in the grooves, forming a particle defect. Normal grooves have smooth surfaces and therefore appear as high-brightness long rectangles in the image. However, the particles used in sandblasting have a rough surface and are darker in color, resulting in a lower grayscale value.

[0055] (3) Stone stuck: Similar to the particle defect, a larger sandblasted stone is left in the LC slot, forming a stone stuck defect. 230 is a stone stuck defect. The LC slot and the base surface are not in the same plane, and there is a random noise texture on the surface. The grayscale value of the stone stuck area is lower than that of a normal workpiece.

[0056] (4) Burrs: The casting process accuracy during production can easily lead to burrs in the angled area. The burr defects are mainly distributed at the four right angles of the LC slot, with 240 being a burr defect. The workpiece with burr defects has rounded corners in areas that should be right angles, and the edges are more rounded than those of normal workpieces.

[0057] (5) Over-material: Similar to burr defects, over-material defects often occur in the SFP base end area. 250 is a over-material defect. The area with this defect will have extra material, the grayscale value will change, and the originally relatively smooth surface will have more edges.

[0058] Figure 3 The flowchart of a method 300 for detecting defects in a base of a small pluggable light-emitting and receiving module based on machine vision according to an embodiment of the present invention is shown. The method 300 is suitable for execution in a computing device (such as the aforementioned computing device 100). Figure 3 As shown, method 300 begins at step 310 .

[0059] In step 310 , an image of the optical module base is acquired.

[0060] According to an embodiment of the present invention, an imaging platform is constructed, wherein the constructed imaging platform includes an industrial camera, a ring light source, a detection tray and a guide rail. The industrial camera can be a CCD industrial camera or a CMOS industrial camera. The CCD working camera has great disadvantages compared to the CMOS industrial camera in terms of cost, power consumption, acquisition speed and integration. Therefore, the solution of the present invention selects a CMOS industrial camera. Based on the above-mentioned characteristic analysis of common defects in the optical module base, the defect characteristics can be extracted based on the edge information and grayscale information of the image. Therefore, a single-channel camera is more suitable than a color camera. At the same time, using a single-channel camera to collect grayscale images can increase the equipment acquisition efficiency and equipment stability. Pixel resolution is an important consideration for the specific selection of industrial cameras. Generally, the higher the resolution, the clearer the image and the higher the accuracy of defect detection. However, too large a resolution will reduce the speed of the image processing algorithm, resulting in a decrease in detection speed. Optionally, considering multiple factors such as sensor type, pixel resolution, and defect characteristics of the optical module base, the camera model STC-MCS2041POE is selected.

[0061] In order to improve the overall performance of the machine vision system, it is necessary to select a suitable lens. The type of lens can be a fixed-focus lens or a telecentric lens. Compared with the fixed-focus lens, the telecentric lens adopts a parallel optical path design, its imaging distortion is smaller, the depth of field has a smaller impact on the magnification, and there is no parallax. Moreover, since the defects of the optical module base of the detection object of the present invention are not distributed in the same plane, and compared with the fixed-focus lens, the telecentric lens has the characteristics of ultra-low distortion, high resolution, and ultra-wide depth of field, it can meet the needs of collecting clear images of the surface of the optical module base and complete subsequent detection tasks. At the same time, it can adapt to harsh industrial production environments and improve the robustness of the system. Therefore, the present invention selects a telecentric lens and equips the telecentric lens on the industrial camera. To facilitate loading and unloading of optical module base inspection components, the lens's working distance must be greater than 200mm. If the lens's camera mount is incompatible with the camera, an adapter is required, which typically affects imaging quality. Therefore, a C-mount lens is preferred. The optical module base measures 55.5mm by 13.5mm, a significant difference between its length and width. Therefore, considering imaging both optical module bases simultaneously and taking into account guide rail movement errors, the selected lens's field of view should be no less than 60mm by 40mm. Furthermore, the optical module base has different surfaces to inspect, and considering its 10mm height, the lens's depth of field should be no less than 10mm. Optionally, the telecentric lens model OPT-TS0206-228-AL, which meets these requirements, can be selected.

[0062] In machine vision systems, light sources illuminate the object being inspected. To reduce the complexity of subsequent algorithms and improve image quality, the features of the object being inspected must be highlighted as much as possible during illumination. This requires selecting an appropriate light source, including fluorescent lamps, halogen lamps, incandescent lamps, and LEDs. Compared to other light sources, LEDs offer advantages such as stable illumination, adjustable intensity, high cost-effectiveness for small-area illumination, and customizable light source shape and size. The imaging system of the optical module base is designed to operate under real-world working conditions, so the selected light source needs to be stable and provide sustainable illumination. Furthermore, the optical module base is relatively small, requiring a small illumination area. Therefore, LEDs are suitable for imaging illumination of the optical module base. LED light sources can be divided into ring light sources, coaxial light sources, back light sources and spherical integrating light sources, etc. The specific light source shape is selected according to the size of the workpiece to be inspected, the characteristics of the surface defects and the working distance. The surface defects of the optical module base are widely distributed, among which the stone defect and other defects are not on the same surface, so the coaxial light source is not applicable; although the material shortage defect is distributed in the edge area of ​​the optical module base, the optical module base has a certain thickness, and the back light source cannot highlight its defect characteristics, so the back light source is not used; the spherical integrating light source can perform uniform light beam illumination, which can highlight the uneven surface, but the volume of the spherical integrating light source is generally large, which is not conducive to the loading and unloading of the optical module base, so the spherical integrating light source is not suitable for the solution of the present invention; the ring light source can also uniformly illuminate the workpiece to be inspected, and is smaller in volume than the spherical integrating light source, can highlight the edge and height information, and meets the imaging requirements of the optical module base, so the ring light source can be selected to provide light source for the industrial camera when collecting images.

[0063] To improve the efficiency of surface defect detection and achieve rapid, automated imaging, the imaging platform of the present invention requires a pallet based on the structure of the inspection object and its surface defect characteristics, as well as a mobile device to achieve automated imaging of the optical module base within the same pallet. To facilitate pallet processing, the groove on the pallet that supports the optical module base has rounded corners. The groove is designed to be 17.5mm x 58mm, slightly larger than the optical module base to facilitate loading and unloading operations. Figure 4 FIG. 1 shows a schematic diagram of a detection tray according to an embodiment of the present invention. Figure 4As shown, the inspection tray can meet the needs of batch imaging of optical module bases. The same tray can hold 40 optical module base images and simultaneously capture two optical module base images, which is divided into 20 imaging sessions. This tray can keep the inspection object in a relatively stable position, maximizing the level of the inspected parts, reducing the difficulty of image acquisition and subsequent detection algorithm implementation, and increasing the speed of image acquisition. Of course, the spacing between the optical module bases placed in the same tray can be adjusted, so the same tray can also hold other numbers of optical module bases, and the number of optical module bases included in the collected optical module base images can also be one or more.

[0064] According to an embodiment of the present invention, a test tray is moved by guide rails so that the optical module base on the test tray is positioned at a capture position for an industrial camera. Optionally, the guide rails are cross-shaped, enabling the test tray to automatically move to an appropriate capture position so that the industrial camera can capture images of the optical module base at that capture position. The cross-shaped guide rails enable vertical movement within a plane, driving the tray along a predetermined image capture path, thereby achieving high-quality imaging of the surfaces of the batch of optical module bases within the tray.

[0065] In step 320 , the image of the optical module base is pre-processed.

[0066] Since the input original image to be detected (i.e., the image of the optical module base obtained in step 310) may include one or more optical module bases, wherein each optical module base is not placed in a standard horizontal position, taking the obtained image of the optical module base including two optical module bases as an example for explanation, it is necessary to segment the input original image to be detected into two separate images, and perform tilt correction on each of them to obtain a standard single optical module base image, thereby increasing the robustness of the solution. According to an embodiment of the present invention, first, the portion containing the optical module base is segmented from the image of the optical module base as a base image. Then, the base image is tilt-corrected.

[0067] Since the optical module base differs significantly from the background information in the grayscale domain, it is considered to extract the optical module base based on grayscale threshold segmentation. Specifically, a grayscale segmentation threshold is set to divide the image of the optical module base into two parts: one with a grayscale value greater than the grayscale segmentation threshold and the other with a grayscale value less than or equal to the grayscale segmentation threshold. Optionally, the grayscale segmentation threshold is set to t, where the proportion of pixels with grayscale values ​​greater than t in the entire image is w1, and their grayscale mean is u1; the proportion of pixels with grayscale values ​​less than or equal to t in the entire image is w0, and their grayscale mean is u0; the total grayscale mean u can be obtained as:

[0068] u=w1×u1+w0×u0

[0069] The variance between the portion having a grayscale value greater than the grayscale segmentation threshold and the portion having a grayscale value less than or equal to the grayscale segmentation threshold is determined.

[0070] σ 2 =w0×(u0-u)+w1(u1-u)

[0071] Combining the above two equations, we can get:

[0072] σ 2 =w0×w1(u1-u0) 2

[0073] The grayscale segmentation threshold is iterated over different values ​​to determine the grayscale segmentation threshold value at which the variance is maximized. The threshold at which the maximum variance is calculated by iterating over all grayscale values ​​is used as the grayscale segmentation threshold, where the difference between the two parts is maximized. The grayscale segmentation scheme of the present invention achieves good segmentation results for images with a small difference in foreground and background area, and is not affected by changes in overall brightness.

[0074] The image of the optical module base is binarized using the grayscale segmentation threshold value when the variance is maximum.

[0075] According to one embodiment of the present invention, the background of the binarized image is removed to obtain a base image. In the binarized image, the detection tray is processed as a background in the image and is rendered black. Therefore, the base image can be obtained by removing the black background portion of the binarized image.

[0076] Although the grayscale segmentation scheme can remove most of the background information in the original image, some impurities on the detection tray in the background are also divided into targets, and there are some holes in the segmentation map. Figure 5 FIG. 1 shows a schematic diagram of a binary image according to an embodiment of the present invention. Figure 5 As shown, after the input image is binarized, some impurities may still be present on the inspection tray, and the optical module base may also have some holes. Therefore, according to another embodiment of the present invention, the image is further processed. Specifically, the binarized image is subjected to noise removal. In the denoised image, the coordinate information of the maximum circumscribed matrix of the base image is calculated. Based on the obtained coordinate information, the base image is segmented from the image of the optical module base. Optionally, the noise in the binarized image is removed through erosion, dilation, opening, or closing operations, and the holes in the white areas are filled. Figure 6 FIG. 1 shows a schematic diagram of image segmentation of an optical module base according to an embodiment of the present invention. Figure 6As shown, in the image after noise removal, the coordinate information of the maximum circumscribed matrix of the base image in the entire image is calculated. Based on the obtained coordinate information, the base image is segmented from the image of the optical module base.

[0077] Through the above processing steps, a single optical module base image is obtained. However, each image has a certain degree of tilt, so tilt correction is required. Alternatively, Hough transforms can be used for tilt correction. However, when using Hough transforms for line detection, it is necessary to traverse all points in the image and convert them to Hough space, which is time-consuming. Therefore, to efficiently perform tilt correction on each optical module base image, a tilt correction method based on minimizing projection length can be used. Since the binary image of the optical module base is relatively regular, its vertical projection length can, to a certain extent, indicate whether it is horizontally positioned.

[0078] According to an embodiment of the present invention, a base image is binarized to obtain a binary image of the base. The binary image of the base is then rotated at multiple angles. At each angle, the vertical projection length of the binary image of the base is determined. The angle corresponding to the minimum projection length is used as a correction angle for the base image. The base image is then corrected using the correction angle.

[0079] Optionally, since the present invention adopts a detection tray to carry the optical module base, its tilt angle will not exceed 5°. Figure 7 The diagram shows the corresponding relationship between the tilt angle and its vertical projection length according to an embodiment of the present invention. The binary image of the horizontally placed optical module base is rotated within [-5:0.1:5], and its corresponding vertical projection length is as follows: Figure 7 As shown, when the rotation angle is 0°, that is, the optical module base is placed horizontally, the corresponding projection length is the smallest. Therefore, whether it is a tilt correction angle can be determined by calculating the projection length of the binary image of the optical module base in the vertical direction.

[0080] Specifically, the binary image is rotated within the range [-5, 5] with a step size of 0.1°. All the transformed images are vertically accumulated to obtain the corresponding one-dimensional matrix A. The length L of matrix A at each angle is compared. The rotation angle corresponding to the minimum L is the tilt correction angle.

[0081] Figure 8A schematic diagram illustrates tilt correction of an optical module base image based on a minimum projection length according to one embodiment of the present invention. After performing tilt correction on the single optical module base image (i.e., the base image) obtained above, some background information around the corrected image needs to be removed. By finding the coordinates of the minimum circumscribed matrix of the connected region of the binary image and performing further cropping, a standard image of the single optical module base can be obtained as the tilt-corrected image.

[0082] In step 330, at least one region of interest (ROI) is extracted from the pre-processed image according to the defect distribution position of the optical module base.

[0083] Since the characteristics and locations of the five defects on the surface of the optical module base are different, multiple feature extraction schemes are required to complete the detection task. Therefore, corresponding ROI extraction is performed according to the distribution location of each defect, and the obtained preprocessed image is divided into different detection blocks. Figure 9 FIG. 1 shows a schematic diagram of detection block division according to an embodiment of the present invention. Figure 9 As shown in the figure, the preprocessed image is divided into five detection blocks based on the location distribution and characteristics of each defect and prior information, corresponding to the five defect classifications on the optical module base surface. Among them, 910 corresponds to two detection blocks including upper and lower edges for detecting missing material; 920 corresponds to two detection blocks including upper and lower grooves for detecting particle defects; 930 corresponds to the detection block including the head portion for detecting multiple material defects; 940 corresponds to the detection block including two LC grooves for detecting stone defects; and 950 corresponds to the four detection blocks located at the four right angles above the LC for detecting burr defects.

[0084] In step 340 , the extracted region of interest is used as a detection block for defect detection to obtain the defect type of the detection block.

[0085] According to an embodiment of the present invention, if the aspect ratio of the detection block is greater than a predetermined value, the detection block is subdivided into a plurality of sub-detection blocks, and the sub-detection blocks with defects are determined therefrom, that is, the long strip detection blocks 910 and 920 are subdivided. The sub-detection blocks with defects are binarized. The sub-detection blocks after binarization are smoothed. The holes on the sub-detection blocks after smoothing are filled to generate a standard template. The sub-detection blocks after smoothing are image-differentiated with the standard template. In the result of the image difference, the height peak of the connected area is used to determine whether the defect type is a material shortage defect, and the length, height and area of ​​the white connected area are used to determine whether the defect type is a particle defect. The predetermined value can be set according to the actual situation, for example, it can be set to a value greater than 1.5, or it can be set to a value greater than 2.

[0086] Since the aspect ratio of the missing material detection block is too large, it is first further subdivided, and the 910 and 920 detection blocks are subdivided. For example, they are divided into 10 sub-detection blocks of the same size. At the same time, in order to ensure that cropping does not affect the extraction of the defective area, there are overlapping areas between the 10 subdivided sub-detection blocks. Figure 10 The figure shows a schematic diagram of feature extraction of a material shortage defect according to an embodiment of the present invention. Since the edge of the detection block with a material shortage defect is concave and has a black shadow, unlike the normal edge which has a high gray value on the image, the gray value of the material shortage area is low. Figure 10 As shown, 1010 is the original detection block image, and 1020 is the sub-detection block after subdivision. Using the grayscale segmentation threshold value at the maximum variance, the sub-detection block is binarized. The defective area with a low grayscale value is classified as background, resulting in the binarized sub-detection block image shown in 1030. An opening operation is used to remove small burrs at the edges, and a closing operation is used to fill holes. The final smoothed image is shown in 1040. The closing operation in morphology is used to fill holes and generate a standard template (shown in 1050). The specific operation is as follows: the images of the binarized sub-detection blocks are accumulated horizontally, and the maximum value N after accumulation is used to construct a horizontal structure factor of [1×N]. The images of the binarized sub-detection blocks are opened to generate a standard template. The image of the smoothed sub-detection block is then subtracted from the standard template to extract defect features. 1060 shows the result of the image subtraction. In addition to the material shortage features to be extracted, there are also some discontinuous line segments. This is because the edge of the optical module base is not completely smooth and does not fall within the defect range. The difference between these noise features and defect features lies in the difference in the height of the white pixel values. Therefore, the peak height of the connected area is used as the basis for determining whether a material shortage defect exists. Optionally, the peak height of the connected area is compared with a set first predetermined value. If it is greater than the first predetermined value, it indicates a material shortage defect; otherwise, there is no material shortage defect. The selection of the first predetermined value can be set according to actual conditions.

[0087] Figure 11 FIG. 1 shows a schematic diagram of feature extraction of particle defects according to an embodiment of the present invention. Figure 11As shown, 1110 is the original detection block image, and 1120 is the subdivided sub-detection block. Using the grayscale segmentation threshold value at the maximum variance, the sub-detection block is binarized, resulting in the binarized sub-detection block image shown in 1130. An opening operation is used to remove fine edge burrs, and a closing operation is used to fill holes. The final smoothed image is shown in 1140. A closing operation in morphology is used to fill holes and generate a standard template (shown in 1050). Specifically, the binarized sub-detection block images are horizontally accumulated. The maximum value N after the accumulation is used to construct a horizontal structure factor of [1×N]. An opening operation is then performed on the binarized sub-detection block images to generate the standard template. Extracting particle defect regions does not require image registration; image difference can be used directly for defect extraction. Particle defect features are similar to those of material shortage defects, so the feature extraction method for material shortage defects is used to extract particle defects. Due to the shadows on the upper and lower edges of the fine groove, there are also some discontinuous line segments in the defect feature map extracted by the final differential operation. The defect feature results are extracted by performing image differential between the smoothed image of the sub-detection block and the standard template. 1160 shows the result of the image differential. Noise features and defect features are easy to distinguish. Image information such as the length, height, and area of ​​the white connected area can be used as a basis for determining whether a particle defect exists. Optionally, thresholds for information such as the length, height, and area of ​​the white connected area can be set according to actual conditions. For example, when the length of the white connected area is greater than a second predetermined value, the height is greater than a third predetermined value, and / or the area is greater than a fourth predetermined value, it is determined that a particle defect exists. The specific values ​​of the second predetermined value, the third predetermined value, and the fourth predetermined value can be set according to actual conditions.

[0088] According to an embodiment of the present invention, if the aspect ratio of the detection block is less than or equal to a predetermined value, the detection block is binarized. The ratio of the area of ​​white pixels in the binarized detection block to the area of ​​its minimum circumscribed matrix is ​​used to determine whether the defect type is a multi-material defect. The ratio of the area of ​​white pixels in the binarized detection block to the area of ​​the entire detection block is used to determine whether the defect type is a stone defect. Local binary pattern processing is performed on the detection block. The grayscale statistical distribution of the image after local binary pattern processing is used to determine whether the defect type is a burr defect.

[0089] Figure 12 FIG. 1 shows a schematic diagram of feature extraction of multi-material defects according to an embodiment of the present invention. Figure 12As shown, 1210 is the original image of a detection block with multiple material defects. After binarization, the image of the detection block after binarization is obtained, as shown in 1220. 1230 is the original image of a detection block without defects. The grayscale values ​​of the multiple material defect areas in the detection block differ from those of normal samples, so threshold segmentation can be used to extract the defect features. 1240 is the image of the detection block after binarization of the detection block without defects. The areas selected in 1220 and 1240 are the minimum circumscribed matrix of the connected areas. It can be seen that the white areas in the binary image of the defect-free multi-material detection block almost fill the entire minimum circumscribed matrix, while the defective samples have large holes. Therefore, the ratio of the white pixel area in the binary image of the multi-material detection block to the area of ​​its minimum circumscribed matrix can be used as a basis for determining whether it contains multiple material defects. Optionally, if the ratio of the white pixel area of ​​the white connected area in the binarized detection block to the area of ​​its minimum circumscribed matrix is ​​less than a fifth predetermined value, the detection block is determined to have multiple material defects. Otherwise, the detection block is determined to not have multiple material defects. The specific value of the fifth predetermined value can be set according to actual conditions, and the present invention does not impose any limitation on this.

[0090] The present invention extracts a region of interest from a preprocessed image and extracts a stone defect detection block, so image registration is no longer necessary. Figure 13 A schematic diagram illustrates feature extraction for a sticking stone defect according to one embodiment of the present invention. 1310 represents a defective detection block, and 1330 represents a non-defective detection block. The grayscale value of the sticking stone defect region in the detection block differs from that of a normal sample, and the sticking stone defect region accounts for a significant portion of the detection block. Therefore, threshold segmentation can be used to extract its features, eliminating the need for image differencing. The detection blocks are binarized. 1320 represents the binary image of the defective sticking stone detection block, and 1340 represents the binary image of the non-defective sticking stone detection block. It can be seen that the total number of black pixels in the detection block with a sticking stone defect far exceeds that in a normal sample. Therefore, the ratio of the white pixel area in the binarized detection block image to the entire detection block area can be used to determine whether a sticking stone defect exists. Optionally, if the ratio of the white pixel area in the binarized detection block to the entire detection block area is less than a sixth predetermined value, the detection block is determined to have a sticking stone defect; otherwise, the detection block is determined to have no sticking stone defect. The specific value of the sixth predetermined value can be set based on actual conditions and is not limited by the present invention.

[0091] Since the burr defects on the base of the optical module involved in the present invention are at the four right angles of the LC slot, the gap between the background and the defect is not obvious, and the effect of extracting the burr defect features by binarization processing based on threshold segmentation is not ideal. The present invention determines whether the defect type is a burr defect by performing local binary pattern (LBP) processing on the detection block. LBP is an operator used to describe local texture features and can be used to extract local texture features of an image. Its essence is to replace the value of a pixel with its gradient information about the surrounding pixels, and compare the center pixel value with the eight pixels of its eight surrounding neighborhoods as a threshold. If it is greater than the center element, it is marked as 1, otherwise it is 0. The resulting 8-bit binary number is converted to decimal, which is the LBP code. The LBP image can be drawn by replacing the center pixel with the LBP value. Its coding calculation formula is as follows:

[0092]

[0093]

[0094] Among them, i c is the center gray value, i p is the neighborhood grayscale value.

[0095] Figure 14 FIG. 1 shows a schematic diagram of feature extraction of burr defects according to an embodiment of the present invention. Figure 14 As shown, 1410 is an image of a defect-free detection block, 1420 is an image of a defect-free detection block after LBP operator processing, 1440 is an image of a detection block with a burr defect, and 1450 is an image of a detection block with a burr defect after LBP operator processing. After LBP operator processing, the originally complex texture features of the detection block with a burr defect are more intuitively reflected in the image in the form of grayscale pixel values. Due to the inclined rounded edges of the burr defect, the distribution of the number of grayscale pixels in the defective and defect-free detection blocks after LBP operator processing is different. Taking the area with a grayscale value of 212 as an example, only pixels with a grayscale value of 212 in the two LBP images are retained. The filtered results are shown in 1430 and 1460. 1430 is the result of the filtered detection block without a defect, and 1460 is the result of the filtered detection block with a burr defect. It can be concluded that after LBP operator processing, the defective burr detection block has more areas with a grayscale value of 212 than the non-defective detection block, and there is a difference in the grayscale distribution between the two. Therefore, the grayscale statistical distribution of the LBP image can be used as the basis for defect determination.

[0096] Optionally, for the image processed by the local binary pattern, pixels having a grayscale value equal to a seventh predetermined value are retained. If white pixels are found in the filtered result, the detection block is determined to have a burr defect. Otherwise, the detection block is determined to have no burr defect. The specific value of the seventh predetermined value can be set according to actual circumstances, for example, 212.

[0097] According to the technical solution of the present invention, different detection methods and detection standards are used for different detection blocks, which has strong pertinence and high detection accuracy.

[0098] In step 350, an algorithm corresponding to the defect type is used to extract a feature vector of the detection block.

[0099] According to an embodiment of the present invention, if the defect type is a material shortage defect, a feature vector of the detection block is constructed based on the height peak of the connected area, the ratio of the height peak to the height mean of the connected area, and the area of ​​the connected area.

[0100] If the defect type is a particle defect, a feature vector of the detection block is constructed based on the height peak of the white connected area, the ratio of the height peak to the height mean of the white connected area, and the area of ​​the white connected area.

[0101] If the defect type is a multi-material defect, a feature vector of the detection block is constructed based on the area of ​​the white pixel and the ratio of the area of ​​the white pixel to the area of ​​its minimum circumscribed matrix.

[0102] If the defect type is a stone defect, a feature vector of the detection block is constructed based on the area of ​​white pixels and the number of connected regions composed of white pixels.

[0103] If the defect type is a burr defect, a feature vector of the detection block is constructed based on the number of pixels of each gray value in the image processed by the local binary pattern.

[0104] In step 360 , the feature vector is input into the trained classification model to determine whether a defect corresponding to the defect type exists in the detection block.

[0105] Optionally, the trained classification model is a support vector machine (SVM), and the kernel function of the SVM is a Gaussian kernel function.

[0106] According to an embodiment of the present invention, a feature vector is input into an SVM classifier to find the optimal hyperplane. A nonlinear SVM is selected as the classifier, and the kernel function is set to a Gaussian kernel function. The optimal hyperplane is obtained using training data. Five trained SVM classifiers are then able to classify the input feature vectors and identify five types of defects. A feature vector constructed based on the features of a detection block with a missing material defect is input into the trained classification model to determine whether the detection block has a missing material defect. A feature vector constructed based on the features of a detection block with a particle defect is input into the trained classification model to determine whether the detection block has a particle defect. A feature vector constructed based on the features of a detection block with a multi-material defect is input into the trained classification model to determine whether the detection block has a multi-material defect. A feature vector constructed based on the features of a detection block with a stone defect is input into the trained classification model to determine whether the detection block has a stone defect. A feature vector constructed based on the features of a detection block with a burr defect is input into the trained classification model to determine whether the detection block has a burr defect. Further confirming the presence of defects using the classification model improves detection accuracy and effectively avoids misjudgments.

[0107] SVM is a classification model based on supervised learning. Its most basic model is used to solve the optimal hyperplane that can correctly divide training samples and maximize the geometric interval between positive and negative samples. Figure 15 FIG. 1 shows a schematic diagram of a two-dimensional linear support vector machine according to an embodiment of the present invention. Figure 15 As shown in the figure, the straight line L w×x+b=0 is the optimized hyperplane for two-dimensional classification samples, l1 and l2 are two straight lines parallel to the straight line L, the positive samples are all on the l1 side, and the negative samples are all on the l2 side, and half of the distance between l1 and l2 is the geometric maximum interval between positive and negative samples.

[0108] For completely linearly separable data, the support vectors are the sample points on l1 and l2. At this time, the SVM solution problem can be transformed into an optimization problem: solving the constraints.

[0109] y i ((w·i+b))≥1

[0110] Objective function:

[0111]

[0112] However, completely linearly separable sample data rarely exists in reality. However, linearly inseparable sample data may be separable in high-dimensional space. In this case, a kernel function is used to increase the dimensionality of the sample data, mapping the low-dimensional data samples to a high-dimensional separable feature space. Finally, the optimal hyperplane is solved based on the representation of the data in the high-dimensional space.

[0113] Optionally, after determining whether a defect corresponding to the defect type exists in the detection block, the defect is presented for the detection block determined to have a defect. If the detection block has a defect and is subdivided into multiple sub-detection blocks, the defective sub-detection block is displayed on the image of the optical module base. If the detection block has a defect and is not subdivided into multiple sub-detection blocks, the detection block is displayed on the image of the optical module base. Optionally, the defect type of the defective detection block is marked on the image of the optical module base. In this way, the defects on the optical module base can be presented clearly and intuitively, allowing the user to clearly know the location and type of the defect, so that corresponding remedial measures can be taken for different types of defects on the optical module base.

[0114] According to the technical solution of the present invention, by preprocessing the image of the optical module base, the portion of the optical module base image containing the optical module base is cropped from the background, and the tilted optical module base is corrected. Since it is difficult to achieve a completely horizontal placement of the optical module base in reality, the impact of the tilted placement of the optical module base can be avoided by correcting the image of the optical module base. The region of interest is extracted based on the defect distribution of the optical module base. Based on the unique defect distribution of the optical module base, the location where the defect may occur is used as the region of interest, which improves detection efficiency and saves detection time. By performing defect detection on the detection block in the region of interest, the defect type of the detection block can be obtained. The feature vector of the detection block is extracted using an algorithm corresponding to the defect type, which is highly targeted and has a high detection accuracy. The classification model is used to determine whether the detection block has defects, further improving the detection accuracy and effectively avoiding the occurrence of misjudgments.

[0115] The various techniques described herein may be implemented in conjunction with hardware or software, or a combination thereof. Thus, the methods and apparatus of the present invention, or certain aspects or portions of the methods and apparatus of the present invention, may be implemented in the form of program codes (i.e., instructions) embedded in a tangible medium, such as a removable hard disk, a USB flash drive, a floppy disk, a CD-ROM, or any other machine-readable storage medium, wherein when the program is loaded into a machine such as a computer and executed by the machine, the machine becomes an apparatus for practicing the present invention.

[0116] When the program code is executed on a programmable computer, the mobile terminal generally includes a processor, a storage medium readable by the processor (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device. The memory is configured to store the program code, and the processor is configured to execute the method for detecting defects in the base of a small pluggable light-emitting and light-receiving module of the present invention according to the instructions in the program code stored in the memory.

[0117] By way of example and not limitation, readable media include readable storage media and communication media. Readable storage media store information such as computer-readable instructions, data structures, program modules, or other data. Communication media typically embody computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and include any information delivery medium. Combinations of any of the above are also included within the scope of readable media.

[0118] In the description provided herein, the algorithms and displays are not inherently related to any particular computer, virtual system, or other device. Various general-purpose systems may also be used in conjunction with the examples of the present invention. Based on the above description, it is apparent that the structure required for constructing such systems is well understood. In addition, the present invention is not directed to any specific programming language. It should be understood that various programming languages ​​may be utilized to implement the present invention described herein, and the description of specific languages ​​above is provided for the purpose of disclosing the preferred embodiment of the present invention.

[0119] In the description provided herein, a large number of specific details are described. However, it is understood that embodiments of the present invention can be practiced without these specific details. In some instances, well-known methods, structures, and techniques are not shown in detail so as not to obscure the understanding of this description.

[0120] Similarly, it should be understood that in order to streamline the present disclosure and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together into a single embodiment, figure, or description thereof. However, this disclosed method should not be interpreted as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim. Rather, as reflected in the claims below, inventive aspects lie in less than all the features of the individual embodiments disclosed above. Accordingly, the claims that follow the detailed description are hereby expressly incorporated into this detailed description, with each claim standing on its own as a separate embodiment of the invention.

[0121] Those skilled in the art will appreciate that the modules, units, or components of the devices in the examples disclosed herein may be arranged in the device described in the embodiment, or alternatively may be located in one or more devices different from the devices in the examples. The modules in the foregoing examples may be combined into one module or further divided into multiple submodules.

[0122] Those skilled in the art will appreciate that the modules in the devices in the embodiments may be adaptively changed and arranged in one or more devices different from the embodiments. The modules or units or components in the embodiments may be combined into one module or unit or component, and in addition may be divided into multiple submodules or subunits or subcomponents. All features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device disclosed herein may be combined in any combination, except that at least some of such features and / or processes or units are mutually exclusive. Unless expressly stated otherwise, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) may be replaced by an alternative feature providing the same, equivalent or similar purpose.

[0123] Furthermore, those skilled in the art will appreciate that although some embodiments described herein include certain features included in other embodiments but not other features, combinations of features from different embodiments are intended to be within the scope of the present invention and to form different embodiments. For example, in the claims below, any of the claimed embodiments may be used in any combination.

[0124] In addition, some of the embodiments are described herein as methods or combinations of method elements that can be implemented by a processor of a computer system or by other devices that perform the functions described. Thus, a processor having the necessary instructions for implementing the method or method element forms a device for implementing the method or method element. Furthermore, the elements described herein of the device embodiments are examples of devices for implementing the functions performed by the elements for the purpose of implementing the invention.

[0125] As used herein, unless otherwise specified, the use of ordinal numbers "first," "second," "third," etc. to describe common objects merely indicates that different instances of similar objects are involved and are not intended to imply that the objects so described must have a given order in time, space, ranking, or in any other manner.

[0126] Although the present invention has been described with respect to a limited number of embodiments, it will be apparent to those skilled in the art, having benefit of the foregoing description, that other embodiments are contemplated within the scope of the invention thus described. Furthermore, it should be noted that the language used in this specification has been selected primarily for readability and didactic purposes, rather than for the purpose of explaining or limiting the subject matter of the present invention. Consequently, many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the appended claims. The disclosure of the present invention is intended to be illustrative and not restrictive of the scope of the invention, which is defined by the appended claims.

Claims

1. A method for detecting defects in a base of a small pluggable light-emitting and light-receiving module, the method comprising: Acquire an image of the optical module base; Preprocessing the image of the optical module base; Extracting at least one region of interest from the preprocessed image according to the defect distribution position of the optical module base; Performing defect detection on the extracted region of interest as a detection block to obtain a defect type of the detection block; extracting a feature vector of the detection block using an algorithm corresponding to the defect type; Inputting the feature vector into a trained classification model to determine whether a defect corresponding to the defect type exists in the detection block; The step of performing defect detection on the extracted region of interest as a detection block to obtain the defect type of the detection block includes: If the aspect ratio of the detection block is greater than a predetermined value, then: subdividing the inspection block into a plurality of sub-inspection blocks, and determining a sub-inspection block having defects therein; performing binarization processing on the defective sub-detection block; Smoothing the sub-detection blocks after binarization processing; Fill the holes on the smoothed sub-detection blocks to generate a standard template; Performing image difference between the smoothed sub-detection block and the standard template; In the image difference results, the height peak of the connected area is used to determine whether the defect type is a material shortage defect, and the length, height, and area of ​​the white connected area are used to determine whether the defect type is a particle defect. If the aspect ratio of the detection block is less than or equal to a predetermined value, then: performing a binarization process on the detection block; By comparing the area of ​​white pixels in the binary detection block to the area of ​​its minimum circumscribed matrix, it is determined whether the defect type is a multi-material defect. The ratio of the white pixel area in the detection block after binarization to the area of ​​the entire detection block is used to determine whether the defect type is a stone defect. performing local binary pattern processing on the detection block; The grayscale statistical distribution of the image after local binary pattern processing is used to determine whether the defect type is a burr defect.

2. The method according to claim 1, wherein The preprocessing of the image of the optical module base includes: Segmenting a portion including the optical module base as a base image from the image of the optical module base; The base image is tilt-corrected.

3. The method according to claim 2, wherein: The step of segmenting the portion including the optical module base as a base image from the image of the optical module base includes: Setting a grayscale segmentation threshold, dividing the image of the optical module base into two parts with a grayscale value greater than the grayscale segmentation threshold and a grayscale value less than or equal to the grayscale segmentation threshold; Determine the variance between a portion whose grayscale value is greater than the grayscale segmentation threshold and a portion whose grayscale value is less than or equal to the grayscale segmentation threshold; Traversing the grayscale segmentation threshold at different values, and determining the value of the grayscale segmentation threshold when the variance is maximum; Binarizing the image of the optical module base using the grayscale segmentation threshold value when the variance is maximum; According to the binarized image, a base image is segmented from the image of the optical module base.

4. The method according to claim 3, wherein: Segmenting a base image from the image of the optical module base according to the binarized image includes: Perform noise removal on the binarized image; In the image after noise removal, calculating coordinate information of the maximum circumscribed matrix of the base image; A base image is segmented from the image of the optical module base according to the coordinate information.

5. The method according to claim 3, wherein Segmenting a base image from the image of the optical module base according to the binarized image includes: The background of the binarized image is removed to obtain the base image.

6. The method according to any one of claims 2 to 5, wherein The performing tilt correction on the base image includes: Binarizing the base image to obtain a binary image of the base; Rotating and transforming the binary image of the base at multiple angles; At each angle, determining the projection length of the binary image of the base in the vertical direction; The angle corresponding to the minimum projection length is used as the correction angle of the base image; The base image is corrected using the correction angle.

7. The method according to any one of claims 1 to 5, wherein The extracting the feature vector of the detection block by using an algorithm corresponding to the defect type includes: If the defect type is a material shortage defect, a feature vector of the detection block is constructed based on a height peak of a connected region, a ratio of the height peak to a height mean of the connected region, and an area of ​​the connected region.

8. The method according to any one of claims 1 to 5, wherein The extracting the feature vector of the detection block by using an algorithm corresponding to the defect type includes: If the defect type is a particle defect, a feature vector of the detection block is constructed based on the height peak of the white connected area, the ratio of the height peak to the height mean of the white connected area, and the area of ​​the white connected area.

9. The method according to any one of claims 1 to 5, wherein The extracting the feature vector of the detection block by using an algorithm corresponding to the defect type includes: If the defect type is a multi-material defect, a feature vector of the detection block is constructed based on the area of ​​the white pixel point and the ratio of the area of ​​the white pixel point to the area of ​​the minimum circumscribed matrix thereof.

10. The method according to any one of claims 1 to 5, wherein The extracting the feature vector of the detection block by using an algorithm corresponding to the defect type includes: If the defect type is a stone-sticking defect, a feature vector of the detection block is constructed based on the area of ​​white pixels and the number of connected regions formed by white pixels.

11. The method according to any one of claims 1 to 5, wherein The extracting the feature vector of the detection block by using an algorithm corresponding to the defect type includes: If the defect type is a burr defect, a feature vector of the detection block is constructed based on the number of pixels of each gray value in the image processed by the local binary pattern.

12. The method according to any one of claims 1 to 5, further comprising: If the inspection block has defects and is subdivided into a plurality of sub-inspection blocks, the sub-inspection blocks with defects are displayed on the image of the optical module base; If the inspection block has defects and is not subdivided into a plurality of sub-inspection blocks, the inspection block is displayed on the image of the optical module base.

13. The method according to any one of claims 1 to 5, wherein The acquiring of the image of the optical module base includes: Constructing an imaging platform, the imaging platform including an industrial camera, a ring light source, a detection tray and a guide rail, the industrial camera being equipped with a telecentric lens, and the ring light source being used to provide light for the industrial camera when capturing images; Move the detection tray by the guide rail so that the optical module base on the detection tray is located at the acquisition position of the industrial camera; The image of the optical module base at the acquisition position is acquired by the industrial camera.

14. The method according to any one of claims 1 to 5, wherein The trained classification model is a support vector machine, and the kernel function of the support vector machine is a Gaussian kernel function.

15. A computing device comprising: at least one processor; as well as A memory storing program instructions, wherein the program instructions are configured to be executed by the at least one processor, and the program instructions include instructions for executing the method according to any one of claims 1 to 14. 16 . A readable storage medium storing program instructions, wherein when the program instructions are read and executed by a computing device, the computing device is caused to execute the method according to claim 1 .

Images