General machine vision detection system for semiconductor industry

Abstract

The invention discloses a general machine vision detection system for the semiconductor industry, and belongs to the technical field of machine vision detection. The system integrates the functions of image acquisition, processing, measurement, calibration alignment, logic control, communication and the like, is suitable for various semiconductor industry scenes such as wafer detection, chip packaging detection, pin detection and the like, and remarkably improves the universality and the integration level of the system; a graphical drag-and-drop type process design is adopted, so that the debugging and deployment period is shortened; through N-point calibration and distortion correction functions, the coordinate conversion precision and the image quality are improved, the calibration and alignment module can directly provide precise coordinates for the manipulator, and the collaborative operation efficiency is improved; logic control functions of conditional branching, circulation and parallel execution are supported, a complex detection process can be flexibly constructed, and multi-step detection requirements are met; a multi-protocol communication interface and a perfect calibration and alignment tool are provided, the collaborative operation process with external equipment can be simplified, and the calibration precision and the alignment accuracy are improved.

Description

Technical Field

[0001] This application relates to the field of machine vision inspection technology, and in particular to a general machine vision inspection system for the semiconductor industry. Background Technology

[0002] With the rapid development of the 3C (computer, communication, and consumer electronics) industry and the semiconductor manufacturing industry, the application scenarios of visual inspection for product appearance, size, and position are constantly expanding, and the market's reliance on visual inspection technology is increasing. In the semiconductor production process, from wafer manufacturing and chip packaging to final product assembly, high-precision and high-reliability visual inspection methods are required to ensure product quality. Visual inspection has become an indispensable key link in the large-scale production of the semiconductor industry.

[0003] While various machine vision inspection technologies and equipment exist in the market, significant technical shortcomings remain in practical applications: First, existing vision systems are mostly dedicated devices designed for specific inspection scenarios, with relatively independent functional modules. This makes it difficult to quickly adapt to the multi-variety, small-batch production model of the semiconductor industry, lacking cross-scenario versatility and flexible adjustment capabilities. Second, the operation of traditional vision inspection software relies on professional technicians writing scripts or configuring complex parameters, which not only increases the skill threshold for operators but also leads to excessively long system debugging cycles, hindering rapid deployment and product changeover on production lines. Third, in high-precision semiconductor inspection scenarios, existing systems often face a contradiction between inspection accuracy and response speed. During high-precision inspection, system processing efficiency decreases, failing to meet the real-time inspection needs of high-speed production lines. Fourth, existing systems are weak in logical management aspects such as inspection process arrangement, condition judgment, and loop control, making it difficult to implement complex multi-step, multi-condition inspection logic. Fifth, in scenarios involving collaborative operation with robotic arms, the calibration tools of existing systems are cumbersome to operate and lack sufficient calibration accuracy, directly affecting the mapping accuracy between pixel coordinates and machine coordinates, thereby reducing overall inspection and operational accuracy. Summary of the Invention

[0004] This application provides a general-purpose machine vision inspection system for the semiconductor industry. The technical solution is as follows: On the one hand, a general machine vision inspection system for the semiconductor industry is provided, including an image acquisition module, an image processing module, a calibration and alignment module, a logic control module, a data and communication module, and a user interface module; The image acquisition module is used to connect to the camera and acquire image data; the image processing module is used to preprocess and detect and identify the image data; the calibration and alignment module is used to establish the mapping relationship between pixel coordinates and mechanical coordinates and correct image distortion; the logic control module is used to program the detection process and implement logical judgment; the data and communication module is used to process the detection data and communicate with external devices; and the user interface module is used to provide a graphical operation interface. The detection method based on the general machine vision inspection system for the semiconductor industry includes: Create a testing workflow through the user interface module and drag and drop the required tools to the workflow bar; Configure camera parameters and connect the device via the image acquisition module, and set the image acquisition mode; Select the corresponding image processing tool through the image processing module; The detection logic is constructed by calling logic tools through the logic control module; The mapping relationship between pixel coordinates and mechanical coordinates is established through the calibration and alignment module; The detection process is executed sequentially, including image acquisition, image processing, and logical judgment, and the detection results are output. The data and communication module enables the recording, display, storage, and communication with external devices of detection data.

[0005] Optionally, the image acquisition module supports the access of area scan cameras and line scan cameras, and the image acquisition modes include specified image acquisition, folder sequence acquisition, and real-time camera acquisition.

[0006] Optionally, the preprocessing submodule of the image processing module includes an adaptive multi-scale morphological filtering algorithm. The algorithm performs 3×3, 5×5, and 7×7 multi-scale structuring element decomposition on the image, performs morphological opening and closing operations on each scale, and dynamically selects the optimal scale based on the local gray-level variance.

[0007] Optionally, the detection and recognition submodule of the image processing module includes a multi-feature fusion template matching algorithm. The algorithm extracts the shape contour, local gray-level distribution, and LBP texture features of the template to construct a joint feature vector. Affine transformation parameter estimation is introduced during the matching process.

[0008] Optionally, the calibration and alignment module includes a subpixel edge extraction tool and an SVM-based defect classifier. The subpixel edge extraction tool employs the Canny-Zernike moment algorithm, and the defect classifier is used to extract the geometric and texture features of the defect region and identify the defect type.

[0009] Optionally, the calibration and alignment module includes a nonlinear distortion calibration algorithm and a multi-camera collaborative calibration mode. The nonlinear distortion calibration algorithm is based on a convolutional neural network to construct a distortion correction model and completes the correction by taking a calibration board image. The multi-camera collaborative calibration mode achieves a unified coordinate system for multiple cameras by matching feature points in the common viewing area, and introduces feedback data from the robotic arm for dynamic error compensation.

[0010] Optionally, the logic control module includes a visual logic flowchart editor and a parallel processing module. The visual logic flowchart editor supports dragging and dropping conditional branches, loops, jumps, and parallel logic blocks to build and detect the process, and provides breakpoint setting, variable monitoring, and real-time debugging functions for single-step execution of the process. The parallel processing module enables multiple independent tasks in the process to be executed simultaneously through the parallel block tool.

[0011] Optionally, the logic control module includes an event-driven process execution mechanism that supports event-driven process branches triggered by image readiness, communication receipt, and IO.

[0012] Optionally, the calibration and alignment module includes intelligent ROI dynamic focusing technology, which automatically adjusts the position and size of the ROI based on the detection results of the previous frame, and combines Kalman filtering to predict the target's motion trajectory.

[0013] Optionally, the communication interface module of the data and communication module supports TCP / IP protocol, serial port protocol, Modbus protocol and MitsubishiMC protocol for data transmission and reception with external devices.

[0014] This application discloses a general-purpose machine vision inspection system for the semiconductor industry, belonging to the field of machine vision inspection technology. The system integrates image acquisition, processing, measurement, calibration and alignment, logic control, and communication functions, and is applicable to various semiconductor industry scenarios such as wafer inspection, chip packaging inspection, and pin inspection, significantly improving the system's versatility and integration. It adopts a graphical drag-and-drop workflow design, eliminating the need for complex code writing, lowering the operational threshold, and shortening the debugging and deployment cycle. Through N-point calibration and distortion correction functions, it improves coordinate transformation accuracy and image quality. Combined with a high-speed data processing mechanism, it can meet the response requirements of high-speed production lines while ensuring high-precision inspection. Furthermore, the calibration and alignment modules can directly provide precise coordinates for the robotic arm, improving collaborative operation efficiency. It supports conditional branching, looping, and parallel execution logic control functions, allowing for flexible construction of complex inspection processes to adapt to multi-step inspection needs. It provides multi-protocol communication interfaces and comprehensive calibration and alignment tools, simplifying collaborative operation processes with external devices and improving calibration accuracy and alignment precision. Attached Figure Description

[0015] Figure 1 A flowchart of a general machine vision inspection method for the semiconductor industry is shown. Figure 2 This is a schematic diagram illustrating the process of applying the present invention to semiconductor chip pin detection and robotic arm alignment; Figure 3 This is a schematic diagram illustrating the collaborative operation of "template matching - rectangular measurement - calibration alignment - logical judgment" in this invention. Detailed Implementation

[0016] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0017] In this article, "multiple" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0018] This invention discloses a general-purpose machine vision inspection method and system for the semiconductor industry, relating to the field of machine vision inspection technology. The system includes modules for image acquisition, image processing, calibration and alignment, logic control, data and communication, and a user interface. The method includes steps such as creating an inspection process, configuring camera parameters, selecting image processing tools, constructing inspection logic, performing calibration and alignment operations, outputting results and alignment coordinates, and recording and storing data. This invention improves the system's versatility and integration, lowers the operational threshold, balances inspection accuracy and response speed, optimizes the collaborative alignment effect with external devices, and adapts to the multi-scenario inspection needs of the semiconductor industry.

[0019] The general-purpose machine vision inspection system for the semiconductor industry provided by the various embodiments of this application aims to solve the problems of low system integration, insufficient versatility and flexibility, and difficulty in adapting to the inspection needs of multiple products in existing machine vision inspection technologies for the semiconductor industry. It also addresses the high operational threshold, requiring professional personnel to configure parameters or write scripts, resulting in low debugging and deployment efficiency. Furthermore, it improves the ability to balance inspection accuracy and response speed, failing to meet the high-precision inspection needs of high-speed production lines. Additionally, it further enhances the collaborative operation effect by addressing the problems of weak logic control functions, difficulty in constructing complex inspection processes, and imperfect calibration and alignment tools, resulting in cumbersome operation and low accuracy.

[0020] Example 1 like Figure 1The diagram shows a schematic of a general-purpose machine vision inspection system for the semiconductor industry. In the diagram, 10 is the image acquisition module, 20 is the image processing module, 30 is the calibration and alignment module, 40 is the logic control module, 50 is the data and communication module, and 60 is the user interface module. All modules are connected through a system bus. The calibration and alignment module is additionally connected to an external robot (marked 70) through a data interface.

[0021] Specifically, the system includes an image acquisition module, an image processing module, a calibration and alignment module, a logic control module, a data and communication module, and a user interface module. The functions of each module are as follows.

[0022] The image acquisition module supports area scan cameras and line scan cameras via GigE interface and USB 3.0 interface, and provides three image acquisition modes: specified image file acquisition, folder sequence image acquisition, and real-time camera acquisition. It can transfer the acquired raw image data to the image processing module. The image processing module includes a preprocessing submodule and a detection and recognition submodule. The preprocessing submodule can perform filtering, binarization, morphological operations (erosion, dilation), and image enhancement. The detection and recognition submodule integrates template matching, geometric measurement (circle, line, rectangle, arc measurement), speckle analysis, character recognition, and QR code recognition tools. The calibration and alignment module provides line-to-line distance measurement, fitted circle measurement, coordinate mapping, N-point calibration, and distortion correction functions. It can establish the mapping relationship between pixel coordinates and mechanical coordinates, correct camera lens distortion, and achieve precise alignment between the object being inspected and external equipment (such as a robotic arm). The logic control module supports conditional branching (If-Else If-Else structure), loop execution (For loop, While loop), parallel block execution, and script editing operations, enabling the orchestration of detection processes and complex logical judgments; The data and communication module includes a variable management submodule and a communication interface submodule. The variable management submodule supports variable definition, variable setting, and array operations, and can store intermediate and result data during the detection process. The communication interface submodule supports TCP / IP protocol, serial port protocol, Modbus protocol, and MitsubishiMC protocol, and can send and receive data with external devices such as PLCs and robotic arms. The user interface module provides a graphical workflow design main workspace, tool icon bar, and parameter setting panel; the main workspace can display the detection workflow bar, image window, and data log; the tool icon bar categorizes and displays visual tools and logical tools; the parameter setting panel can be dynamically displayed when the user double-clicks a workflow tool and allows configuration of tool parameters.

[0023] Example 2 Furthermore, corresponding to the detection method of this system, the embodiments of this application also include the core process of a general machine vision inspection method for the semiconductor industry, such as... Figure 2 The diagram shows the process of applying the present invention to semiconductor chip pin detection and robotic arm alignment. The steps in the diagram are as follows: "System initialization and process creation → Camera connection and parameter configuration → Chip positioning (template matching) → Pin spacing measurement (rectangular measurement) → Calibration and alignment setting → Logic judgment and result / alignment command output → Data recording and visualization".

[0024] Correspondingly, Figure 3 This is a schematic diagram illustrating the collaborative operation of "template matching - rectangle measurement - calibration and alignment - logical judgment" in this invention. In the diagram, 101 represents the template matching tool, 102 the rectangle measurement tool, 103 the calibration and alignment tool, 104 the data checking tool, 105 the conditional branching tool, 106 the text sending (OK signal) tool, 107 the alignment coordinate sending tool, and 108 the text sending (NG alarm) tool. The output terminals (center X, center Y, angle) of the template matching tool are connected to the input terminals (ROI reference position) of the rectangle measurement tool and the input terminals (image coordinates) of the calibration and alignment tool, respectively. The output terminal (distance) of the rectangle measurement tool is connected to the input terminal of the data checking tool. The output terminal (mechanical coordinates) of the calibration and alignment tool is connected to the input terminal of the alignment coordinate sending tool. The output terminal (comprehensive judgment) of the data checking tool is connected to the judgment condition input port of the conditional branching tool. The specific steps include the following:

[0025] Step 1: Create a new inspection solution through the user interface module. Drag and drop the required tools such as "Image Acquisition", "Template Matching", "Geometric Measurement", and "Data Inspection" into the workflow bar of the main work area to build the inspection workflow. Step 2: Select and connect the target camera using the "Camera Settings" tool in the image acquisition module, configure the camera exposure time and gain parameters, click the "Connect" and "Acquire Image" buttons, and confirm that the image window can display a clear image of the detected object. Step 3: Select the corresponding processing tool through the image processing module; if you need to locate the detection object, select the "Template Matching" tool, draw the template area in the current image, and save the standard template; if you need to measure the size of the detection object, select the "Geometric Measurement" tool and set the measurement parameters. Step 4: Use the logic control module to call conditional branching and looping tools to build detection logic according to detection requirements, such as setting the logical relationship of "execute geometric measurement after successful template matching" and "trigger alarm when geometric measurement result exceeds tolerance"; Step 5: Perform calibration and alignment preparation operations through the calibration and alignment module; if it is necessary to convert pixel units to physical units, use the "Measurement Calibration" tool to take an image of the standard calibration board and obtain the pixel equivalent (e.g., 0.02mm / pixel); if it is necessary to correct lens distortion, use the "Distortion Correction" tool to perform the correction operation; if it is necessary to coordinate with the robot arm, use the "N-point Calibration" tool to establish the mapping relationship between pixel coordinates and robot arm coordinates. Step 6: Run the inspection process. The system sequentially performs image acquisition, image processing, and logical judgment operations, and outputs the inspection results (such as the position coordinates of the inspected object, size data, and pass / fail judgment results). At the same time, the system converts the coordinates of the inspected object into mechanical coordinates through the calibration and alignment module and sends them to the robot arm to achieve precise alignment. Step 7: Display the detection results in real time to the user interface data log using the "Data Recording" tool of the data and communication module, save the detection results (including chip ID, detection size, judgment result, and timestamp) to the local hard drive using the "CSV Storage" tool, and send the detection results and alignment coordinate data to the PLC and robot through the communication interface submodule.

[0026] Therefore, in this embodiment, the system integrates image acquisition, processing, measurement, calibration and alignment, logic control, and communication functions, making it adaptable to various scenarios in the semiconductor industry, such as wafer inspection, chip packaging inspection, and pin inspection, thus improving the system's versatility and integration. It adopts a graphical drag-and-drop workflow design, eliminating the need for complex code writing, lowering the operational threshold, and shortening the debugging and deployment cycle. Through N-point calibration and distortion correction functions, it improves coordinate transformation accuracy and image quality. Combined with a high-speed data processing mechanism, it can meet the response requirements of high-speed production lines while ensuring high-precision detection. Furthermore, the calibration and alignment modules can directly provide precise coordinates for the robotic arm, improving collaborative operation efficiency. It supports logic control functions such as conditional branching, looping, and parallel execution, allowing for flexible construction of complex detection processes to adapt to multi-step detection needs. It provides multi-protocol communication interfaces and comprehensive calibration and alignment tools, simplifying collaborative operation processes with external devices and improving calibration accuracy and alignment precision.

[0027] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0028] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware, or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk. The above descriptions are merely optional embodiments of this application and are not intended to limit the application. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A general-purpose machine vision inspection system for the semiconductor industry, characterized in that, It includes an image acquisition module, an image processing module, a calibration and alignment module, a logic control module, a data and communication module, and a user interface module; The image acquisition module is used to connect to the camera and acquire image data; the image processing module is used to preprocess and detect and identify the image data; the calibration and alignment module is used to establish the mapping relationship between pixel coordinates and mechanical coordinates and correct image distortion; the logic control module is used to program the detection process and implement logical judgment; the data and communication module is used to process the detection data and communicate with external devices; and the user interface module is used to provide a graphical operation interface. The detection method based on the general machine vision inspection system for the semiconductor industry includes: Create a testing workflow through the user interface module and drag and drop the required tools to the workflow bar; Configure camera parameters and connect the device via the image acquisition module, and set the image acquisition mode; Select the corresponding image processing tool through the image processing module; The detection logic is constructed by calling logic tools through the logic control module; The mapping relationship between pixel coordinates and mechanical coordinates is established through the calibration and alignment module; The detection process is executed sequentially, including image acquisition, image processing, and logical judgment, and the detection results are output. The data and communication module enables the recording, display, storage, and communication with external devices of detection data.

2. The system according to claim 1, characterized in that, The image acquisition module supports the access of area scan cameras and line scan cameras, and the image acquisition modes include specified image acquisition, folder sequence acquisition, and real-time camera acquisition.

3. The system according to claim 1, characterized in that, The preprocessing submodule of the image processing module includes an adaptive multi-scale morphological filtering algorithm. The algorithm performs 3×3, 5×5, and 7×7 multi-scale structuring element decomposition on the image, performs morphological opening and closing operations on each scale, and dynamically selects the optimal scale based on the local gray-level variance.

4. The system according to claim 1, characterized in that, The detection and recognition submodule of the image processing module includes a multi-feature fusion template matching algorithm. The algorithm extracts the shape contour, local gray-level distribution, and LBP texture features of the template to construct a joint feature vector. Affine transformation parameter estimation is introduced during the matching process.

5. The system according to claim 1, characterized in that, The calibration and alignment module includes a subpixel edge extraction tool and an SVM-based defect classifier. The subpixel edge extraction tool uses the Canny-Zernike moment algorithm, and the defect classifier is used to extract the geometric and texture features of the defect region and identify the defect type.

6. The system according to claim 1, characterized in that, The calibration and alignment module includes a nonlinear distortion calibration algorithm and a multi-camera collaborative calibration mode. The nonlinear distortion calibration algorithm is based on a convolutional neural network to construct a distortion correction model and completes the correction by taking a calibration board image. The multi-camera collaborative calibration mode achieves a unified coordinate system for multiple cameras by matching feature points in the common viewing area, and introduces feedback data from the robotic arm for dynamic error compensation.

7. The system according to claim 1, characterized in that, The logic control module includes a visual logic flowchart editor and a parallel processing module. The visual logic flowchart editor supports dragging and dropping conditional branches, loops, jumps, and parallel logic blocks to build and detect the process, and provides breakpoint setting, variable monitoring, and real-time debugging functions for single-step execution of the process. The parallel processing module enables multiple independent tasks in the process to be executed simultaneously through the parallel block tool.

8. The system according to claim 1, characterized in that, The logic control module includes an event-driven process execution mechanism that supports event-driven process branches triggered by image readiness, communication receipt, and IO.

9. The system according to claim 1, characterized in that, The calibration and alignment module includes intelligent ROI dynamic focusing technology, which automatically adjusts the position and size of the ROI based on the detection results of the previous frame, and combines Kalman filtering to predict the target's motion trajectory.

10. The system according to claim 1, characterized in that, The communication interface module of the data and communication module supports TCP / IP protocol, serial port protocol, Modbus protocol and MitsubishiMC protocol for sending and receiving data with external devices.

Images