A mechanical arm automatic sorting method and system for quartz wafers
Patent Information
- Application Number
- CN202611190747.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-06
- Publication Date
- 2026-10-30
AI Technical Summary
本发明通过高斯拉普拉斯算子提取图像中的边缘信息。通过自适应增强系数
根据局部对比度动态调整增强强度。这一处理使得原本透明、低纹理的石英晶片在图像中的轮廓显著性大幅提升，为后续目标检测模型提供了高质量输入，从源头保障了识别精度。由此，本发明有效克服了石英晶片透明表面导致的特征稀缺问题，使目标检测模型能够清晰识别晶片边缘与轮廓。
Smart Images
Figure CN123010609A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of quartz wafer sorting technology, and more particularly to an automated sorting method and system for quartz wafers using a robotic arm. Background Technology
[0002] Quartz crystals utilize the piezoelectric effect to convert mechanical vibrations into stable electrical signals, providing a precise clock reference for microprocessors and digital circuits. They are core components for frequency control and signal filtering. Their outstanding features include extremely high frequency stability, excellent quality factor, and good long-term reliability, maintaining a constant resonant frequency over a wide temperature range. Furthermore, quartz crystals are small in size, low in power consumption, and have strong anti-interference capabilities, making them flexibly adaptable to demanding applications such as 5G communication, automotive electronics, consumer electronics, and aerospace. They are an indispensable "frequency-fixed cornerstone" for modern electronic systems.
[0003] Current physical manufacturing technologies cannot guarantee that every quartz wafer will be identical. Furthermore, to ensure product quality, quartz wafers with irregular shapes, scratches, contaminants, or other defects must be removed, necessitating sorting. Currently, sorting is primarily done manually or using traditional image processing techniques. Both methods have several drawbacks: First, manual sorting is inefficient, labor-intensive, and susceptible to human error, leading to wasted resources. Second, traditional image algorithms have poor generalization ability and insufficient robustness in feature extraction, making them ill-suited to the feature scarcity of the transparent surface of quartz wafers, and also exhibiting low accuracy in recognizing irregularly shaped or defective wafers.
[0004] The above content is only used to help understand the technical solution of the present invention and does not represent an admission that the above content is prior art. Summary of the Invention
[0005] The present invention provides a robotic arm automatic sorting method for quartz wafers, comprising: acquiring raw visual images of quartz wafers; The original visual image is preprocessed to obtain a preprocessed visual image for distortion correction and edge contour enhancement. The target detection model is invoked to identify targets in the preprocessed visual image, and the category and image coordinates of the quartz crystal are output. Calculate the actual position coordinates of the quartz crystal in the world coordinate system based on the image coordinates; The kinematic model is solved in reverse based on the actual position coordinates in order to control the movement of the robotic arm.
[0006] Furthermore, the chessboard grid marking plate is fixed to the working surface; Calibration is performed using the MATLAB camera calibration toolbox to obtain the intrinsic parameter matrix, extrinsic parameter matrix, and distortion parameters. The cv2.remap function of the OpenCV library is called to perform distortion correction on the original visual image based on the intrinsic parameter matrix, extrinsic parameter matrix, and distortion parameters to obtain the distortion-corrected image; The distortion-reduced image is processed according to the edge contour enhancement formula to obtain a preprocessed visual image.
[0007] Furthermore, the formula for edge contour enhancement is:
[0008] For preprocessing visual images; Spatial domain coordinates; The image after distortion correction; For the Gaussian Laplace operator; The standard deviation of the Gaussian function; This is the adaptive enhancement coefficient.
[0009] Furthermore, the preprocessed visual image is converted into an input feature map and then fed into the object detection model; The input feature map is processed according to the feature extraction formula to obtain the output feature map; The output feature map is decoded to obtain the quartz crystal's category and image coordinates.
[0010] Furthermore, the feature extraction formula is as follows:
[0011] Output feature map; This is the sample number for the current batch; The channel number of the input feature map; The channel number of the output feature map; The height index of the output feature map; This is the width index of the output feature map; This represents the traversal offset of the convolution kernel in the height direction; This represents the offset of the convolution kernel in the width direction. Input the number of channels; The kernel size; To connect the first The input channel and the first The kernel weight parameters for each output channel; Input feature map; The convolution stride; For fill size; For the first The bias term for each output channel.
[0012] Furthermore, the size relationship between the input feature map and the output feature map satisfies:
[0013]
[0014] This is the height of the output feature map; This is the width of the output feature map; The height of the input feature map; This is the width of the output feature map.
[0015] Furthermore, it also includes optimization steps for the object detection model: Define a loss function and minimize its value to update the convolutional kernel weight parameters of the object detection model; The loss function is:
[0016] The number of positive samples; Total number of categories; Number the sample; The category label number; For the first The sample at the th The true binary annotation on each label; For the first The sample at the th Predicted probabilities on each label.
[0017] This invention also provides an automated robotic arm sorting system for quartz wafers, employing the method described above. It includes: an image acquisition module, a calculation and processing module, a control module, an execution module, and a communication module. The image acquisition module includes a monocular camera for acquiring raw visual images; The calculation and processing module is used to preprocess and identify the original visual image in order to calculate the actual position coordinates based on the acquired quartz crystal type and image coordinates. The execution module includes interconnected robotic arms, stepper motor drivers, and electromagnet-based grippers; The control module is used to solve the kinematic model inversely based on the actual position coordinates in order to control the actions of the execution module; The communication module is used to maintain communication between the execution module and the control module.
[0018] Compared with the prior art, the present invention has the following advantages: This invention utilizes the Gaussian Laplace operator Extract edge information from the image. Use adaptive enhancement coefficients. The enhancement intensity is dynamically adjusted based on local contrast. This process significantly improves the saliency of the originally transparent, low-texture quartz wafer's outline in the image, providing high-quality input for subsequent object detection models and ensuring recognition accuracy from the source. Therefore, this invention effectively overcomes the feature scarcity problem caused by the transparent surface of quartz wafers, enabling object detection models to clearly identify the wafer's edges and outlines.
[0019] This invention uses binary cross-entropy calculation based on the Sigmoid output, with each category's prediction independent of others. When quartz crystals are stacked or occluded, the target detection model does not lower the confidence of a particular target due to the requirement that the sum of class probabilities must be 1. Instead, it can simultaneously assign reasonable high confidence to multiple categories, thus significantly improving the recognition accuracy in complex scenes. This allows the same predicted location to be assigned multiple category labels simultaneously. It avoids the adverse effects of class competition caused by Softmax forced normalization on small or occluded targets, resulting in a target detection model with better generalization ability and robustness. Consequently, it maintains a high recognition rate and low false detection rate even under different lighting, backgrounds, transparency, and placement postures.
[0020] This invention achieves layer-by-layer abstraction of wafer features through feature extraction formulas. Simultaneously, it controls the height and width of the feature map after each convolutional layer using size relationship formulas. Therefore, when using a monocular camera, this invention can maintain sufficient spatial resolution while ensuring feature extraction depth, preventing the loss of positional information of small-sized wafers as the network depth increases, ultimately achieving industrial-grade accuracy with a positioning error ≤2mm.
[0021] This invention uses only a monocular camera, rather than binocular vision or 3D sensors, for image acquisition, and achieves precise positioning by combining checkerboard calibration and coordinate mapping technology. In terms of hardware, it only requires conventional industrial components such as a camera, an embedded computing platform (e.g., Jetson TX2), a DSP control board, and a SCARA robotic arm; it does not require expensive 3D scanners or high-precision laser sensors. The overall solution offers advantages in both performance and cost, and can be flexibly adapted to automated sorting scenarios for other small, precision parts. Attached Figure Description
[0022] Figure 1 Overall process diagram. Detailed Implementation
[0023] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.
[0024] Example 1: This invention provides an automated sorting method for quartz wafers using a robotic arm, referring to... Figure 1 , Figure 1 This is a schematic diagram of the process of the present invention. It includes the following steps: Print a 6x8 checkerboard calibration board with squares measuring 30mm x 30mm. Fix the checkerboard calibration board to the work surface. Take 40 calibration images from different angles using a monocular camera. Use the Zhang Zhengyou checkerboard calibration method and the MATLAB camera calibration toolbox to perform calibration and obtain the intrinsic parameter matrix, extrinsic parameter matrix, and distortion parameters (k1, k2, k3, p1, p2).
[0025] The raw visual image of the quartz crystal is acquired using a monocular camera. Based on the OpenCV library, the cv2.remap function is used to perform distortion correction on the raw visual image according to the intrinsic parameter matrix, extrinsic parameter matrix, and distortion parameters to obtain the distortion-corrected image.
[0026] The distortion-corrected image is processed according to the edge contour enhancement formula to obtain a preprocessed visual image. Specifically, the edge contour enhancement formula is:
[0027] For preprocessing visual images; Spatial domain coordinates; The image after distortion correction; For the Gaussian Laplace operator; The standard deviation of the Gaussian function; This is the adaptive enhancement coefficient.
[0028] In summary, using the Gaussian Laplace operator Extract edge information from the image. Use adaptive enhancement coefficients. The enhancement intensity is dynamically adjusted based on local contrast. This process significantly improves the saliency of the originally transparent, low-texture quartz wafer's outline in the image, providing high-quality input for subsequent object detection models and ensuring recognition accuracy from the source. Therefore, this invention effectively overcomes the feature scarcity problem caused by the transparent surface of quartz wafers, enabling object detection models to clearly identify the wafer's edges and outlines.
[0029] A YOLOv11-based object detection model performs object recognition on preprocessed visual images, outputting the category and image coordinates of quartz wafers. The overall network of the object detection model consists of three main parts: a Backbone feature extraction module, a Neck multi-scale feature fusion module, and a Head multi-branch detection output module. The Backbone is responsible for extracting deep semantic features of the wafer, the Neck enhances the perception of wafers of different sizes through multi-scale fusion, and the Head branch outputs both the category probability and bounding box coordinates. These three components work together to achieve an end-to-end mapping from the original image to "what shape it is and where it is located." This eliminates the need for manually designed feature rules and optimizes for transparent, low-texture quartz wafers.
[0030] The specific recognition process is as follows: the preprocessed visual image is converted into an input feature map and input into the target detection model; the input feature map is processed according to the feature extraction formula to obtain an output feature map; the output feature map is decoded to obtain the category and image coordinates of the quartz crystal. The feature extraction formula is:
[0031] Output feature map; This is the sample number for the current batch; The channel number of the input feature map; The channel number of the output feature map; The height index of the output feature map; This is the width index of the output feature map; This represents the traversal offset of the convolution kernel in the height direction; This represents the offset of the convolution kernel in the width direction. Input the number of channels; The kernel size; To connect the first The input channel and the first The kernel weight parameters for each output channel; Input feature map; The convolution stride; For fill size; For the first The bias term for each output channel.
[0032] in, The superscript (1) indicates that the feature map is in the first convolutional stage of the network.
[0033] Meanwhile, the size relationship between the input feature map and the output feature map satisfies:
[0034]
[0035] This is the height of the output feature map; This is the width of the output feature map; The height of the input feature map; This is the width of the output feature map; The kernel size; For fill size; is the convolution stride.
[0036] In summary, the feature extraction formula achieves layer-by-layer abstraction of the wafer's features. Simultaneously, the size relationship formula controls the height and width of the feature map after each convolutional layer. Therefore, when using a monocular camera, this invention can maintain sufficient spatial resolution while ensuring feature extraction depth, preventing the loss of positional information of small-sized wafers as the network depth increases, ultimately achieving industrial-grade accuracy with a positioning error ≤2mm.
[0037] After obtaining the type and image coordinates of the quartz wafer, the actual position coordinates of the quartz wafer in the world coordinate system are calculated based on the image coordinates.
[0038] The kinematic model is solved in reverse based on the actual position coordinates in order to control the movement of the robotic arm.
[0039] It also includes an optimization step for the object detection model, which optimizes the model during the training phase. The specific method is as follows: Define a loss function and minimize its value to update the convolutional kernel weight parameters of the object detection model; The loss function is:
[0040] The number of positive samples; Total number of categories; Number the sample; The category label number; For the first The sample at the th The true binary annotation on each label; For the first The sample at the th Predicted probabilities on each label.
[0041] In summary, the aforementioned process calculates binary cross-entropy based on the Sigmoid output, with each category's prediction independent of others. When quartz crystals are stacked or occluded, the object detection model does not lower the confidence of a particular target due to the requirement that the sum of class probabilities must be 1. Instead, it can simultaneously assign reasonable high confidence to multiple categories, thus significantly improving recognition accuracy in complex scenes. This allows the same predicted location to be assigned multiple category labels simultaneously. It avoids the adverse effects of class competition caused by Softmax forced normalization on small or occluded targets, resulting in superior generalization ability and robustness for the object detection model. Consequently, it maintains high recognition rates and low false detection rates even under varying lighting, backgrounds, transparency, and placement orientations.
[0042] In summary, this invention can effectively replace manual sorting, thereby greatly improving sorting efficiency, reducing labor intensity, and avoiding misjudgments caused by human factors. Furthermore, it effectively optimizes the characteristics of quartz wafers, thus enhancing its generalization ability and robustness in this scenario.
[0043] Example 2: An automated robotic arm sorting system for quartz wafers, employing the method described in Example 1, includes: an image acquisition module, a calculation and processing module, a control module, an execution module, and a communication module. The image acquisition module uses a monocular camera supporting USB 2.0 and OTG communication protocols, outputting 1080P resolution images at a frame rate of 30 frames per second, with a lens focal length of 6mm, and transmitting data using YUV format encoding. It also features a combination of a ring LED light source and a coaxial light source to provide ample illumination. This allows the image acquisition module to capture raw visual images.
[0044] Meanwhile, the monocular camera was calibrated using the MATLAB camera calibration toolbox. The calibration board was a 6×8 checkerboard with a grid of 30mm×30mm squares. Forty calibration images were taken from different angles to obtain the intrinsic parameters, extrinsic parameters, and distortion parameters (k1, k2, k3, p1, p2).
[0045] The computational processing module, using the Jetson TX2 embedded platform, is used to preprocess and identify the raw visual images, calculating the actual position coordinates based on the acquired quartz crystal type and image coordinates.
[0046] The execution module employs a SCARA robotic arm, stepper motor drivers, and an electromagnet-based gripper to address the thin and fragile nature of quartz wafers, preventing damage from mechanical grippers. Specifically, the gripping force is adjusted by a DSP via PWM to regulate the electromagnet current, achieving flexible gripping. Furthermore, the SCARA robotic arm is a 3-DOF structure with a repeatability of 0.05mm, a load capacity of 1.5kg, encoders at its joints, and is driven by a 57-type stepper motor.
[0047] The control module uses a DSP28335 development board with a main frequency of 150MHz. It integrates an external memory interface, 18-channel EPWM, and a 16-channel 12-bit ADC, supporting SCI, CAN, and SPI communication. It is used to inversely solve the kinematic model based on the actual position coordinates to control the actions of the execution module. Specifically, the EPWM interface output, through an interrupt response mechanism, enables the coordinated operation of SCI communication, EPWM output, and the relay module, ensuring precise execution of the sorting action.
[0048] The communication module uses a TTL to RS232 serial port converter to maintain communication between the execution module and the control module.
[0049] In summary, this invention uses only a monocular camera, rather than binocular vision or 3D sensors, for image acquisition, and achieves precise positioning by combining checkerboard calibration and coordinate mapping technology. Hardware-wise, it only requires conventional industrial components such as a camera, an embedded computing platform (e.g., Jetson TX2), a DSP control board, and a SCARA robotic arm, without expensive 3D scanners or high-precision laser sensors. The overall solution offers both performance and cost advantages and can be flexibly adapted to automated sorting scenarios for other small, precision parts.
[0050] Furthermore, it should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.
[0051] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.
[0052] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as read-only memory (ROM) / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.
[0053] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.
[0054] It is understood that the system provided in the embodiments of the present invention corresponds to the method provided in the embodiments of the present invention, and the explanation, examples and beneficial effects of the relevant content can be referred to the corresponding parts of the above methods.
Claims
1. A robotic arm-based automatic sorting method for quartz wafers, characterized in that, include: Obtain the raw visual image of the quartz crystal wafer; The original visual image is preprocessed to obtain a preprocessed visual image for distortion correction and edge contour enhancement. The target detection model is invoked to perform target recognition on the preprocessed visual image, and the category and image coordinates of the quartz crystal are output. Calculate the actual position coordinates of the quartz wafer in the world coordinate system based on the image coordinates; The kinematic model is solved inversely based on the actual position coordinates to control the movement of the robotic arm.
2. The robotic arm automatic sorting method for quartz wafers as described in claim 1, characterized in that, Fix the chessboard grid calibration plate to the working surface; Calibration is performed using the MATLAB camera calibration toolbox to obtain the intrinsic parameter matrix, extrinsic parameter matrix, and distortion parameters. The cv2.remap function of the OpenCV library is called to perform distortion correction on the original visual image based on the intrinsic parameter matrix, extrinsic parameter matrix, and distortion parameters to obtain the distortion-corrected image; The distortion-reduced image is processed according to the edge contour enhancement formula to obtain the preprocessed visual image.
3. The robotic arm automatic sorting method for quartz wafers as described in claim 2, characterized in that, The edge contour enhancement formula is as follows: The The preprocessed visual image; The Spatial domain coordinates; The The image after distortion correction; The For the Gaussian Laplace operator; The The standard deviation of the Gaussian function; The This is the adaptive enhancement coefficient.
4. The robotic arm automatic sorting method for quartz wafers as described in claim 1, characterized in that, The preprocessed visual image is converted into an input feature map and then input into the target detection model; The input feature map is processed according to the feature extraction formula to obtain the output feature map; The output feature map is decoded to obtain the category and image coordinates of the quartz wafer.
5. The robotic arm automatic sorting method for quartz wafers as described in claim 4, characterized in that, The feature extraction formula is: The The output feature map; The This is the sample number for the current batch; The The channel number of the input feature map; The The channel number of the output feature map; The The height index of the output feature map; The The width index of the output feature map; The This represents the traversal offset of the convolution kernel in the height direction; The This represents the offset of the convolution kernel in the width direction. The Input the number of channels; The The kernel size; The To connect the first The input channel and the first The kernel weight parameters for each output channel; The The input feature map; The The convolution stride; The For fill size; The For the first The bias term for each output channel.
6. The robotic arm automatic sorting method for quartz wafers as described in claim 4, characterized in that, The size relationship between the input feature map and the output feature map satisfies: The The height of the output feature map; The The width of the output feature map; The The height of the input feature map; The The width of the output feature map.
7. The robotic arm automatic sorting method for quartz wafers as described in claim 1, characterized in that, It also includes optimization steps for the target detection model: Define a loss function, minimize the loss value of the loss function, and update the convolution kernel weight parameters of the object detection model; The loss function is: The The number of positive samples; The Total number of categories; The Number the sample; The The category label number; The For the first The sample at the th The true binary annotation on each label; The For the first The sample at the th Predicted probabilities on each label.
8. An automated robotic arm sorting system for quartz wafers, characterized in that, The method described in any one of claims 1 to 7 comprises: an image acquisition module, a calculation and processing module, a control module, an execution module, and a communication module; The image acquisition module includes a monocular camera for acquiring raw visual images; The calculation and processing module is used to preprocess and identify the original visual image to calculate the actual position coordinates based on the acquired quartz wafer type and image coordinates. The execution module includes interconnected robotic arms, stepper motor drivers, and electromagnet-adhesive grippers; The control module is used to perform inverse kinematic model solving based on the actual position coordinates in order to control the action of the execution module; The communication module is used to maintain communication between the execution module and the control module.