Chip visual positioning method
By improving the Backbone, Neck, and Head parts of the Yolov8 model and adding angle information and loss functions, the problems of high computational cost, slow speed, insufficient accuracy and stability of chip visual localization methods are solved, and fast, accurate and universal chip localization is achieved.
Patent Information
- Application Number
- CN202510916371.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-03
- Publication Date
- 2025-10-31
AI Technical Summary
Existing chip vision positioning methods involve large computational loads, slow positioning speed, insufficient accuracy and stability, and poor versatility, making them unsuitable for different chip models.
By improving the Yolov8 model, replacing its backbone with a MobileNetV2 network, the neck with an improved Rep-PAN module, and adding angle information to the head and angle loss to the loss function, the model is trained to adapt to different chip models. The improved Yolov8 model is then used to predict the chip's position and angle information.
With a relatively small computational load, it improves the speed, accuracy, and stability of chip positioning, and has high versatility, enabling it to adapt to positioning different types of chips.
Smart Images
Figure CN120876599A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to visual positioning methods, and more particularly to a chip visual positioning method. Background Technology
[0002] With the rapid development of information technology, chips, as core components of electronic devices or integrated circuit systems, are becoming increasingly important in terms of performance, reliability, and stability. In modern electronic devices, chips are widely used in smartphones, computers, servers, and IoT devices, with demand continuing to grow. However, this also brings unprecedented challenges to chip testing. Traditional chip testing methods rely on manual labor and simple equipment, which have significant shortcomings in meeting mass production demands. On the one hand, manual testing is inefficient, with operators spending a lot of time on chip loading and unloading, equipment parameter settings, and result recording and analysis, resulting in long testing cycles. On the other hand, manual testing suffers from poor accuracy and reliability; human factors can cause errors, leading to defective products entering the market or misjudging qualified chips.
[0003] To address the problems of traditional chip testing methods, automated chip testing technology has emerged. Introducing automated testing equipment can effectively improve testing efficiency and accuracy, while reducing labor costs and the risk of human error. Continuity and stability are particularly important during the testing process; therefore, it is necessary to ensure accurate alignment between the chip and the test socket, based on a fixed test socket. Thus, accurately positioning the chip and aligning it with the test socket is a crucial step in the chip testing process.
[0004] Currently, the mainstream method for chip positioning is a visual positioning method based on image feature matching. This method pre-sets a chip image with high positional accuracy as a template. By judging the similarity of features between the image of the chip under test and the template, and deeply analyzing the positional relationship between the two, the required position and angle information is obtained. Finally, based on the obtained position and angle information, the actuator is controlled to adjust the position of the chip under test, aligning it with the test socket. This visual positioning method typically uses edge, corner, and shape parameters as matching features to obtain effective feature information for matching, resulting in low computational load and faster chip positioning. However, this method only utilizes partial grayscale information when judging feature similarity, which makes errors more obvious during feature matching, negatively impacting positioning accuracy. Furthermore, it requires selecting a chip image as a template, limiting its applicability to chips with the same model as the template, resulting in poor versatility. The choice of matching element directly affects positioning quality, and overall stability needs improvement. Summary of the Invention
[0005] The technical problem to be solved by the present invention is to provide a chip visual positioning method that has a small amount of computation, can speed up chip positioning, improve positioning accuracy and stability, and has high versatility.
[0006] The technical solution adopted by this invention to solve the above-mentioned technical problems is as follows: a chip visual positioning method, which first obtains the required position and angle information through visual positioning technology, and then controls the actuator to adjust the position of the chip under test according to the obtained position and angle information to achieve chip visual positioning. The specific way to obtain the required position and angle information through visual positioning technology is as follows: first, the YOLOv8 model is improved to obtain an improved YOLOv8 model, and then the improved YOLOv8 model is trained to obtain a trained improved YOLOv8 model. Then, the image of the chip to be positioned on the test stand is acquired, and the chip image is preprocessed into a format that the trained improved YOLOv8 model can recognize and then input into the trained improved YOLOv8 model. The trained improved YOLOv8 model predicts the chip image within it and obtains the coordinates of the four corner points of the prediction box and the predicted angle output. Finally, the required position and angle information is calculated according to the coordinates of the four corner points of the prediction box to avoid positioning errors caused by prediction angle deviation.
[0007] The improved Yolov8 model is obtained by making the following improvements:
[0008] 1. Replace the feature extraction network in the Backbone part of the Yolov8 model with the MobileNetV2 network;
[0009] 2. Replace the feature fusion module of the Neck part of the Yolov8 model with an improved Rep-PAN module. The improved Rep-PAN module is obtained by making the following improvements to the Rep-PAN module:
[0010] (1) Replace the RepBlock module of the Rep-PAN module with a multi-head attention module;
[0011] (2) Replace the 3×3 convolution kernel in the Rep-PAN module with a 1×1 convolution kernel;
[0012] Third, add angle information to the detection information of the Head part of the Yolov8 model, so that the detection information of the Head part includes three aspects: classification, regression and angle.
[0013] Fourth, add an angle loss to the loss function of the Yolov8 model, so that the loss function of the improved Yolov8 model includes classification loss, regression loss and angle loss calculation.
[0014] Compared with existing technologies, the advantages of this invention lie in improving the YOLOv8 model to obtain an improved YOLOv8 model for prediction. This model outputs the coordinates of the four corner points of the prediction box and the predicted angles. The required position and angle information are then calculated based on the coordinates of the four corner points, avoiding positioning errors caused by prediction angle deviations. Specifically, replacing the feature extraction network in the backbone of the YOLOv8 model with a MobileNetV2 network accelerates the detection speed of the improved YOLOv8 model. Furthermore, replacing the feature fusion module in the Neck part of the YOLOv8 model with an improved Rep-PAN module that includes a multi-head attention module and 1×1 convolutional kernels allows the improved YOLOv8 model to fully utilize feature extraction information, improving the accuracy and stability of chip positioning. This invention reduces the number of parameters and computational load, accelerating the operation of the improved YOLOv8 model. It adds angle information to the detection information in the Head part of the YOLOv8 model, making the Head part's detection information include classification, regression, and angle information. It also adds angle loss to the loss function of the YOLOv8 model, making the improved YOLOv8 model's loss function include classification loss, regression loss, and angle loss calculations. This allows the improved YOLOv8 model to predict chip location information. Furthermore, by training the improved YOLOv8 model with data from different chip models, it can achieve location prediction capabilities for different chip models. Therefore, this invention, while having a smaller computational load and accelerating chip positioning speed, also improves positioning accuracy and stability, and has high versatility.
[0015] Furthermore, the specific process of calculating the required position and angle information based on the coordinates of the four corner points of the prediction box is as follows:
[0016] Step S1: Record the coordinates of the four corner points of the prediction box as (x1, y1), (x2, y2), (x3, y3), and (x4, y4), respectively. Among them, (x1, y1) and (x3, y3) are a pair of diagonal coordinates, (x2, y2) and (x4, y4) are a pair of diagonal coordinates, x1, x2, x3, and x4 represent the x-coordinates of the four corner points of the prediction box, and y1, y2, y3, and y4 represent the y-coordinates of the four corner points of the prediction box.
[0017] The coordinates of the center point (x, y) are obtained from the coordinates of the four corner points of the prediction box. center y center ), center point coordinates (xcenter y center This is the required location information, where the x-coordinate of the center point is... center The ordinate y is calculated using formula (1). center The result is obtained by calculation using formula (2):
[0018]
[0019] Step S2: Calculate the arc angle (Angle) that the chip needs to rotate in the horizontal plane using formula (3):
[0020]
[0021] Where π represents the value of a circle, and arctan represents the arctangent function;
[0022] Step S3: Calculate the rotation angle δ using formula (4). The rotation angle δ is the required angle information.
[0023] δ=Angle*180 / π (4)
[0024] Here, * represents the multiplication operator.
[0025] Furthermore, based on the obtained position and angle information, the actuator is controlled to adjust the position of the chip under test, and the specific process of achieving chip visual positioning is as follows:
[0026] Step A1: Take the center point of the captured chip image as the reference point and mark its coordinates as follows:
[0027] Step A2, Calculation and (x) center y center The offset between these two coordinates;
[0028] Step A3: The actuator moves the chip on the test stand according to the offset obtained in step A2, so that the center of the chip is completely aligned with the reference point. That is, the center coordinates of the prediction box obtained by the trained improved Yolov8 model for the current chip image are...
[0029] Step A4: If δ is negative, the actuator rotates the chip on the test socket counterclockwise around the center point of the captured chip image, with a rotation angle of |δ|. If δ is positive, the actuator rotates the chip on the test socket clockwise around the center point of the captured chip image, with a rotation angle of δ. || represents the absolute value sign.
[0030] Furthermore, the specific process of training the improved Yolov8 model to obtain the trained improved Yolov8 model is as follows:
[0031] Step B1: Set the image acquisition device in the same position and acquire images of each type of chip that needs to be located at different positions on the test stand. When taking images of each type of chip, deflect the chip located at a certain position to change its orientation so as to take images of the chip at different orientations at the same center position to meet the subsequent training requirements. At this time, several chip images including images of various types of chips are obtained.
[0032] Step B2: Use X-AnyLabeling annotation software to annotate each chip image. The annotation content includes the coordinates of the four corner points of the chip in each chip image and the corresponding chip model. Save the annotation content of each chip image as its annotation file to obtain several annotated images and several annotation files of the annotated images.
[0033] Step B3: Crop each labeled image to place the chip in the center of the image, resulting in several cropped images;
[0034] Step B4: Perform grayscale processing on each cropped image to obtain several grayscale images;
[0035] Step B5: Scale each grayscale image to a pixel size of 640*640 to obtain several scaled images. Use these scaled images to form the first sample set.
[0036] Step B6: Randomly select a portion of scaled images from the first sample set, and randomly flip the selected portion of scaled images vertically or horizontally to obtain several flipped images. These several flipped images constitute the second sample set.
[0037] Step B7: Select flipped images from the second sample set multiple times, selecting multiple flipped images each time, and stitching all the selected flipped images together so that the target, i.e. the chip, in all the selected flipped images is stitched into one of the flipped images to form an enhanced image. The annotation files of all the selected flipped images are merged into one annotation file as the annotation file of the enhanced image. At this time, several enhanced images and corresponding annotation files are obtained.
[0038] Step B8: Take all scaled images from the first sample set, all flipped images from the second sample set, and all enhanced images from the second sample set as sample images to form a dataset;
[0039] Step B9: Randomly divide the sample images in the dataset into a training set and a validation set, wherein the ratio of the number of sample images in the training set to the number of sample images in the validation set is... This is the floor symbol;
[0040] Step B10: After setting the training parameters, train the improved Yolov8 model. After each epoch of training on the training set, test it using the validation set and observe the detection accuracy of the currently trained improved Yolov8 model. If its overall loss function does not decrease compared to the previous epoch, the training is complete, and the current improved Yolov8 model is the trained improved Yolov8 model. If its loss function continues to decrease compared to the previous epoch, continue training.
[0041] Furthermore, in the loss function of the improved Yolov8 model, the classification loss uses binary cross-entropy loss to evaluate whether the improved Yolov8 model's prediction of the chip model is correct, the regression loss uses CIOU loss to measure the difference between the predicted box and the true box, where CIOU loss is based on IOU loss and also considers the difference in center point distance and aspect ratio; the angle loss uses mean squared error loss to calculate the average of the squares of the difference between the predicted angle and the true angle.
[0042] Furthermore, the overall loss function of the improved Yolov8 model is expressed by formula (5):
[0043] L=λ cls L cls +λ reg L reg +λ angle L angle (5)
[0044] Where L represents the overall loss function value of the improved Yolov8 model, L cls L represents the classification loss. reg L represents the regression loss. angle λ represents the angle loss. cls λ represents the classification loss weight, with a value of 0.2. reg λ represents the regression loss weight, with a value of 0.4. angle This represents the angle loss weight, with a value of 0.4.
[0045] Classification loss L cls Regression loss L reg and angle loss L angle The results were obtained by using equations (6), (7), and (8) respectively:
[0046]
[0047] Where N is the number of sample images in the training set, y i p represents the model number labeled in the i-th sample image of the training set. i Let represent the probability of predicting the model number for the i-th sample image in the training set. Let A represent the intersection-union ratio (IoU) between the predicted bounding box and the ground truth bounding box in the i-th sample image of the training set, where A i B represents the predicted bounding box output by the improved Yolov8 model when predicting the i-th sample image. i Label the ground truth bounding box formed by the four coordinate points for the i-th sample image. This represents the center point of the predicted bounding box in the i-th sample image. its center point The square of the Euclidean distance between them, c i ν is the diagonal length of the smallest closure region containing the predicted bounding box and its ground truth bounding box of the i-th sample image, α is a weight parameter with a value of 0.1, and ν i It is a term used to measure the difference in aspect ratio between the predicted bounding box and the ground truth bounding box in the i-th sample image, through... Calculated This represents the width of the ground truth bounding box in the i-th sample image. w represents the height of the ground truth bounding box in the i-th sample image. i h represents the width of the predicted bounding box for the i-th sample image. i It represents the height of the predicted bounding box for the i-th sample image; θ is the true rotation angle of the i-th sample image in the training set. i That is the corresponding prediction angle. Attached Figure Description
[0048] Figure 1 This is a flowchart of the chip visual positioning method of the present invention;
[0049] Figure 2 This describes the training and testing process of the chip visual positioning method in this embodiment of the invention.
[0050] Figure 3 This is a theoretical schematic diagram illustrating the calculation of the center point coordinates in an embodiment of the present invention;
[0051] Figure 4 This is a theoretical schematic diagram of calculating the rotation angle in an embodiment of the present invention. Detailed Implementation
[0052] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments.
[0053] Example 1: As Figure 1As shown, a chip visual positioning method first obtains the required position and angle information through visual positioning technology, and then controls the actuator to adjust the position of the chip under test according to the obtained position and angle information to achieve chip visual positioning. The specific method of obtaining the required position and angle information through visual positioning technology is as follows: first, the YOLOv8 model is improved to obtain an improved YOLOv8 model, and then the improved YOLOv8 model is trained to obtain a trained improved YOLOv8 model. Then, the image of the chip to be positioned on the test stand is acquired, and the chip image is preprocessed into a format that the trained improved YOLOv8 model can recognize before being input into the trained improved YOLOv8 model. The trained improved YOLOv8 model predicts the chip image within it and obtains the coordinates of the four corner points of the prediction box and the predicted angle output. Finally, the required position and angle information is calculated according to the coordinates of the four corner points of the prediction box to avoid positioning errors caused by prediction angle deviation.
[0054] The improved Yolov8 model is obtained by making the following improvements:
[0055] 1. Replace the feature extraction network in the backbone of the Yolov8 model with the MobileNetV2 network;
[0056] 2. Replace the feature fusion module of the Neck part of the Yolov8 model with an improved Rep-PAN module. The improved Rep-PAN module is obtained by making the following improvements to the Rep-PAN module:
[0057] (1) Replace the RepBlock module of the Rep-PAN module with the multi-head attention module;
[0058] (2) Replace the 3×3 convolution kernel in the Rep-PAN module with a 1×1 convolution kernel;
[0059] Third, add angle information to the detection information of the Head part of the Yolov8 model, so that the detection information of the Head part includes three aspects: classification, regression and angle.
[0060] Fourth, add angular loss to the loss function of the Yolov8 model, so that the loss function of the improved Yolov8 model includes the calculation of classification loss, regression loss and angular loss.
[0061] In this embodiment, the feature extraction network in the Backbone part of the YOLOv8 model is replaced with a MobileNetV2 network, which accelerates the detection speed of the improved YOLOv8 model. The feature fusion module in the Neck part of the YOLOv8 model is replaced with an improved Rep-PAN module with a multi-head attention module and a 1×1 convolutional kernel, enabling the improved YOLOv8 model to fully utilize feature extraction information, improve the accuracy and stability of chip localization, reduce the number of parameters and computation, and speed up the operation of the improved YOLOv8 model. Angle information is added to the detection information in the Head part of the YOLOv8 model, so that the detection information in the Head part includes classification, regression, and angle information. Angle loss is added to the loss function of the YOLOv8 model, so that the loss function of the improved YOLOv8 model includes classification loss, regression loss, and angle loss calculation, thereby enabling the improved YOLOv8 model to predict the chip position information. In addition, by training the improved YOLOv8 model with data from different models of chips, the improved YOLOv8 model can be made capable of predicting the position of different models of chips.
[0062] Example 2: This example is basically the same as Example 1, except that: in this example, as Figure 3 and Figure 4 As shown, the specific process of calculating the required position and angle information based on the coordinates of the four corner points of the prediction box is as follows:
[0063] Step S1: Record the coordinates of the four corner points of the prediction box as (x1, y1), (x2, y2), (x3, y3), and (x4, y4), respectively. Among them, (x1, y1) and (x3, y3) are a pair of diagonal coordinates, (x2, y2) and (x4, y4) are a pair of diagonal coordinates, x1, x2, x3, and x4 represent the x-coordinates of the four corner points of the prediction box, and y1, y2, y3, and y4 represent the y-coordinates of the four corner points of the prediction box.
[0064] The coordinates of the center point (x, y) are obtained from the coordinates of the four corner points of the prediction box. center y center ), center point coordinates (x center y center This is the required location information, where the x-coordinate of the center point is... center The ordinate y is calculated using formula (1). center The result is obtained by calculation using formula (2):
[0065]
[0066] Step S2: Calculate the arc angle (Angle) that the chip needs to rotate in the horizontal plane using formula (3):
[0067]
[0068] Where π represents the value of a circle, and arctan represents the arctangent function;
[0069] Step S3: Calculate the rotation angle δ using formula (4). The rotation angle δ is the required angle information.
[0070] δ=Angle*180 / π (4)
[0071] Here, * represents the multiplication operator.
[0072] Example 3: This example is basically the same as Example 2, except that: In this example, the specific process of controlling the actuator to adjust the position of the chip under test based on the obtained position and angle information to achieve chip visual positioning is as follows:
[0073] Step A1: Take the center point of the captured chip image as the reference point and mark its coordinates as follows:
[0074] Step A2, Calculation and (x) center y center The offset between these two coordinates;
[0075] Step A3: The actuator moves the chip on the test stand according to the offset obtained in step A2, so that the center of the chip is perfectly aligned with the reference point. That is, the center coordinates of the prediction box obtained by the trained improved Yolov8 model for the current chip image are...
[0076] Step A4: If δ is negative, the actuator rotates the chip on the test socket counterclockwise around the center point of the captured chip image, with a rotation angle of |δ|. If δ is positive, the actuator rotates the chip on the test socket clockwise around the center point of the captured chip image, with a rotation angle of δ. || represents the absolute value sign.
[0077] Example 4: This example is basically the same as Example 3, except that: in this example, as Figure 2 As shown, the specific process of training the improved Yolov8 model to obtain the trained improved Yolov8 model is as follows:
[0078] Step B1: Set the image acquisition device in the same position and acquire images of each type of chip that needs to be located at different positions on the test stand. When taking images of each type of chip, deflect the chip located at a certain position to change its orientation so as to take images of the chip at different orientations at the same center position to meet the subsequent training requirements. At this time, several chip images including images of various types of chips are obtained.
[0079] Step B2: Use X-AnyLabeling annotation software to annotate each chip image. The annotation content includes the coordinates of the four corner points of the chip in each chip image and the corresponding chip model. Save the annotation content of each chip image as its annotation file to obtain several annotated images and several annotation files of the annotated images.
[0080] Step B3: Crop each labeled image to place the chip in the center of the image, resulting in several cropped images;
[0081] Step B4: Perform grayscale processing on each cropped image to obtain several grayscale images;
[0082] Step B5: Scale each grayscale image to a pixel size of 640*640 to obtain several scaled images. Use these scaled images to form the first sample set.
[0083] Step B6: Randomly select a portion of scaled images from the first sample set, and randomly flip the selected portion of scaled images vertically or horizontally to obtain several flipped images. These several flipped images constitute the second sample set.
[0084] Step B7: Select flipped images from the second sample set multiple times, selecting multiple flipped images each time, and stitching all the selected flipped images together so that the target, i.e. the chip, in all the selected flipped images is stitched into one of the flipped images to form an enhanced image. The annotation files of all the selected flipped images are merged into one annotation file as the annotation file of the enhanced image. At this time, several enhanced images and corresponding annotation files are obtained.
[0085] Step B8: Take all scaled images from the first sample set, all flipped images from the second sample set, and all enhanced images from the second sample set as sample images to form a dataset;
[0086] Step B9: Randomly divide the sample images in the dataset into a training set and a validation set, wherein the ratio of the number of sample images in the training set to the number of sample images in the validation set is... This is the floor symbol;
[0087] Step B10: After setting the training parameters, train the improved Yolov8 model. After each epoch of training on the training set, test it using the validation set and observe the detection accuracy of the currently trained improved Yolov8 model. If its overall loss function does not decrease compared to the previous epoch, the training is complete, and the current improved Yolov8 model is the trained improved Yolov8 model. If its loss function continues to decrease compared to the previous epoch, continue training.
[0088] In this embodiment, the loss function of the improved Yolov8 model uses binary cross-entropy loss for classification loss to evaluate whether the improved Yolov8 model's prediction of chip model is correct, CIOU loss for regression loss to measure the difference between the predicted box and the true box, where CIOU loss is based on IOU loss and also considers the difference in center point distance and aspect ratio; and mean squared error loss for angle loss to calculate the average of the squares of the difference between the predicted angle and the true angle.
[0089] In this embodiment, the overall loss function of the improved Yolov8 model is expressed by formula (5):
[0090] L=λ cls L cls +λ reg L reg +λ angle L angle (5)
[0091] Where L represents the overall loss function value of the improved Yolov8 model, L cls L represents the classification loss. reg L represents the regression loss. angle λ represents the angle loss. cls λ represents the classification loss weight, with a value of 0.2. reg λ represents the regression loss weight, with a value of 0.4. angle This represents the angle loss weight, with a value of 0.4.
[0092] Classification loss L cls Regression loss L reg and angle loss L angle The results were obtained by using equations (6), (7), and (8) respectively:
[0093]
[0094] Where N is the number of sample images in the training set, y i p represents the model number labeled in the i-th sample image of the training set. i Let represent the probability of predicting the model number for the i-th sample image in the training set. Let A represent the intersection-union ratio (IoU) between the predicted bounding box and the ground truth bounding box in the i-th sample image of the training set, where A i B represents the predicted bounding box output when the improved Yolov8 model predicts the i-th sample image. i Label the ground truth bounding box formed by the four coordinate points for the i-th sample image. This represents the center point of the predicted bounding box in the i-th sample image. its center point The square of the Euclidean distance between them, c i ν is the diagonal length of the smallest closure region containing the predicted bounding box and its ground truth bounding box of the i-th sample image, α is a weight parameter with a value of 0.1, and ν i It is a term used to measure the difference in aspect ratio between the predicted bounding box and the ground truth bounding box in the i-th sample image, through... Calculated This represents the width of the ground truth bounding box in the i-th sample image. w represents the height of the ground truth bounding box in the i-th sample image. i h represents the width of the predicted bounding box for the i-th sample image. i It represents the height of the predicted bounding box for the i-th sample image; θ is the true rotation angle of the i-th sample image in the training set. i That is the corresponding prediction angle.
[0095] In this embodiment, by introducing a variety of data augmentation strategies, the performance and robustness of the improved Yolov8 model in chip detection tasks are effectively improved; at the same time, by combining various loss function designs, the feature learning and discrimination capabilities of the improved Yolov8 model are further enhanced.
[0096] To verify the performance of the chip vision localization method of this invention, an improved Yolov8 model was constructed using Python code in PyCharm software. System experiments were conducted on a dataset of multiple chip models. This dataset contains 5467 training images, 1561 validation images, and 782 test images. The improved Yolov8 model adopted a 200-round training strategy, with 64 images input per round. An early stopping mechanism was introduced during training: if the overall loss function value of the validation set did not decrease further within 5 consecutive rounds, training was terminated early, and the weights of the improved Yolov8 model with the best performance on the validation set were retained. Accuracy and detection time were used as the main evaluation metrics for the improved Yolov8 model's performance. Accuracy measures the proportion of correct predictions made by the improved Yolov8 model across all images, while detection time reflects the average processing speed (in milliseconds) of the improved Yolov8 model per image. On the aforementioned dataset, the improved Yolov8 model achieved an average accuracy of 96.1%, with an average detection time of 83 milliseconds per image. Experimental results show that the improved Yolov8 model has high detection accuracy and real-time performance in chip vision localization tasks, and can achieve efficient and reliable chip recognition and localization.
[0097] In summary, the chip visual positioning method of the present invention improves the Yolov8 model. By using the improved Yolov8 model for chip visual positioning, it can accelerate chip positioning speed with less computation, while also improving positioning accuracy and stability. It also has high versatility and has broad application prospects in the field of chip positioning and detection.
Claims
1. A chip visual positioning method, comprising first obtaining the required position and angle information through visual positioning technology, and then controlling an actuator to adjust the position of the chip under test according to the obtained position and angle information, thereby achieving chip visual positioning, characterized in that... The specific method for obtaining the required position and angle information through visual positioning technology is as follows: First, the YOLOv8 model is improved to obtain an improved YOLOv8 model. Then, the improved YOLOv8 model is trained to obtain a trained improved YOLOv8 model. Next, the chip image to be positioned on the test stand is acquired. The chip image is preprocessed into a format that the trained improved YOLOv8 model can recognize and then input into the trained improved YOLOv8 model. The trained improved YOLOv8 model predicts the chip image within it and obtains the coordinates of the four corner points of the prediction box and the predicted angle output. Finally, the required position and angle information is calculated based on the coordinates of the four corner points of the prediction box to avoid positioning errors caused by prediction angle deviation. The improved Yolov8 model is obtained by making the following improvements:
1. Replace the feature extraction network in the Backbone part of the Yolov8 model with the MobileNetV2 network; 2. Replace the feature fusion module of the Neck part of the Yolov8 model with an improved Rep-PAN module. The improved Rep-PAN module is obtained by making the following improvements to the Rep-PAN module: (1) Replace the RepBlock module of the Rep-PAN module with a multi-head attention module; (2) Replace the 3×3 convolution kernel in the Rep-PAN module with a 1×1 convolution kernel; Third, add angle information to the detection information of the Head part of the Yolov8 model, so that the detection information of the Head part includes three aspects: classification, regression and angle. Fourth, add an angle loss to the loss function of the Yolov8 model, so that the loss function of the improved Yolov8 model includes classification loss, regression loss and angle loss calculation.
2. The chip visual positioning method according to claim 1, characterized in that... The specific process of calculating the required position and angle information based on the coordinates of the four corner points of the prediction box is as follows: Step S1: Record the coordinates of the four corner points of the prediction box as (x1, y1), (x2, y2), (x3, y3), and (x4, y4), respectively. Among them, (x1, y1) and (x3, y3) are a pair of diagonal coordinates, (x2, y2) and (x4, y4) are a pair of diagonal coordinates, x1, x2, x3, and x4 represent the x-coordinates of the four corner points of the prediction box, and y1, y2, y3, and y4 represent the y-coordinates of the four corner points of the prediction box. The coordinates of the center point (x, y) are obtained from the coordinates of the four corner points of the prediction box. center y center ), center point coordinates (x center y center This is the required location information, where the x-coordinate of the center point is... center The ordinate y is calculated using formula (1). center The result is obtained by calculation using formula (2): Step S2: Calculate the arc angle (Angle) that the chip needs to rotate in the horizontal plane using formula (3): Where π represents the value of a circle, and arctan represents the arctangent function; Step S3: Calculate the rotation angle δ using formula (4). The rotation angle δ is the required angle information. δ=Angle*180 / π (4) Here, * represents the multiplication operator.
3. The chip visual positioning method according to claim 2, characterized in that... The specific process of controlling the actuator to adjust the position of the chip under test based on the obtained position and angle information, and realizing chip visual positioning, is as follows: Step A1: Take the center point of the captured chip image as the reference point and mark its coordinates as follows: Step A2, Calculation and (x) center y center The offset between these two coordinates; Step A3: The actuator moves the chip on the test stand according to the offset obtained in step A2, so that the center of the chip is completely aligned with the reference point. That is, the center coordinates of the prediction box obtained by the trained improved Yolov8 model for the current chip image are... Step A4: If δ is negative, the actuator rotates the chip on the test socket counterclockwise around the center point of the captured chip image, with a rotation angle of |δ|. If δ is positive, the actuator rotates the chip on the test socket clockwise around the center point of the captured chip image, with a rotation angle of δ. || represents the absolute value sign.
4. The chip visual positioning method according to claim 1, characterized in that... The specific process of training the improved Yolov8 model to obtain the trained improved Yolov8 model is as follows: Step B1: Set the image acquisition device in the same position and acquire images of each type of chip that needs to be located at different positions on the test stand. When taking images of each type of chip, deflect the chip located at a certain position to change its orientation so as to take images of the chip at different orientations at the same center position to meet the subsequent training requirements. At this time, several chip images including images of various types of chips are obtained. Step B2: Use X-AnyLabeling annotation software to annotate each chip image. The annotation content includes the coordinates of the four corner points of the chip in each chip image and the corresponding chip model. Save the annotation content of each chip image as its annotation file to obtain several annotated images and several annotation files of the annotated images. Step B3: Crop each labeled image to place the chip in the center of the image, resulting in several cropped images; Step B4: Perform grayscale processing on each cropped image to obtain several grayscale images; Step B5: Scale each grayscale image to a pixel size of 640*640 to obtain several scaled images. Use these scaled images to form the first sample set. Step B6: Randomly select a portion of scaled images from the first sample set, and randomly flip the selected portion of scaled images vertically or horizontally to obtain several flipped images. These several flipped images constitute the second sample set. Step B7: Select flipped images from the second sample set multiple times, selecting multiple flipped images each time, and stitching all the selected flipped images together so that the target, i.e. the chip, in all the selected flipped images is stitched into one of the flipped images to form an enhanced image. The annotation files of all the selected flipped images are merged into one annotation file as the annotation file of the enhanced image. At this time, several enhanced images and corresponding annotation files are obtained. Step B8: Take all scaled images from the first sample set, all flipped images from the second sample set, and all enhanced images from the second sample set as sample images to form a dataset; Step B9: Randomly divide the sample images in the dataset into a training set and a validation set, wherein the ratio of the number of sample images in the training set to the number of sample images in the validation set is... This is the floor symbol; Step B10: After setting the training parameters, train the improved Yolov8 model. After each epoch of training on the training set, test it using the validation set and observe the detection accuracy of the currently trained improved Yolov8 model. If its overall loss function does not decrease compared to the previous epoch, the training is complete, and the current improved Yolov8 model is the trained improved Yolov8 model. If its loss function continues to decrease compared to the previous epoch, continue training.
5. The chip visual positioning method according to claim 4, characterized in that... In the loss function of the improved Yolov8 model, the classification loss uses binary cross-entropy loss to evaluate whether the improved Yolov8 model's prediction of the chip model is correct, the regression loss uses CIOU loss to measure the difference between the predicted box and the true box, where CIOU loss is based on IOU loss and also considers the difference in center point distance and aspect ratio; the angle loss uses mean squared error loss to calculate the average of the squares of the difference between the predicted angle and the true angle.
6. The chip visual positioning method according to claim 5, characterized in that... The overall loss function of the improved Yolov8 model is expressed by formula (5): L=λ cls L cls +λ reg L reg +λ angle L angle (5) Where L represents the overall loss function value of the improved Yolov8 model, L cls L represents the classification loss. reg L represents the regression loss. angle λ represents the angle loss. cls λ represents the classification loss weight, with a value of 0.
2. reg λ represents the regression loss weight, with a value of 0.
4. angle This represents the angle loss weight, with a value of 0.
4. Classification loss L cls Regression loss L reg and angle loss L angle The results were obtained by using equations (6), (7), and (8) respectively: Where N is the number of sample images in the training set, y i p represents the model number labeled in the i-th sample image of the training set. i Let represent the probability of predicting the model number for the i-th sample image in the training set. Let A represent the intersection-union ratio (IoU) between the predicted bounding box and the ground truth bounding box in the i-th sample image of the training set, where A i B represents the predicted bounding box output by the improved Yolov8 model when predicting the i-th sample image. i Label the ground truth bounding box formed by the four coordinate points for the i-th sample image. Center represents the center point of the predicted bounding box in the i-th sample image. Ai its center point The square of the Euclidean distance between them, c i ν is the diagonal length of the smallest closure region containing the predicted bounding box and its ground truth bounding box of the i-th sample image, α is a weight parameter with a value of 0.1, and ν i It is a term used to measure the difference in aspect ratio between the predicted bounding box and the ground truth bounding box in the i-th sample image, through... Calculated This represents the width of the ground truth bounding box in the i-th sample image. w represents the height of the ground truth bounding box in the i-th sample image. i h represents the width of the predicted bounding box for the i-th sample image. i It represents the height of the predicted bounding box for the i-th sample image; θ is the true rotation angle of the i-th sample image in the training set. i That is the corresponding prediction angle.