Chip graph code identification method based on multi-order feature interaction

Abstract

The invention provides a chip graph code identification method based on multi-order feature interaction. The method comprises the following steps: collecting chip image data and preprocessing the chip image data; a feature extraction module of a YOLOv8 model is used for extracting image features, and the perception ability of a complex background is enhanced through multi-order feature interaction; a target detection module of YOLOv8 is used for detecting and classifying coding areas; post-processing the detection result to obtain a final coding area and category information; according to the chip image coding recognition method, by introducing multi-order feature interaction and a self-attention mechanism, the recognition precision of a chip image coding area is improved, the real-time performance and robustness of the model are optimized, and high-precision and high-efficiency chip image coding recognition can be achieved in a complex environment. Through accurate fitting of the geometric shape of the chip coding area, the size of the coding area can be efficiently estimated, and powerful support is provided for chip quality detection and automatic production.

Description

Technical Field

[0001] The present invention relates to the field of semiconductor technology, and in particular to a chip image coding recognition method based on multi-order feature interaction. Background Art

[0002] With the continuous development of the semiconductor industry, the design and manufacture of chips have become more and more complex. Traditional chip image processing technology mainly relies on static image feature extraction and simple image analysis methods, which have problems such as poor adaptability to complex backgrounds, low processing accuracy, and slow speed. Especially in the field of chip image code recognition, traditional methods have difficulty processing images with high density and complex backgrounds, and lack a deep understanding of different feature levels in the image.

[0003] The target detection algorithms in the existing technology (such as the YOLO series) have achieved good results in common object detection tasks, but their application in chip image code recognition still faces challenges in recognition accuracy and processing speed. Therefore, how to use deep learning algorithms to improve the automatic code recognition accuracy of chip images and optimize real-time performance has become a technical problem that needs to be solved urgently. Summary of the invention

[0004] In view of the shortcomings of the prior art, the purpose of the present invention is to provide a chip image code recognition method based on multi-order feature interaction to solve the problems raised in the above background technology. The present invention improves the recognition accuracy of the chip image coding area and optimizes the real-time and robustness of the model by introducing multi-order feature interaction and self-attention mechanism, and can achieve high-precision and high-efficiency chip image code recognition in complex environments.

[0005] In order to achieve the above object, the present invention is implemented by the following technical solution: a chip image coding recognition method based on multi-order feature interaction includes the following steps:

[0006] Step 1: Collect chip image data and perform preprocessing;

[0007] Step 2: Use the feature extraction module of the YOLOv8 model to extract image features and enhance the perception of complex backgrounds through multi-order feature interaction;

[0008] Step 3: Use the YOLOv8 target detection module to detect and classify the coding area;

[0009] Step 4: Post-process the detection results to obtain the final coding area and category information;

[0010] Step 5: Output the encoding recognition result.

[0011] Furthermore, in the step 1, a high-precision chip image acquisition device is used to capture the coding area on the chip surface, and the chip image acquisition device includes a high-resolution camera and an RGB-D sensor.

[0012] Furthermore, the collected images are processed by denoising, enhancement, and size standardization to ensure the high quality and consistency of the input data; for chip images under different lighting conditions, adaptive image enhancement technology is used to ensure the contrast and clarity of the image.

[0013] Furthermore, the step 2 is used to extract multi-level and multi-scale information in the chip image, and to strengthen the attention to the coding area based on the self-attention mechanism, so as to improve the recognition accuracy and robustness of the chip image coding area.

[0014] Furthermore, the YOLOv8 model adopts an anchor-free framework and pseudo-supervision technology to improve the detection accuracy of the model in various environments.

[0015] Furthermore, the YOLOv8 model is used to detect the coding regions in the chip image, and a convolutional neural network is used to classify the types of coding regions. The YOLOv8 model is used to automatically distinguish different types of coding regions and output the location information and category information of each coding region.

[0016] Furthermore, in step 4, a non-maximum suppression (NMS) algorithm is used to remove redundant detection frames and optimize the positioning accuracy of the remaining frames.

[0017] Furthermore, the coding recognition result output information includes the location information and category information of each coding region, providing a reliable recognition result for a subsequent automated chip detection system.

[0018] Furthermore, it also includes a training data set and an enhancement process, wherein a large-scale chip image data set is used in the training process, the chip image data set contains coding areas of different types and sizes, and data enhancement technology is used to simulate various complex production environments and image deformations to improve the generalization ability of the model; the `EMA-SlideLoss` loss function is used in the training process.

[0019] Furthermore, it also includes combining image processing methods with deep learning technology. The image processing methods include geometric morphology analysis and edge detection, and using prior geometric information to constrain the model to further improve recognition accuracy and reliability.

[0020] Beneficial effects of the present invention:

[0021] 1. The present invention deeply fuses low-level features (such as edges and textures) with high-level features (such as geometric shapes and symbol structures) in chip images through a multi-order feature interaction mechanism. Multi-order feature interaction enables the model to better capture the detailed information of the small coding area on the chip surface, thereby improving the accuracy of chip image code recognition.

[0022] 2. The present invention adopts a lightweight deep learning network structure and combines it with an efficient multi-scale feature extraction module to reduce computational complexity and improve reasoning speed. In addition, computational pruning and quantization optimization techniques are used to enable the model to run efficiently on devices with low computing resources and meet the needs of real-time chip detection systems.

[0023] 3. The present invention introduces a self-attention mechanism, which enables the model to dynamically adjust the focus of different areas. This mechanism not only improves the accuracy of the model for the coding area, but also effectively ignores irrelevant areas when there is an interfering background in the image, thereby improving the recognition accuracy of the chip coding area.

[0024] 4. The present invention combines prior geometric information and improves the recognition accuracy of the model on different coding areas by fusing the constraints of the chip surface geometry and coding information. By introducing shape constraints into the target detection model, the model can recognize different types of coding areas and correctly classify and locate them.

[0025] 5. The present invention uses a feature pyramid network (FPN) to extract features of different scales in chip images through multi-scale feature learning. Through this multi-scale feature fusion, the model can accurately detect small and large coding areas at different levels, ensuring that all types of coding areas on the chip surface can be efficiently identified.

[0026] 6. The present invention adopts an end-to-end training process, from chip image acquisition, preprocessing, feature extraction, to coding area detection and classification, and finally outputs the recognition result. Through this end-to-end training strategy, the model can automatically learn the features most relevant to the chip coding area, thereby improving the overall performance and generalization ability of the model. Especially in industrial chip production lines, the end-to-end optimization method can ensure that the recognition system has high stability and accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 It is a flow chart of the chip image code recognition method based on multi-order feature interaction of the present invention;

[0028] Figure 2 This is a schematic diagram of the YOLOv8 model structure;

[0029] Figure 3 It is a schematic diagram of the multi-order feature interaction module of the present invention. DETAILED DESCRIPTION

[0030] In order to make the technical means, creative features, objectives and effects achieved by the present invention easy to understand, the present invention is further explained below in conjunction with specific implementation methods.

[0031] See also Figures 1 to 3 , the present invention provides the following technical solutions: a chip image coding recognition method based on multi-order feature interaction, which improves the recognition accuracy of the chip image coding area and optimizes the real-time and robustness of the model by introducing multi-order feature interaction and self-attention mechanism. Compared with traditional image processing methods, the present invention has significant advantages in the accuracy, robustness and real-time performance of chip image coding recognition. It is widely used in automated detection tasks in the fields of chip design and manufacturing. By accurately fitting the geometric shape of the chip coding area, the size of the coding area can be efficiently estimated, providing strong support for chip quality detection and automated production.

[0032] This embodiment provides the following specific scheme of the method:

[0033] 1. Data collection and preprocessing

[0034] In order to achieve high-precision image acquisition under different chip production environments, the chip image code recognition method of the present invention requires full-scale image acquisition and preprocessing of the chip surface to ensure the accuracy of subsequent analysis.

[0035] Image acquisition: The present invention uses a high-resolution industrial camera (such as an RGB-D sensor or a CCD camera with a microscope function) for image acquisition to obtain subtle coding information on the chip surface. To ensure the comprehensive capture of the coding area, the image acquisition device must have a flexible angle adjustment function and be able to scan the chip surface from multiple angles and heights to avoid missing any coding information. For chips of different sizes, such as tiny IC chips or large integrated circuits, cameras and lenses of different specifications are used to obtain images with required clarity and resolution.

[0036] Lighting adjustment and control: In order to deal with the problems of uneven light and reflection that may occur in the production environment, the present invention adds a lighting adjustment module during the image acquisition process. By using devices such as adjustable LED light sources and polarizers, the influence of light reflection and highlight areas is reduced, thereby improving the clarity of the coding area.

[0037] Image preprocessing: The captured images are subjected to denoising, contrast enhancement, and size standardization to ensure image quality and consistency. Gaussian filtering, bilateral filtering and other denoising methods are used to reduce the impact of chip background noise, and the edges of the coding area are sharpened to improve image contrast and make the coding area clearer and easier to identify. In particular, histogram equalization is used to enhance regional details for tiny text, patterns and other information that may be contained in the coding area. In addition, in order to cope with different lighting conditions, brightness and contrast adaptive adjustment technology is further used to ensure consistent visibility of the coded information in all captured images.

[0038] 2. Feature extraction and multi-order feature interaction

[0039] The shape, size and position of the coding area on the chip surface vary, and the background is complex and diverse. Traditional feature extraction methods are difficult to accurately capture the information of these subtle coding areas. Therefore, the present invention adopts a multi-order feature interaction mechanism to improve the model's recognition effect on the chip coding area.

[0040] Improvement of YOLOv8 model: This paper adjusts the YOLOv8 model to adapt to the complex structure of chip images. Through the anchor-free framework design, the YOLOv8 model is enhanced in small target detection and can better capture the small coding area on the chip surface. The model extracts multi-level features of the chip image through a multi-layer convolutional network, where low-level features retain the details (such as texture, edge) of the coding area, while high-level features capture more abstract coding shapes and patterns.

[0041] Multi-order feature interaction: To further improve the model's accurate recognition of the chip coding area, the present invention introduces a multi-order feature interaction module, which weights and fuses feature information at different levels through a self-attention mechanism. This module enables the model to adaptively adjust according to local details and global features in the actual image, focusing on the key information of the coding area in the chip image while ignoring background noise or interference from non-coding areas. This mechanism ensures that the model can accurately extract core features regardless of how the shape or position of the coding area changes.

[0042] 3. Coding region detection and classification

[0043] The detection module of the present invention can automatically detect and classify different types of coding areas distributed on the chip surface. In order to cope with various forms of coding (such as identifiers, batch numbers, two-dimensional codes, etc.) in chip images, the classification module is specially optimized.

[0044] Coding region detection: YOLOv8's multi-scale detection strategy enables it to detect coding regions of different sizes at the same time. The model first generates multiple candidate boxes to cover possible coding regions. By introducing high- and low-level feature interactions, the model can effectively handle tiny coding regions (such as small logos or patterns) and large coding regions (such as QR codes or batch numbers) on the chip surface. This multi-scale strategy ensures that coding regions can be effectively detected at different sizes and under complex backgrounds.

[0045] Coding type classification: The content of the coding area on the chip surface usually contains a variety of information, such as product number, manufacturing batch, QR code or barcode. The present invention improves the classification module of YOLOv8 and combines the multi-layer classifier in the convolutional neural network to make a category judgment for each detected coding area and output the corresponding category label and confidence. The classifier uses the Softmax function to estimate the probability of the category of each coding area, ensuring that the model can quickly and accurately classify according to the production requirements of the chip.

[0046] 4. Post-processing and result output

[0047] In order to ensure the accuracy and uniqueness of the code recognition result, the present invention adopts a non-maximum suppression (NMS) algorithm and an accurate position information optimization module.

[0048] Post-processing: After detecting and obtaining candidate frames, the non-maximum suppression (NMS) algorithm is used to remove duplicate candidate frames. NMS removes candidate frames with high overlap by calculating the overlap ratio (IoU) between candidate frames, and only retains the optimal frame of each coding region. The post-processing module ensures that the boundaries of the coding regions in the final output are clear and non-overlapping, which improves the accuracy of the detection results.

[0049] Output results: The test results after post-processing optimization contain the location information, category label and confidence of each coding area. In order to meet the needs of actual industrial environments, these results are output to the chip detection database or display system in real time. The output data format is compatible with multiple interfaces, which is convenient for subsequent production inspection, quality monitoring and other applications, ensuring efficient and accurate quality control of every link in chip production.

[0050] 5. Model training and optimization

[0051] In order to achieve high-precision requirements for chip image coding recognition, the present invention combines multiple optimization strategies in the model training process to ensure that the model has strong adaptability to actual chip images.

[0052] Dataset construction and enhancement: The model is trained using a dataset containing a large number of chip coding images, which cover coding areas of different types and sizes and various background conditions. In order to further improve the generalization and robustness of the model, data enhancement techniques (such as rotation, scaling, translation, color jitter, etc.) are used to simulate image transformations under different lighting conditions and production environments. The diversity of the dataset ensures the applicability of the model in practical applications.

[0053] Training strategy: During the training process, the present invention adopts an adaptive learning rate strategy and combines it with the Adam optimizer to accelerate model convergence and improve stability. In order to improve the accuracy of the model, regularization technology and early stopping mechanism are also introduced to ensure that overfitting is avoided during the training process. For model parameters, the present invention adopts automatic parameter adjustment technology to ensure that the model maintains the best performance in different types of chip code recognition tasks.

[0054] 6. Performance evaluation and experimental verification

[0055] In order to verify the practical application effect of the present invention, a detailed experimental evaluation was carried out on the encoding recognition performance of chip images.

[0056] Accuracy evaluation: In the chip image code recognition task, by comparing with the traditional method, the high accuracy of the present invention under complex background, illumination changes and various coding forms is verified experimentally. The experimental results show that the present invention is superior to the existing methods in key indicators such as accuracy, recall rate, F1 score, etc., especially in the recognition of small coding areas and robustness under background interference.

[0057] Real-time verification: The real-time performance of the present invention was tested in an industrial environment, and the results showed that the model can complete the detection and classification of chip image coding areas within milliseconds. The real-time test results prove that the present invention can meet the real-time processing requirements of modern automated production lines and provide strong technical support for chip manufacturing and quality control processes.

[0058] This example also provides formulas used in the model training and experimental verification parts to enhance the mathematical basis of the implementation details:

[0059] 1. Loss function: During the training process, the EMA-SlideLoss loss function is used, which can better handle the problem of sample imbalance and focus on difficult samples. It is defined as:

[0060] \[

[0061] L_{\text{loss}}=\sum_{i

[0062] =1}^{N}\left(\alpha\cdot\max(0,1-y_i\cdot

[0063] \hat{y_i})+\beta\cdot\left(1-\frac{TP}{TP+FN}

[0064] \right)\right)

[0065] \]

[0066] Among them, \(y_i\) is the actual label, \(\hat{y_i}\) is the predicted value, \(TP\) and \(FN\) are TruePositive and False Negative respectively, and \(\alpha\) and \(\beta\) are hyperparameter adjustment coefficients.

[0067] 2. Object detection accuracy evaluation: In the object detection task, the performance of the model is evaluated by calculating `Mean Intersection over Union (MIoU)`, `Average Precision (AP)`, `Mean Average Precision (mAP)`, and `F1-score`:

[0068] \[\text{MIoU}=\frac{\text{Area Overlap}}{\text{Area Union}}

[0069] =\frac{TP}{FP+TP+FN}\]

[0070] TP is true positive, FP is false positive, and FN is false negative.

[0071] 3. Chip coding area size estimation (geometric shape fitting): In the chip image coding area recognition, a method similar to fruit size estimation is used to extract the size information of the coding area through geometric shape fitting. For the coding area that is approximately spherical, the least squares method (Least Squares) is used for geometric shape fitting:

[0072] (x) 2 +(yb) 2 +(zc) 2 =R 2

[0073] Among them, \((a,b,c)\) is the center coordinate of the geometric shape, \(R\) is the radius, and \((x,y,z)\) is each data point in the encoding area. By optimizing these parameters, the size of the encoding area can be accurately estimated.

[0074] In addition, in order to deal with coding areas with large shape changes (such as irregular long strips or elliptical areas), the ellipse fitting method is used. This method uses an iterative algorithm (such as -John ellipse fitting) precisely determines the major and minor axes of the coding region, resulting in a more accurate size estimate.

[0075] The basic principles and main features of the present invention and the advantages of the present invention are shown and described above. For those skilled in the art, it is obvious that the present invention is not limited to the details of the above exemplary embodiments, and the present invention can be implemented in other specific forms without departing from the spirit or basic characteristics of the present invention.

[0076] In addition, it should be understood that although the present specification is described according to implementation modes, not every implementation mode contains only one independent technical solution. This description of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment may also be appropriately combined to form other implementation modes that can be understood by those skilled in the art.

Claims

1. A chip image coding recognition method based on multi-order feature interaction, characterized in that: The following steps are involved: Step 1: Collect chip image data and perform preprocessing; Step 2: Use the feature extraction module of the YOLOv8 model to extract image features and enhance the perception of complex backgrounds through multi-order feature interaction; Step 3: Use the YOLOv8 target detection module to detect and classify the coding area; Step 4: Post-process the detection results to obtain the final coding area and category information; Step 5: Output the encoding recognition result.

2. The chip image code recognition method based on multi-order feature interaction according to claim 1 is characterized in that: In the step 1, a high-precision chip image acquisition device is used to capture the coding area on the chip surface, and the chip image acquisition device includes a high-resolution camera and an RGB-D sensor.

3. The chip image code recognition method based on multi-order feature interaction according to claim 2 is characterized in that: The collected images are processed by denoising, enhancement, size standardization, etc. to ensure the high quality and consistency of the input data; for chip images under different lighting conditions, adaptive image enhancement technology is used to ensure the contrast and clarity of the image.

4. The chip image code recognition method based on multi-order feature interaction according to claim 1 is characterized in that: The step 2 is used to extract multi-level and multi-scale information in the chip image, and to strengthen the attention to the coding area based on the self-attention mechanism, so as to improve the recognition accuracy and robustness of the chip image coding area.

5. The chip image code recognition method based on multi-order feature interaction according to claim 4 is characterized in that: The YOLOv8 model adopts an anchor-free framework and pseudo-supervision technology to improve the detection accuracy of the model in various environments.

6. The chip image code recognition method based on multi-order feature interaction according to claim 1 is characterized in that: The YOLOv8 model is used to detect the coding regions in the chip image, and a convolutional neural network is used to classify the types of coding regions. The YOLOv8 model is used to automatically distinguish different types of coding regions and output the location information and category information of each coding region.

7. The chip image code recognition method based on multi-order feature interaction according to claim 1 is characterized in that: In the step 4, a non-maximum suppression (NMS) algorithm is used to remove redundant detection frames and optimize the positioning accuracy of the remaining frames.

8. The chip image code recognition method based on multi-order feature interaction according to claim 7 is characterized in that: The coding recognition result output information includes the location information and category information of each coding region, providing a reliable recognition result for the subsequent automated chip detection system.

9. The chip image code recognition method based on multi-order feature interaction according to claim 1, characterized in that: It also includes a training data set and an enhancement process. A large-scale chip image data set is used in the training process. The chip image data set contains coding areas of different types and sizes, and data enhancement technology is used to simulate various complex production environments and image deformations to improve the generalization ability of the model. The `EMA-SlideLoss` loss function is used in the training process.

10. The chip image code recognition method based on multi-order feature interaction according to claim 1, characterized in that: It also includes combining image processing methods with deep learning technology. The image processing methods include geometric morphology analysis and edge detection, and using prior geometric information to constrain the model to further improve recognition accuracy and reliability.

Images