Integrated circuit marking detection methods, apparatus, computer equipment, readable storage media, and program products

By introducing a deep learning object detection network model and an attention mechanism layer, the problems of low accuracy and efficiency in integrated circuit logo detection are solved, and efficient automatic recognition of integrated circuit logos is achieved.

CN118314327BActive Publication Date: 2026-11-10CHINA ELECTRONICS RELIABILITY AND ENVIRONMENTAL TESTING INSTITUTE ((THE FIFTH INSTITUTE OF ELECTRONICS MINISTRY OF INDUSTRY AND INFORMATION TECHNOLOGY) (CHINA SAIBAO LABORATORY)
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
CN202410515602.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-04-26
Publication Date
2026-11-10
Estimated Expiration
2044-04-26

AI Technical Summary

Technical Problem

Existing technologies for integrated circuit logo detection and recognition suffer from poor accuracy and low efficiency.

Method used

A deep learning-based object detection network model is adopted. By acquiring image datasets, adding labels, training the object detection model, and introducing an attention mechanism layer into the model, the accuracy of feature extraction and recognition is improved.

Benefits of technology

It achieves efficient automatic recognition of integrated circuit logos, solving the problems of discontinuous recognition process and difficulty in feature extraction in traditional methods, and significantly improving recognition accuracy and efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118314327B_ABST
    Figure CN118314327B_ABST
Patent Text Reader

Abstract

This application relates to a method, apparatus, computer device, computer-readable storage medium, and computer program product for detecting integrated circuit markings. The method involves: acquiring an image dataset including sample images containing marking objects of corresponding integrated circuits; adding corresponding marking labels to the sample images; using the marking labels to locate and annotate the marking objects in the sample images; acquiring a training set based on the sample images and the marking labels, and training an initial neural network using the training set to obtain a target detection model; the target detection model including convolutional layers and attention mechanism layers; acquiring a target image of the integrated circuit to be detected, inputting the target image into the target detection model, and obtaining the marking detection result of the integrated circuit to be detected. This method can significantly improve the efficiency and accuracy of integrated circuit marking recognition.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of target detection technology, and in particular to an integrated circuit mark detection method, apparatus, computer equipment, computer-readable storage medium, and computer program product. Background Technology

[0002] Integrated circuit manufacturer logos serve as direct identifiers of integrated circuit production information, playing a crucial role in automated quality inspection and defect detection. Logo detection and recognition technology involves processing input images to locate the logo area and determine its specific category. However, in current inspection environments, traditional manual visual inspection methods are largely used for detecting and recognizing integrated circuit logos, which is inefficient and cannot guarantee recognition accuracy.

[0003] Currently, the recognition of integrated circuit logos suffers from poor accuracy and low efficiency. Summary of the Invention

[0004] Therefore, it is necessary to provide an integrated circuit logo detection method, apparatus, computer equipment, computer-readable storage medium, and computer program product that can improve the accuracy and efficiency of integrated circuit logo recognition in response to the above-mentioned technical problems.

[0005] In a first aspect, this application provides a method for detecting integrated circuit markings, including:

[0006] Obtain the image dataset; the image dataset includes sample images, which contain the marker objects of the corresponding sample integrated circuits;

[0007] Add corresponding labels to the sample images; the labels are used to locate and annotate the marked objects in the sample images.

[0008] A training set is obtained based on sample images and labeling, and the initial neural network is trained using the training set to obtain an object detection model; the object detection model includes convolutional layers and attention mechanism layers;

[0009] The image of the integrated circuit to be tested is acquired, and the image is input into the target detection model to obtain the mark detection result of the integrated circuit to be tested.

[0010] In one embodiment, obtaining the image dataset includes:

[0011] Obtain the raw dataset of integrated circuit surface images; the raw dataset includes integrated circuit surface images of multiple integrated circuit categories;

[0012] For each integrated circuit surface image in the original dataset, bilateral filtering and histogram equalization are performed to obtain the original dataset after image preprocessing.

[0013] Based on the number of images for each integrated circuit category in the original dataset, image data augmentation is performed on the preprocessed original dataset to obtain the image dataset.

[0014] In one embodiment, based on the number of images for each integrated circuit category in the original dataset, image data augmentation is performed on the preprocessed original dataset to obtain an image dataset, including:

[0015] The number of images for each integrated circuit category in the original dataset is compared with the standard number threshold. It is estimated that the integrated circuit categories included in the original dataset are compared with the preset category labels to determine the first integrated circuit category that is missing in the original dataset and the second integrated circuit category whose number of images is less than the standard number threshold.

[0016] Based on the first and second integrated circuit categories, a diffusion model is used to augment the image data of the preprocessed original dataset to obtain the image dataset.

[0017] In one embodiment, the label includes a bounding box and category information; adding the corresponding label to the sample image includes:

[0018] Determine the label format based on the input data format of the initial neural network;

[0019] Obtain integrated circuit-related information corresponding to the sample image; integrated circuit-related information is used to characterize the distinctive features of the corresponding integrated circuit;

[0020] Based on integrated circuit-related information, determine the marked areas and manufacturer categories of the sample images;

[0021] Based on the marked area and manufacturer category, add bounding boxes and category information in label format to the sample image; the bounding boxes are used to label the marked area, and the category information is used to characterize the manufacturer category.

[0022] In one embodiment, after adding a bounding box in label format and category information to the sample image based on the marked region and manufacturer category of the sample image, the method further includes:

[0023] Perform at least one round of data detection on the bounding boxes and category information in the sample images, as well as the data format of the bounding boxes and category information;

[0024] If, in each round of data detection, the bounding box and category information are determined to be in label format, the bounding box matches the label region, and the category information matches the manufacturer category, then the step of obtaining the training set based on the sample image and the label is executed.

[0025] In one embodiment, the image to be detected is input into a target detection model to obtain the mark detection result of the integrated circuit to be detected, including:

[0026] The image to be detected is input into the object detection model, and the image features of the image to be detected are obtained through the convolutional layer located before the attention mechanism layer in the object detection model.

[0027] The image features are processed through the attention mechanism layer of the object detection model to obtain a global feature map;

[0028] The global feature map is processed by the convolutional layer located after the attention mechanism layer in the object detection model to obtain the mark detection result of the integrated circuit to be detected.

[0029] Secondly, this application also provides an integrated circuit mark detection device, comprising:

[0030] The acquisition module is used to acquire an image dataset; the image dataset includes sample images, and the sample images include the marker objects of the corresponding sample integrated circuits;

[0031] The annotation module is used to add corresponding labels to sample images; the labels are used to locate and annotate the label objects in the sample images.

[0032] The training module is used to obtain a training set based on sample images and labels, and to train the initial neural network based on the training set to obtain an object detection model; the object detection model includes convolutional layers and attention mechanism layers;

[0033] The detection module is used to acquire the image of the integrated circuit to be detected, input the image into the target detection model, and obtain the mark detection result of the integrated circuit to be detected.

[0034] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0035] Obtain an image dataset; the image dataset includes sample images, and the sample images include the marker objects of the corresponding sample integrated circuits;

[0036] Add corresponding labels to the sample images; the labels are used to locate and annotate the marked objects in the sample images.

[0037] A training set is obtained based on the sample images and the label, and an initial neural network is trained based on the training set to obtain an object detection model; the object detection model includes convolutional layers and attention mechanism layers;

[0038] A test image of the integrated circuit to be tested is acquired, and the test image is input into the target detection model to obtain the mark detection result of the integrated circuit to be tested.

[0039] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0040] Obtain an image dataset; the image dataset includes sample images, and the sample images include the marker objects of the corresponding sample integrated circuits;

[0041] Add corresponding labels to the sample images; the labels are used to locate and annotate the marked objects in the sample images.

[0042] A training set is obtained based on the sample images and the label, and an initial neural network is trained based on the training set to obtain an object detection model; the object detection model includes convolutional layers and attention mechanism layers;

[0043] A test image of the integrated circuit to be tested is acquired, and the test image is input into the target detection model to obtain the mark detection result of the integrated circuit to be tested.

[0044] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, performs the following steps:

[0045] Obtain an image dataset; the image dataset includes sample images, and the sample images include the marker objects of the corresponding sample integrated circuits;

[0046] Add corresponding labels to the sample images; the labels are used to locate and annotate the marked objects in the sample images.

[0047] A training set is obtained based on the sample images and the label, and an initial neural network is trained based on the training set to obtain an object detection model; the object detection model includes convolutional layers and attention mechanism layers;

[0048] A test image of the integrated circuit to be tested is acquired, and the test image is input into the target detection model to obtain the mark detection result of the integrated circuit to be tested.

[0049] The aforementioned integrated circuit mark detection method, apparatus, computer equipment, computer-readable storage medium, and computer program product acquire an image dataset; the image dataset includes sample images, each containing a mark object of a corresponding sample integrated circuit; corresponding mark labels are added to the sample images; the mark labels are used to locate and annotate the mark objects in the sample images; a training set is acquired based on the sample images and the mark labels, and an initial neural network is trained using the training set to obtain a target detection model; the target detection model includes convolutional layers and attention mechanism layers; a target image of the integrated circuit to be detected is acquired, and the target image is input into the target detection model to obtain the mark detection result of the integrated circuit to be detected. The deep learning-based target detection network model can effectively solve the problems of complex steps, difficult feature extraction, and discontinuous recognition process in traditional algorithms, and can greatly improve the recognition efficiency and accuracy of integrated circuit marks. Attached Figure Description

[0050] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0051] Figure 1 This is an application environment diagram of an integrated circuit mark detection method in one embodiment;

[0052] Figure 2 This is a flowchart illustrating an integrated circuit mark detection method in one embodiment;

[0053] Figure 3 This is a schematic diagram illustrating the effect of bilateral filtering in one embodiment;

[0054] Figure 4 This is a schematic diagram of the histogram equalization effect in one embodiment;

[0055] Figure 5 This is a schematic diagram illustrating the process of adding labels to a sample image in one embodiment;

[0056] Figure 6 This is a schematic diagram of the attention mechanism layer architecture in one embodiment;

[0057] Figure 7 This is a schematic diagram of the channel attention mechanism submodule architecture in one embodiment;

[0058] Figure 8 This is a schematic diagram of the spatial attention mechanism submodule architecture in one embodiment;

[0059] Figure 9 A schematic diagram of the YOLO-v8 model architecture with GAM added in one embodiment;

[0060] Figure 10 This is a structural block diagram of an integrated circuit mark detection device in one embodiment;

[0061] Figure 11 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0062] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0063] The integrated circuit mark detection method provided in this application embodiment can be applied to, for example... Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104 or located on the cloud or other network servers. Terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, projection devices, etc. Portable wearable devices can include smartwatches, smart bracelets, head-mounted devices, etc. Head-mounted devices can be virtual reality (VR) devices, augmented reality (AR) devices, smart glasses, etc. Server 104 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0064] In one exemplary embodiment, such as Figure 2 As shown, a method for detecting integrated circuit markings, also known as integrated circuit logos, is provided. This method is applied to... Figure 1 Taking terminal 102 as an example, the explanation includes the following steps 202 to 208. Wherein:

[0065] Step 202: Obtain the image dataset; the image dataset includes sample images, and the sample images include the marker objects of the corresponding sample integrated circuits.

[0066] Optionally, the integrated circuit logo image can be acquired and analyzed to generate an original image dataset.

[0067] Step 204: Add corresponding labels to the sample image; the labels are used to locate and annotate the objects in the sample image.

[0068] Optionally, since training the deep detection network model requires a large amount of labeled data, professional annotation software is used to define and label the integrated circuit logo area, and a label is added to each sample image in the image dataset.

[0069] Step 206: Obtain a training set based on the sample images and labels, and train the initial neural network based on the training set to obtain the object detection model; the object detection model includes convolutional layers and attention mechanism layers.

[0070] Optionally, an initial neural network can be trained using an image dataset with added labels to obtain an object detection model. The initial neural network may be, but is not limited to, a YOLO-v8 model.

[0071] Step 208: Obtain the image of the integrated circuit to be tested, input the image into the target detection model, and obtain the mark detection result of the integrated circuit to be tested.

[0072] Optionally, an image of the surface of the integrated circuit to be tested is captured, and a trained target detection model is used to locate and detect the marker information in the image to obtain the final marker detection result.

[0073] In the aforementioned integrated circuit logo detection method, an image dataset is acquired; the image dataset includes sample images, each containing the logo object of a corresponding integrated circuit; corresponding logo labels are added to the sample images; the logo labels are used to locate and annotate the logo objects in the sample images; a training set is acquired based on the sample images and the logo labels, and an initial neural network is trained using the training set to obtain a target detection model; the target detection model includes convolutional layers and attention mechanism layers; a target image of the integrated circuit to be detected is acquired, and the target image is input into the target detection model to obtain the logo detection result of the integrated circuit to be detected. This method enables computers to automatically and effectively locate the position of the integrated circuit logo and classify it, solving both the problem of low efficiency in manual visual inspection and the problem of complex and redundant steps in traditional image detection algorithms, greatly improving the recognition efficiency and accuracy of integrated circuit logos.

[0074] In one embodiment, obtaining an image dataset includes: obtaining an original dataset of integrated circuit surface images; the original dataset includes integrated circuit surface images of multiple integrated circuit categories; performing bilateral filtering denoising and histogram equalization on each integrated circuit surface image in the original dataset to obtain a preprocessed original dataset; comparing the number of images of each integrated circuit category in the original dataset with a standard number threshold, and comparing the expected integrated circuit categories included in the original dataset with preset category labels to determine the first missing integrated circuit category and the second integrated circuit category whose number of images is less than the standard number threshold; and using a diffusion model to augment the image data of the preprocessed original dataset according to the first and second integrated circuit categories to obtain an image dataset.

[0075] The diffusion model can be, but is not limited to, the DDPM algorithm.

[0076] Optionally, the biggest challenge in the research and application of logo detection and recognition technology is the lack of datasets. A survey and analysis of existing logo image datasets reveals that current databases in this field include BelgaLogos, FlickrLogos, and LogoDet-3K, covering logo categories such as sports brands, automotive brands, food brands, and apparel brands. As shown in Table 1, most existing open-source logo image datasets are concentrated in databases of everyday product brands and are generally comprehensive datasets, with very few logo image datasets specific to any particular field.

[0077] Table 1 Overview of Existing Logo Image Datasets

[0078]

[0079] As shown in Table 1, a usable image dataset is lacking in the field of integrated circuit logo detection. Therefore, this invention analyzes the characteristics of integrated circuit surface images, studies the relevant image preprocessing procedures and methods, and performs bilateral filtering denoising and histogram equalization on the original images to eliminate the problem of low contrast between the logo and background due to lighting effects. Figure 3 and Figure 4 As shown, the image filtering effect and histogram equalization effect are respectively demonstrated.

[0080] Next, this invention performed image sample statistics on the original acquired database, revealing an imbalance in the number of samples among different logo categories. Therefore, to address the issues of insufficient image data for some manufacturer logos and overall data imbalance, research was conducted on an image data augmentation algorithm based on the DDPM algorithm to create an image dataset that meets the training requirements of deep learning.

[0081] In this embodiment, the image of the logo on the surface of the integrated circuit is first acquired, and the original image is preprocessed to eliminate the quality problems caused by noise and low contrast. The image dataset is augmented using the DDPM algorithm to eliminate the model training bias caused by imbalanced samples.

[0082] In one embodiment, the label includes a bounding box and category information; adding a corresponding label to a sample image includes: determining the label format based on the input data format of the initial neural network; obtaining integrated circuit-related information corresponding to the sample image; the integrated circuit-related information is used to characterize the marking features of the corresponding integrated circuit; determining the marking region and manufacturer category of the sample image based on the integrated circuit-related information; adding a bounding box and category information of the label format to the sample image based on the marking region and manufacturer category; the bounding box is used to mark the marking region, and the category information is used to characterize the manufacturer category.

[0083] Furthermore, after adding bounding boxes and category information in label format to the sample image based on the marker region and manufacturer category of the sample image, the process further includes: performing at least one round of data detection on the bounding boxes and category information in the sample image, as well as the data format of the bounding boxes and category information; and, if in each round of data detection it is determined that the data format of the bounding boxes and category information is the label format, the bounding boxes match the marker regions, and the category information matches the manufacturer category, then the step of obtaining a training set based on the sample image and the marker labels is executed.

[0084] Optionally, after image sample amplification, the logo objects in the images need to be annotated according to the input image data format requirements specified by the target detection model. Integrated circuit logo detection aims to train a detection model to locate and identify logo regions in images; therefore, this project will not annotate character information in the images. To determine the accuracy of the logo annotations, we identified the manufacturer for each image in the test records and annotated the logo regions. The data annotation process is as follows: Figure 5 As shown. First, using the relevant information in the inspection report, the bounding boxes and corresponding categories of the LOGO areas in the images are labeled; then, each annotated image undergoes at least two checks on data format and information, discarding unqualified images and making modifications, until all images are accurately annotated.

[0085] In this embodiment, prior knowledge is used to accurately label the amplified image samples to meet the training requirements of the neural network.

[0086] In one embodiment, inputting the image to be detected into a target detection model to obtain the mark detection result of the integrated circuit to be detected includes: inputting the image to be detected into the target detection model, obtaining image features of the image to be detected through a convolutional layer located before the attention mechanism layer in the target detection model; processing the image features through the attention mechanism layer of the target detection model to obtain a global feature map; and processing the global feature map through a convolutional layer located after the attention mechanism layer in the target detection model to obtain the mark detection result of the integrated circuit to be detected.

[0087] Optionally, the object detection model can adopt the YOLO-v8 model. As the most advanced object detection network currently available, YOLO-v8 combines various improvements from previous generations, offering better network architecture and stability, and achieving superior generalization ability on benchmark datasets. The YOLO-v8 model uses the C2f module instead of the C3 module in the YOLO-v5 model, which can obtain richer gradient flow information while maintaining lightweight design, effectively solving the problems of inaccurate localization and missed object detection in the YOLO-v5 model. However, integrated circuit logo images have issues such as large differences in morphological features among some intra-class samples and high similarity between inter-class samples. To address these issues, the model needs to fully learn global image feature information and extract deep detail feature information. Therefore, this embodiment needs to introduce an attention mechanism to improve the YOLO-v8 model to meet the application requirements for detection accuracy and efficiency.

[0088] To reduce information diffusion during feature extraction by convolutional networks and increase the global interaction depth to facilitate the extraction of deeper target information, this embodiment uses a Global Attention Machanism (GAM) layer to improve the YOLO-v8 model. For example... Figure 6 As shown, GAM is a lightweight module consisting of channel attention mechanism and spatial attention mechanism.

[0089] The channel attention mechanism can be divided into the following three steps: ① After the original feature map is input, GAM rearranges and exchanges the dimensional order of the three-dimensional feature map; ② The rearranged feature map is passed through a multilayer perceptron, which transforms the three-dimensional feature map into a one-dimensional vector, which is then weighted and distributed through a fully connected layer, and then transformed back into a three-dimensional feature map for deep feature extraction; ③ The above feature maps are restored and rearranged.

[0090] The spatial attention mechanism submodule consists of two convolutional layers. First, it compresses the number of spatial channels in the feature map, and then restores the number of channels through a second convolution. Based on the fusion of spatial information, it makes the model pay more attention to spatial information. In this improvement, the spatial attention mechanism uses a compression ratio of r=4.

[0091] In summary, the channel attention mechanism submodule in GAM amplifies the spatial-channel dependency of feature maps through channel shuffling; the spatial attention mechanism submodule uses two convolutional layers to fuse spatial information in order to focus on spatial information, so that the feature maps after GAM retain the original feature information while allowing the network to focus on deep global information.

[0092] In this embodiment, considering both detection accuracy and efficiency, a GAM module was inserted into the YOLO-v8 model to further improve performance. The structure of the improved YOLO-v8 model is as follows: Figure 9 As shown in the diagram, GAM modules are inserted after the backbone network and feature map extraction. Specifically, in the backbone network, this embodiment inserts an attention mechanism before the Spatial Pyramid Pooling-Fast (SPPF) layer to ensure focus on target features without compromising the feature extraction of the backbone network. A GAM module is inserted before the feature map input feature enhancement extraction network to provide a global feature map for subsequent network feature extraction, thus improving the overall model's detection accuracy. Optional parameters for the YOLO-v8 model are shown in Table 2.

[0093] Table 2 Optional parameters for the YOLO-v8 model

[0094]

[0095] In this embodiment, the feature extraction capability of the YOLO-v8 model is enhanced by adding an attention mechanism, which helps the neural network learn deeper and more representative target features, thereby increasing the accuracy of the target detection task.

[0096] In one exemplary embodiment, an integrated circuit mark detection method includes:

[0097] The process involves obtaining a raw dataset of integrated circuit surface images. This dataset includes surface images of multiple integrated circuit categories. For each integrated circuit surface image in the raw dataset, bilateral filtering for denoising and histogram equalization are performed to obtain a preprocessed raw dataset. The number of images for each integrated circuit category in the raw dataset is compared with a standard threshold. The expected number of integrated circuit categories in the raw dataset is then compared with preset category labels to identify the first missing integrated circuit category and the second integrated circuit category whose image count is less than the standard threshold. Based on the first and second integrated circuit categories, a diffusion model is used to augment the image data in the preprocessed raw dataset, resulting in an image dataset. The image dataset includes sample images, each containing the corresponding marker object of the sample integrated circuit.

[0098] Based on the input data format of the initial neural network, the label format is determined; relevant information about integrated circuits corresponding to the sample images is obtained; the relevant information about integrated circuits is used to characterize the marking features of the corresponding integrated circuits; based on the relevant information about integrated circuits, the marking regions and manufacturer categories of the sample images are determined; based on the marking regions and manufacturer categories, bounding boxes and category information in label format are added to the sample images; the bounding boxes are used to label the marking regions, and the category information is used to characterize the manufacturer categories. The bounding boxes and category information together serve as the label.

[0099] Perform at least one round of data detection on the bounding boxes and category information in the sample images, as well as the data format of the bounding boxes and category information. If in each round of data detection it is determined that the data format of the bounding boxes and category information is the label format, the bounding boxes match the label regions, and the category information matches the manufacturer category, then obtain the training set based on the sample images and label information.

[0100] The initial neural network is trained using the training set to obtain the object detection model; the object detection model includes convolutional layers and attention mechanism layers.

[0101] The process involves acquiring an image of the integrated circuit to be detected, inputting the image into a target detection model, obtaining image features of the image through a convolutional layer before the attention mechanism layer in the target detection model, processing the image features through the attention mechanism layer of the target detection model to obtain a global feature map, and processing the global feature map through a convolutional layer after the attention mechanism layer in the target detection model to obtain the mark detection result of the integrated circuit to be detected.

[0102] In this embodiment, by introducing image augmentation and target detection models, the integrated circuit logo area is located and the integrated circuit logo category is output, thereby identifying the integrated circuit logo information. In integrated circuit logo detection, there are problems such as the variety of manufacturers and unclear printed patterns. Using traditional feature extraction and classification methods for matching and identification results in high complexity, slow speed, and poor accuracy. Therefore, this invention uses YOLO-v8 as the baseline model to locate and detect the logo area in integrated circuit identification images. However, due to the small size of integrated circuit samples, problems such as low image resolution and low contrast exist during logo detection. To address the difficulty of detecting small targets, this patent introduces an attention mechanism into the YOLO-v8 model. Utilizing the characteristics of the attention mechanism, the model focuses on more important key features and filters relatively important information from a large amount of data, enhancing the attention to underlying features and strengthening the focus on small targets, thereby improving the model's detection accuracy. Experimental results show that the proposed algorithm can solve the problem of detecting and locating small target logo areas. Compared with existing detection algorithms, the average accuracy is effectively improved, demonstrating significant advantages.

[0103] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0104] Based on the same inventive concept, this application also provides an integrated circuit mark detection apparatus for implementing the aforementioned integrated circuit mark detection method. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, the specific limitations in one or more embodiments of the integrated circuit mark detection apparatus provided below can be found in the limitations of the integrated circuit mark detection method described above, and will not be repeated here.

[0105] In one exemplary embodiment, such as Figure 10 As shown, an integrated circuit mark detection device 1000 is provided, including: an acquisition module 1001, an annotation module 1002, a training module 1003, and a detection module 1004, wherein:

[0106] The acquisition module 1001 is used to acquire an image dataset; the image dataset includes sample images, and the sample images include the marker objects of the corresponding sample integrated circuits.

[0107] The annotation module 1002 is used to add corresponding labels to the sample image; the labels are used to locate and annotate the label objects in the sample image.

[0108] The training module 1003 is used to obtain a training set based on sample images and label, and to train the initial neural network based on the training set to obtain an object detection model; the object detection model includes convolutional layers and attention mechanism layers.

[0109] The detection module 1004 is used to acquire the image of the integrated circuit to be detected, input the image into the target detection model, and obtain the mark detection result of the integrated circuit to be detected.

[0110] In one embodiment, the acquisition module 1001 is further configured to acquire an original dataset of integrated circuit surface images; the original dataset includes integrated circuit surface images of multiple integrated circuit categories; for each integrated circuit surface image in the original dataset, bilateral filtering denoising and histogram equalization operations are performed to obtain the original dataset after image preprocessing; according to the number of images of each integrated circuit category in the original dataset, image data augmentation is performed on the original dataset after image preprocessing to obtain an image dataset.

[0111] In one embodiment, the acquisition module 1001 is further configured to compare the number of images of each integrated circuit category in the original dataset with a standard number threshold, and compare the integrated circuit categories included in the original dataset with preset category labels to determine the first integrated circuit category missing in the original dataset and the second integrated circuit category whose number of images is less than the standard number threshold; based on the first integrated circuit category and the second integrated circuit category, a diffusion model is used to amplify the image data of the image preprocessed original dataset to obtain an image dataset.

[0112] In one embodiment, the label includes a bounding box and category information; the annotation module 1002 is further configured to determine the label format based on the input data format of the initial neural network; obtain integrated circuit-related information corresponding to the sample image; the integrated circuit-related information is used to characterize the marking features of the corresponding integrated circuit; determine the marking region and manufacturer category of the sample image based on the integrated circuit-related information; add a bounding box and category information in the label format to the sample image based on the marking region and manufacturer category; the bounding box is used to mark the marking region, and the category information is used to characterize the manufacturer category.

[0113] In one embodiment, the training module 1003 is further configured to perform at least one round of data detection on the bounding boxes and category information in the sample image, as well as the data format of the bounding boxes and category information; and to obtain a training set based on the sample image and the label if each round of data detection determines that the data format of the bounding boxes and category information is a label format, the bounding boxes match the label regions, and the category information matches the manufacturer category.

[0114] In one embodiment, the detection module 1004 is further configured to input the image to be detected into the target detection model, obtain the image features of the image to be detected through the convolutional layer located before the attention mechanism layer in the target detection model; process the image features through the attention mechanism layer of the target detection model to obtain a global feature map; and process the global feature map through the convolutional layer located after the attention mechanism layer in the target detection model to obtain the mark detection result of the integrated circuit to be detected.

[0115] Each module in the aforementioned integrated circuit marking detection device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0116] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 11 As shown, this computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores image data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When executed by the processor, the computer program implements an integrated circuit mark detection method.

[0117] Those skilled in the art will understand that Figure 11The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0118] In an exemplary embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to perform the following steps: acquiring an image dataset; the image dataset includes sample images, each sample image containing a marker object of a corresponding sample integrated circuit; adding corresponding marker labels to the sample images; the marker labels are used to locate and annotate the marker objects in the sample images; acquiring a training set based on the sample images and the marker labels, and training an initial neural network based on the training set to obtain an object detection model; the object detection model includes convolutional layers and attention mechanism layers; acquiring a target image of the integrated circuit to be detected, inputting the target image into the object detection model, and obtaining the marker detection result of the integrated circuit to be detected.

[0119] In one embodiment, when the processor executes the computer program, it further performs the following steps: acquiring a raw dataset of integrated circuit surface images; the raw dataset includes integrated circuit surface images of multiple integrated circuit categories; performing bilateral filtering denoising and histogram equalization operations on each integrated circuit surface image in the raw dataset to obtain a preprocessed raw dataset; and performing image data augmentation on the preprocessed raw dataset according to the number of images of each integrated circuit category in the raw dataset to obtain an image dataset.

[0120] In one embodiment, when the processor executes the computer program, it further performs the following steps: comparing the number of images for each integrated circuit category in the original dataset with a standard number threshold, comparing the expected integrated circuit categories included in the original dataset with preset category labels, and determining the first integrated circuit category missing in the original dataset and the second integrated circuit category whose number of images is less than the standard number threshold; based on the first integrated circuit category and the second integrated circuit category, using a diffusion model to perform image data augmentation on the image preprocessed original dataset to obtain an image dataset.

[0121] In one embodiment, the label includes a bounding box and category information. When the processor executes the computer program, it further performs the following steps: determining the label format based on the input data format of the initial neural network; obtaining integrated circuit-related information corresponding to the sample image; the integrated circuit-related information is used to characterize the marking features of the corresponding integrated circuit; determining the marking region and manufacturer category of the sample image based on the integrated circuit-related information; adding a bounding box and category information of the label format to the sample image based on the marking region and manufacturer category; the bounding box is used to mark the marking region, and the category information is used to characterize the manufacturer category.

[0122] In one embodiment, when the processor executes the computer program, it further performs the following steps: performing at least one round of data detection on the bounding boxes and category information in the sample image, as well as the data format of the bounding boxes and category information; and, if in each round of data detection it is determined that the data format of the bounding boxes and category information is a label format, the bounding boxes match the label regions, and the category information matches the manufacturer category, then performing the step of obtaining a training set based on the sample image and the label.

[0123] In one embodiment, when the processor executes the computer program, it further performs the following steps: inputting the image to be detected into the target detection model, obtaining image features of the image to be detected through a convolutional layer in the target detection model before the attention mechanism layer; processing the image features through the attention mechanism layer of the target detection model to obtain a global feature map; and processing the global feature map through a convolutional layer in the target detection model after the attention mechanism layer to obtain the mark detection result of the integrated circuit to be detected.

[0124] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When executed by a processor, the computer program performs the following steps: acquiring an image dataset; the image dataset includes sample images, each containing a marker object of a corresponding sample integrated circuit; adding corresponding marker labels to the sample images; the marker labels are used to locate and annotate the marker objects in the sample images; acquiring a training set based on the sample images and the marker labels, and training an initial neural network based on the training set to obtain an object detection model; the object detection model includes convolutional layers and attention mechanism layers; acquiring a target image of the integrated circuit to be detected, inputting the target image into the object detection model, and obtaining the marker detection result of the integrated circuit to be detected.

[0125] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: acquiring a raw dataset of integrated circuit surface images; the raw dataset includes integrated circuit surface images of multiple integrated circuit categories; performing bilateral filtering denoising and histogram equalization operations on each integrated circuit surface image in the raw dataset to obtain a preprocessed raw dataset; and performing image data augmentation on the preprocessed raw dataset according to the number of images of each integrated circuit category in the raw dataset to obtain an image dataset.

[0126] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: comparing the number of images for each integrated circuit category in the original dataset with a standard number threshold, comparing the expected integrated circuit categories included in the original dataset with preset category labels, and determining the first integrated circuit category missing in the original dataset and the second integrated circuit category whose number of images is less than the standard number threshold; based on the first integrated circuit category and the second integrated circuit category, using a diffusion model to perform image data augmentation on the image preprocessed original dataset to obtain an image dataset.

[0127] In one embodiment, the label includes a bounding box and category information. When the computer program is executed by the processor, it further performs the following steps: determining the label format based on the input data format of the initial neural network; obtaining integrated circuit-related information corresponding to the sample image; the integrated circuit-related information is used to characterize the marking features of the corresponding integrated circuit; determining the marking region and manufacturer category of the sample image based on the integrated circuit-related information; adding a bounding box and category information of the label format to the sample image based on the marking region and manufacturer category; the bounding box is used to mark the marking region, and the category information is used to characterize the manufacturer category.

[0128] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: performing at least one round of data detection on the bounding boxes and category information in the sample image, as well as the data format of the bounding boxes and category information; and, if in each round of data detection it is determined that the data format of the bounding boxes and category information is a label format, the bounding boxes match the label regions, and the category information matches the manufacturer category, then performing the step of obtaining a training set based on the sample image and the label.

[0129] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: inputting the image to be detected into the target detection model, obtaining image features of the image to be detected through a convolutional layer in the target detection model before the attention mechanism layer; processing the image features through the attention mechanism layer of the target detection model to obtain a global feature map; and processing the global feature map through a convolutional layer in the target detection model after the attention mechanism layer to obtain the mark detection result of the integrated circuit to be detected.

[0130] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, performs the following steps: acquiring an image dataset; the image dataset includes sample images, each containing a marker object of a corresponding sample integrated circuit; adding corresponding marker labels to the sample images; the marker labels are used to locate and annotate the marker objects in the sample images; acquiring a training set based on the sample images and the marker labels, and training an initial neural network based on the training set to obtain an object detection model; the object detection model includes convolutional layers and attention mechanism layers; acquiring a target image of the integrated circuit to be detected, inputting the target image into the object detection model, and obtaining the marker detection result of the integrated circuit to be detected.

[0131] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: acquiring a raw dataset of integrated circuit surface images; the raw dataset includes integrated circuit surface images of multiple integrated circuit categories; performing bilateral filtering denoising and histogram equalization operations on each integrated circuit surface image in the raw dataset to obtain a preprocessed raw dataset; and performing image data augmentation on the preprocessed raw dataset according to the number of images of each integrated circuit category in the raw dataset to obtain an image dataset.

[0132] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: comparing the number of images for each integrated circuit category in the original dataset with a standard number threshold, comparing the expected integrated circuit categories included in the original dataset with preset category labels, and determining the first integrated circuit category missing in the original dataset and the second integrated circuit category whose number of images is less than the standard number threshold; based on the first integrated circuit category and the second integrated circuit category, using a diffusion model to perform image data augmentation on the image preprocessed original dataset to obtain an image dataset.

[0133] In one embodiment, the label includes a bounding box and category information. When the computer program is executed by the processor, it further performs the following steps: determining the label format based on the input data format of the initial neural network; obtaining integrated circuit-related information corresponding to the sample image; the integrated circuit-related information is used to characterize the marking features of the corresponding integrated circuit; determining the marking region and manufacturer category of the sample image based on the integrated circuit-related information; adding a bounding box and category information of the label format to the sample image based on the marking region and manufacturer category; the bounding box is used to mark the marking region, and the category information is used to characterize the manufacturer category.

[0134] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: performing at least one round of data detection on the bounding boxes and category information in the sample image, as well as the data format of the bounding boxes and category information; and, if in each round of data detection it is determined that the data format of the bounding boxes and category information is a label format, the bounding boxes match the label regions, and the category information matches the manufacturer category, then performing the step of obtaining a training set based on the sample image and the label.

[0135] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: inputting the image to be detected into the target detection model, obtaining image features of the image to be detected through a convolutional layer in the target detection model before the attention mechanism layer; processing the image features through the attention mechanism layer of the target detection model to obtain a global feature map; and processing the global feature map through a convolutional layer in the target detection model after the attention mechanism layer to obtain the mark detection result of the integrated circuit to be detected.

[0136] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0137] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0138] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0139] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for detecting integrated circuit markings, characterized in that, The method includes: Obtain an image dataset; the image dataset includes sample images, and the sample images include the marker objects of the corresponding sample integrated circuits; Add corresponding labels to the sample images; the labels are used to locate and annotate the marked objects in the sample images. A training set is obtained based on the sample images and the label, and an initial neural network is trained based on the training set to obtain an object detection model. The object detection model is a YOLO-v8 model, which includes convolutional layers and a global attention mechanism layer. The global attention mechanism layer includes a channel attention mechanism submodule and a spatial attention mechanism submodule. The global attention mechanism layer is set before the fast spatial pyramid pooling layer and the feature map input feature enhancement extraction network of the YOLO-v8 model. Acquire an image of the integrated circuit to be tested, input the image into the target detection model, and obtain the mark detection result of the integrated circuit to be tested; The step of inputting the image to be detected into the target detection model to obtain the mark detection result of the integrated circuit to be detected includes: The image to be detected is input into the target detection model, and the image features of the image to be detected are obtained through the convolutional layer located before the attention mechanism layer in the target detection model. The image features are processed through the attention mechanism layer of the target detection model to obtain a global feature map; The global feature map is processed by the convolutional layer located after the attention mechanism layer in the target detection model to obtain the mark detection result of the integrated circuit to be detected.

2. The method according to claim 1, characterized in that, The acquisition of the image dataset includes: Obtain the raw dataset of integrated circuit surface images; the raw dataset includes integrated circuit surface images of multiple integrated circuit categories; For each integrated circuit surface image in the original dataset, bilateral filtering denoising and histogram equalization are performed to obtain the original dataset after image preprocessing. Based on the number of images for each integrated circuit category in the original dataset, image data augmentation is performed on the preprocessed original dataset to obtain the image dataset.

3. The method according to claim 2, characterized in that, The image dataset is obtained by augmenting the preprocessed original dataset with image data based on the number of images for each integrated circuit category in the original dataset, including: The number of images for each integrated circuit category in the original dataset is compared with a standard number threshold. The integrated circuit categories included in the original dataset are compared with preset category labels to determine the first missing integrated circuit category and the second integrated circuit category whose number of images is less than the standard number threshold in the original dataset. Based on the first integrated circuit category and the second integrated circuit category, a diffusion model is used to augment the image data of the preprocessed original dataset to obtain the image dataset.

4. The method according to claim 1, characterized in that, The label includes a bounding box and category information; adding the corresponding label to the sample image includes: The label format is determined based on the input data format of the initial neural network; Obtain integrated circuit-related information corresponding to the sample image; the integrated circuit-related information is used to characterize the distinctive features of the corresponding integrated circuit; Based on the integrated circuit-related information, determine the marked areas and manufacturer categories of the sample image; Based on the marked area and the manufacturer category, a bounding box and category information in the label format are added to the sample image; the bounding box is used to label the marked area, and the category information is used to characterize the manufacturer category.

5. The method according to claim 4, characterized in that, After adding the bounding box and category information in the label format to the sample image based on the marked region and manufacturer category of the sample image, the method further includes: At least one round of data detection is performed on the bounding boxes and category information in the sample images, as well as the data format of the bounding boxes and category information; If in each round of data detection the bounding box and category information data format is determined to be the label format, the bounding box matches the label region, and the category information matches the manufacturer category, then the step of obtaining the training set based on the sample image and the label is executed.

6. An integrated circuit marking detection device, characterized in that, The device includes: An acquisition module is used to acquire an image dataset; the image dataset includes sample images, and the sample images include marker objects of corresponding sample integrated circuits; An annotation module is used to add corresponding labels to the sample image; the labels are used to locate and annotate the label objects in the sample image. The training module is used to obtain a training set based on the sample images and the label, and to train the initial neural network based on the training set to obtain an object detection model. The object detection model is a YOLO-v8 model including convolutional layers and a global attention mechanism layer. The global attention mechanism layer includes a channel attention mechanism submodule and a spatial attention mechanism submodule. The global attention mechanism layer is set before the fast spatial pyramid pooling layer and the feature map input feature enhancement extraction network of the YOLO-v8 model. The detection module is used to acquire the image of the integrated circuit to be detected, input the image to be detected into the target detection model, and obtain the mark detection result of the integrated circuit to be detected. The detection module is further configured to input the image to be detected into the target detection model, obtain image features of the image to be detected through a convolutional layer in the target detection model before the attention mechanism layer, process the image features through the attention mechanism layer of the target detection model to obtain a global feature map, and process the global feature map through a convolutional layer in the target detection model after the attention mechanism layer to obtain the mark detection result of the integrated circuit to be detected.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 5.

9. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 5.

Citation Information

Patent Citations

  • Training method and system based on Yolov5 traffic sign detection and recognition model

    CN115273017A