Defect classification method and apparatus based on micro-display devices, and storage medium

Abstract

Disclosed in the present application are a defect classification method and apparatus based on micro-display devices, and a storage medium, which are used for improving the defect detection efficiency for micro-display devices. The present application comprises: acquiring images to be subjected to detection; monitoring statuses of a CPU and a GPU; locating minimum detection unit regions; performing initial defect detection; selecting images to be subjected to re-detection; transmitting the images to be subjected to re-detection to a re-detection module; performing re-detection on the images to be subjected to re-detection, and generating a re-detection analysis result; and performing defect classification and summarization on micro-display devices.

Description

A defect classification method, apparatus, and storage medium based on microdisplay devices Technical Field

[0001] This application relates to the field of microdisplay device inspection, and more particularly to a defect classification method, apparatus, and storage medium based on microdisplay devices. Background Technology

[0002] With the rapid development of industrial and internet technologies, the industry's requirements for the quality of industrial products are becoming increasingly refined and standardized. In the display panel industry, the quality of products directly affects their competitiveness.

[0003] With the gradual integration of micro-projectors and wearable devices into real-world production and daily life, the application prospects of micro-display devices are becoming increasingly broad. In particular, the application of micro-display devices in fields such as virtual reality (VR), augmented reality (AR), and micro-projectors makes the next generation of active-emitting devices extremely promising.

[0004] Microdisplay devices can be viewed as miniaturized LEDs, possessing advantages such as self-illumination, low power consumption, independent pixel control, and ultra-high resolution. Building upon this foundation, and combining it with nanowire technology, microdisplay devices can be fabricated into flexible devices for application in wearable devices. This has led to a proliferation of research directions for microdisplay devices, but consequently, the demands and requirements for these devices are also gradually increasing.

[0005] Currently, the fabrication process of microdisplay devices still faces certain technical challenges. For example, structural defects on the surface and inside of microdisplay devices can severely impact the performance and lifespan of the entire device. Existing defect detection methods for microdisplay devices often focus on visual appearance inspection. This type of visual inspection frequently requires manual review to classify detected NG (non-compliant) wafers. Defect types include, but are not limited to, good products, damage, missing parts, scratches, and cross-contamination. Each defect typically has unique characteristics, but there are also defects with similar appearances, or even two or more defect features existing on a single wafer area. To accelerate defect detection efficiency, machine first inspection combined with manual review is often used. However, in real-world scenarios, to meet tight production schedules, high-speed aerial cameras are generally used for image acquisition. This acquisition method captures images of microdisplay devices in a very short time, placing extremely high demands on the performance of the entire defect classification system. Currently, neither manual nor machine review can simultaneously detect multiple targets, failing to fully utilize the performance of CPUs and GPUs, leading to a decrease in defect detection efficiency. Technical issues

[0006] This application discloses a defect classification method, apparatus, and storage medium based on microdisplay devices, which can improve the defect detection efficiency of microdisplay devices. Technical solutions

[0007] The first aspect of this application discloses a defect classification method based on microdisplay devices, including:

[0008] Images of the microdisplay device to be tested are acquired using a high-speed aerial camera;

[0009] Real-time monitoring of the operating status of the CPU (Central Processing Unit) and the GPU (Graphics Processing Unit);

[0010] When the CPU running state reaches the first preset condition, the image to be detected and the template information are transmitted to the detection thread. The template information is the template feature information extracted from the defect-free image of the micro-display device.

[0011] The positioning module is used in conjunction with template information to locate the smallest detection unit region on the image to be detected. The smallest detection unit region is the effective region in the image to be detected.

[0012] Perform initial defect detection on the smallest detection unit region of the image to be detected and generate initial detection analysis results;

[0013] Images to be re-inspected are selected from the images to be inspected based on the initial inspection analysis results;

[0014] When the GPU's operating status reaches the second preset condition, the image to be re-inspected is transmitted to the GPU's re-inspection module;

[0015] The re-inspection module re-inspects the images to be re-inspected and generates re-inspection analysis results.

[0016] Based on the initial inspection analysis results and the re-inspection analysis results, the defects of the micro-display devices were classified and summarized.

[0017] Optionally, the steps for performing initial defect detection on the smallest detection unit region of the image to be detected and generating initial detection analysis results include:

[0018] Set the target convolution kernel based on the wafer position information in the template information;

[0019] Set the non-detection areas on the image to be detected to black;

[0020] The target convolution kernel is used to perform convolution emphasis processing on the smallest detection unit region on the image to be detected;

[0021] The blob algorithm is used to analyze the smallest detection unit region after convolutional emphasis processing, and to generate defect feature information of the smallest detection unit region.

[0022] Based on the defect characteristic information, a defect comparison analysis is performed to generate the first inspection analysis results.

[0023] Optionally, the steps for re-examining the image to be re-examined and generating re-examination analysis results through the re-examination module include:

[0024] The image to be re-examined is input into the input layer of the DPN classification network model. The DPN classification network model includes an input layer, an initial convolutional layer, at least two DPN blocks, a fully connected layer, and an output layer. The DPN block includes a feature extraction module and at least two DPU modules.

[0025] The initial convolutional layer of the DPN classification network model extracts primary features from the image to be re-examined, and then inputs the feature extraction module of at least two DPN blocks to extract secondary features.

[0026] Secondary features are processed and transmitted simultaneously through residual connections and dense connections of DPU blocks to generate fused features;

[0027] The fusion features extracted and fused from the DPU block are passed through a fully connected layer to vectorize the fusion features, and then classified through the output layer to generate the re-examination analysis results.

[0028] Optionally, the step of using the localization module in conjunction with template information to locate the smallest detection unit region on the image to be detected includes:

[0029] Based on the template information, at least one coarse bounding box is determined on the image to be detected;

[0030] Determine the initial center coordinates of the coarse positioning bounding box region;

[0031] Generate an attention matrix for each pixel in the coarse localization box of the image to be detected;

[0032] Pixels with an attention level less than a preset attention threshold in each coarse positioning bounding box are filtered out based on the attention matrix;

[0033] The offset center coordinate position is calculated by using the pixel positions of the remaining pixels in the coarse positioning box;

[0034] The offset data is calculated based on the initial center coordinates and the offset center coordinates. The offset data includes the offset distance and offset direction.

[0035] When the offset data does not meet the preset distance conditions, the position of the coarse localization box on the image to be detected is adjusted according to the offset data, and the initial center coordinate position of the new coarse localization box is re-determined, the attention matrix is ​​calculated, pixels are filtered out, the offset center coordinate position is calculated, and the offset data is analyzed.

[0036] When the offset data meets the preset distance conditions, the smallest detection unit area is determined based on the coarse positioning box on the current image to be detected.

[0037] Optionally, when the CPU operating state reaches a first preset condition, the step of transmitting the image to be detected and template information to the detection thread includes:

[0038] The CPU idle detection thread is determined based on the CPU's operating status;

[0039] When the number of idle detection threads of the CPU is greater than the preset first number of threads, the image to be detected and the template information are transmitted to the detection thread. The template information is the template feature information extracted from the defect-free image of the micro-display device.

[0040] When the GPU's operating state reaches the second preset condition, the steps for transmitting the image to be re-inspected to the GPU's re-inspection module include:

[0041] The idle detection thread of the GPU is determined based on the GPU's running status;

[0042] When the number of idle detection threads of the GPU is greater than the preset number of second threads, and the number of images to be re-inspected reaches the preset number of re-inspections, the images to be re-inspected are sent to the GPU's re-inspection module.

[0043] Optionally, after the step of acquiring the image to be inspected from the microdisplay device using a high-speed camera and before the step of real-time monitoring of the operating status of the CPU and GPU, the defect classification method further includes:

[0044] The image to be detected is subjected to Gaussian filtering for noise reduction.

[0045] Optionally, the step of performing Gaussian filtering noise reduction on the image to be detected includes:

[0046] Obtain the standard deviation, average gray level, and number of pixels of the image to be detected;

[0047] The center threshold is calculated based on the image standard deviation, average gray level, and number of pixels.

[0048] Determine the location of pixels in the image to be detected whose grayscale value is not less than the center threshold;

[0049] The center coordinate position is calculated based on the position of the pixel whose gray value is not less than the center threshold.

[0050] Obtain the spatial domain standard deviation of the image to be detected;

[0051] Calculate the spatial domain Gaussian filter kernel based on the center coordinate position, spatial domain standard deviation, and the first preset Gaussian function;

[0052] Obtain the standard deviation of the color gamut of the image to be detected;

[0053] Calculate the color gaussian filter kernel based on the center coordinate position, the standard deviation of the color gaussian ... function, and the second preset Gaussian function.

[0054] Gaussian filtering is performed on the image to be detected based on the spatial domain Gaussian filter kernel and the color domain Gaussian filter kernel.

[0055] The second aspect of this application discloses a defect classification device based on a microdisplay device, comprising:

[0056] The acquisition unit is used to acquire images of the microdisplay device to be tested using a high-speed camera.

[0057] The monitoring unit is used to monitor the operating status of the CPU (Central Processing Unit) and the GPU (Graphics Processing Unit) in real time.

[0058] The first transmission unit is used to transmit the image to be detected and the template information to the detection thread when the CPU running state reaches the first preset condition. The template information is the template feature information extracted from the defect-free image of the micro-display device.

[0059] The positioning unit is used to locate the smallest detection unit region on the image to be detected using the positioning module in combination with template information. The smallest detection unit region is the effective region in the image to be detected.

[0060] The first generation unit is used to perform initial defect detection in the smallest detection unit region of the image to be detected and generate initial detection analysis results.

[0061] The filtering unit generates images to be re-inspected from the images to be inspected based on the initial inspection analysis results;

[0062] The second transmission unit is used to transmit the image to be re-inspected to the re-inspection module of the GPU when the GPU's operating state reaches the second preset condition.

[0063] The second generation unit is used to re-examine the image to be re-examined through the re-examination module and generate re-examination analysis results.

[0064] The summary unit is used to classify and summarize the defects of micro-display devices based on the initial inspection analysis results and the re-inspection analysis results.

[0065] Optionally, the first generating unit is used for:

[0066] Set the target convolution kernel based on the wafer position information in the template information;

[0067] Set the non-detection areas on the image to be detected to black;

[0068] The target convolution kernel is used to perform convolution emphasis processing on the smallest detection unit region on the image to be detected;

[0069] The blob algorithm is used to analyze the smallest detection unit region after convolutional emphasis processing, and to generate defect feature information of the smallest detection unit region.

[0070] Based on the defect characteristic information, a defect comparison analysis is performed to generate the first inspection analysis results.

[0071] Optionally, the second generating unit is used for:

[0072] The image to be re-examined is input into the input layer of the DPN classification network model. The DPN classification network model includes an input layer, an initial convolutional layer, at least two DPN blocks, a fully connected layer, and an output layer. The DPN block includes a feature extraction module and at least two DPU modules.

[0073] The initial convolutional layer of the DPN classification network model extracts primary features from the image to be re-examined, and then inputs the feature extraction module of at least two DPN blocks to extract secondary features.

[0074] Secondary features are processed and transmitted simultaneously through residual connections and dense connections of DPU blocks to generate fused features;

[0075] The fusion features extracted and fused from the DPU block are passed through a fully connected layer to vectorize the fusion features, and then classified through the output layer to generate the re-examination analysis results.

[0076] Optionally, the positioning unit is used for:

[0077] Based on the template information, at least one coarse bounding box is determined on the image to be detected;

[0078] Determine the initial center coordinates of the coarse positioning bounding box region;

[0079] Generate an attention matrix for each pixel in the coarse localization box of the image to be detected;

[0080] Pixels with an attention level less than a preset attention threshold in each coarse positioning bounding box are filtered out based on the attention matrix;

[0081] The offset center coordinate position is calculated by using the pixel positions of the remaining pixels in the coarse positioning box;

[0082] The offset data is calculated based on the initial center coordinates and the offset center coordinates. The offset data includes the offset distance and offset direction.

[0083] When the offset data does not meet the preset distance conditions, the position of the coarse localization box on the image to be detected is adjusted according to the offset data, and the initial center coordinate position of the new coarse localization box is re-determined, the attention matrix is ​​calculated, pixels are filtered out, the offset center coordinate position is calculated, and the offset data is analyzed.

[0084] When the offset data meets the preset distance conditions, the smallest detection unit area is determined based on the coarse positioning box on the current image to be detected.

[0085] Optionally, the first transmission unit is used for:

[0086] The CPU idle detection thread is determined based on the CPU's operating status;

[0087] When the number of idle detection threads of the CPU is greater than the preset first number of threads, the image to be detected and the template information are transmitted to the detection thread. The template information is the template feature information extracted from the defect-free image of the micro-display device.

[0088] Optionally, the second transmission unit includes:

[0089] The idle detection thread of the GPU is determined based on the GPU's running status;

[0090] When the number of idle detection threads on the GPU is greater than the preset number of second threads, and the number of images to be re-inspected reaches the preset number of re-inspections, the images to be re-inspected are sent to the GPU's re-inspection module.

[0091] Optionally, the defect classification device may also include:

[0092] The Gaussian noise reduction unit is used to perform Gaussian filtering on the image to be detected for noise reduction. The Gaussian noise reduction unit is electrically connected to both the acquisition unit and the monitoring unit.

[0093] Optionally, the Gaussian noise reduction unit is used for:

[0094] Obtain the standard deviation, average gray level, and number of pixels of the image to be detected;

[0095] The center threshold is calculated based on the image standard deviation, average gray level, and number of pixels.

[0096] Determine the location of pixels in the image to be detected whose grayscale value is not less than the center threshold;

[0097] The center coordinate position is calculated based on the position of the pixel whose gray value is not less than the center threshold.

[0098] Obtain the spatial domain standard deviation of the image to be detected;

[0099] Calculate the spatial domain Gaussian filter kernel based on the center coordinate position, spatial domain standard deviation, and the first preset Gaussian function;

[0100] Obtain the standard deviation of the color gamut of the image to be detected;

[0101] Calculate the color gaussian filter kernel based on the center coordinate position, the standard deviation of the color gaussian ... function, and the second preset Gaussian function.

[0102] Gaussian filtering is performed on the image to be detected based on the spatial domain Gaussian filter kernel and the color domain Gaussian filter kernel.

[0103] A third aspect of this application provides a defect classification device based on a microdisplay device, comprising:

[0104] Processor, memory, input / output units, and bus;

[0105] The processor is connected to memory, input / output units, and a bus;

[0106] The memory stores a program, which the processor calls to execute, such as the first aspect and any optional defect classification method of the first aspect.

[0107] The fourth aspect of this application provides a computer-readable storage medium on which a program is stored, which, when executed on a computer, performs the first aspect and any optional defect classification method of the first aspect. Beneficial effects

[0108] In this application, a high-speed aerial camera first acquires an image of the microdisplay device to be inspected. Then, the operating status of the CPU (Central Processing Unit) and GPU (Graphics Processing Unit) is monitored in real time. When the CPU operating status reaches a first preset condition, the image to be inspected and template information are transmitted to the inspection thread. The template information consists of template feature information extracted from the defect-free image of the microdisplay device. A positioning module, combined with the template information, locates the smallest detection unit region on the image to be inspected. The smallest detection unit region is the effective region in the image to be inspected, thus determining the effective detection area and reducing the computational load of the initial inspection and re-inspection systems. A first-stage defect inspection is performed on the smallest detection unit region of the image to be inspected, generating an initial inspection analysis result. Then, based on the initial inspection analysis result, images to be re-inspected are selected from the images to be inspected. These images are either images of the smallest detection unit regions whose initial inspection defect features do not meet the judgment criteria, or images corresponding to the smallest detection unit regions with at least two initial inspection feature defects. When the GPU operating status reaches a second preset condition, the image to be re-inspected is transmitted to the GPU's re-inspection module. The re-inspection module performs a re-inspection on the image to be re-inspected, generating a re-inspection analysis result. After identifying the images to be re-inspected, the re-inspection module performs a re-inspection of these images, generating re-inspection analysis results. Finally, based on the initial inspection analysis results and the re-inspection analysis results, the defects of the micro-display devices are classified and summarized.

[0109] By monitoring the operating status of the CPU and GPU, when the CPU status indicates that it can allocate enough detection threads for the initial inspection, multiple images to be inspected and their corresponding template information are sent to different detection threads, allowing multiple images to be inspected simultaneously on different threads. After the initial inspection of the images to be inspected, some images that have met the inspection standards are filtered out, and the remaining images are sent to the re-inspection module for further inspection. When both the GPU operating status and the re-inspection start conditions are met, the images to be re-inspected are sent to the GPU's re-inspection module for re-inspection.

[0110] In the above process, multiple images to be detected can be detected simultaneously in different detection threads, and the initial inspection and re-inspection of different images to be detected can also be run simultaneously. The CPU runs the initial inspection and the GPU runs the re-inspection. While achieving simultaneous detection of multiple targets, the performance of the CPU and GPU can be fully utilized, further improving the efficiency of the entire defect classification system. Attached Figure Description

[0111] Figure 1 is a schematic diagram of an embodiment of the defect classification method based on micro-display devices in this application;

[0112] Figure 2 is a schematic diagram of an embodiment of the first stage of the defect classification method based on micro-display devices in this application;

[0113] Figure 3 is a schematic diagram of an embodiment of the second stage of the defect classification method based on micro-display devices in this application;

[0114] Figure 4 is a schematic diagram of an embodiment of the third stage of the defect classification method based on micro-display devices in this application;

[0115] Figure 5 is a schematic diagram of an embodiment of the fourth stage of the defect classification method based on micro-display devices in this application;

[0116] Figure 6 is a schematic diagram of an embodiment of the defect classification device based on micro-display devices of this application;

[0117] Figure 7 is a schematic diagram of another embodiment of the defect classification device based on microdisplay devices of this application. Embodiments of the present invention

[0118] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0119] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0120] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0121] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0122] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0123] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0124] In the field of microdisplay device manufacturing, defect detection often focuses on the visual appearance inspection of microdisplay devices. This visual appearance inspection frequently requires manual review to classify detected NG (non-compliant) wafers. Defect types include, but are not limited to, good products, damage, missing parts, scratches, and cross-contamination. Each defect typically has its unique characteristics, but there are also defect features that appear similar, or two or more defect features may exist on a single wafer area. To accelerate defect detection efficiency, machine first inspection combined with manual review is often used. However, in real-world scenarios, to meet tight production schedules, high-speed aerial cameras are generally used for image acquisition. This acquisition method captures images of microdisplay devices in a very short time, placing extremely high demands on the performance of the entire defect classification system. Currently, neither manual nor machine review can simultaneously detect multiple targets, failing to fully utilize the performance of CPUs and GPUs, leading to a decrease in defect detection efficiency.

[0125] Based on this, this application discloses a defect classification method, apparatus and storage medium based on microdisplay devices, which can improve the defect detection efficiency of microdisplay devices.

[0126] The technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0127] The method described in this application can be applied to servers, devices, terminals, or other devices with logical processing capabilities; therefore, this application does not limit its application. For ease of description, the following description uses a terminal as the executing entity.

[0128] Please refer to Figure 1. This application provides an embodiment of a defect classification method based on microdisplay devices, including:

[0129] 101. Acquire images of the microdisplay device to be tested using a high-speed aerial camera;

[0130] In this embodiment, the high-speed acquisition of microdisplay devices using a drone camera is mainly accomplished by the image acquisition module.

[0131] The image acquisition module mainly includes a platform, a pressing device, a drone camera, a point light source, a displacement component, and a lens. Before acquisition, the parameters of the entire image acquisition module need to be set, such as exposure time, displacement speed, acquisition interval, and lens aperture and focal length. Different operating modes are set for different microdisplay devices, and the parameters will differ between these modes.

[0132] Specifically, after adjusting the standard parameters, the terminal fixes the aperture and focal length of the lens to meet the working conditions of the high-speed aerial photography system. Then, it adjusts the positional relationship between the platform and the aerial camera, and fixes the point light source on one side of the aerial camera and the lens so that the illumination of the micro-display device by the point light source is as uniform as possible during each acquisition process. Finally, it saves the working parameters of the aerial camera itself.

[0133] After completing the above adjustments, the terminal acquires the image to be tested of the microdisplay device using a high-speed flying camera.

[0134] 102. Monitor the operating status of the CPU (Central Processing Unit) and the GPU (Graphics Processing Unit) in real time;

[0135] 103. When the CPU running state reaches the first preset condition, the image to be detected and the template information are transmitted to the detection thread. The template information is the template feature information extracted from the defect-free image of the micro-display device.

[0136] In this embodiment, the current operating status of the CPU and GPU is monitored, and the image to be detected is sent to the system's first inspection module for first inspection through different detection threads according to the current operating status of the CPU. Subsequent re-inspection will be started according to the current status of the GPU.

[0137] By monitoring the status of the CPU and GPU, this approach addresses the problems of limited monitoring functionality, incomplete monitoring, and poor real-time performance in existing technologies. Specifically, this embodiment primarily utilizes a performance monitoring module, which includes a data acquisition unit, a data processing unit, and a data display unit.

[0138] The data acquisition unit is used to collect real-time performance metrics such as CPU and GPU utilization, temperature, frequency, and power consumption. The data acquisition unit includes sensor interfaces, operating system interfaces, and application programming interfaces (APIs). It acquires CPU and GPU performance data from hardware sensors, the operating system kernel, and applications. Specifically, this data acquisition unit obtains data through various methods.

[0139] In this embodiment, the terminal uses the System.Diagnostics namespace to obtain CPU utilization. Then, the terminal uses the OpenHardwareMonitor library to obtain CPU and GPU temperature, frequency, and power consumption information. The terminal uses the NVAPI library to obtain NVIDIA GPU utilization and video memory usage.

[0140] In this embodiment, the data processing unit analyzes and processes the collected data to generate a comprehensive performance report. Specifically, the data processing unit can perform analytical operations such as data filtering, data storage, and data analysis. Data filtering is mainly used to clean and format the collected data. Data storage is used to store historical data, and data analysis is used to calculate comprehensive performance indicators such as the average utilization rate, maximum temperature, and frequency fluctuation of the CPU and GPU.

[0141] In this embodiment, the data display unit is used to display the CPU and GPU performance status in real time in text form. The data display unit includes a recording module and an alarm module. The recording module is used to display real-time and historical CPU and GPU performance data in text form, while the alarm module is used to send alarm information to the system when the CPU and GPU performance exceeds a preset threshold.

[0142] In this embodiment, the terminal acquires template information of the microdisplay device. Before inspection, the terminal collects images of normal, defect-free microdisplay devices and extracts wafer feature information from these images to generate template information in preparation for subsequent inspection.

[0143] 104. Use the positioning module in conjunction with template information to locate the smallest detection unit region on the image to be detected. The smallest detection unit region is the effective region in the image to be detected.

[0144] In this embodiment, in order to reduce the amount of computation, the terminal first needs to use the positioning module to locate the position area of ​​all wafers on the image to be detected according to the template information. The wafer area is the smallest detection unit area, and the other areas are set as background areas.

[0145] 105. Perform initial defect inspection on the smallest detection unit region of the image to be inspected and generate initial inspection analysis results;

[0146] The terminal can perform initial defect detection using traditional methods on the smallest detection unit region of the image to be inspected, or it can use convolutional neural networks, or various image recognition methods, which are not limited here. Through initial detection, the information of defect features on the smallest detection unit region can be determined, and a preliminary judgment of the defect information can be made.

[0147] Specifically, during the detection process, the terminal can compare the information of defect features with the labels of specific features to calculate the probability that the defect feature belongs to the current specific feature. If no matching defect feature is found after comparing all specific features, it means that the smallest detection unit area is defect-free.

[0148] Based on the comparison method described above, the terminal can perform initial defect classification of the wafer region (smallest detection unit region) according to the characteristics (area, grayscale ratio, aspect ratio, etc.) of the corresponding defect region. This involves identifying the smallest detection unit region without defect features, the smallest detection unit region with obvious defect features, the smallest detection unit region with indistinct defect features, and the smallest detection unit region that may contain multiple defects.

[0149] 106. Based on the initial inspection analysis results, select images to be re-inspected from the images to be inspected;

[0150] The terminal determines the images to be re-inspected based on the initial inspection analysis results. Specifically, after step 105, the terminal can determine the smallest detection unit area without defect features, the smallest detection unit area with obvious defect features, the smallest detection unit area with indistinct defect features, and the smallest detection unit area that may have multiple defects. The terminal selects the images corresponding to the smallest detection unit areas with indistinct defect features and the images corresponding to the smallest detection unit areas that may have multiple defects, and determines them as the images that need to be re-inspected.

[0151] It should be noted that the image to be re-examined here can be the complete image to be detected, or it can be the smallest detection area cropped from the image to be detected as the image to be re-examined.

[0152] 107. When the GPU's operating status reaches the second preset condition, the image to be re-inspected is transmitted to the GPU's re-inspection module;

[0153] As the CPU's detection thread continuously generates initial detection analysis results, the detection thread will receive new images to be detected for initial detection as needed, until all images to be detected have been detected. At the same time, the terminal will generate more and more images to be re-inspected based on the initial detection analysis structure. When the number of images to be re-inspected accumulates to a certain amount, it can be determined whether the current GPU operating status has reached the second preset condition.

[0154] In this embodiment, while performing the initial system check using the CPU, the terminal determines whether the current GPU operating state meets the second preset condition and whether the preset conditions for entering the system for review are met. If met, the terminal inputs the image to be reviewed into the system review module for GPU artificial intelligence network model inference. If not met, the existing image to be reviewed is saved, and the above process is repeated until the conditions are met.

[0155] 108. The re-inspection module performs a re-inspection of the image to be re-inspected and generates re-inspection analysis results;

[0156] 109. Based on the initial inspection analysis results and the re-inspection analysis results, classify and summarize the defects of micro-display devices.

[0157] The terminal categorizes and summarizes defects in micro-display devices based on the initial inspection and re-inspection analysis results. This primarily involves integrating the initial and re-inspection results and saving them to the corresponding storage or display area. The terminal generates a data list (containing all feature information for each wafer) based on the initial inspection analysis results and image IDs. It then obtains the image IDs and wafer indices for all re-inspection categories. Next, the terminal retrieves the re-inspection analysis results and replaces them in the data list according to the image IDs and wafer indices corresponding to each re-inspection category, finally outputting the data list.

[0158] In this embodiment, a high-speed aerial camera first acquires an image of the microdisplay device to be inspected. Then, the operating status of the CPU (Central Processing Unit) and GPU (Graphics Processing Unit) is monitored in real time. Based on the CPU's operating status, the image to be inspected and template information are transmitted to the inspection thread. The template information consists of template feature information extracted from the defect-free image of the microdisplay device. A positioning module, combined with the template information, locates the smallest detection unit region on the image to be inspected. The smallest detection unit region is the effective region in the image to be inspected, thus determining the effective detection area and reducing the computational load of the initial inspection and re-inspection systems. Initial defect inspection is performed on the smallest detection unit region of the image to be inspected, generating an initial inspection analysis result. Then, based on the initial inspection analysis result, images to be re-inspected are selected from the images to be inspected. These images are those where the initial inspection defect features do not meet the judgment criteria for the smallest detection unit region, or images corresponding to the smallest detection unit regions with at least two initial inspection feature defects. Based on the GPU's operating status, the images to be re-inspected are transmitted to the GPU's re-inspection module. After determining the images to be re-inspected, the re-inspection module performs a re-inspection on the images to be re-inspected, generating a re-inspection analysis result. Finally, based on the initial inspection analysis results and the re-inspection analysis results, the defects of the micro-display devices were classified and summarized.

[0159] By monitoring the operating status of the CPU and GPU, when the CPU status indicates that it can allocate enough detection threads for the initial inspection, multiple images to be inspected and their corresponding template information are sent to different detection threads, allowing multiple images to be inspected simultaneously on different threads. After the initial inspection of the images to be inspected, some images that have met the inspection standards are filtered out, and the remaining images are sent to the re-inspection module for further inspection. When both the GPU operating status and the re-inspection start conditions are met, the images to be re-inspected are sent to the GPU's re-inspection module for re-inspection.

[0160] In the above process, multiple images to be detected can be detected simultaneously in different detection threads, and the initial inspection and re-inspection of different images to be detected can also be run simultaneously. The CPU runs the initial inspection and the GPU runs the re-inspection. While achieving simultaneous detection of multiple targets, the performance of the CPU and GPU can be fully utilized, further improving the efficiency of the entire defect classification system.

[0161] Please refer to Figures 2, 3, 4, and 5. This application provides an embodiment of a defect classification method based on microdisplay devices, including:

[0162] As shown in Figure 2, 201. The image of the microdisplay device to be tested is acquired by a high-speed flying camera;

[0163] Step 201 in this embodiment is similar to step 101 in the previous embodiment, and will not be repeated here.

[0164] 202. Obtain the standard deviation, average gray level, and number of pixels of the image to be detected;

[0165] 203. Calculate the center threshold based on the image standard deviation, average gray level, and number of pixels;

[0166] 204. Determine the locations of pixels in the image to be detected whose grayscale values ​​are not less than the center threshold;

[0167] 205. Calculate the center coordinate position based on the positions of pixels whose grayscale values ​​are not less than the center threshold;

[0168] 206. Obtain the spatial domain standard deviation of the image to be detected;

[0169] 207. Calculate the spatial domain Gaussian filter kernel based on the center coordinates, spatial domain standard deviation, and the first preset Gaussian function;

[0170] 208. Obtain the standard deviation of the color gamut of the image to be detected;

[0171] 209. Calculate the color gamut Gaussian filter kernel based on the center coordinate position, color gamut standard deviation, and the second preset Gaussian function;

[0172] 210. Apply Gaussian filtering to the image to be detected based on the spatial domain Gaussian filter kernel and the color domain Gaussian filter kernel;

[0173] Images captured by industrial-grade drone cameras often contain various types of noise. To ensure image quality, noise reduction processing is essential. In this embodiment, Gaussian filtering is primarily used, while mean filtering can be applied as needed. If mean filtering is used, a specific method involves applying a 3×3 mean filtering window to the image.

[0174] In this embodiment, the core Gaussian filtering processing method can be a traditional approach, such as using a Gaussian function as a filter to perform weighted averaging on the signal to smooth the signal and reduce noise, thereby filtering the image to be detected.

[0175] Gaussian filtering can be a conventional form of Gaussian filtering. However, in the case of images on microdisplay devices, the use of high-speed aerial cameras for rapid acquisition presents structural differences from traditional image acquisition methods, placing higher demands on the image quality. Noise generated during the aerial shooting process reduces the effectiveness of traditional Gaussian filtering for noise reduction. Specifically, during aerial shooting, the short shooting time and the relative displacement between the camera and the microdisplay device can easily generate new noise.

[0176] This embodiment proposes a Gaussian filtering method specifically designed to handle this type of aerial photography noise, achieving significant noise reduction. First, the standard deviation S, average gray level M, and number of pixels N of the image to be detected are obtained. Then, the center threshold D is calculated based on the standard deviation S, average gray level M, and number of pixels N. The formula for calculating the standard deviation is as follows:

[0177] The center threshold D is calculated as follows:

[0178] in, The variance adjustment coefficient is a positive number. It is determined based on the average gray level M. First, the average gray level of historically acquired normal, defect-free microdisplay device images is used as a reference average gray level. Then, the difference between the average gray level of the image to be tested and the reference average gray level is compared. The variance adjustment coefficient is generated proportionally based on the magnitude of this difference. It should be noted that the variance adjustment coefficient can also be manually set by comparing the difference between the average gray level and the reference average gray level. Generally, the larger the difference, the larger the variance adjustment coefficient, and vice versa. If the difference between the average gray level of the image to be tested and the reference average gray level is within an acceptable preset range, the variance adjustment coefficient is set to 1.

[0179] After the terminal calculates the center threshold, it determines the positions of pixels in the image to be detected whose gray values ​​are not less than the center threshold. These pixels are the pixels in the wafer region. The terminal calculates the center coordinates based on the positions of the pixels with gray values ​​not less than the center threshold. Specifically, it can be obtained by fitting the center point data of these pixels or by calculating the centroid of the region. The fitting method can be of various types.

[0180] After determining the center coordinates, Gaussian filtering is applied to the image to be detected in both the spatial and color domains.

[0181] Specifically, the spatial domain standard deviation δ of the image to be detected is first obtained. α Then, based on the center coordinate position (k,l) and the standard deviation δ of the spatial domain... α and the first preset Gaussian function w s Calculate the spatial domain Gaussian filter kernel (i,j,k,l). The formula is as follows:

[0182] Where (i,j) represents the position of a pixel in the image to be detected whose gray value is not less than the center threshold.

[0183] Obtain the standard deviation δ of the color gamut of the image to be detected. r Based on the center coordinate position (k,l) and the standard deviation of the color gamut δ r Second preset Gaussian function w g Calculate the color gaussian filter kernel (i,j,k,l). The formula is as follows:

[0184] Where (i,j) represents the pixel position in the image to be detected whose gray value is not less than the center threshold. f(k,l) is the gray value at the center coordinate position, f(i,j) is the gray value at the pixel position whose gray value is not less than the center threshold, and || is the Euclidean distance.

[0185] Finally, Gaussian filtering is applied to the image to be detected based on the spatial domain Gaussian filter kernel and the color domain Gaussian filter kernel. Specifically, the target Gaussian filter kernel is obtained by multiplying the functions of the spatial domain Gaussian filter kernel and the color domain Gaussian filter kernel:

[0186] w z (i,j,k,l)=w s (i,j,k,l)×w g (i,j,k,l)

[0187] Then through w z (i,j,k,l) ​​performs convolution processing on the image to be detected. This filtering and noise reduction method analyzes the color domain and spatial domain of the image and combines it with a well-designed Gaussian function to generate a target Gaussian filter kernel for the aerial photography system, thereby improving the noise reduction effect on the image of the aerial photography system.

[0188] After step 210, proceed to step 211.

[0189] As shown in Figure 3, 211. Real-time monitoring of the operating status of the CPU central processing unit and the GPU graphics processing unit;

[0190] Step 212 in this embodiment is similar to step 102 in the previous embodiment, and will not be repeated here.

[0191] 212. Determine the CPU idle detection thread based on the CPU's operating status;

[0192] 213. When the number of idle detection threads of the CPU is greater than the preset first number of threads, the image to be detected and the template information are transmitted to the detection thread. The template information is the template feature information extracted from the defect-free image of the micro-display device.

[0193] The first preset condition includes comprehensive performance indicators such as CPU utilization, average utilization, maximum temperature, and frequency fluctuation. Only when this preset condition is met can the initial inspection proceed. The terminal determines the number of detection threads available for allocation based on the CPU's operating status and the aforementioned comprehensive performance indicators. When the number of available detection threads exceeds the preset first number, the image to be inspected and template information are transmitted to the detection thread. The template information consists of template feature information extracted from the defect-free image of the microdisplay device. The remaining threads wait until the conditions are met before being distributed.

[0194] 214. Determine at least one coarse bounding box on the image to be detected based on the template information;

[0195] 215. Determine the initial center coordinates of the coarse positioning frame region;

[0196] 216. Generate an attention matrix for each pixel in the coarse localization box of the image to be detected;

[0197] 217. Based on the attention matrix, filter out pixels in each coarse positioning box whose attention is less than the preset attention threshold;

[0198] 218. Calculate the offset center coordinates using the pixel positions of the remaining pixels in the coarse positioning frame;

[0199] 219. Calculate the offset data based on the initial center coordinate position and the offset center coordinate position. The offset data includes the offset distance and offset direction. Determine whether the offset data meets the preset distance conditions. If it does not meet the conditions, proceed to step 220. If it does meet the conditions, proceed to step 221.

[0200] 220. When the offset data does not meet the preset distance conditions, adjust the position of the coarse positioning box on the image to be detected according to the offset data, and redetermine the initial center coordinate position of the new coarse positioning box, calculate the attention matrix, filter pixels, calculate the offset center coordinate position, and analyze the offset data; that is, after adjusting the position of the coarse positioning box on the image to be detected according to the offset data in step 220, return to step 215 and repeat steps 215 and the steps after 215.

[0201] 221. When the offset data meets the preset distance condition, determine the smallest detection unit area based on the coarse positioning box on the current image to be detected;

[0202] The template information contains multiple calibration boxes. The terminal first determines at least one coarse positioning box on the image to be inspected based on the calibration boxes in the template information. Initially, the coarse positioning box may completely cover the smallest detection unit region (wafer region) on the image to be inspected, or it may have offsets in different directions. At this point, it is necessary to determine the initial center coordinates of the coarse positioning box region. The initial center position is calculated by using the coordinates of the four corners of the coarse positioning box on the image to be inspected.

[0203] Next, the Dropout module generates an attention matrix for each pixel in the coarse bounding box of the image to be detected. The Dropout module includes BatchNorm-2*2DefConv-ReLU and BatchNorm-2*2DefConv-SigMiod. First, the coarse bounding box region in the image to be detected is input into BatchNorm-2*2DefConv-ReLU for processing, and then its output is input into BatchNorm-2*2DefConv-SigMiod to generate an attention matrix of the same size as the coarse bounding box region.

[0204] After generating the attention matrix, the terminal filters out pixels with attention values ​​less than a preset attention threshold in each coarse localization bounding box, leaving only the matching pixels. Most of the pixels with matching attention values ​​belong to the wafer region (the smallest detection unit region), so the offset center coordinates need to be calculated using the pixel positions of the remaining pixels in the coarse localization bounding box (this can be done using a centroid algorithm). After calculating the offset center coordinates, the terminal calculates offset data based on the initial center coordinates and the offset center coordinates. The offset data includes the offset distance and offset direction; that is, the error between the initial center coordinates and the actual wafer region is determined based on the offset center coordinates, and the initial center coordinates are then repositioned.

[0205] When the offset data does not meet the preset conditions, the terminal adjusts the position of the coarse bounding box on the image to be detected based on the offset data, and redetermines the initial center coordinates of the new coarse bounding box, calculates the attention matrix, filters pixels, calculates the offset center coordinates, and analyzes the offset data. When the distance between the two is not within the preset range, it is necessary to adjust the initial center point coordinates based on the offset data and redraw the coarse bounding box for detection.

[0206] When the offset data meets the preset conditions, the terminal determines the smallest detection unit area and the background area based on the coarse positioning box on the current image to be detected. Specifically, when the distance between the two is within the preset range, it means that the current coarse positioning box and the actual wafer area position are basically consistent, and subsequent detection can be performed.

[0207] After step 221, proceed to step 222.

[0208] As shown in Figure 4, 222. Set the target convolution kernel according to the wafer position information on the template information;

[0209] 223. Set the non-detection areas on the image to be detected to black;

[0210] 224. Use the target convolution kernel to perform convolution emphasis processing on the smallest detection unit region on the image to be detected;

[0211] 225. Analyze the smallest detection unit region after convolutional emphasis processing using the blob algorithm to generate defect feature information of the smallest detection unit region;

[0212] 226. Perform defect comparison analysis based on defect feature information to generate first-pass inspection analysis results;

[0213] In this embodiment, since the wafer spacing is the same and periodic, the terminal sets the target convolution kernel according to the wafer position information on the template information. The setting of the convolution kernel mainly includes determining the size of the detection convolution kernel and the relevant parameters of the convolution kernel.

[0214] The terminal sets the background area on the image to be detected to black, so that the detection of irrelevant background areas is reduced in the subsequent initial inspection process, thus reducing the amount of computation.

[0215] The terminal uses a target convolution kernel to perform convolutional emphasis processing on the smallest detection unit region on the image to be detected, and obtains the corresponding economic features. Then, the terminal analyzes and extracts the features of the convolutionally emphasized defect region through the blob algorithm, and calculates the defect feature information of the defect region.

[0216] The terminal classifies defects on the wafer region based on defect feature information (defect area information, grayscale ratio of the defect region, aspect ratio of the defect region, etc.) and generates initial inspection analysis results. Wafer regions with indistinct defect features or those containing multiple defect feature types are marked as re-inspection categories.

[0217] The terminal cuts out the entire wafer area that needs to be re-inspected based on the wafer's location coordinates, generating an image to be re-inspected, in preparation for subsequent re-inspection.

[0218] In this embodiment, the terminal can save the already sorted categories, count the number of images to be re-inspected and the number of re-inspection categories, and repeatedly execute the above-mentioned first inspection module and process to minimize the number of re-inspections.

[0219] After step 226, proceed to step 227.

[0220] As shown in Figure 5, 227. Based on the initial inspection analysis results, select the images to be re-inspected from the images to be inspected;

[0221] Step 227 in this embodiment is similar to step 106 in the previous embodiment, and will not be repeated here.

[0222] 228. Determine the GPU's idle detection thread based on the GPU's running status;

[0223] 229. When the number of idle detection threads of the GPU is greater than the preset number of second threads, and the number of images to be re-inspected reaches the preset number of re-inspections, the images to be re-inspected are sent to the GPU's re-inspection module.

[0224] The second preset condition includes comprehensive performance indicators such as GPU temperature, frequency, power consumption information, average utilization, maximum temperature, frequency fluctuation, utilization rate, and memory usage.

[0225] In this embodiment, while performing the initial system check using the CPU, the terminal determines whether the current GPU operating status meets the comprehensive performance indicators and whether it meets the preset entry conditions for system review (e.g., whether it meets the requirement of 5000 wafer thumbnails). If the conditions are met, the terminal inputs the image to be reviewed into the system review module for GPU AI network model inference. If the conditions are not met, the existing image to be reviewed is saved, and the above process is repeated until the conditions are met.

[0226] 230. Input the image to be re-examined into the input layer of the DPN classification network model. The DPN classification network model includes an input layer, an initial convolutional layer, at least two DPN blocks, a fully connected layer, and an output layer. The DPN block includes a feature extraction module and at least two DPU modules.

[0227] 231. Extract primary features from the image to be re-examined through the initial convolutional layer of the DPN classification network model, and then input the feature extraction module of at least two DPN blocks to extract secondary features.

[0228] 232. By simultaneously processing and transmitting secondary features through residual connections and dense connections of the DPU block, fused features are generated;

[0229] 233. The fusion features extracted and fused from the DPU block are passed through a fully connected layer to vectorize the fusion features, and then classified through the output layer to generate the re-examination analysis results.

[0230] In this embodiment, the terminal uses the DPN classification network model for re-inspection. DPN (Dual Path Network) is a deep neural network architecture that combines the advantages of ResNet (Residual Network) and DenseNet (Dense Convolutional Network), and is applied to image defect classification and recognition of the micro-display device in this embodiment.

[0231] The core of the DPN classification network model constructed in this embodiment lies in the construction of residual paths and dense paths. Through parallel residual paths and dense paths, more efficient feature extraction and transmission are achieved, thereby improving the performance and stability of the network.

[0232] Specifically, the architecture of the DPN classification network model mainly includes an input layer, an initial convolutional layer, multiple DPN blocks, a fully connected layer, and an output layer.

[0233] The input layer is mainly used to receive input image data.

[0234] The initial convolutional layer is mainly used to perform the initial convolution operation to extract low-level features.

[0235] Each DPN block consists of multiple Dual Path Units (DPUs). The Dual Path Unit is the core component of DPN. Each DPU includes two parallel paths: a residual path and a dense path. The residual path preserves input features, avoids the vanishing gradient problem through skip connections, and directly passes the input features to the output. The parallel dense path connects the output of each layer with the outputs of all preceding layers, forming dense connections and promoting the reuse of features. The functionality of this DPN block will be described in detail later.

[0236] Fully connected layers are used to vectorize the features extracted by convolutional layers for classification and can work in conjunction with the output layer.

[0237] The output layer is used to output the classification results of defect features through an activation function (such as Softmax).

[0238] The architecture of the DPN classification network model in this embodiment is mainly capable of feature extraction, feature fusion, and classification decision-making.

[0239] When the input image to be detected passes through the initial convolutional layer, features are extracted once, and then it enters multiple DPN blocks for further secondary feature extraction. Feature extraction is one of the functions of the DPN blocks.

[0240] In each DPN block, the DPU processes and transmits features simultaneously through residual connections and dense connections, thereby achieving efficient feature fusion.

[0241] Features extracted and fused by DPN blocks are passed through a fully connected layer and finally classified through the output layer.

[0242] This DPN classification network model enables efficient feature extraction and propagation. Specifically, by preserving input features through residual paths and directly passing them to the output, it avoids the vanishing and exploding gradient problems and supports deeper network structures. Simultaneously, dense paths promote the multiple utilization of features, improving feature propagation efficiency and network performance.

[0243] Secondly, this embodiment enables deeper network training by combining residual connections and dense connections in DPN, thereby improving the accuracy of defect classification.

[0244] Furthermore, the DPN classification network model can utilize features multiple times, and the dense connection paths ensure that the output of each layer is directly connected to the subsequent layers, enhancing the multiple utilization and sharing of features and improving the expressive power of the network model.

[0245] The DPN classification network model can reduce parameters and improve computational efficiency. Although the network depth increases, the efficient feature transfer and sharing mechanism of the path enables DPN to maintain high performance while reducing the number of parameters and overall computational complexity.

[0246] Before training, a dataset is collected, including training images of wafer defects of different defect types on the microdisplay device. The collected wafer defect training images are then used for pre-training to obtain the trained DPN classification network model.

[0247] 234. Based on the initial inspection analysis results and the re-inspection analysis results, classify and summarize the defects of micro-display devices.

[0248] Step 234 in this embodiment is similar to step 109 in the previous embodiment, and will not be repeated here.

[0249] In this embodiment, the image of the microdisplay device to be tested is first acquired by a high-speed aerial camera.

[0250] Obtain the standard deviation, average gray level, and number of pixels of the image to be detected. Calculate the center threshold based on the standard deviation, average gray level, and number of pixels. Determine the positions of pixels in the image to be detected whose gray level is not less than the center threshold. Calculate the center coordinate position based on the positions of pixels with gray levels not less than the center threshold. Obtain the spatial domain standard deviation of the image to be detected. Calculate the spatial domain Gaussian filter kernel based on the center coordinate position, spatial domain standard deviation, and a first preset Gaussian function. Obtain the color domain standard deviation of the image to be detected. Calculate the color domain Gaussian filter kernel based on the center coordinate position, color domain standard deviation, and a second preset Gaussian function. Apply Gaussian filtering to the image to be detected based on the spatial domain Gaussian filter kernel and the color domain Gaussian filter kernel.

[0251] Then, the operating status of the CPU (Central Processing Unit) and GPU (Graphics Processing Unit) is monitored in real time. The CPU's idle detection threads are determined based on its operating status. When the number of idle detection threads exceeds a preset first number, the image to be detected and template information are transmitted to the detection threads. The template information consists of template feature information extracted from the defect-free image of the microdisplay device.

[0252] Based on the template information, at least one coarse bounding box is determined on the image to be detected. The initial center coordinates of the coarse bounding box region are determined. An attention matrix is ​​generated for each pixel in the coarse bounding box of the image to be detected. Pixels in each coarse bounding box with an attention value less than a preset attention threshold are filtered out based on the attention matrix. The offset center coordinates are calculated using the pixel positions of the remaining pixels in the coarse bounding box. Offset data, including offset distance and offset direction, is calculated based on the initial center coordinates and the offset center coordinates. When the offset data does not meet the preset distance condition, the position of the coarse bounding box on the image to be detected is adjusted according to the offset data, and the initial center coordinates of the new coarse bounding box are determined, the attention matrix is ​​calculated, pixel filtering is performed, the offset center coordinates are calculated, and the offset data is analyzed. When the offset data meets the preset distance condition, the smallest detection unit region is determined based on the coarse bounding boxes on the current image to be detected.

[0253] The target convolution kernel is set according to the wafer position information in the template information. Non-detection areas in the image to be detected are set to black. The target convolution kernel is used to perform convolutional emphasis processing on the smallest detection unit region in the image to be detected. The smallest detection unit region after convolutional emphasis processing is analyzed using the blob algorithm to generate defect feature information of the smallest detection unit region. Defect comparison analysis is performed based on the defect feature information to generate the initial inspection analysis result. Then, images to be re-inspected are selected from the images to be detected based on the initial inspection analysis result. These re-inspected images are images of smallest detection unit regions whose initial inspection defect features do not meet the judgment criteria, or images corresponding to smallest detection unit regions with at least two initial inspection feature defects. The idle detection threads of the GPU are determined based on the GPU's running status. When the number of idle detection threads of the GPU is greater than the preset second thread number, and the number of images to be re-inspected reaches the preset re-inspection number, the images to be re-inspected are sent to the GPU's re-inspection module. The images to be re-inspected are input into the input layer of the DPN classification network model. The DPN classification network model includes an input layer, an initial convolutional layer, at least two DPN blocks, a fully connected layer, and an output layer. Each DPN block includes a feature extraction module and at least two DPU modules. The DPN classification network model extracts primary features from the image to be re-examined through its initial convolutional layers. Secondary features are then extracted by feature extraction modules in at least two DPN blocks. These secondary features are simultaneously processed and transmitted through residual and dense connections in the DPU blocks to generate fused features. The fused features extracted and fused by the DPU blocks are then passed through fully connected layers to vectorize them, and finally, the output layer performs classification to generate the re-examination analysis results.

[0254] Finally, based on the initial inspection analysis results and the re-inspection analysis results, the defects of the micro-display devices were classified and summarized.

[0255] By monitoring the operating status of the CPU and GPU, when the CPU status indicates that it can allocate enough detection threads for the initial inspection, multiple images to be inspected and their corresponding template information are sent to different detection threads, allowing multiple images to be inspected simultaneously on different threads. After the initial inspection of the images to be inspected, some images that have met the inspection standards are filtered out, and the remaining images are sent to the re-inspection module for further inspection. When both the GPU operating status and the re-inspection start conditions are met, the images to be re-inspected are sent to the GPU's re-inspection module for re-inspection.

[0256] In the above process, multiple images to be detected can be detected simultaneously in different detection threads, and the initial inspection and re-inspection of different images to be detected can also be run simultaneously. The CPU runs the initial inspection and the GPU runs the re-inspection. While achieving simultaneous detection of multiple targets, the performance of the CPU and GPU can be fully utilized, further improving the efficiency of the entire defect classification system.

[0257] Secondly, in this embodiment, the DPN classification network model can efficiently extract and transfer features. Specifically, it preserves input features through residual paths and directly transfers them to the output, avoiding the vanishing and exploding gradient problems and supporting deeper network structures. Simultaneously, dense paths promote the multiple uses of features, improving feature transfer efficiency and network performance.

[0258] Secondly, this embodiment combines residual connections and dense connections in DPN to train deeper networks, thereby improving defect classification accuracy. Furthermore, the DPN classification network model can utilize features multiple times; the dense connection paths ensure that the output of each layer directly connects to subsequent layers, enhancing feature reuse and sharing, and improving the model's expressive power. The DPN classification network model reduces parameters and computational efficiency. While increasing network depth, the efficient feature transfer and sharing mechanism through the paths allows DPN to maintain high performance while reducing the number of parameters and overall computational complexity.

[0259] Secondly, by constructing color domain and spatial domain filtering kernels through the designed Gaussian kernel function, noise in the captured images of microdisplay devices can be removed more effectively.

[0260] Please refer to Figure 6. This application provides an embodiment of a defect classification device based on a microdisplay device, comprising:

[0261] The acquisition unit 601 is used to acquire the image to be tested of the microdisplay device using a high-speed camera.

[0262] The Gaussian noise reduction unit 602 is used to perform Gaussian filtering noise reduction processing on the image to be detected;

[0263] Optionally, the Gaussian noise reduction unit 602 is used for:

[0264] Obtain the standard deviation, average gray level, and number of pixels of the image to be detected;

[0265] The center threshold is calculated based on the image standard deviation, average gray level, and number of pixels.

[0266] Determine the location of pixels in the image to be detected whose grayscale value is not less than the center threshold;

[0267] The center coordinate position is calculated based on the position of the pixel whose gray value is not less than the center threshold.

[0268] Obtain the spatial domain standard deviation of the image to be detected;

[0269] Calculate the spatial domain Gaussian filter kernel based on the center coordinate position, spatial domain standard deviation, and the first preset Gaussian function;

[0270] Obtain the standard deviation of the color gamut of the image to be detected;

[0271] Calculate the color gaussian filter kernel based on the center coordinate position, the standard deviation of the color gaussian ... function, and the second preset Gaussian function.

[0272] Gaussian filtering is performed on the image to be detected based on the spatial domain Gaussian filter kernel and the color domain Gaussian filter kernel.

[0273] The monitoring unit 603 is used to monitor the operating status of the CPU central processing unit and the GPU graphics processing unit in real time.

[0274] The first transmission unit 604 is used to transmit the image to be detected and template information to the detection thread when the CPU running state reaches the first preset condition. The template information is template feature information extracted from the defect-free image of the micro-display device.

[0275] Optionally, the first transmission unit 604 is used for:

[0276] The CPU idle detection thread is determined based on the CPU's operating status;

[0277] When the number of idle detection threads of the CPU is greater than the preset first number of threads, the image to be detected and the template information are transmitted to the detection thread. The template information is the template feature information extracted from the defect-free image of the microdisplay device.

[0278] The positioning unit 605 is used to locate the smallest detection unit region on the image to be detected by combining the positioning module with the template information. The smallest detection unit region is the effective region in the image to be detected.

[0279] Optionally, the positioning unit 605 is used for:

[0280] Based on the template information, at least one coarse bounding box is determined on the image to be detected;

[0281] Determine the initial center coordinates of the coarse positioning bounding box region;

[0282] Generate an attention matrix for each pixel in the coarse localization box of the image to be detected;

[0283] Pixels with an attention level less than a preset attention threshold in each coarse positioning bounding box are filtered out based on the attention matrix;

[0284] The offset center coordinate position is calculated by using the pixel positions of the remaining pixels in the coarse positioning box;

[0285] The offset data is calculated based on the initial center coordinates and the offset center coordinates. The offset data includes the offset distance and offset direction.

[0286] When the offset data does not meet the preset distance conditions, the position of the coarse localization box on the image to be detected is adjusted according to the offset data, and the initial center coordinate position of the new coarse localization box is re-determined, the attention matrix is ​​calculated, pixels are filtered out, the offset center coordinate position is calculated, and the offset data is analyzed.

[0287] When the offset data meets the preset distance conditions, the smallest detection unit area is determined based on the coarse positioning box on the current image to be detected.

[0288] The first generation unit 606 is used to perform initial defect detection in the smallest detection unit region of the image to be detected and generate initial detection analysis results.

[0289] Optionally, the first generating unit 606 is used for:

[0290] Set the target convolution kernel based on the wafer position information in the template information;

[0291] Set the non-detection areas on the image to be detected to black;

[0292] The target convolution kernel is used to perform convolution emphasis processing on the smallest detection unit region on the image to be detected;

[0293] The blob algorithm is used to analyze the smallest detection unit region after convolutional emphasis processing, and to generate defect feature information of the smallest detection unit region.

[0294] Based on the defect characteristic information, a defect comparison analysis is performed to generate the first inspection analysis results.

[0295] The filtering unit 607 generates images to be re-inspected from the images to be inspected based on the initial inspection analysis results;

[0296] The second transmission unit 608 is used to transmit the image to be re-inspected to the re-inspection module of the GPU when the GPU's operating state reaches the second preset condition.

[0297] Optionally, the second transmission unit 608 is used for:

[0298] The idle detection thread of the GPU is determined based on the GPU's running status;

[0299] When the number of idle detection threads of the GPU is greater than the preset number of second threads, and the number of images to be re-inspected reaches the preset number of re-inspections, the images to be re-inspected are sent to the GPU's re-inspection module.

[0300] The second generation unit 609 is used to re-examine the image to be re-examined through the re-examination module and generate re-examination analysis results.

[0301] Optionally, the second generating unit 609 is used for:

[0302] The image to be re-examined is input into the input layer of the DPN classification network model. The DPN classification network model includes an input layer, an initial convolutional layer, at least two DPN blocks, a fully connected layer, and an output layer. The DPN block includes a feature extraction module and at least two DPU modules.

[0303] The initial convolutional layer of the DPN classification network model extracts primary features from the image to be re-examined, and then inputs the feature extraction module of at least two DPN blocks to extract secondary features.

[0304] Secondary features are processed and transmitted simultaneously through residual connections and dense connections of DPU blocks to generate fused features;

[0305] The fusion features extracted and fused from the DPU block are passed through a fully connected layer to vectorize the fusion features, and then classified through the output layer to generate the re-examination analysis results.

[0306] The summary unit 610 is used to classify and summarize the defects of micro-display devices based on the initial inspection analysis results and the re-inspection analysis results.

[0307] Please refer to Figure 7. This application provides a defect classification device based on a microdisplay device, comprising:

[0308] Processor 701, memory 702, input / output unit 703, and bus 704.

[0309] The processor 701 is connected to the memory 702, the input / output unit 703, and the bus 704.

[0310] The memory 702 stores a program, and the processor 701 calls the program to execute the defect classification method shown in Figures 1, 2 and 3, 4 and 5.

[0311] This application provides a computer-readable storage medium on which a program is stored. When the program is executed on a computer, it performs the defect classification method shown in Figures 1, 2, 3, 4, and 5.

[0312] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0313] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between devices or units through some interfaces, and may be electrical, mechanical, or other forms.

[0314] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0315] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0316] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A defect classification method based on microdisplay devices, comprising: Images of the microdisplay device to be tested are acquired using a high-speed aerial camera; Real-time monitoring of the operating status of the CPU (Central Processing Unit) and the GPU (Graphics Processing Unit); When the CPU running state reaches the first preset condition, the image to be detected and the template information are transmitted to the detection thread. The template information is template feature information extracted from the defect-free image of the micro-display device. The positioning module is used in conjunction with the template information to locate the smallest detection unit region on the image to be detected, and the smallest detection unit region is the effective region in the image to be detected. Perform initial defect detection on the smallest detection unit region of the image to be detected, and generate initial detection analysis results; Based on the initial inspection analysis results, images to be re-inspected are selected from the images to be inspected; When the GPU's operating state reaches the second preset condition, the image to be re-inspected is transmitted to the GPU's re-inspection module; The re-inspection module performs a re-inspection on the image to be re-inspected and generates a re-inspection analysis result. Based on the initial inspection analysis results and the re-inspection analysis results, the defects of the micro-display devices are classified and summarized.

2. The defect classification method according to claim 1, wherein, The step of performing initial defect detection on the smallest detection unit region of the image to be detected and generating initial detection analysis results includes: Set the target convolution kernel based on the wafer position information in the template information; Set the non-detection areas on the image to be detected to black; The target convolution kernel is used to perform convolutional emphasis processing on the smallest detection unit region in the image to be detected; The smallest detection unit region after convolutional emphasis processing is analyzed using the blob algorithm to generate defect feature information of the smallest detection unit region. Based on the defect feature information, a defect comparison analysis is performed to generate the first inspection analysis result.

3. The defect classification method according to claim 1, wherein, The step of performing a re-inspection on the image to be re-inspected through the re-inspection module and generating a re-inspection analysis result includes: The image to be re-examined is input into the input layer of the DPN classification network model. The DPN classification network model includes an input layer, an initial convolutional layer, at least two DPN blocks, a fully connected layer, and an output layer. The DPN block includes a feature extraction module and at least two DPU modules. The initial convolutional layer of the DPN classification network model extracts primary features from the image to be re-examined, and then inputs the feature extraction module of at least two DPN blocks to extract secondary features. The secondary features are processed and transmitted simultaneously through the residual connections and dense connections of the DPU block to generate fused features; The fusion features extracted and fused from the DPU block are passed through the fully connected layer to enable the fusion... The combined features are vectorized and then classified through the output layer to generate the re-examination analysis results.

4. The defect classification method according to claim 1, wherein, The step of using the positioning module in conjunction with the template information to locate the smallest detection unit region on the image to be detected includes: At least one coarse bounding box is determined on the image to be detected based on the template information; Determine the initial center coordinates of the coarse positioning frame region; An attention matrix is ​​generated for each pixel in the coarse localization box of the image to be detected; Pixels with an attention level less than a preset attention threshold in each coarse positioning frame are filtered out based on the attention matrix. The offset center coordinate position is calculated based on the pixel positions of the remaining pixels in the coarse positioning frame; Offset data is calculated based on the initial center coordinate position and the offset center coordinate position, and the offset data includes the offset distance and offset direction; When the offset data does not meet the preset distance conditions, the position of the coarse positioning box on the image to be detected is adjusted according to the offset data, and the initial center coordinate position of the new coarse positioning box is determined, the attention matrix is ​​calculated, pixel points are filtered out, the offset center coordinate position is calculated, and the offset data is analyzed. When the offset data meets the preset distance conditions, the smallest detection unit region is determined based on the coarse positioning box on the current image to be detected.

5. The defect classification method according to claim 1, wherein, The step of transmitting the image to be detected and the template information to the detection thread when the CPU operation state reaches the first preset condition includes: The CPU idle detection thread is determined based on the CPU operating status; When the number of idle detection threads of the CPU is greater than the preset first number of threads, the image to be detected and the template information are transmitted to the detection thread. The template information is template feature information extracted from the defect-free image of the microdisplay device. When the GPU's operating state reaches the second preset condition, the image to be re-inspected is transmitted to the GPU's re-inspection module, including: The idle detection thread of the GPU is determined based on the GPU's operating status; When the number of idle detection threads of the GPU is greater than the preset second number of threads, and the number of images to be re-inspected reaches the preset re-inspection number, the images to be re-inspected are transmitted to the GPU's re-inspection module.

6. The defect classification method according to claim 1, wherein, After the step of acquiring the image to be inspected from the microdisplay device using a high-speed camera and before the step of real-time monitoring of the operating status of the CPU and GPU, the defect classification method further includes: The image to be detected is subjected to Gaussian filtering for noise reduction.

7. The defect classification method according to claim 6, wherein, The step of performing Gaussian filtering noise reduction on the image to be detected includes: Obtain the standard deviation, average gray level, and number of pixels of the image to be detected; The center threshold is calculated based on the image standard deviation, average gray level, and number of pixels. Determine the positions of pixels in the image to be detected whose grayscale value is not less than the center threshold; The center coordinate position is calculated based on the position of the pixel whose gray value is not less than the center threshold. Obtain the spatial domain standard deviation of the image to be detected; Calculate the spatial domain Gaussian filter kernel based on the center coordinate position, the spatial domain standard deviation, and the first preset Gaussian function; Obtain the standard deviation of the color gamut of the image to be detected; Calculate the color gaussian filter kernel based on the center coordinate position, the color gaussian standard deviation, and the second preset Gaussian function; The image to be detected is subjected to Gaussian filtering based on the spatial domain Gaussian filter kernel and the color domain Gaussian filter kernel.

8. A defect classification device based on a microdisplay device, comprising: The acquisition unit is used to acquire images of the microdisplay device to be tested using a high-speed camera. The monitoring unit is used to monitor the operating status of the CPU central processing unit and the GPU graphics processing unit in real time. The first transmission unit is used to transmit the image to be detected and the template information to the detection thread when the CPU running state reaches the first preset condition. The template information is template feature information extracted from the defect-free image of the microdisplay device. A positioning unit is used to locate the smallest detection unit region on the image to be detected using the positioning module in conjunction with the template information, wherein the smallest detection unit region is the effective region in the image to be detected. The first generation unit is used to perform initial defect detection on the smallest detection unit region of the image to be detected and generate initial detection analysis results. The filtering unit generates images to be re-inspected from the images to be detected based on the initial inspection analysis results; The second transmission unit is used to transmit the image to be re-inspected to the re-inspection module of the GPU when the GPU's operating state reaches the second preset condition. The second generation unit is used to re-examine the image to be re-examined through the re-examination module and generate re-examination analysis results; The summarization unit is used to classify and summarize the defects of the micro-display device based on the initial inspection analysis results and the re-inspection analysis results.

9. The defect classification device according to claim 8, wherein, The first generation unit is used for: Set the target convolution kernel based on the wafer position information in the template information; Set the non-detection areas on the image to be detected to black; The target convolution kernel is used to perform convolution emphasis processing on the smallest detection unit region on the image to be detected; The blob algorithm is used to analyze the smallest detection unit region after convolutional emphasis processing, and to generate defect feature information of the smallest detection unit region. Based on the defect characteristic information, a defect comparison analysis is performed to generate the first inspection analysis results.

10. The defect classification device according to claim 8, wherein, The second generation unit is used for: The image to be re-examined is input into the input layer of the DPN classification network model. The DPN classification network model includes an input layer, an initial convolutional layer, at least two DPN blocks, a fully connected layer, and an output layer. Each DPN block includes a feature extraction module. Block and at least two DPU modules; The initial convolutional layer of the DPN classification network model extracts primary features from the image to be re-examined, and then inputs the feature extraction module of at least two DPN blocks to extract secondary features. Secondary features are processed and transmitted simultaneously through residual connections and dense connections of DPU blocks to generate fused features; The fusion features extracted and fused from the DPU block are passed through a fully connected layer to vectorize the fusion features, and then classified through the output layer to generate the re-examination analysis results.

11. The defect classification device according to claim 8, wherein, The positioning unit is used for: Based on the template information, at least one coarse bounding box is determined on the image to be detected; Determine the initial center coordinates of the coarse positioning bounding box region; Generate an attention matrix for each pixel in the coarse localization box of the image to be detected; Pixels with an attention level less than a preset attention threshold in each coarse positioning bounding box are filtered out based on the attention matrix; The offset center coordinate position is calculated by using the pixel positions of the remaining pixels in the coarse positioning box; The offset data is calculated based on the initial center coordinates and the offset center coordinates. The offset data includes the offset distance and offset direction. When the offset data does not meet the preset distance conditions, the position of the coarse localization box on the image to be detected is adjusted according to the offset data, and the initial center coordinate position of the new coarse localization box is re-determined, the attention matrix is ​​calculated, pixels are filtered out, the offset center coordinate position is calculated, and the offset data is analyzed. When the offset data meets the preset distance conditions, the smallest detection unit area is determined based on the coarse positioning box on the current image to be detected.

12. The defect classification device according to claim 8, wherein, The first transmission unit is used for: The CPU idle detection thread is determined based on the CPU's operating status; When the number of idle detection threads of the CPU is greater than the preset first number of threads, the image to be detected and the template information are transmitted to the detection thread. The template information is the template feature information extracted from the defect-free image of the micro-display device. The second transmission unit is used for: The idle detection thread of the GPU is determined based on the GPU's running status; When the number of idle detection threads on the GPU is greater than the preset number of second threads, and the number of images to be re-inspected reaches the preset number of re-inspections, the images to be re-inspected are sent to the GPU's re-inspection module.

13. The defect classification device according to claim 8, wherein, The defect classification device further includes: The Gaussian noise reduction unit is used to perform Gaussian filtering noise reduction processing on the image to be detected. The Gaussian noise reduction unit is electrically connected to the acquisition unit and the monitoring unit respectively.

14. The defect classification device according to claim 13, wherein, The Gaussian noise reduction unit is used for: Obtain the standard deviation, average gray level, and number of pixels of the image to be detected; The center threshold is calculated based on the image standard deviation, average gray level, and number of pixels. Determine the location of pixels in the image to be detected whose grayscale value is not less than the center threshold; The center coordinate position is calculated based on the position of the pixel whose gray value is not less than the center threshold. Obtain the spatial domain standard deviation of the image to be detected; Calculate the spatial domain Gaussian filter kernel based on the center coordinate position, spatial domain standard deviation, and the first preset Gaussian function; Obtain the standard deviation of the color gamut of the image to be detected; Calculate the color gaussian filter kernel based on the center coordinate position, the standard deviation of the color gaussian ... function, and the second preset Gaussian function. Gaussian filtering is performed on the image to be detected based on the spatial domain Gaussian filter kernel and the color domain Gaussian filter kernel.

15. A defect classification system based on microdisplay devices, comprising: A defect classification device based on microdisplay devices and a worktable for placing the microdisplay under test, wherein the defect classification system based on microdisplay devices includes: The acquisition unit is used to acquire images of the microdisplay device to be tested using a high-speed camera. The monitoring unit is used to monitor the operating status of the CPU central processing unit and the GPU graphics processing unit in real time. The first transmission unit is used to transmit the image to be detected and the template information to the detection thread when the CPU running state reaches the first preset condition. The template information is template feature information extracted from the defect-free image of the microdisplay device. A positioning unit is used to locate the smallest detection unit region on the image to be detected using the positioning module in conjunction with the template information, wherein the smallest detection unit region is the effective region in the image to be detected. The first generation unit is used to perform initial defect detection on the smallest detection unit region of the image to be detected and generate initial detection analysis results. The filtering unit generates images to be re-inspected from the images to be detected based on the initial inspection analysis results; The second transmission unit is used to transmit the image to be re-inspected to the re-inspection module of the GPU when the GPU's operating state reaches the second preset condition. The second generation unit is used to re-examine the image to be re-examined through the re-examination module and generate re-examination analysis results; The summarization unit is used to classify and summarize the defects of the micro-display device based on the initial inspection analysis results and the re-inspection analysis results.

16. The defect classification system according to claim 15, wherein, The first generation unit is used for: Set the target convolution kernel based on the wafer position information in the template information; Set the non-detection areas on the image to be detected to black; The target convolution kernel is used to perform convolution emphasis processing on the smallest detection unit region on the image to be detected; The blob algorithm is used to analyze the smallest detection unit region after convolutional emphasis processing, and to generate defect feature information of the smallest detection unit region. Based on the defect characteristic information, a defect comparison analysis is performed to generate the first inspection analysis results.

17. The defect classification system according to claim 15, wherein, The second generation unit is used for: The image to be re-examined is input into the input layer of the DPN classification network model. The DPN classification network model includes an input layer, an initial convolutional layer, at least two DPN blocks, a fully connected layer, and an output layer. The DPN block includes a feature extraction module and at least two DPU modules. The initial convolutional layer of the DPN classification network model extracts primary features from the image to be re-examined, and then inputs the feature extraction module of at least two DPN blocks to extract secondary features. Secondary features are processed and transmitted simultaneously through residual connections and dense connections of DPU blocks to generate fused features; The fusion features extracted and fused from the DPU block are passed through a fully connected layer to vectorize the fusion features, and then classified through the output layer to generate the re-examination analysis results.

18. The defect classification system according to claim 15, wherein, The positioning unit is used for: Based on the template information, at least one coarse bounding box is determined on the image to be detected; Determine the initial center coordinates of the coarse positioning bounding box region; Generate an attention matrix for each pixel in the coarse localization box of the image to be detected; Pixels with an attention level less than a preset attention threshold in each coarse positioning bounding box are filtered out based on the attention matrix; The offset center coordinate position is calculated by using the pixel positions of the remaining pixels in the coarse positioning box; The offset data is calculated based on the initial center coordinates and the offset center coordinates. The offset data includes the offset distance and offset direction. When the offset data does not meet the preset distance conditions, the position of the coarse localization box on the image to be detected is adjusted according to the offset data, and the initial center coordinate position of the new coarse localization box is re-determined, the attention matrix is ​​calculated, pixels are filtered out, the offset center coordinate position is calculated, and the offset data is analyzed. When the offset data meets the preset distance conditions, the smallest detection unit area is determined based on the coarse positioning box on the current image to be detected.

19. The defect classification system according to claim 15, wherein, The first transmission unit is used for: The CPU idle detection thread is determined based on the CPU's operating status. When the number of CPU idle detection threads is greater than the preset first thread number, the image to be detected and template information are transmitted to the detection thread. The template information is template feature information extracted from the defect-free image of the microdisplay device. The second transmission unit is used for: The idle detection thread of the GPU is determined based on the GPU's operating status; when the number of idle detection threads of the GPU is greater than the preset number of second threads, and the number of images to be re-inspected reaches the preset number of re-inspections, the images to be re-inspected are sent to the GPU's re-inspection module.

20. The defect classification system according to claim 15, wherein, The defect classification system also includes: The Gaussian noise reduction unit is used to perform Gaussian filtering noise reduction processing on the image to be detected. The Gaussian noise reduction unit is electrically connected to the acquisition unit and the monitoring unit respectively.

Images