Defect detection method and device for optical element, computer equipment and storage medium

Abstract

The invention relates to a defect detection method and device for an optical element, computer equipment and a storage medium. The method comprises the following steps: acquiring a target image of an optical element; based on the trained defect detection model, performing defect detection on the target image to obtain a defect identification frame corresponding to at least one detection target and a corresponding defect type label; the defect detection model is obtained based on deep learning model training; and carrying out fusion processing on the defect identification frames of the labels with the same defect type to obtain a defect area in the optical element. By adopting the method, adjacent or overlapped same-class defect areas can be effectively merged, so that a complete defect contour and a complete distribution range can be accurately restored.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of optical sheet defect detection, in particular to an optical sheet defect detection method and device, computer equipment and a storage medium. BACKGROUND

[0002] With the continuous development of semiconductor manufacturing and optical sheet detection technology, visual defect detection technology based on deep learning has emerged. This technology has the characteristics of high automation, strong adaptability and the ability to handle complex scenes, and has gradually become the mainstream direction in the field of industrial quality inspection.

[0003] In the traditional technology, defects on the surface of the optical sheet (such as scratches, edge collapse, ink contamination, etc.) are usually detected by manual visual inspection or traditional image processing algorithms. Manual detection relies on experience and needs to be checked one by one with the help of magnifying equipment, which is inefficient and easily affected by subjective factors. Traditional algorithms are usually based on edge detection, threshold segmentation, morphological operations and other methods, which are sensitive to defect morphology, illumination changes and other conditions, and have poor robustness.

[0004] However, the current target detection method based on deep learning has the following problems when processing images such as optical sheets with high resolution, large defect size difference, and irregular shape (such as long scratches that span the diagonal of the image): large-scale defects may cause the target box to contain too much background information, interfering with model feature learning; small-scale defects (such as fine scratches) have a very small pixel ratio in the entire image and are easily ignored or missed; the preset (Anchor) of the target box is difficult to consider both large and small defects, affecting the detection accuracy and convergence stability, so any of them will have the problem of inaccurate detection of optical sheet defects. SUMMARY

[0005] Therefore, an optical sheet defect detection method, device, computer equipment and storage medium are provided, which can improve the accuracy of defect detection and improve the detection accuracy of optical sheets.

[0006] In a first aspect, the present application provides an optical sheet defect detection method, comprising:

[0007] obtaining a target image of an optical sheet;

[0008] performing defect detection on the target image based on a trained defect detection model to obtain a defect recognition box corresponding to at least one detection target and a corresponding defect type label; the defect detection model is trained based on a deep learning model;

[0009] fusing the defect recognition boxes of the same defect type label to obtain a defect area in the optical sheet.

[0010] In one of the embodiments, the defect recognition boxes of the same defect type label are fused to obtain a defect region in the optical wafer, comprising:

[0011] determining a reference distance between different reference recognition boxes; the reference recognition boxes are defect recognition boxes of the same defect type label;

[0012] performing connected processing on the reference recognition boxes with a reference distance less than a preset distance threshold to obtain a defect region in the optical wafer.

[0013] In one of the embodiments, a target image of the optical wafer is obtained, comprising:

[0014] obtaining a grid image of the optical wafer;

[0015] setting pixel values of each pixel point in the grid image according to position distribution between each pixel point and different grid lines in the grid image to obtain the target image of the optical wafer.

[0016] In one of the embodiments, setting pixel values of each pixel point in the grid image according to position distribution between each pixel point and different grid lines in the grid image, comprising:

[0017] setting a first pixel point in the grid image as a first pixel value, and setting a second pixel point in the grid image as a second pixel value;

[0018] wherein the first pixel point is a pixel point within a preset distance range of each grid line; the second pixel point is a pixel point outside the preset distance range of each grid line; and the second pixel value is greater than the first pixel value.

[0019] In one of the embodiments, the defect detection model is obtained by training in the following manner:

[0020] obtaining a sample image and a standard defect recognition box of a sample wafer;

[0021] performing defect detection on the sample image based on the defect detection model to be trained to obtain at least one sample defect recognition box;

[0022] for each sample defect recognition box, determining a loss adjustment coefficient according to a distance between the sample defect recognition box and a sample grid line; and

[0023] determining an initial loss value according to a position difference between the sample defect recognition box and a corresponding standard defect recognition box;

[0024] determining a target loss value according to a product between the loss adjustment coefficient and the initial loss value;

[0025] adjusting model parameters of the defect detection model according to the target loss value to train the defect detection model.

[0026] In one embodiment, the loss adjustment coefficient is determined according to distances between the sample defect bounding box and the sample grid line, including:

[0027] For each vertex in the sample defect bounding box, a vertical distance from the vertex to the sample grid line is determined.

[0028] The loss adjustment coefficient is determined according to mean square errors of the vertical distances corresponding to different vertices.

[0029] In a second aspect, the present application further provides a defect detection device for an optical component, including:

[0030] An acquisition module is configured to acquire a target image of the optical component.

[0031] An identification module is configured to perform defect detection on the target image based on a trained defect detection model to obtain a defect bounding box corresponding to at least one detection target and a corresponding defect type label; the defect detection model is trained based on a deep learning model.

[0032] A fusion module is configured to perform fusion processing on the defect bounding boxes of the same defect type label to obtain a defect region in the optical component.

[0033] In a third aspect, the present application further provides a computer device including a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0034] A target image of the optical component is acquired.

[0035] Defect detection is performed on the target image based on a trained defect detection model to obtain a defect bounding box corresponding to at least one detection target and a corresponding defect type label; the defect detection model is trained based on a deep learning model.

[0036] Fusion processing is performed on the defect bounding boxes of the same defect type label to obtain a defect region in the optical component.

[0037] In a fourth aspect, the present application further provides a computer readable storage medium having a computer program stored thereon, and the computer program is executed by a processor to implement the following steps:

[0038] A target image of the optical component is acquired.

[0039] Defect detection is performed on the target image based on a trained defect detection model to obtain a defect bounding box corresponding to at least one detection target and a corresponding defect type label; the defect detection model is trained based on a deep learning model.

[0040] The defect recognition boxes with the same defect type label are fused to obtain a defect area in the optical component.

[0041] In a fifth aspect, the present application provides a computer program product comprising a computer program which, when executed by a processor, implements the following steps:

[0042] obtaining a target image of the optical component;

[0043] performing defect detection on the target image based on the trained defect detection model to obtain a defect recognition box corresponding to at least one detection target and a corresponding defect type label; the defect detection model is obtained based on a deep learning model;

[0044] fusing the defect recognition boxes with the same defect type label to obtain a defect area in the optical component.

[0045] The defect detection method, device, computer device and storage medium of the optical component can accurately locate the defect position and identify the defect type by obtaining a target image of the optical component and performing defect detection using a deep learning model. The same type of defect recognition box is introduced into the fusion process, which can effectively merge adjacent or overlapping defects of the same type, thereby accurately restoring the complete defect contour and distribution range, significantly improving the detection integrity and positioning accuracy of small, intermittent or narrow defects (such as scratches), enhancing the overall perception ability of multi-scale defects under complex background, and further improving the accuracy of optical component detection. BRIEF DESCRIPTION OF DRAWINGS

[0046] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related art, the drawings needed to be used in the embodiment or related art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without creative labor.

[0047] Figure 1 An application environment diagram of a defect detection method of an optical component provided by the present embodiment;

[0048] Figure 2 A flowchart of a first defect detection method of an optical component provided by the present embodiment;

[0049] Figure 3A A flowchart of a step of obtaining a defect area provided by the present embodiment;

[0050] Figure 3B A flowchart provided by the present embodiment;

[0051] Figure 4A A flowchart of a step of acquiring a target image is provided for the embodiment;

[0052] Figure 4B A schematic diagram of different detection algorithm results is provided for the embodiment;

[0053] Figure 4C A start and end point schematic diagram is provided for the embodiment;

[0054] Figure 4D A sampling result schematic diagram is provided for the embodiment;

[0055] Figure 4E A different gradient insertion effect diagram is provided for the embodiment;

[0056] Figure 5 A flowchart of a step of training a defect detection model is provided for the embodiment;

[0057] Figure 6A A flowchart of a second optical element defect detection method is provided for the embodiment;

[0058] Figure 6B A flowchart of a method of training a defect detection model is provided for the embodiment;

[0059] Figure 7 A structural block diagram of an optical element defect detection device is provided for the embodiment;

[0060] Figure 8 An internal structure diagram of a computer device is provided for the embodiment. DETAILED DESCRIPTION

[0061] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and not to limit the present application.

[0062] The optical element defect detection method provided by the embodiments of the present application can be applied to, for example, Figure 1In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104 or placed on a cloud or other network server. The computer device acquires the target image of the optical element; based on a trained defect detection model, it performs defect detection on the target image, obtaining at least one defect identification box and corresponding defect type label corresponding to the detected target; the defect detection model is trained based on a deep learning model; defect identification boxes with the same defect type label are fused to obtain the defect region in the optical element. The computer device can be a terminal or a server. Terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can be smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, etc. Portable wearable devices can be smartwatches, smart bracelets, head-mounted devices, etc. Server 104 can be implemented using a standalone server or a server cluster composed of multiple servers.

[0063] In one exemplary embodiment, such as Figure 2 As shown, a defect detection method for optical components is provided, which can be applied to... Figure 1 Taking a computer device as an example, the explanation includes the following steps S201 to S203. Wherein:

[0064] S201 acquires the target image of the optical element.

[0065] Optical components refer to precision elements used in optical systems. They typically have minute dimensions and complex surface structures, and are prone to surface defects such as scratches, chipping, and corrosion during manufacturing due to machining or chemical treatments, affecting their optical performance and quality. The target image refers to the surface image of the optical component acquired by an image acquisition device (such as a high-resolution camera). After preprocessing (such as adjusting gridded pixel values), this image is used as input to a deep learning model for defect detection, outputting defect bounding boxes and type labels.

[0066] Optionally, the method of obtaining the target image of the optical element can be: directly obtaining the target image of the optical element from a local database; or connecting the computer device to the camera device and responding to the target image of the optical element transmitted by the camera device; there are various specific methods for obtaining the target image of the optical element, which are not limited in this embodiment.

[0067] S202 performs defect detection on the target image based on the trained defect detection model, and obtains at least one defect recognition box and corresponding defect type label for the detected target.

[0068] The defect detection model is trained using a deep learning model. Through training, it learns the characteristic patterns of surface defects in optical components, enabling automated defect identification and classification of input target images. Defect detection refers to the process of identifying and locating abnormal features such as chipping, scratches, and corrosion on the surface of optical components using image processing techniques or deep learning algorithms. The detection target refers to the region or object in the target image that may contain defects; the model analyzes and determines whether it contains surface anomalies with abnormal features. The defect recognition box is the rectangular bounding box output by the defect detection model, used to accurately mark the location of detected defects in the target image and associate it with the corresponding defect type label. The defect type label is a text identifier generated by the model after classifying the detected target, used to distinguish different defect types, such as scratches, cracks, color ink stains, and inner ring ink fading.

[0069] In some embodiments, a target image is input to a defect detection model; the defect detection model performs defect detection on the target image and identifies at least one detection target from the target image; a defect recognition box is added to the detection target; and the defect type of the detection target is determined, and a defect type label matching the defect type is added to the detection target.

[0070] S203 performs fusion processing on the defect identification boxes of the same defect type labels to obtain the defect area in the optical element.

[0071] The fusion process refers to merging multiple adjacent or overlapping defect boxes with the same defect type label detected by the model into a unified defect region through distance threshold judgment and connectivity analysis, in order to eliminate redundant boxes and optimize the defect localization results. The defect region refers to the continuous defect range in the target image determined by connected component analysis after fusion processing, containing the actual defect coverage area with complete semantic information.

[0072] In some embodiments, different defect identification boxes are classified according to defect type labels to obtain at least one defect classification group. If there is only one defect identification box in a defect classification group, it indicates that there is only one defect identification box under that defect type label, with no other defect identification boxes, and no scattered detection results. Therefore, the defect area corresponding to the defect identification box is taken as the defect area of ​​the optical element under the corresponding defect type label. If there are at least two defect identification boxes in a defect classification group, it indicates that there are multiple defect identification boxes under that defect type label, and there may be a lack of continuity in the detection results. Therefore, the defect areas corresponding to the defect identification boxes are fused to obtain a complete target area. This target area is taken as the defect area of ​​the optical element under the corresponding defect type label.

[0073] The aforementioned defect detection method for optical components acquires target images of the optical components and uses a deep learning model for defect detection, which can accurately locate the defect location and identify the defect type. By introducing fusion processing of defect recognition boxes of the same type, adjacent or overlapping defect regions of the same type can be effectively merged, thereby accurately restoring the complete defect contour and distribution range. This significantly improves the detection completeness and positioning accuracy of small, discontinuous or narrow defects (such as scratches), enhances the overall perception capability of multi-scale defects in complex backgrounds, and further improves the accuracy of optical component detection.

[0074] Figure 3A This is a flowchart illustrating the steps for obtaining the defect region in one embodiment. Since there are various types of defects, and different defect types exhibit different manifestations, for continuous defects, the defect identification boxes may be relatively dispersed within a certain area. Therefore, to obtain the true defect region of continuous defects in the target image, connectivity analysis is performed on the defect identification boxes at a specific threshold distance. The merged defect identification boxes are then used as the defect region in the optical element. Based on this, this embodiment refines the steps in the above embodiment of fusing defect identification boxes with the same defect type label to obtain the defect region in the optical element, including the following steps:

[0075] S301 determines the reference distance between different reference recognition boxes.

[0076] Among them, the reference identification box is the defect identification box with the same defect type label; the reference distance refers to the Euclidean distance between the corresponding vertices (such as the center point or the nearest vertex) of two different defect identification boxes under the same defect type label, which is used to quantify the spatial proximity of the identification boxes.

[0077] In some embodiments, different defect recognition boxes in the target image are traversed, and defect recognition boxes with the same defect type label are used as reference recognition boxes; the reference distance between different reference recognition boxes is determined.

[0078] It should be noted that, when there are multiple reference bounding boxes, to further ensure that the defect type labels among the reference bounding boxes are the same or similar, this embodiment can also use a disjoint-set data structure to detect adjacent boxes. A disjoint-set data structure (Union-Find) is a tree-like data structure primarily used to handle the merging and querying of disjoint sets.

[0079] Optionally, the reference distance between different reference boxes can be determined in the following ways: either by determining the Euclidean distance between different reference boxes and using the Euclidean distance as the reference distance; or by determining the straight-line distance between different reference boxes and using the straight-line distance as the reference distance. There are various ways to determine the reference distance between different reference boxes, and this embodiment does not limit this one.

[0080] S302 performs connectivity processing on reference recognition boxes whose reference distance is less than a preset distance threshold to obtain the defect region in the optical element.

[0081] The preset distance threshold can be understood as a pre-set value in object detection or image processing, used to determine whether two adjacent bounding boxes (such as defect boxes or target boxes) meet the spatial proximity condition. When the Euclidean distance between the corresponding vertices (such as the center point or the nearest boundary point) of two boxes is less than this threshold, they are considered to belong to the same connected region and need to be merged. It should be noted that the larger the preset distance threshold, the more reference bounding boxes can be merged. Connectivity processing refers to the operation of merging adjacent bounding boxes that meet the preset distance threshold condition into a connected region. The connectivity of the boxes is dynamically maintained through data structures such as Union-Find, automatically aggregating all mutually adjacent boxes, and finally generating complete connected components (such as defect regions or target clusters) to eliminate redundant detection and improve region integrity.

[0082] In some embodiments, the relationship between the reference distance between different reference recognition boxes and a preset distance threshold is determined; for any reference recognition box whose reference distance is less than the preset distance threshold, each reference recognition box is connected to obtain the defect region in the optical element.

[0083] For example, the process of determining the relationship between the reference distance and the preset distance threshold between different reference recognition boxes can be implemented in the following way: taking reference recognition box 1 as... Reference box 2 is Taking the distance threshold as an example, when the reference identification box 1 and the reference identification box 2 satisfy the following formula (1), it is determined that the reference identification box 1 and the reference identification box 2 can be connected.

[0084] (1)

[0085] Among them, bool h Characterizes the intersection of the reference bounding boxes in the vertical direction, bool w This characterizes the intersection of the reference bounding boxes in the horizontal direction. It should be noted that if boolean... h and bool w If both are true, then reference frame 1 and reference frame 2 are determined to be adjacent. Therefore, reference frame 1 and reference frame 2 can be connected.

[0086] For example, such as Figure 3BThe diagram shows a connection process. In Figure (A), the defect detection result before connection processing is performed. Reference boxes labeled "Crack" and "Scratch" (represented by reference boxes 1, 2, 3, and 5) are present in (A). Reference boxes 1, 2, and 3 are adjacent to each other. Clearly, reference boxes 1, 2, and 3 are close enough to be connected. Therefore, connection processing is performed on reference boxes 1, 2, and 3 to obtain reference box 4 as shown in Figure (B). Furthermore, since the crack type reference box exists only once, connection processing is unnecessary for this type of reference box. Reference box 5 is far from reference boxes 1-3; therefore, it can be determined that reference box 5 and reference boxes 1-3 do not belong to the same defect region, and thus, connection processing is also unnecessary for this type of reference box.

[0087] In the above embodiments, by calculating the reference distance between reference recognition boxes of the same defect type label and performing connectivity processing on recognition boxes with a distance less than a preset threshold, adjacent or overlapping defect areas are effectively merged, avoiding interference from redundant detection boxes, thereby significantly improving the integrity and accuracy of defect area localization.

[0088] Figure 4A This is a flowchart illustrating the steps for acquiring a target image in one embodiment. This embodiment refines the steps for acquiring the target image of the optical element in the above embodiments, including the following steps:

[0089] S401 acquires a grid image of the optical element.

[0090] Among them, a grid image is an image representation that divides the original image into regular or irregular grid structures (such as rectangular, hexagonal, and other grid units). Each grid unit describes its local content through pixel values, labels, or feature vectors, and is used to quantify spatial distribution, simplify calculations, or assist in target localization (such as grid analysis in defect detection).

[0091] In some embodiments, the original image of the optical element is acquired; based on a preset sampling ratio, the original image is divided to obtain a grid image of the optical element.

[0092] It should be noted that defects in the original optical image may exist in the diagonal region. If a rectangular bounding box is used for annotation, the target will be the entire width and height of the image, including a significant amount of background, making it difficult to learn the true defect features. If the annotation is done using a rotated target detection box, the learned features will be target boxes with small widths and large lengths. When such defect samples are few, the model struggles to fit these anomalous samples, easily leading to missed detections. If segmentation is used for annotation, the model can only learn from this high-resolution image through image compression or cropping; otherwise, it risks a series of problems such as memory exhaustion. However, scratches are inherently small, and image compression can lead to the loss of scratch defect information, resulting in incomplete or unsegmented scratch regions. Image cropping and segmentation significantly increase the inference time per image. Therefore, the training method using guided annotation not only satisfies the need for fast inference on the entire image but also decouples the defect information from the original image. By using guided annotation, the problem of excessively large defects in anomalous samples is effectively solved, normalizing all targets to the maximum possible range and greatly improving the model's fit to defect sizes. Figure 4B The diagrams show the results of different detection algorithms, the results of surface defects on optical components after auxiliary guidance annotation, and the visualization results of defects using different annotation methods. It can be seen that there is a significant difference between rotating target detection and rectangular target detection for scratch targets; the maximum length of the scratch target exceeds 50% of the image's length and width, while the smallest target is only about 2%, making it difficult to simultaneously set prior boxes for both large and small targets. Instance segmentation not only faces the problem of difficult scratch annotation but also consumes more GPU memory and inference time compared to detection methods. The auxiliary guidance annotation box of this invention, through a preset normalized grid, limits the size of the annotated target to a certain range, effectively solving the problem of target variability by defining the boundary range of the auxiliary guidance annotation box.

[0093] Based on this, this embodiment divides the original image according to a preset sampling ratio to obtain a grid image of the optical element. That is, taking the original image with a width and height of (w, h) pixels as an example, the original image is divided into several layers, and sampling is performed in the horizontal and vertical directions with sampling points of m and n respectively. The sampling ratio of the horizontal and vertical segments is shown in the following formula (2):

[0094] (2)

[0095] Among them, t iw t represents the proportion of the horizontal sampling segment. ih is the proportion of the vertical sampling segment, m is the number of horizontal sampling points, and n is the number of vertical sampling points.

[0096] For example, such as Figure 4CThe starting and ending point diagram shown below uses the following formula (3) to limit all labeled coordinates to the corresponding coordinates for each sampling segment. Within the scope.

[0097] (3)

[0098] Where x1 is the starting point of the horizontal axis, x2 is the ending point of the horizontal axis, y1 is the starting point of the vertical axis, y2 is the ending point of the vertical axis, w is the width of the original image, h is the height of the original image, and t... iw t represents the proportion of the horizontal sampling segment. ih x represents the proportion of the vertical sampling segment. iw Let y be the coordinates of the horizontal sampling segment. ih The coordinates of the longitudinal sampling segment.

[0099] For example, such as Figure 4D The schematic diagram of the sampling results shown illustrates this example. In this embodiment, sampling points are set uniformly at equal intervals in both the horizontal and vertical directions, with 5 sampling points used as an example. Specifically, in the above calculation, m=5, n=5, the starting point is 0, and the ending points are w and h respectively, thus obtaining a grid image. Furthermore, this embodiment can guide manual data annotation based on the grid image to obtain corresponding supervised learning labels for target detection. The annotation range is limited to within the grid area, with the maximum annotation box being the sampling grid box. Defects exceeding the grid are annotated according to their actual defect locations, including but not limited to rectangular annotations, thus obtaining a manually annotated image.

[0100] S402 sets the pixel value of each pixel in the grid image according to the positional distribution of each pixel and different grid lines in the grid image, and obtains the target image of the optical element.

[0101] In a grid image, a pixel can be understood as the smallest unit that constitutes a digital image. Each pixel represents local information of the image at that point through its position coordinates (such as row and column numbers in a two-dimensional plane) and corresponding numerical values ​​(such as grayscale values ​​or RGB channel values). Grid lines can be understood as regular or irregular frames formed by virtual or actual drawn straight lines in a grid image, used to divide the image into several grid units (such as rectangular or hexagonal regions), each corresponding to a local region. Positional distribution describes the relative positional relationship of pixels or grid units in image space, including their distance from grid lines, the topological relationship of adjacent grids (such as whether they are adjacent), or the distribution pattern of pixels within a grid unit (such as center or edge). Pixel values ​​can be understood as the numerical information carried by a pixel, used to represent the brightness (grayscale image) or color (color image, such as RGB channel values) of that point, and are the basic data for calculation and analysis in image processing. The target image can be understood as the final image generated after specific processing (such as gridding analysis, pixel value adjustment, defect enhancement, etc.), whose pixel values ​​or structure are optimized to highlight target features (such as defects, edges) or meet specific task requirements (such as detection, classification).

[0102] It should be noted that, as Figure 4E The different gradient insertion effects shown illustrate that deep learning typically learns high-order semantic information in images by constructing convolutional neural network models. This information is mostly reflected in sets with the same color or texture features. Therefore, to guide the model to learn boundary-based defect texture features, a gradient insertion method is used to truncate the existing large-scale defect features. This results in the loss of some defect feature information within a small range, while maintaining the integrity of the large-scale defects over a larger range, thus decoupling the defect features. The decoupled features possess complete semantic information within the sampled cell, but are disconnected between adjacent cells, yet still exhibit clear feature representation across the entire image. Therefore, based on this decoupling method, the features learned by the model are the feature boundaries within a single sampled cell, rather than cross-cell bounding boxes. However, it is important to note that the inserted gradient cannot completely cover the gradient information corresponding to the original defect texture. Therefore, gradient insertion requires setting prior positions and pixels to avoid covering the original defects and to ensure the gradient information is easily learned by the model. The insertion position must coincide with the position of the auxiliary guidance annotation. The processing effects after different gradient insertions are shown in the figure below. Based on this, the following two optional implementation methods are given.

[0103] In one alternative embodiment, the vertical distance between each pixel and the grid line is determined; a pixel reference value matching the vertical distance is determined; and the pixel reference value is used as the pixel value of each pixel in the grid image to obtain the target image of the optical element.

[0104] In one optional embodiment, a first pixel in the grid image is set to a first pixel value (e.g., 0), and a second pixel in the grid image is set to a second pixel value (e.g., 225); wherein the first pixel is a pixel within a preset distance range of each grid line; the second pixel is a pixel outside the preset distance range of each grid line; and the second pixel value is greater than the first pixel value. The advantage of this setting is that by setting a lower pixel value for the first pixel closer to the grid line and a higher pixel value for the second pixel farther from the grid line, a gradual gradient effect from the defect edge to the background area is constructed, effectively enhancing the clarity and visual contrast of the defect boundary, while avoiding information loss caused by hard boundaries, thereby significantly improving the defect detection model's ability to identify small or blurry defects.

[0105] For example, as shown in the following formula (4), the coordinates from the start point to the end point of the horizontal axis are C xs C xe The coordinates from the starting point to the ending point of the vertical axis are C. ys C ye Taking C as an example, based on the following formulas (5) and (6), xs C ys Pixels within a neighborhood of n pixels are assigned a value of 0, and the first pixel outside this range is assigned a value of 255.

[0106] (4)

[0107] Among them, C xs Insert the starting coordinates for the horizontal axis, C xe To insert the endpoint coordinates on the horizontal axis, C ys Insert the starting coordinates for the vertical axis, C ye Insert the endpoint coordinates for the vertical axis, x iw Let y be the coordinates of the horizontal sampling segment. ih is the coordinate of the vertical sampling segment, w is the width of the original image, and h is the height of the original image.

[0108] (5)

[0109] (6)

[0110] Among them, C xs Insert the starting coordinates for the horizontal axis, C xe To insert the endpoint coordinates on the horizontal axis, C ys Insert the starting coordinates for the vertical axis, C ye Insert the endpoint coordinates for the vertical axis, x iw Let y be the coordinates of the horizontal sampling segment. ih is the coordinate of the vertical sampling segment, and n is a preset prior value representing the number of pixel translations.

[0111] In the above embodiments, by setting pixel values ​​based on the grid line position distribution, pixels closer to the grid line are assigned lower values ​​and pixels farther away from the grid line are assigned higher values, which effectively enhances the contrast between the defect area and the background, while preserving the complete semantic information of the defect, thereby significantly improving the accuracy of subsequent defect detection and the convergence efficiency of model training.

[0112] Figure 5 This is a flowchart illustrating the steps of training a defect detection model in one embodiment. This embodiment refines the above embodiment and includes the following steps:

[0113] S501 acquires sample images and standard defect recognition boxes for sample elements.

[0114] In machine learning or image processing, a sample patch can be understood as a small, representative image region extracted from the original image, typically used to provide local detail information (such as texture, edges, or defect fragments) during model training. A sample image can be understood as complete image data used for model training or testing, containing the target object (such as a product surface) and its potential defects or other features, serving as input data for the algorithm to analyze and extract features. A standard defect detection box can be understood as a rectangular or polygonal bounding box, manually annotated or generated by a high-precision algorithm, that precisely defines the location of defects in the sample image. It serves as ground truth for model training, measuring the deviation between the model's detection results and the actual defect locations.

[0115] It should be noted that the implementation method of obtaining the sample image of the sample element in this embodiment is the same as or similar to the implementation method of obtaining the target image of the sample element in step S201 of the above embodiment, and will not be described in detail in this embodiment.

[0116] S502 performs defect detection on sample images based on the defect detection model to be trained, and obtains at least one sample defect recognition box.

[0117] It should be noted that the implementation method of performing defect detection on sample images based on the defect detection model to be trained in this embodiment to obtain at least one sample defect recognition box is the same as or similar to the implementation method of step S202 in the above embodiment, which performs defect detection on target images based on the trained defect detection model to obtain at least one defect recognition box and corresponding defect type label corresponding to the detected target. This embodiment will not elaborate on this.

[0118] For each sample defect identification box, S503 determines the loss adjustment coefficient based on the distance between the sample defect identification box and the sample grid line.

[0119] In this model, the sample defect bounding box can be understood as a rectangular or polygonal bounding box generated by the defect detection model in the sample image, used to mark the location of detected defects. Its coordinates and size reflect the model's prediction of the defect region. Sample grid lines can be understood as predefined regular or irregular lines in the sample image (such as the boundary lines of rectangular, hexagonal, or other grid cells), used to divide the image into multiple local regions, aiding in the analysis of the spatial relationship between defects and the grid structure (such as their distance from the grid lines). The loss adjustment coefficient can be understood as a weighting factor dynamically calculated during model training based on the spatial relationship between the sample defect bounding box and the sample grid lines (such as the root mean square error of the vertical distance from vertices to grid lines). It is used to adjust the loss function for the defect detection task, making the model focus more on the detection accuracy of specific regions (such as easily missed areas near grid lines).

[0120] In one alternative embodiment, for each sample defect identification box, the distance between the sample defect identification box and the sample grid line is determined; the mean of all distances is used as the loss adjustment coefficient.

[0121] In one optional embodiment, for each vertex in the sample defect identification box, the vertical distance from the vertex to the sample grid line is determined; the loss adjustment coefficient is determined based on the mean squared error of the vertical distances corresponding to different vertices. The advantage of this approach is that by calculating the mean squared error of the vertical distances from each vertex of the sample defect identification box to the sample grid line to determine the loss adjustment coefficient, the degree of deviation between the defect box and the grid structure can be dynamically quantified. This allows the model to assign higher weights to defects closer to the grid lines (more susceptible to interference) during training, thereby effectively enhancing the model's ability to locate small defects in complex layouts and improving detection accuracy and anti-interference capabilities.

[0122] For example, for each sample defect identification box, the distance between the sample defect identification box and the sample grid line can be determined according to the following formula (7), with the target image bounding box as... and auxiliary guide bounding box as The sample defect identification box is Let's take an example to illustrate:

[0123] (7)

[0124] Where, d left Let d be the distance in the direction closest to the starting point of the horizontal axis. right Let d be the distance in the direction closest to the endpoint of the horizontal axis. top Let d be the distance in the direction closest to the endpoint of the vertical axis. bottom x is the distance in the direction closest to the starting point of the vertical axis. iw Let y be the coordinates of the horizontal sampling segment. ih The coordinates of the longitudinal sampling segment.

[0125] In some embodiments, the loss adjustment coefficient is determined based on the mean squared error of the vertical distances corresponding to different vertices by: determining the mean squared error of the vertical distances corresponding to different vertices; and determining the loss adjustment coefficient with the natural constant as the base and the mean squared error and a preset intensity factor as the exponent.

[0126] For example, the mean square error of the vertical distances corresponding to different vertices can be determined according to the following formula (8):

[0127] (8)

[0128] in, For the mean squared error, d left Let d be the distance in the direction closest to the starting point of the horizontal axis. right Let d be the distance in the direction closest to the endpoint of the horizontal axis. top Let d be the distance in the direction closest to the endpoint of the vertical axis. bottom This represents the distance in the direction closest to the starting point of the vertical axis.

[0129] For example, the loss adjustment factor can be determined according to the following formula (9):

[0130] (9)

[0131] in, This is the loss adjustment factor. Intensity factor The mean squared error is used. It should be noted that by introducing a focus intensity factor to control the sensitivity of boundary errors, the initial training steps are set to 0 to allow the model to converge quickly. After a certain number of steps, the intensity factor is gradually increased to expand the learning degree of the four edges in the model training.

[0132] S504 determines the initial loss value based on the positional difference between the sample defect identification box and the corresponding standard defect identification box.

[0133] The initial loss value can be understood as the loss function value calculated on sample data (such as sample images and their labeled defect recognition boxes) based on the initial model parameters at the beginning of model training (i.e., the first iteration or before any parameter updates). It is used to measure the degree of deviation between the model's initial prediction results and the true labels.

[0134] In some embodiments, the positional difference between the sample defect identification box and the corresponding standard defect identification box is determined; based on a preset loss function, an initial loss value is determined according to the positional difference. Optionally, the preset loss function may be the Complete Intersection over Union (CIoU) loss function, or it may be the regression loss function in the You Only Look Once (YOLO) detector. The specific loss function is not limited in this embodiment.

[0135] It should be noted that in this embodiment, the process of determining the loss adjustment coefficient based on the distance between the sample defect identification box and the sample grid line in S503 for each sample defect identification box, and the process of determining the initial loss value based on the positional difference between the sample defect identification box and the corresponding standard defect identification box in S504, can be executed synchronously or sequentially. This embodiment does not limit this.

[0136] S505 determines the target loss value based on the product of the loss adjustment factor and the initial loss value.

[0137] In some embodiments, the product of the loss adjustment factor and the initial loss value is used as the target loss value. For example, as shown in formula (10) below:

[0138] (10)

[0139] Where Loss is the target loss value, Loss yolo The initial loss value, This is the loss adjustment factor.

[0140] S506 adjusts the model parameters of the defect detection model based on the target loss value to train the defect detection model.

[0141] In some embodiments, based on a preset adjustment function, the target parameters of the defect detection model are determined according to the target loss value; the target parameters are used as the model parameters of the defect detection model in the current training phase to train the defect detection model.

[0142] In the above embodiments, a loss adjustment coefficient based on the distance between the sample defect identification box and the sample grid line is introduced to dynamically adjust the weight of defects at different locations, so that the model pays more attention to defects that are easy to miss, such as those near the grid line. At the same time, the initial loss value is calculated by combining the positional difference of the standard identification box, and finally the target loss value is determined by the product of the two to optimize the model parameters. This effectively improves the identification accuracy and robustness of the defect detection model for defects in complex backgrounds and edge areas.

[0143] In one embodiment, this embodiment provides an optional method for defect detection of optical components, using the application of this method to a server as an example for illustration. Figure 6A As shown, the method includes the following steps:

[0144] The S601A acquires a grid image of the optical element.

[0145] S602A sets the first pixel in the grid image to the first pixel value and sets the second pixel in the grid image to the second pixel value to obtain the target image of the optical element.

[0146] The first pixel is a pixel within a preset distance range of each grid line; the second pixel is a pixel outside the preset distance range of each grid line; and the value of the second pixel is greater than the value of the first pixel.

[0147] Based on the trained defect detection model, the S603A performs defect detection on the target image and obtains at least one defect recognition box and corresponding defect type label for the detected target.

[0148] The defect detection model is trained based on a deep learning model.

[0149] S604A determines the reference distance between different reference recognition boxes.

[0150] Among them, the reference identification box is the defect identification box with the same defect type label.

[0151] The S605A performs connectivity processing on reference recognition boxes whose reference distance is less than a preset distance threshold to obtain the defect area in the optical element.

[0152] In one embodiment, this embodiment provides an optional method for training a defect detection model, using the application of this method to a server as an example for illustration. For example... Figure 6B As shown, the method includes the following steps:

[0153] The S601B acquires sample images and standard defect recognition boxes for sample elements.

[0154] The S602B uses a defect detection model to be trained to detect defects in sample images and obtain at least one sample defect recognition box.

[0155] For each sample defect identification box, S603B determines a loss adjustment coefficient based on the distance between the sample defect identification box and the sample grid line; and determines an initial loss value based on the positional difference between the sample defect identification box and the corresponding standard defect identification box.

[0156] For each vertex in the sample defect identification box, S604B determines the vertical distance from the vertex to the sample grid line.

[0157] S605B determines the loss adjustment coefficient based on the mean square error of the vertical distances corresponding to different vertices.

[0158] The S606B adjusts the model parameters of the defect detection model based on the target loss value to train the defect detection model.

[0159] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0160] Based on the same inventive concept, this application also provides an optical component defect detection device for implementing the aforementioned optical component defect detection method. The solution provided by this device is similar to the implementation described in the above method; therefore, the specific limitations in one or more optical component defect detection device embodiments provided below can be found in the limitations of the optical component defect detection method described above, and will not be repeated here.

[0161] In one exemplary embodiment, such as Figure 7 As shown, a defect detection device for optical components is provided, comprising: an acquisition module 10, an identification module 11, and a fusion module 12, wherein:

[0162] Acquisition module 10 is used to acquire the target image of the optical element;

[0163] The recognition module 11 is used to perform defect detection on the target image based on the trained defect detection model, and obtain at least one defect recognition box and corresponding defect type label corresponding to the detected target; the defect detection model is trained based on a deep learning model.

[0164] The fusion module 12 is used to fuse defect identification boxes with the same defect type label to obtain the defect area in the optical element.

[0165] In some embodiments, the fusion module 12 is further configured to determine the reference distance between different reference identification boxes; the reference identification boxes are defect identification boxes with the same defect type label; and the reference identification boxes with reference distances less than a preset distance threshold are connected to obtain the defect region in the optical element.

[0166] In some embodiments, the acquisition module 10 is further configured to acquire a grid image of the optical element; and set the pixel value of each pixel in the grid image according to the positional distribution between each pixel and different grid lines in the grid image to obtain the target image of the optical element.

[0167] In some embodiments, the acquisition module 10 is further configured to set a first pixel in the grid image to a first pixel value, and to set a second pixel in the grid image to a second pixel value; wherein the first pixel is a pixel within a preset distance range of each grid line; the second pixel is a pixel outside the preset distance range of each grid line; and the second pixel value is greater than the first pixel value.

[0168] In some embodiments, the defect detection device for optical components further includes: a training module for acquiring sample images and standard defect recognition boxes of sample components; performing defect detection on the sample images based on a defect detection model to be trained to obtain at least one sample defect recognition box; determining a loss adjustment coefficient for each sample defect recognition box based on the distance between the sample defect recognition box and the sample grid line; determining an initial loss value based on the positional difference between the sample defect recognition box and the corresponding standard defect recognition box; determining a target loss value based on the product of the loss adjustment coefficient and the initial loss value; and adjusting the model parameters of the defect detection model based on the target loss value to train the defect detection model.

[0169] In some embodiments, the training module is further configured to determine the vertical distance from each vertex in the sample defect identification box to the sample grid line; and to determine the loss adjustment coefficient based on the mean square error of the vertical distances corresponding to different vertices.

[0170] Each module in the aforementioned defect detection device for optical components can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0171] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 8As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network. When the computer program is executed by the processor, it implements a defect detection method for optical components.

[0172] Those skilled in the art will understand that Figure 8 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0173] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0174] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0175] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0176] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0177] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0178] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0179] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for detecting defects in optical components, characterized in that, The method includes: Acquire the target image of the optical element; Based on the trained defect detection model, defects are detected in the target image to obtain at least one defect recognition box and corresponding defect type label for the detected target; the defect detection model is trained based on a deep learning model. Defect identification boxes with the same defect type label are fused to obtain the defect region in the optical element.

2. The method according to claim 1, characterized in that, The process of fusing defect identification boxes labeled with the same defect type to obtain the defect region in the optical element includes: Determine the reference distance between different reference identification boxes; the reference identification boxes are defect identification boxes with the same defect type label. Connectivity processing is performed on reference identification boxes whose reference distance is less than a preset distance threshold to obtain the defect region in the optical element.

3. The method according to claim 1, characterized in that, The acquisition of the target image of the optical element includes: Obtain the grid image of the optical element; Based on the positional distribution of each pixel in the grid image with respect to different grid lines, the pixel value of each pixel in the grid image is set to obtain the target image of the optical element.

4. The method according to claim 3, characterized in that, The step of setting the pixel value of each pixel in the grid image based on the positional distribution of each pixel in the grid image with different grid lines includes: Set the first pixel in the grid image to a first pixel value, and set the second pixel in the grid image to a second pixel value; Wherein, the first pixel is a pixel within a preset distance range of each grid line; the second pixel is a pixel outside the preset distance range of each grid line; and the value of the second pixel is greater than the value of the first pixel.

5. The method according to any one of claims 1-4, characterized in that, The defect detection model was trained using the following method: Obtain sample images and standard defect recognition boxes for sample elements; Based on the defect detection model to be trained, defect detection is performed on the sample image to obtain at least one sample defect recognition box; For each sample defect identification box, a loss adjustment coefficient is determined based on the distance between the sample defect identification box and the sample grid lines; and, The initial loss value is determined based on the positional difference between the sample defect identification box and the corresponding standard defect identification box; The target loss value is determined based on the product of the loss adjustment coefficient and the initial loss value; The model parameters of the defect detection model are adjusted based on the target loss value to train the defect detection model.

6. The method according to claim 5, characterized in that, The step of determining the loss adjustment coefficient based on the distance between the sample defect identification box and the sample grid line includes: For each vertex in the sample defect identification box, determine the vertical distance from the vertex to the sample grid line; The loss adjustment coefficient is determined based on the mean squared error of the vertical distances corresponding to different vertices.

7. A defect detection device for optical components, characterized in that, The device includes: The acquisition module is used to acquire the target image of the optical element. The recognition module is used to perform defect detection on the target image based on the trained defect detection model, and obtain at least one defect recognition box and corresponding defect type label corresponding to the detected target; the defect detection model is trained based on a deep learning model. The fusion module is used to fuse defect identification boxes with the same defect type labels to obtain the defect area in the optical element.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images