Defect detection method, electronic device, storage medium, and program product

Abstract

The present disclosure relates to the technical field of detection, and in particular to the technical fields of computer vision and image processing, and provides a defect detection method, an electronic device, a storage medium, and a program product. The defect detection method comprises: acquiring an initial image of an object to be detected in a target process stage, the target process stage being associated with a defect type to be detected and a region type to be detected; performing defect detection on the initial image to obtain a plurality of pieces of defect information for said defect type; performing region detection on the initial image to obtain at least one piece of region-of-interest information for said region type; on the basis of the plurality of pieces of defect information and the at least one piece of region-of-interest information, determining target defect information from among the plurality of pieces of defect information; and generating a defect annotation image on the basis of the position of a target defect corresponding to the target defect information in the initial image, wherein the position of the target defect overlaps at least one region-of-interest.

Description

Defect detection methods, electronic devices, storage media and software products Technical Field

[0001] This disclosure relates to the field of detection technology, and in particular to the fields of computer vision and image processing technology, specifically to defect detection methods, electronic devices, storage media and program products. Background Technology

[0002] Liquid crystal display (LCD) panels are widely used in electronic devices such as mobile phones and computers due to their low power consumption, low radiation, and low heat generation. However, the manufacturing process for LCD panels is complex. When hidden defects that are not easily observed, such as scratches, burrs, chipping, and bubbles, occur, it becomes difficult to meet the quality requirements of LCD panels.

[0003] Summary of the Invention

[0004] This disclosure provides a defect detection method, electronic device, storage medium, and program product.

[0005] According to a first aspect, this disclosure provides a defect detection method, comprising: acquiring an initial image of an object to be detected in a target process segment; associating the target process segment with a defect type to be detected and a region type to be detected; performing defect detection on the initial image to obtain multiple defect information for the defect type to be detected; performing region detection on the initial image to obtain at least one region of interest information for the region type to be detected; determining target defect information from the multiple defect information based on the multiple defect information and the at least one region of interest information; and generating a defect marker image based on the position of the target defect corresponding to the target defect information in the initial image; wherein the position of the target defect overlaps with at least one region of interest.

[0006] According to a second aspect, this disclosure provides an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the aforementioned defect detection method.

[0007] According to a third aspect, this disclosure provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause a computer to perform the above-described defect detection method.

[0008] According to the fourth aspect, this disclosure provides a computer program product, including a computer program that, when executed by a processor, implements the above-described defect detection method. Attached Figure Description

[0009] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0010] Figure 1 is an application scenario diagram of the defect detection method according to an embodiment of this disclosure;

[0011] Figure 2 is a flowchart of a defect detection method according to an embodiment of this disclosure;

[0012] Figure 3 is a schematic diagram of overlapping regions according to an embodiment of this disclosure;

[0013] Figure 4A is a schematic diagram of a defect marking image according to an embodiment of this disclosure;

[0014] Figure 4B is a schematic diagram of an initial defect marking image according to another embodiment of this disclosure;

[0015] Figure 4C is a schematic diagram showing the defect marking image and the initial defect marking image according to an embodiment of this disclosure;

[0016] Figure 4D is a schematic diagram showing the defect marking image and the initial defect marking image according to another embodiment of this disclosure.

[0017] Figure 5 is a schematic diagram summarizing the detection results of the defect detection method of this disclosure on multiple detection objects of the same raw material;

[0018] Figure 6 is a schematic diagram summarizing the detection results of multiple test objects in the same batch of raw materials using the defect detection method of this embodiment of the present disclosure;

[0019] Figure 7 is a schematic diagram of system interaction for performing the defect detection method according to the embodiments of this disclosure;

[0020] Figure 8 is a schematic diagram illustrating the interaction between a defect detection platform and a client cloud page for executing the defect detection method of this disclosure embodiment; and

[0021] Figure 9 is a block diagram of an electronic device used to perform the defect detection method according to the embodiments of this disclosure. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this disclosure. Based on the described embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure. It should be noted that throughout the accompanying drawings, the same elements are represented by the same or similar reference numerals. In the following description, some specific embodiments are used for descriptive purposes only and should not be construed as limiting this disclosure in any way, but are merely examples of embodiments of this disclosure. Conventional structures or configurations will be omitted where they may cause confusion in understanding this disclosure. It should be noted that the shapes and dimensions of the components in the figures do not reflect actual size and proportion, but are only schematic representations of the embodiments of this disclosure.

[0023] Unless otherwise defined, the technical or scientific terms used in the embodiments of this disclosure shall have the ordinary meaning as understood by those skilled in the art. The terms "first," "second," and similar words used in the embodiments of this disclosure do not indicate any order, quantity, or importance, but are merely used to distinguish different components.

[0024] The manufacturing process of LCD panels is complex, including processes such as coating, etching, development, panel assembly, die filling and sealing, and driver chip installation. Each process step can potentially produce defects that affect quality, and each step can generate a variety of defect types, such as photoresist residue, edge chipping, scratches, burrs, and bubbles. Compared to defects in other components, such as assembly misalignment or dimensional (shape, color) deviations, defects in LCD panels are more subtle and difficult to detect.

[0025] With the development of artificial intelligence technology and the increasing maturity of computer vision technology, using object detection models to detect images of liquid crystal display panels to identify the aforementioned defects can reduce the difficulty of detecting these latent defects. The recognition capability of object detection models is based on deep learning of the image features (e.g., shape, color, texture, etc.) of various defects in sample images. When applying object detection models to identify images of liquid crystal display panels, any area that matches the image features of a defect will be identified by the model as a defect.

[0026] However, this is not the case in actual production lines. For example, black spots in the image of an LCD panel are only considered quality defects if they appear in the driving circuit area of ​​the LCD panel.

[0027] Therefore, this disclosure provides a defect detection method that marks defects corresponding to defect regions overlapping with the region of interest as target defects in the initial image of the object to be detected, further improving the accuracy of defect detection in specific areas of the object. By associating process segments with the types of defects and regions to be detected, targeted defect detection for different process segments is achieved. This allows users to flexibly configure the types of defects and regions to be detected according to the quality requirements of different process segments on the production line, improving the adaptability of the defect detection equipment to application scenarios.

[0028] Figure 1 is an application scenario diagram of the defect detection equipment and method according to an embodiment of this disclosure.

[0029] As shown in Figure 1, the exemplary framework 100 may include: a defect detection device 101, a production line 102, and an object to be detected 103.

[0030] According to embodiments of this disclosure, the defect detection device 101 may include an image acquisition device 101_1 and a processor 101_2. The image acquisition device 101_1 is configured to acquire an initial image of an object to be inspected located in a target process segment on a production line. The processor 101_2 is configured to, in response to detecting multiple defect information corresponding to a type of defect to be detected and at least one region of interest information corresponding to a type of region to be detected, process the multiple defect information and the at least one region of interest information, determine a target defect from the multiple defects, mark the position of the target defect in the initial image, and generate a defect-marked image for display.

[0031] According to embodiments of this disclosure, the object to be tested 103 can be any semi-finished liquid crystal display panel placed on production line 102. The target process segment can be any process segment in the liquid crystal display panel production process, such as: coating process segment, etching process segment, developing process segment, panel assembly process segment, crystal filling and sealing process segment, driver chip mounting process segment, etc. Each process segment can correspond to one processing equipment or multiple processing equipment. When a certain process segment corresponds to multiple processing equipment, due to differences in the processing equipment itself, such as equipment model, service life, etc., the finished product may also have some defects caused by the equipment itself when performing the same processing operation. Therefore, the target process segment can also correspond to any processing equipment in any process segment of the liquid crystal display panel production process.

[0032] For example, the defect detection device 101 can be installed on one side of the production line 102 so that the image acquisition device 101_1 in the defect detection device 101 can acquire an initial image of the object 103 to be inspected in the vertical direction. Then, the processor 101_2 of the defect detection device 101 performs defect detection and region detection on the initial image. Upon detecting multiple defect information corresponding to the type of defect to be detected and at least one region of interest information corresponding to the type of region to be detected, the processor processes the multiple defect information and the at least one region of interest information to determine the target defect from the multiple defects; it marks the position of the target defect in the initial image and generates a defect-marked image. The defect-marked image is then displayed on the display panel 101_3 of the defect detection device.

[0033] Figure 2 schematically illustrates a flowchart of a defect detection method according to an embodiment of the present disclosure.

[0034] As shown in Figure 2, the method 200 may include operations S210 to S250.

[0035] In operation S210, an initial image of the object to be inspected in the target process section is acquired.

[0036] In operation S220, defect detection is performed on the initial image to obtain multiple defect information for the type of defect to be detected.

[0037] In operation S230, region detection is performed on the initial image to obtain at least one region of interest information for the type of region to be detected.

[0038] In operation S240, the target defect information is determined from the multiple defect information based on multiple defect information and at least one region of interest information.

[0039] In operation S250, a defect marker image is generated based on the position of the target defect corresponding to the target defect information in the initial image. According to embodiments of this disclosure, the target process segment is associated with a defect type to be detected and a region type to be detected. Both the defect type to be detected and the region type to be detected associated with the target process segment can be configured according to the actual quality requirements of the production line.

[0040] For example, the types of defects to be detected associated with the coating process section can include edge chipping, burrs, and bubbles. The types of areas to be detected associated with the coating process section can include driving circuit areas, light-emitting pixel areas, and solder joint areas.

[0041] For example, in a panel assembly process segment with multiple parallel processing machines, the LCD panels produced by a certain processing machine often show photoresist residue in the light-emitting pixel area during manual re-inspection. Therefore, photoresist residue can be added when configuring the defect type to be detected associated with the panel assembly process segment, and light-emitting pixel area can be added when configuring the area type to be detected associated with the panel assembly process segment.

[0042] According to embodiments of this disclosure, a pre-trained target detection model can be used to perform defect detection on an initial image to obtain defect detection results. The defect detection results may include at least the following information: whether a defect corresponding to the type of defect to be detected exists in the initial image, the defect name, the defect type, and the defect location.

[0043] For example, the initial image can be an image of the object to be inspected in the developing section of the production line. Using a target detection model to inspect the initial image, the resulting defect detection result can include: the presence of burrs, with the defect region of the burr defined as (A, B). The defect region of the burr can be of any geometric shape, such as a rectangle, where A represents the coordinates of the lower left vertex of the rectangle, and B represents the coordinates of the upper right vertex.

[0044] For example, object detection models can be R-CNN (Region-Convolutional Neural Network) or Fast R-CNN models trained using a two-stage algorithm. Object detection models can also be YOLO (You Only Look Once) or SSD (Single Shot MultiBox Detector) models trained using a one-stage algorithm.

[0045] According to embodiments of this disclosure, a pre-trained image segmentation model can be used to perform region detection on an initial image to obtain region detection results. The region detection results may include at least the following information: the presence of a region type corresponding to the type of region to be detected in the initial image, the region name, and the region location.

[0046] For example, image segmentation models can be semantic segmentation models, such as FCN (Fully Convolutional Networks), Unet (pixel-level image segmentation network), SegNet (deep fully convolutional image segmentation network), and ResNet (Residual Network).

[0047] For example, the region of interest can be any geometric shape, such as a regular or irregular polygon. The location of the region of interest can be represented as a combination of the coordinates of multiple vertices of the irregular polygon.

[0048] According to embodiments of this disclosure, the target detection model and image segmentation model described above can both be configured in a model library, so that users can configure different models for different process sections according to actual needs, and perform corresponding defect detection tasks and region detection tasks by loading the corresponding models.

[0049] Since in practical applications, only defects existing within the region of interest are identified as defects that need attention, it is possible to determine whether there are defects that need attention in the initial image by judging whether there is overlap between the geometric shapes corresponding to the defect region and the region of interest.

[0050] For example, when both the defect region and the region of interest are rectangular in shape, the corresponding horizontal and vertical projection lines can be determined based on the coordinates of the four vertices of each region. When the corresponding horizontal projection lines intersect and / or the corresponding vertical projection lines intersect, it can be determined that the defect region and the region of interest overlap.

[0051] According to embodiments of this disclosure, the defect region can be a rectangular frame with a minimum area surrounding the defect, and the defect location can be the coordinates of the defect's position within that rectangular frame. Since the shapes of the regions within the object to be inspected vary, the region of interest can be an irregular polygon. The overlap between the defect region and the region of interest can occur when there is at least one intersection point between the rectangular frame region with the minimum area surrounding the defect and the irregular polygon region used to characterize the region of interest.

[0052] According to embodiments of this disclosure, the target defect can be directly marked in the initial image based on its position within the initial image. Alternatively, the initial image can be rotated or cropped according to display requirements before the target defect marking operation is performed. For example, display requirements can be determined based on the screen size of the display device.

[0053] According to embodiments of this disclosure, since the defect region of the target defect overlaps with at least one region of interest, the target defect marked in the defect marking image is a defect within the user's area of ​​interest for that target process segment, further improving the accuracy of defect detection in specific areas of the object to be inspected. By associating process segments with the type of defect to be detected and the type of area to be detected, targeted defect detection for different process segments is achieved. This allows users to flexibly configure the type of defect to be detected and the type of area to be detected according to the quality requirements of different process segments on the production line, improving the adaptability of the defect detection equipment to application scenarios.

[0054] According to embodiments of this disclosure, multiple defect information includes defect regions of multiple defects; a target defect information is determined from the multiple defect information based on the multiple defect information and at least one region of interest information.

[0055] Since the vertices of the overlapping region are either the intersection points of the defect region and the region of interest, or are located within the defect region or the region of interest, after the defect region intersects with the region of interest.

[0056] Therefore, we can first call the `GetCrossPoint()` function to calculate the intersection points between each edge of the defect region and the region of interest. Then, we call the `IsPointInpolygon()` function to determine the points within each of the defect region and the region of interest. Next, we call the `ClockwiseSortPoints()` function to sort the intersection points between each edge and the points within each of the defect region and the region of interest in a counter-clockwise (or clockwise) polar angle order to obtain the final point set. We then calculate the area of ​​the final point set. If the area is greater than 0, it indicates that the defect region and the region of interest overlap. Finally, we determine the defect regions that overlap with the region of interest as the defect regions of the target defect.

[0057] According to the embodiments of this disclosure, the process of performing region overlap operation on the region of interest and the defect region has simple operation logic, does not require excessive hardware resources, and reduces the impact of region overlap operation on detection efficiency.

[0058] Figure 3 is a schematic diagram of overlapping regions according to an embodiment of this disclosure.

[0059] As shown in Figure 3, in embodiment 300, region detection is performed on the initial image 301 of the object to be detected to obtain the type and location of each region of interest. Defect detection is performed on the initial image 302 of the object to be detected to obtain the defect type and defect location.

[0060] According to embodiments of this disclosure, a region overlap operation is performed on the defect regions of multiple defects and at least one region of interest to obtain multiple overlap results, including: binarizing the defect regions of multiple defects and at least one region of interest respectively to generate multiple defect region images and at least one region of interest image; and performing a pixel dot product operation on the multiple defect region images and at least one region of interest image to obtain multiple overlap results.

[0061] For example: First, based on the region detection results, a region of interest image 302 can be generated. The region of interest image 302 may include six luminescent pixel regions 310a to 310f and one driving circuit region 320.

[0062] Then, based on the defect detection results, a defect detection image 303 can be generated. The defect detection image 303 may include a first defect region 331 and a second defect region 332. The types of defects corresponding to the first defect region and the second defect region can be the same or different.

[0063] Next, a pixel-by-pixel product operation is performed on the region of interest image 302 and the defect detection image 303 to obtain an overlap result 304. In the overlap result 304, only the second defect region 332, which overlaps with the driving circuit region 320, is marked, while the first defect region 331, which does not overlap with any of the above seven regions of interest, is not marked in the overlap result 304.

[0064] According to embodiments of this disclosure, the information of the defect region and the region of interest can also be converted into JSON data first. The JSON data may include: the name of the defect, the vertex coordinates of the defect region bounding box, the region name, and all pixel coordinates of the polygonal border enclosing the region.

[0065] Then, you can initialize the map structure arrays that store defect information and the map structure arrays that store area information. For example, the map structure array for defect information can be named `defect_points_map`, and the map structure array for area information can be named `area_points_map`.

[0066] Next, iterate through the JSON data corresponding to the defect information, using the defect name as the key and the coordinates of the defect area as the value, and insert them into the `defect_points_map` as key-value pairs. Similarly, iterate through the JSON data corresponding to the area information, using the area name as the key and the coordinates of the area of ​​interest as the value, and insert them into the `area_points_map` as key-value pairs.

[0067] Next, initialize a map structure array for defect overlap statistics, which can be named: intersect_region. In this map structure array, the key can be a <string, string> pair structure, and the value can be an integer structure representing the number of overlapping pixels.

[0068] Next, we can iterate through each defect name in the `defect_points_map` and fill each defect region (e.g., `defect1`) with a rectangle by calling OpenCV's shape fill function. Then, based on the filled rectangular regions, we iterate through all regions of interest (ROIs) in the `area_points_map` and fill each ROI (e.g., `area1`) with a polygon. Finally, we calculate the number of pixels overlapping between the rectangular and polygonal regions using pixel-dot product. When the number of overlapping pixels is greater than 0, we can insert ("defect1", "area1") as the key and the number of overlapping pixels as the value into `intersect_region`.

[0069] Finally, the defect name in the intersect_region can be used as the target defect.

[0070] According to embodiments of this disclosure, by calling a shape fill function, the defect region and the region of interest are filled respectively to perform pixel dot product operations. This method has lower requirements for hardware resource processing capabilities and is more suitable for defect detection scenarios in production environments.

[0071] According to embodiments of this disclosure, a pixel-by-pixel product operation is performed on multiple defect region images and at least one region of interest image to obtain multiple overlapping results, including: for each defect region image in the multiple defect region images, reading the first gray value of each pixel in each defect region image and the second gray value of each pixel in at least one region of interest image; multiplying each second gray value with each first gray value to obtain multiple overlapping results.

[0072] For example, the region of interest image 302 and the defect detection image 303 can be binary images located in the same background layer. The gray values ​​in both the region of interest and the defect region can be set to 1, while the gray values ​​in the background region can be set to 0. A row-wise AND operation (multiplication of gray values) can be performed on each pixel in the region of interest image 302 and the defect detection image 303. When a non-zero pixel appears in the result, it indicates that the region of interest and the defect region overlap.

[0073] According to embodiments of this disclosure, whether the region of interest and the defect region overlap is determined by pixel dot product operation, which further improves the data processing efficiency of region overlap operation, thereby further improving the detection efficiency.

[0074] According to embodiments of this disclosure, a region overlap operation is performed on the defect regions of multiple defects and at least one region of interest to obtain multiple overlap results, including: combining the defect regions of multiple defects with at least one region of interest to obtain multiple combined regions; binarizing the multiple combined regions to generate multiple combined images; and performing a pixel-by-pixel product operation on the multiple combined images and a predetermined image to obtain multiple overlap results; wherein the region where the predetermined image is located surrounds the multiple combined regions.

[0075] For example, each defect region can be paired with at least one region of interest to obtain a combined region. When a defect region partially overlaps with a region of interest, the combined region can be a polygonal region consisting of the intersection points of the defect and region of interest, and the vertices of the non-intersecting parts of the defect and region of interest. When a defect region and region of interest completely overlap, the combined region can be the area with the larger area between the defect and region of interest.

[0076] Then, each combined region is binarized to obtain a combined image. In the combined image, the grayscale value of the background region can be 0, and the grayscale value of the combined region can be 1. The predetermined image can be an image composed of 1×n pixels, all with a grayscale value of 1. The predetermined image can be ANDed with each row of pixels of the combined image from top to bottom. When the area of ​​the non-zero pixels in the result is less than the sum of the areas of the region of interest and the defect region, it can be determined that the region of interest and the defect region overlap.

[0077] According to embodiments of this disclosure, pixel-by-pixel product operations are performed on multiple combined images and the predetermined image to obtain multiple overlapping results, including the following operations: performing pixel-by-pixel product operations on the predetermined image and each combined image respectively to obtain the overlapping area corresponding to each combined image; and determining that the overlapping result is that the defective region and the region of interest overlap if the overlapping area is less than the sum of the areas of the corresponding defective region and the region of interest.

[0078] According to embodiments of this disclosure, multiple combined images may include M combined images, where M is an integer greater than 1; performing a pixel-by-pixel product operation on the multiple combined images and a predetermined image to obtain multiple overlapping results may include the following operations: performing a pixel-by-pixel product operation on the predetermined image and the m-th combined image to obtain an overlapping area; in response to the overlapping area being less than the sum of the areas of the m-th defect region and the region of interest, determining that the m-th overlapping result is an overlap between the m-th defect region and the region of interest; in response to m being less than M, returning to the operation of performing a pixel-by-pixel product operation on the predetermined image and the m-th combined image, and incrementing m; and in response to m being equal to M, obtaining M overlapping results.

[0079] According to embodiments of this disclosure, performing pixel-by-pixel product operations on a predetermined image and each combined image to obtain the overlapping area corresponding to each combined image includes: reading the third gray value of each pixel in each combined image and the fourth gray value of each pixel in the predetermined image; multiplying each fourth gray value with each third gray value to obtain overlapping area information corresponding to each combined image; and calculating each overlapping area based on the overlapping area information.

[0080] For example, the predetermined image can also be an image that surrounds the combined region and has an area larger than the combined region. Next, the grayscale value of the predetermined image can also be set to 1. Then, the predetermined image is multiplied by the grayscale value of each row of pixels in the combined image. When there is a non-zero value in the result, the area corresponding to the non-zero pixel is the overlapping region. The overlapping area can be calculated based on the pixel coordinates of the overlapping region. Furthermore, when the overlapping area is less than the sum of the areas of the region of interest and the defect region, it can be determined that the region of interest and the defect region overlap.

[0081] According to embodiments of this disclosure, pixel dot product operations are performed between a predetermined image and a combined image to determine whether the defect region overlaps with the region of interest, thereby improving the processing efficiency of the overlap operation.

[0082] To further adapt to the quality requirements of actual production lines, the defect detection equipment provided in this disclosure supports user configuration of defect detection rules.

[0083] According to embodiments of this disclosure, determining target defect information from multiple defect information based on multiple overlapping results includes: determining multiple target regions of interest information that overlap with a target type defect from multiple overlapping results based on the defect type; and determining the information of the target type defect as target defect information in response to the number of multiple target regions of interest information being greater than or equal to a first predetermined threshold.

[0084] For example, by performing region detection on the initial image of the object to be detected, four luminescent pixel regions and one driving circuit region can be obtained. By performing defect detection on the initial image of the object to be detected, four defect regions can be obtained, including three crack defects and one chipped edge defect. The target region of interest information can be the luminescent pixel region information and the driving circuit region, and the target type defect can be a crack defect. In the overlap result, the region overlapping with the defect regions of the three crack defects can include two luminescent pixel regions and one driving circuit region. The first predetermined threshold can be 2. Since 2 = 2 and 1 < 2, it can be determined that the information of the two crack defects that only overlap with the two luminescent pixel regions is the target defect information, while the information of the crack defect that overlaps with the driving circuit region is not the target defect information.

[0085] According to embodiments of this disclosure, target type region of interest information is determined from a plurality of target region of interest information based on the region of interest type; and in response to the number of target type region of interest information being greater than or equal to a second predetermined threshold, information of target type defect is determined as target defect information.

[0086] For example, the second predetermined threshold can be 2, and the target type region of interest can be the luminescent pixel region. Since the number of luminescent pixel regions in the region that overlaps with the defect regions of the three crack defects in the overlapping result is 2, and since 2 = 2, the defect in the crack defect region that overlaps with the luminescent pixel region can be determined as the target defect.

[0087] According to embodiments of this disclosure, based on the type of region of interest, multiple defect information that overlaps with the target type of region of interest is determined from multiple overlapping results; and in response to the number of multiple defect information being greater than or equal to a third predetermined threshold, the multiple defect information is determined to be target defect information.

[0088] For example, the third predetermined threshold can be 2, and the target type region of interest can be the luminescent pixel region. Since in the overlap result, the defect region overlapping with two luminescent pixel regions is two crack defect regions, and the defect region overlapping with one luminescent pixel region is one chipped edge defect region, the number of defect regions intersecting with the target type region of interest is 3. Since 3 > 2, the defects in both the crack defect regions and the chipped edge defect regions overlapping with the luminescent pixel regions can be identified as target defects.

[0089] It should be noted that the first, second, and third predetermined thresholds can all be configured based on the association rules between defect types and region types. The specific configuration rules can be determined based on the actual quality requirements of the production line, and are not specifically limited here.

[0090] According to embodiments of this disclosure, by flexibly configuring the association rules between defect types and region types, the adaptability of the defect detection equipment to the quality requirements of actual production lines is further improved.

[0091] The detection accuracy of the object detection and image segmentation models described above typically depends on the diversity of sample data during the training phase. When performing defect detection and region detection tasks, it is currently necessary to use manual review to verify the detection results of the defect detection equipment. This is to collect data on misclassifications and missed detections from the defect detection equipment, and then iteratively optimize and train the object detection and image segmentation models to improve their accuracy.

[0092] However, since the types of defects and areas to be detected associated with the process section are configured by the user based on the actual quality requirements of the production line, it is difficult to determine the specific reasons for misjudgments and omissions during the review stage. Therefore, an initial defect marker image can be generated based on the positions of multiple defects corresponding to multiple defect information in the initial image, and the defect marker image can be compared and displayed with the initial defect marker image.

[0093] The defect marking image and the initial defect image described above will be compared and explained below with reference to Figures 4A and 4B.

[0094] Figure 4A is a schematic diagram of a defect marking image according to an embodiment of this disclosure.

[0095] As shown in Figure 4A, the schematic diagram of the detection result 400A shows the defect marking image described above. In the defect marking image 400A, the object to be detected 401 includes two driving circuit regions 410a to 410b and four light-emitting pixel regions 420a to 420d.

[0096] Defect region 431, representing a residual photoresist defect, is displayed in luminescent pixel region 420a. Defect region 432, representing a crack defect, is displayed in luminescent pixel region 420b. Defect region 433, representing a bubble defect, is displayed in luminescent pixel region 420c. The defect regions 431 (residual photoresist), 432 (crack), and 433 (bubble) are all marked with defect detection boxes, indicating that these three defects are defects within the user's region of interest or a specific region.

[0097] The defect area 434 in the driving circuit area 410a is not marked by a defect detection box, indicating that the defect is not within the area of ​​interest of interest to the user. This can be understood as the defect existing within the driving circuit area and not affecting the quality of the liquid crystal display panel; therefore, it is not treated as a defect.

[0098] Figure 4B is a schematic diagram of the initial defect marking image of an embodiment of this disclosure.

[0099] As shown in Figure 4B, the initial defect marking image 400B displays the initial defect marking image described above. The difference from the detection result schematic diagram 400A is that the defect region 434 in the driving circuit region 410a is also marked by the defect detection box.

[0100] According to embodiments of this disclosure, the defect marking image 400A and the initial defect marking image 400B can be displayed in different positions on the same interface. For example, the defect marking image 400A can be displayed in the left area of ​​the interface, and the initial defect marking image 400B can be displayed in the right area of ​​the interface, so as to compare the image content.

[0101] Figures 4C to 4D are schematic diagrams showing the defect marking image and the initial defect marking image according to an embodiment of the present disclosure.

[0102] As shown in Figure 4C, the display interface 400C may include a thumbnail 400a of the defect marking image and a thumbnail 400b of the initial defect marking image displayed in the upper half of the interface.

[0103] When the user clicks on the thumbnail 400b of the initial defect marker image, the initial defect marker image 400B is displayed in the lower half of the interface.

[0104] As shown in Figure 4D, the display interface 400D may include a thumbnail 400a of the defect mark image displayed in the upper left corner and a comparison button 402 displayed in the upper right corner.

[0105] When the user clicks the comparison button 402, the initial defect marking image 400B is displayed in the lower half of the interface.

[0106] During the manual review stage, when defect region 434 was identified as a missed defect region, a comparison of Figures 4A and 4B shows that defect region 434 was filtered out due to the user-configured rules linking the region of interest to the defect region, rather than being missed by the object detection model or image segmentation model. Therefore, the data corresponding to defect region 434 does not need to be used as missed data for optimizing the training of the object detection model.

[0107] According to embodiments of this disclosure, based on the results of manual review, the positions of all defects in the initial image are marked so as to be compared with the defect mark image of the target defect marks obtained after the region overlap is filtered, so as to determine the specific reasons for the misjudgment or omission of defects.

[0108] To further improve the adaptability of the defect detection method to different process stages, the defect detection method provided in this disclosure, which offers the type of defect to be detected and / or the type of region to be detected associated with the target process stage, can be flexibly configured by the user. Therefore, based on the source of the object to be detected, the defect information to be processed and the region of interest to be processed can be determined from multiple defect information and at least one region of interest information, respectively.

[0109] According to embodiments of this disclosure, the source of the object to be tested can be the process segment that produces the object to be tested, the processing equipment that produces the object to be tested, or the raw material used to produce the object to be tested.

[0110] For example, if a certain process section includes N processing machines, and during the manual re-inspection of the objects to be inspected produced by the nth processing machine, it is found that there are cases of missed detection of photoresist residue defects, then photoresist residue defects can be added to the defect types to be inspected by the defect detection equipment corresponding to the nth processing machine.

[0111] For example, the types of defects to be detected associated with a certain process segment can include: cracks, clouding, chipping, and bubbles. Users can add or reduce the types of defects to be detected according to the actual quality requirements of the production line.

[0112] For example, if cracks of varying degrees and sizes are found in all batches of raw materials during the initial inspection, crack defects can be added to the list of defects to be detected in all process stages corresponding to that batch of raw materials.

[0113] According to embodiments of this disclosure, the defect type and / or region type to be detected can be flexibly configured according to the source of the object to be detected. The defect detection strategy can be adjusted in a timely manner according to the changes in the status of raw materials, process sections, and processing equipment on the actual production line, so as to improve the detection accuracy.

[0114] Since object detection models or image segmentation models output confidence scores corresponding to the results, and these confidence scores characterize the reliability of the output results, it is also possible to determine the defect information to be processed and the region of interest to be processed from the multiple defect information and at least one region of interest information respectively, based on the confidence scores corresponding to each of the multiple defect information and at least one region of interest information. According to embodiments of this disclosure, the confidence scores can also be flexibly configured by the user. For example, the defect detection results of the object to be detected in a certain process section may include: bubbles (confidence score: 20%) and cracks (confidence score: 80%).

[0115] According to embodiments of this disclosure, the model detection results are filtered based on confidence levels, which can further improve the accuracy of the detection results while reducing the amount of data processing for region overlap operations and improving detection efficiency.

[0116] Since bubble-type defects are often misjudged during the manual review stage, users can increase the confidence threshold for bubble defects to improve the accuracy of defect detection.

[0117] According to embodiments of this disclosure, after generating a defect-marked image, the method further includes: obtaining a manual review result of the defect-marked image; and, in response to the manual review result including misjudged defect information, adjusting the confidence level of the target detection model for the misjudged defect information based on the misjudged defect information.

[0118] For example, the confidence threshold for bubble defects can be adjusted from 20% to 70%.

[0119] According to embodiments of this disclosure, the accuracy of defect detection can be improved by adjusting the confidence level of misjudged defect information based on the results of manual review.

[0120] Since the target detection model and the image segmentation model have different processing speeds for the same image, the defect detection method of this disclosure embodiment can perform defect detection and region detection asynchronously on the initial image to obtain multiple defect information and at least one region of interest information, respectively.

[0121] For example, when the object detection model has completed defect detection of the initial image, but the image segmentation model has not yet completed region detection of the initial image, the object detection model can be used to perform defect detection on the next batch of initial images to reduce the waiting time of the object detection model.

[0122] According to embodiments of this disclosure, by asynchronously performing defect detection and region detection, the waiting time during model inference is reduced, further reducing the waste of hardware resources during idle time, improving data processing efficiency, and further improving defect detection efficiency.

[0123] The sizes of LCD panels vary on actual production lines. For LCD panels that exceed the acquisition range of the image acquisition device, images can be acquired in sections, and then the test results can be summarized.

[0124] According to embodiments of this disclosure, in scenarios where the object to be detected is large, initial images can be acquired by region. Therefore, the initial images may include multiple images corresponding to multiple acquisition locations. The method may further include: generating alarm information and displaying the alarm information in response to the total number of target defects within the same type of region of interest corresponding to multiple images exceeding an alarm threshold.

[0125] For example, the object to be inspected can be divided into four regions for initial image acquisition. For each region, a corresponding defect detection result can be obtained. Then, the defect detection results corresponding to the four regions can be summarized. For example, the number of crack defects in the drive circuit region of the initial images corresponding to the first three acquisition regions is 1, while the number of crack defects in the drive circuit region of the initial image corresponding to the fourth acquisition region is 0. Therefore, the number of crack defects in the drive circuit region of the object to be inspected is 3. When the crack defect alarm threshold in the drive circuit region is 2, an alarm message can be generated and displayed to prompt relevant personnel to handle the object to be inspected.

[0126] For example, the initial images of the object to be inspected may include three images. The defect detection result of the first initial image is that there is one crack defect in the driving circuit area; the defect detection result of the second initial image is that there is no defect; and the defect detection result of the third initial image is that there are two crack defects in the driving circuit area. Therefore, the defect detection result of the object to be inspected could be that there are three crack defects in the driving circuit area. When the alarm threshold for crack defects in the driving circuit area is 2, since 3>2, an alarm message indicating that there are crack defects in the driving circuit area of ​​the object to be inspected can be generated and displayed.

[0127] According to embodiments of this disclosure, by summarizing the defect detection results of initial images from different acquisition areas according to the number of defects in the same type of region of interest, the application needs of defect detection scenarios for larger-sized objects to be detected can be met.

[0128] The raw material for LCD panels can be glass. In the initial stage of LCD panel production, the same piece of glass needs to be divided into multiple pieces according to the size requirements of the LCD panel. In the actual production environment, defects on LCD panels may originate from the processing equipment on the production line or from the raw material itself.

[0129] Therefore, the method in this embodiment may further include: acquiring defect marker images of multiple target objects associated with the object to be detected; and stitching together the defect marker image of the object to be detected and the defect marker images of the multiple target objects according to the relative positional relationship of the targets to generate a first fused image, so as to trace the cause of the defect.

[0130] According to embodiments of this disclosure, the raw materials of multiple target objects are the same as the raw materials of the object to be tested, and the relative positional relationship is determined based on the cutting position of the raw materials.

[0131] For example, multiple glass raw materials obtained from the same piece of glass can be identified by the same raw material identifier Glass1. Then, the defect marking images (corresponding defect marking images) of multiple target objects with the same raw material identifier can be restored and stitched together according to their relative positions when the raw materials were cut to generate the first fused image.

[0132] The following section provides a detailed explanation of the summary results of defect marker images for multiple objects to be tested from the same raw material, using Figure 5 as an example.

[0133] Figure 5 is a schematic diagram summarizing the detection results of the defect detection method of this disclosure for multiple detection objects of the same raw material.

[0134] As shown in Figure 5, the schematic diagram 500 summarizing the detection results includes a first fused image 510 and a magnified local image 520. The first fused image 510 is generated by restoring and stitching together defect marker images of multiple target objects with the same raw material identifier according to their relative positions when the raw material was cut. The dashed lines in the first fused image 510 indicate the cutting position of the raw material, and the dots in the first fused image 510 indicate the defect area.

[0135] The upper left corner region where P1 is located in the first fused image 510 can represent an object to be detected, such as a liquid crystal display panel or a semi-finished liquid crystal display panel in a certain process stage. By locally magnifying the upper left corner region where P1 is located, a locally magnified image 520 can be obtained to display the overlapping result 304 in Figure 3.

[0136] According to embodiments of this disclosure, when the frequency of a certain type of target defect in the summary image 410 is greater than a predetermined threshold, or when the defect area of ​​a certain type of target defect is concentrated within a certain defect area, it can be determined that the cause of this type of defect is due to the raw material itself.

[0137] According to embodiments of this disclosure, by summarizing defect marker images of the same raw materials, it is possible to quickly trace whether the cause of the defect is related to the raw materials, so as to adjust the production strategy in a timely manner and reduce the product defect rate on the production line.

[0138] In addition to considering the same raw materials, when tracing the cause of defects, the scope can be further expanded to the same batch of raw materials.

[0139] Therefore, the defect detection method provided in this embodiment may further include: acquiring multiple first fused images for multiple raw materials, wherein the multiple raw materials are from the same batch; and stitching the multiple first fused images together according to the raw material identification order to generate a second fused image.

[0140] For example, the second fused image, the first fused image, and the defect marker image can be displayed on the same page according to the predetermined display positions.

[0141] The following is a detailed explanation of the summary diagram of the results of multiple tests on the same batch of raw materials, with reference to Figure 6.

[0142] Figure 6 is a schematic diagram summarizing the detection results of multiple test objects in the same batch of raw materials using the defect detection method of this embodiment of the present disclosure.

[0143] As shown in Figure 6, the summary diagram of the test results 600 may include: a display image 610 of the same batch of raw materials, a defect marking image 620 of the same batch of raw materials, a thumbnail image 630 of the defect markings of each tested object, a magnified detail image 640 corresponding to the defect marking thumbnail, and defect alarm information 650. The display image 610 of the same batch of raw materials can be obtained using the first fused image generation method described above. The defect marking image 620 of the same batch of raw materials can be obtained using the second fused image generation method described above. The defect marking thumbnail 630 of each tested object and the magnified detail image 640 corresponding to the defect marking thumbnail are both obtained using the defect marking image generation method described above.

[0144] The raw material display diagram 610 for the same batch may include six raw materials, glass1 to glass6, from the same batch. Correspondingly, the defect marking diagram 620 for the same batch of raw materials marks the location of the target defect in each raw material.

[0145] Each defect marker thumbnail 630 for each inspected object includes a defect marker thumbnail corresponding to each initial image of each inspected object. Correspondingly, after selecting one of the defect marker thumbnails, the basic information display area can display relevant information about the inspected object corresponding to that defect marker thumbnail, such as: site name (corresponding to the name of the process section), equipment name, number of defects of each type, product set number, glass number, total number of defects, configuration item name, and resolution, etc. Below the basic information display area, there may also be detection trigger timing information and accuracy information. Detection trigger timing information indicates whether the current detection is passively triggered. Passive triggering usually occurs when product quality does not meet standards, requiring a backtracking of the detection process. Active triggering usually occurs during product manufacturing. Accuracy information is usually an evaluation of the accuracy of the defect detection equipment's detection results after manual review, combined with the results of the manual review.

[0146] According to embodiments of this disclosure, defect marker images of multiple inspection objects can be stored based on raw material information. When a user selects any defect marker thumbnail, the defect marker image corresponding to the thumbnail can be retrieved from the memory, and the location of the target defect is shown in the enlarged detail view 640 corresponding to the defect marker thumbnail. The information in the defect alarm information 650 includes the defect type (Code), alarm information (Alarm), action to be taken (Action), and comment (Comment) corresponding to the enlarged detail view 640.

[0147] According to embodiments of this disclosure, by summarizing defect marker images of the same batch of raw materials, it is possible to more accurately trace whether the cause of the defect is related to the raw materials, so as to adjust the production strategy in a timely manner and reduce the product defect rate on the production line.

[0148] Figure 7 is a schematic diagram of system interaction for performing the defect detection method of the present disclosure embodiments.

[0149] As shown in Figure 7, the exemplary architecture 700 for system interaction may include an image acquisition device 701, a file storage system 702, a MES (Manufacturing Execution System) 703, a defect detection platform 704, and a client cloud page 705.

[0150] Image acquisition device 701 can be installed on the production line of LCD display panel to acquire initial images of the object to be inspected and store them in file storage system 702. Production execution system 703 can retrieve the initial image of the object to be inspected from file storage system 702 when it receives an inspection task from a client, and send the initial image of the object to be inspected to defect detection platform 704.

[0151] The defect detection platform 704 can execute the defect detection method of this disclosure embodiment, respectively calling the defect detection service and the region detection service to perform defect detection and region detection on the initial image of the object to be detected, and perform region overlap calculation on the detection results to generate a defect marking image. Finally, the defect detection platform 704 can store the marked image in the file storage system 702 and display it through the client cloud page 705 so that users can browse the marked image.

[0152] According to embodiments of this disclosure, by marking defects corresponding to defect regions overlapping with the region of interest as target defects in the initial image of the object to be inspected, the accuracy of defect detection in specific regions of the object to be inspected is further improved. By associating process segments with the types of defects and regions to be inspected, targeted defect detection for different process segments is achieved, allowing users to flexibly configure the types of defects and regions to be inspected according to the quality requirements of different process segments on the production line, thereby improving the adaptability of the defect detection method to application scenarios.

[0153] Defect detection services can be implemented based on object detection models, while region detection services can be implemented based on image segmentation models. Since the two models process the initial image at different speeds, to further improve detection efficiency, a model inference service can be introduced into the defect detection platform. This allows for asynchronous invocation of both defect detection and region detection services, thereby reducing waiting time during model inference, further minimizing the waste of hardware resources during downtime, and improving data processing efficiency.

[0154] According to embodiments of this disclosure, before acquiring an initial image of the object to be inspected for the corresponding target process segment, the method further includes: in response to receiving a selection operation for a detection model in the inspection service selection page, loading a target detection model associated with the target process segment to perform defect detection and area detection using the target detection model.

[0155] According to embodiments of this disclosure, the detection service selection page can be a client cloud page.

[0156] Figure 8 is a schematic diagram of the interaction between the defect detection platform and the client cloud page for executing the defect detection method of the present disclosure embodiments.

[0157] As shown in Figure 8, users can select the models to be loaded for the defect detection service and the region detection service on the client cloud page 705. For example, the defect detection service selection box 705_1 and the region detection service selection box 705_2 can be configured separately on the client cloud page 705. The defect detection service selection box 705_1 can include the names of multiple target detection models used to perform the defect detection service. The region detection service selection box 705_2 can include the names of multiple image segmentation models used to perform the region detection service.

[0158] The defect detection platform 704 can load the corresponding models from the model library 704_4 according to the user's selection, so as to execute the defect detection service 704_2 and the area detection service 704_3.

[0159] Model inference service 704_1 can asynchronously call defect detection service 704_2 and region detection service 704_3 to perform defect detection and region detection on the initial image respectively, and use callback functions to receive the defect detection results and region detection results respectively. When the processing rates of defect detection service 704_2 and region detection service 704_3 for the same initial image differ, for example, when defect detection results have been received but region detection results have not been received, the defect detection service can continue to be called to process the next initial image or the next batch of initial images. However, after both defect detection results and region detection results for the same initial image have been received, region overlap calculation 704_5 is then performed to generate the labeled image.

[0160] According to embodiments of this disclosure, by asynchronously calling the defect detection service and the area detection service, the waiting time during the model inference process can be reduced, further reducing the waste of hardware resources during downtime and improving data processing efficiency.

[0161] According to embodiments of the present disclosure, a non-transitory computer-readable storage medium stores computer instructions, wherein the computer instructions are used to cause a computer to perform the method described above.

[0162] According to an embodiment of this disclosure, a computer program product includes a computer program that, when executed by a processor, implements the method described above.

[0163] Figure 9 illustrates a schematic block diagram of an example electronic device 900 that can be used to implement embodiments of the present disclosure. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0164] As shown in Figure 9, device 900 includes a computing unit 901, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 902 or a computer program loaded into random access memory (RAM) 903 from storage unit 908. RAM 903 can also store various programs and data required for the operation of device 900. The computing unit 901, ROM 902, and RAM 903 are interconnected via bus 904. Input / output (I / O) interface 905 is also connected to bus 904.

[0165] Multiple components in device 900 are connected to I / O interface 905, including: input unit 906, such as keyboard, mouse, etc.; output unit 907, such as various types of monitors, speakers, etc.; storage unit 908, such as disk, optical disk, etc.; and communication unit 909, such as network card, modem, wireless transceiver, etc. Communication unit 909 allows device 900 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0166] The computing unit 901 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 901 performs the various methods and processes described above, such as video processing methods. For example, in some embodiments, the video processing method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 908. In some embodiments, part or all of the computer program may be loaded and / or installed on device 900 via ROM 902 and / or communication unit 909. When the computer program is loaded into RAM 903 and executed by the computing unit 901, one or more steps of the video processing method described above may be performed. Alternatively, in other embodiments, the computing unit 901 may be configured to perform video processing methods by any other suitable means (e.g., by means of firmware).

[0167] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transferring data and instructions to the storage system, the at least one input device, and the at least one output device.

[0168] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0169] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0170] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0171] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0172] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, distributed system servers, or servers incorporating blockchain technology.

[0173] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0174] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A defect detection method, comprising: Obtain the initial image of the object to be inspected in the target process section; The target process segment is associated with the type of defect to be detected and the type of area to be detected; Defect detection is performed on the initial image to obtain multiple defect information for the type of defect to be detected; Perform region detection on the initial image to obtain at least one region of interest information for the type of region to be detected; Based on the plurality of defect information and the at least one region of interest information, the target defect information is determined from the plurality of defect information; as well as A defect marker image is generated based on the position of the target defect corresponding to the target defect information in the initial image; wherein the position of the target defect overlaps with the at least one region of interest.

2. The method according to claim 1, wherein: The multiple defect information includes the defect regions of each of the multiple defects; The step of determining the target defect information from multiple defect information and at least one region of interest information includes: Perform a region overlap operation between the defect regions of each of the multiple defects and the at least one region of interest to obtain multiple overlap results; as well as Based on the multiple overlapping results, the target defect information is determined from the multiple defect information.

3. The method according to claim 2, wherein: The method involves performing a region overlap operation on the defect regions of each of the multiple defects and the at least one region of interest to obtain multiple overlap results, including: Binarize the defect regions of each of the plurality of defects and the at least one region of interest respectively to generate multiple defect region images and at least one region of interest image; and The pixel-by-pixel product operation is performed on the multiple defect region images and the at least one region of interest image to obtain the multiple overlapping results.

4. The defect detection method according to claim 3, wherein, The step of performing a pixel-by-pixel product operation on the multiple defect region images and the at least one region of interest image to obtain the multiple overlapping results includes: For each of the plurality of defect region images, the first gray value of each pixel in each defect region image and the second gray value of each pixel in the at least one region of interest image are read respectively; Each of the second grayscale values ​​is multiplied by each of the first grayscale values ​​to obtain the multiple overlapping results.

5. The method according to claim 2, wherein, The method involves performing a region overlap operation on the defect regions of each of the multiple defects and the at least one region of interest to obtain multiple overlap results, including: The defect regions of each of the multiple defects are combined with the at least one region of interest to obtain multiple combined regions; Binarize the multiple combined regions separately to generate multiple combined images; and The multiple combined images are multiplied by pixels and a predetermined image to obtain the multiple overlapping results; wherein the region of the predetermined image surrounds the multiple combined regions.

6. The method according to claim 5, wherein: The step of performing a pixel-by-pixel product operation on the multiple combined images and the predetermined image to obtain multiple overlapping results includes: The predetermined image is then multiplied by each combined image to obtain the overlap area corresponding to each combined image; and In response to the fact that the overlapping area is less than the sum of the areas of the corresponding defect region and the region of interest, the overlapping result is determined to be that the defect region and the region of interest overlap.

7. The method according to claim 6, wherein, The step of performing a pixel-by-pixel product operation between the predetermined image and each combined image to obtain the overlap area corresponding to each combined image includes: Read the third grayscale value of each pixel in each combined image and the fourth grayscale value of each pixel in the predetermined image respectively; The fourth grayscale value is multiplied by the third grayscale value to obtain the overlapping region information corresponding to each combined image; and Based on the information of each overlapping region, the overlapping area of ​​each region is calculated.

8. The method according to any one of claims 2-7, wherein: The multiple overlapping results include multiple overlapping defect regions and regions of interest; The defect information includes the defect type; The step of determining the target defect information from the multiple defect information based on the multiple overlap results includes: Based on the defect type, determine multiple target regions of interest information that overlap with the target type defect from the multiple overlapping results; as well as In response to the number of the plurality of target regions of interest information being greater than or equal to a first predetermined threshold, the information of the target type defect is determined to be the target defect information.

9. The defect detection method according to claim 8, wherein: The target region of interest information includes the region of interest type, and the method further includes: Based on the region of interest type, determine the target type region of interest information from multiple target region of interest information; and In response to the number of target type region of interest information being greater than or equal to a second predetermined threshold, the information of the target type defect is determined to be the target defect information.

10. The method according to any one of claims 2-7, wherein: The multiple overlapping results include multiple overlapping defect regions and regions of interest; the region of interest information includes the region of interest type; Based on the type of region of interest, determine multiple defect information that overlaps with the target type of region of interest from the multiple overlapping results; as well as In response to the number of the plurality of defect information being greater than or equal to a third predetermined threshold, the plurality of defect information is determined to be the target defect information.

11. The method according to any one of claims 1-10, wherein: The step of determining the target defect information from multiple defect information and at least one region of interest information includes: Based on the source of the object to be detected, determine the defect information to be processed and the region of interest to be processed from the plurality of defect information and the at least one region of interest information, respectively; and Based on the defect information to be processed and the region of interest information to be processed, the target defect information is determined from the defect information to be processed.

12. The method according to any one of claims 1-10, wherein: The step of determining the target defect information from multiple defect information and at least one region of interest information includes: Based on the confidence levels of the plurality of defect information and the confidence levels of the at least one region of interest information, determine the defect information to be processed and the region of interest to be processed from the plurality of defect information and the at least one region of interest information, respectively; and Based on the defect information to be processed and the region of interest information to be processed, the target defect information is determined from the defect information to be processed.

13. The method according to any one of claims 1-12, wherein the initial image comprises multiple images for multiple acquisition locations; wherein: The method further includes: An alarm message is generated in response to the total number of target defects in the same type of region of interest corresponding to multiple images exceeding the alarm threshold.

14. The method according to any one of claims 1-13, wherein: The method further includes: Acquire defect marker images of multiple target objects associated with the object to be inspected; wherein the multiple target objects are made of the same raw material as the object to be inspected; and Based on the relative positional relationship of the targets, the defect-marked image of the object to be detected and the multiple target objects are... The respective defect marker images are stitched together to generate a first fused image, wherein the relative positional relationship of the targets is determined based on the cutting position of the raw material.

15. The method of claim 14, wherein: The method further includes: Acquire multiple first fused images of multiple raw materials, wherein the multiple raw materials are from the same batch; and The multiple first fused images are stitched together according to the raw material identification order to generate a second fused image.

16. The method of claim 15, wherein: The method further includes: The second fused image, the first fused image, and the defect marker image are displayed on the same page according to the predetermined display positions.

17. The method according to claim 1, wherein: The method further includes: An initial defect marker image is generated based on the positions of multiple defects corresponding to the multiple defect information in the initial image; and The defect-marked image is compared and displayed with the initial defect-marked image.

18. The method according to any one of claims 1-17, wherein, Before acquiring the initial image of the object to be inspected in the corresponding target process segment, the method further includes: In response to receiving a selection operation for a detection model in the detection service selection page, a target detection model associated with the target process segment is loaded to perform the defect detection and the area detection using the target detection model.

19. The method according to claim 18, wherein, After generating the defect marker image, the method further includes: Obtain the results of manual review of the defect-marked image; In response to the fact that the manual review result includes misjudged defect information, the confidence level of the target detection model for the misjudged defect information is adjusted according to the misjudged defect information.

20. An electronic device, comprising: At least one processor; as well as A memory that is communicatively connected to at least one processor; wherein, The memory stores instructions that can be executed by at least one processor to enable the at least one processor to perform the method of any one of claims 1-19.

21. A non-transitory computer-readable storage medium storing computer instructions, wherein, Computer instructions are used to cause a computer to perform the method according to any one of claims 1-19.

22. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-19.

Images