Surface defect detection method, device, equipment and storage medium thereof

Abstract

The present application discloses a surface defect detection method, apparatus, device and storage medium thereof, and relates to the field of defect detection technology. The method comprises: obtaining a plurality of sub-images based on an initial image, a translation window and a translation step; obtaining a defect detection frame corresponding to each sub-image based on the plurality of sub-images; and obtaining a defect detection result based on each defect detection frame. The present application divides a larger initial image into a plurality of smaller sub-images so that the surface defect detection method in the prior art can be applied to each smaller sub-image. After obtaining the surface defect detection results of each sub-image, the surface defect detection results of each sub-image are fused to form the surface defect detection result of the initial image. This solves the technical problem that the surface defect detection method in the prior art is not applicable to surface defect detection of a larger initial image.

Description

Technical Field

[0001] The present application relates to the field of defect detection technology, and specifically to a surface defect detection method, device, equipment and storage medium thereof. Background Art

[0002] Surface defect detection, a type of machine vision technology, typically involves detecting surface flaws. This technology utilizes computer vision to simulate the functions of human vision, performing image acquisition, processing, and calculations, ultimately enabling the actual detection, control, and application of specific objects. Currently, machine vision-based surface defect detection has widely replaced manual visual inspection in various industrial sectors, such as consumer electronics, automotive, and home appliance industries.

[0003] Because traditional machine vision-based surface defect detection methods are primarily used to detect defects visible to the naked eye, in some applications, these methods are difficult to apply to surfaces with extremely small defect areas. For example, in the application scenario of chip shell surface defect detection, due to the extremely small surface area of the chip shell surface defect, a high-power microscope is required to capture the defect. Generally, a bump defect on a chip shell has less than 3 pixels when photographed with an 8x high-power microscope. In other words, the size of the image captured with a high-power microscope far exceeds the commonly used image resolution range. In other words, it is difficult to detect surface defects on chip shells using traditional surface defect detection methods. Summary of the Invention

[0004] The purpose of this application is to provide a surface defect detection method, device, equipment and storage medium thereof to solve the technical problem that existing surface defect detection methods cannot be applied to products with smaller surface defects.

[0005] To achieve the above objectives, this application provides the following technical solutions:

[0006] In a first aspect, the present application proposes a surface defect detection method, the method comprising:

[0007] Based on an initial image, a translation window and a translation step, a plurality of sub-images are acquired; the length of the translation window is less than the length of the initial image, and the height of the translation window is less than the height of the initial image; the initial image is acquired in advance; the translation window and the translation step are preset or generated based on the initial image; the translation step of the translation window along the length direction of the initial image is less than or equal to the length of the translation window; the translation step of the translation window along the height direction of the initial image is less than or equal to the height of the translation window;

[0008] Based on the multiple sub-images, a defect detection frame corresponding to each sub-image is obtained;

[0009] Based on each defect detection frame, a defect detection result is obtained.

[0010] As a specific solution in the technical solution of the present application, the length of the translation window is less than or equal to 1 / 2 of the length of the initial image and greater than or equal to 1 / 10 of the length of the initial image; the height of the translation window is less than or equal to 1 / 2 of the height of the initial image and greater than or equal to 1 / 10 of the height of the initial image.

[0011] As a specific solution in the technical solution of this application, obtaining defect detection results based on each defect detection frame includes:

[0012] Based on each defect detection frame, at least one defect detection group is obtained; each defect detection group has at least one defect detection frame, and each defect detection frame in each defect detection group has the same defect label;

[0013] Obtaining a first defect detection group; the first defect detection group is a group that has not been subjected to redundant comparison processing among the defect detection groups;

[0014] Obtaining the confidence of each defect detection frame in the first defect detection group;

[0015] Based on the confidence of each defect detection frame, the first defect detection group is updated to obtain a current defect detection group;

[0016] Based on each defect detection frame in each current defect detection group, a defect detection result is obtained.

[0017] As a specific solution in the technical solution of the present application, the updating of the first defect detection group based on the confidence of each defect detection frame and obtaining the current defect detection group includes:

[0018] Based on the confidence of each defect detection frame, a first defect detection frame is obtained; the first defect detection frame is a defect detection frame in the first defect detection group that has not been subjected to interactive comparison processing and has the highest confidence;

[0019] Based on the first defect detection frame, sequentially obtaining, in descending order of confidence, an interaction ratio between each defect detection frame in the first defect detection group and the first defect detection frame;

[0020] If the interaction ratio of the defect detection frame is greater than a first threshold, the defect detection frame is deleted until all defect detection frames in the first defect detection group are interactively compared, and then the first defect detection group is updated to obtain a second defect detection group. The first threshold is preset.

[0021] Based on the second defect detection group, the current defect detection group is obtained.

[0022] As a specific solution in the technical solution of the present application, obtaining the current defect detection group based on the second defect detection group includes:

[0023] Based on the second defect detection group, a second defect detection frame and a third defect detection frame are obtained; the second defect detection frame and the third defect detection frame are defect detection frames in the second defect detection group that have not been subjected to overlapping comparison processing;

[0024] Based on the second defect detection frame and the third defect detection frame, obtaining an overlap ratio of the second defect detection frame and an overlap ratio of the third defect detection frame;

[0025] If the overlap ratio of the third defect detection frame is greater than a second threshold and the overlap ratio of the second defect detection frame is less than the second threshold, the third defect detection frame is deleted; if the overlap ratio of the second defect detection frame is greater than the second threshold and the overlap ratio of the third defect detection frame is less than the second threshold, the second defect detection frame is deleted; after all defect detection frames in the second defect detection group have been overlapped and compared, the second defect detection group is updated to obtain a third defect detection group, and the second threshold is preset;

[0026] Based on the third defect detection group, the current defect detection group is obtained.

[0027] As a specific solution in the technical solution of the present application, obtaining the current defect detection group based on the third defect detection group includes:

[0028] Based on the third defect detection group, a fourth defect detection frame and a fifth defect detection frame are obtained; the fourth defect detection frame and the fifth defect detection frame are defect detection frames in the third defect detection group that have not been merged and compared;

[0029] acquiring a merging ratio of the fourth defect detection frame and the fifth defect detection frame based on the fourth defect detection frame and the fifth defect detection frame;

[0030] If the merging ratio of the fourth defect detection frame and the fifth defect detection frame is greater than a third threshold, updating the historical label values of the fourth defect detection frame and the fifth defect detection frame, and obtaining the current label values of the fourth defect detection frame and the fifth defect detection frame; after all defect detection frames in the third defect detection group are merged and compared, merging all defect detection frames with the same current label value to obtain a fourth defect detection group, where the third threshold is preset;

[0031] Based on the fourth defect detection group, the current defect detection group is obtained.

[0032] In a second aspect, the present application proposes a surface defect detection device, comprising a processing module, wherein the processing module is configured to acquire a plurality of sub-images based on an initial image, a translation window, and a translation step; the length of the translation window is less than the length of the initial image, and the height of the translation window is less than the height of the initial image; the initial image is acquired in advance; the translation window and the translation step are preset or generated based on the initial image; the translation step of the translation window along the length direction of the initial image is less than or equal to the length of the translation window; and the translation step of the translation window along the height direction of the initial image is less than or equal to the height of the translation window;

[0033] and, based on the plurality of sub-images, obtaining a defect detection frame corresponding to each sub-image;

[0034] And, based on each defect detection frame, a defect detection result is obtained.

[0035] As a specific solution in the technical solution of the present application, the length of the translation window is less than or equal to 1 / 2 of the length of the initial image and greater than or equal to 1 / 10 of the length of the initial image; the height of the translation window is less than or equal to 1 / 2 of the height of the initial image and greater than or equal to 1 / 10 of the height of the initial image.

[0036] As a specific solution in the technical solution of the present application, the processing module is further used to obtain at least one defect detection group based on each defect detection frame; each defect detection group has at least one defect detection frame, and the defect labels of the defect detection frames in each defect detection group are the same;

[0037] The acquisition module is further configured to acquire a first defect detection group; the first defect detection group is a group among the defect detection groups that has not undergone redundant comparison processing;

[0038] The acquisition module is further configured to acquire the confidence level of each defect detection frame in the first defect detection group;

[0039] The processing module is further configured to update the first defect detection group based on the confidence level of each defect detection frame to obtain a current defect detection group;

[0040] The processing module is further configured to obtain defect detection results based on each defect detection frame in each current defect detection group.

[0041] As a specific solution in the technical solution of the present application, the acquisition module is further configured to acquire a first defect detection frame based on the confidence of each defect detection frame; the first defect detection frame is a defect detection frame in the first defect detection group that has not been subjected to interactive comparison processing and has the highest confidence;

[0042] The processing module is further configured to obtain, based on the first defect detection frame, an interaction ratio between each defect detection frame in the first defect detection group and the first defect detection frame in descending order of confidence;

[0043] The processing module is further configured to delete the defect detection frame if the interaction ratio of the defect detection frame is greater than a first threshold, until all defect detection frames in the first defect detection group are interactively compared, and then update the first defect detection group to obtain a second defect detection group, wherein the first threshold is preset;

[0044] The processing module is further configured to obtain the current defect detection group based on the second defect detection group.

[0045] As a specific solution in the technical solution of the present application, the acquisition module is further used to acquire a second defect detection frame and a third defect detection frame based on the second defect detection group; the second defect detection frame and the third defect detection frame are defect detection frames in the second defect detection group that have not been subjected to overlapping comparison processing;

[0046] The processing module is further configured to obtain an overlap ratio of the second defect detection frame and an overlap ratio of the third defect detection frame based on the second defect detection frame and the third defect detection frame;

[0047] The processing module is further configured to delete the third defect detection frame if the overlap ratio of the third defect detection frame is greater than a second threshold and the overlap ratio of the second defect detection frame is less than the second threshold; delete the second defect detection frame if the overlap ratio of the second defect detection frame is greater than the second threshold and the overlap ratio of the third defect detection frame is less than the second threshold; and update the second defect detection group to obtain a third defect detection group after all defect detection frames in the second defect detection group have been subjected to overlap comparison processing, wherein the second threshold is preset;

[0048] The processing module is further configured to obtain the current defect detection group based on the third defect detection group.

[0049] As a specific solution in the technical solution of the present application, the acquisition module is further used to acquire a fourth defect detection frame and a fifth defect detection frame based on the third defect detection group; the fourth defect detection frame and the fifth defect detection frame are defect detection frames in the third defect detection group that have not been merged and compared;

[0050] The processing module is further configured to obtain a merging ratio of the fourth defect detection frame and the fifth defect detection frame based on the fourth defect detection frame and the fifth defect detection frame;

[0051] The processing module is further configured to update historical label values of the fourth defect detection frame and the fifth defect detection frame if a merging ratio of the fourth defect detection frame and the fifth defect detection frame is greater than a third threshold, and obtain current label values of the fourth defect detection frame and the fifth defect detection frame; after all defect detection frames in the third defect detection group are merged and compared, merge all defect detection frames with the same current label value to obtain a fourth defect detection group, wherein the third threshold is preset;

[0052] The processing module is further configured to obtain the current defect detection group based on the fourth defect detection group.

[0053] In a third aspect, the present application proposes a surface defect detection device, comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, it implements the surface defect detection method as described in any one of the first aspects.

[0054] In a fourth aspect, the present application proposes a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the surface defect detection method as described in any one of the first aspects.

[0055] Compared with the prior art, the present invention has the following advantages:

[0056] By dividing a large initial image into multiple smaller sub-images, the existing surface defect detection method can be applied to each of the smaller sub-images. After obtaining surface defect detection results for each sub-image, the surface defect detection results of each sub-image are fused to form the surface defect detection result for the initial image. This solves the technical problem that the existing surface defect detection method is not applicable to surface defect detection of large initial images. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 A flow chart of a surface defect detection method proposed in an embodiment of the present application;

[0058] Figure 2A schematic diagram of a method for segmenting an initial image proposed in an embodiment of the present application;

[0059] Figure 3 A schematic diagram of another method for segmenting an initial image proposed in an embodiment of the present application;

[0060] Figure 4 A schematic diagram of the surface defect detection result of the initial image proposed in an embodiment of the present application;

[0061] Figure 5 A schematic structural diagram of a surface defect detection device proposed in an embodiment of the present application;

[0062] Figure 6 This is a schematic structural diagram of a surface defect detection device proposed in an embodiment of the present application. DETAILED DESCRIPTION

[0063] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0064] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0065] In the description and claims of the embodiments of this application, as well as in the accompanying drawings, the terms "first," "second," and the like are used to distinguish similar objects (e.g., the first defect detection group and the second defect detection group are respectively different defect detection groups, the first threshold and the second threshold are respectively different thresholds, and so on), and are not necessarily used to describe a specific order or precedence. It should be understood that the processing modules used in this manner are interchangeable where appropriate, so that the embodiments described herein can be implemented in an order other than that shown or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or device that includes a series of steps or modules is not necessarily limited to those steps or modules that are clearly listed, but may include other steps or modules that are not clearly listed or that are inherent to these processes, methods, products, or devices. The division of modules that appears in the embodiments of the present application is merely a logical division. In actual applications, there may be other division methods. For example, multiple modules can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between each other shown or discussed can be through some interfaces, and the indirect coupling or communication connection between modules can be electrical or other similar forms, which are not limited in the embodiments of the present application. Moreover, the modules or submodules described as separate components may or may not be physically separated, may or may not be physical modules, or may be distributed in multiple circuit modules. Some or all of the modules can be selected according to actual needs to achieve the purpose of the embodiment of the present application.

[0066] It should be clear that in order to solve the technical problems in the background technology, this application proposes an embodiment of a surface defect detection method, specifically, Figure 1 As shown, the surface defect detection method includes:

[0067] Step S100: Acquire multiple sub-images based on the initial image, the translation window and the translation step.

[0068] Specifically, in the embodiments of the present application, the initial image is an image for surface defect detection. The initial image is pre-acquired, and the method for acquiring the initial image is not limited. For example, as described in the background, the initial image may be acquired using a device based on machine vision technology. Of course, the initial image may also be acquired using other image acquisition devices, such as a camera, video recorder, or mobile phone.

[0069] It should be clear that in the embodiment of the present application, the sub-image refers to a partial image belonging to the initial image. In the embodiment of the present application, the method of obtaining the sub-image from the initial image can be any method. Figure 2As shown, in one embodiment of the present application, assuming that an initial image 1 has been obtained, the initial image 1 can be directly divided equally into 9 sub-images 2. Of course, in other embodiments of the present application, other numbers of sub-images such as 4 or 16 can also be obtained, and of course, equal division is not required in other embodiments of the present application.

[0070] Since the larger initial image 1 is divided into multiple smaller sub-images 2, in subsequent surface defect detection, the smaller sub-images 2 can be detected, and then the detection results can be merged to obtain the surface defect detection results of the initial image 1.

[0071] like Figure 2 As shown, if the initial image 1 is directly segmented, a dividing line 3 will be formed between adjacent sub-images 2. As we know from the background art, in the application scenario of chip package surface defect detection, the smallest defect is less than 3 pixels. In other words, when the initial image 1 is segmented, some surface defects are likely to be located at the dividing line 3 between adjacent sub-images 2. In other words, if a small defect happens to fall at the dividing line 3, it may not be detected during the subsequent inspection process, resulting in inaccurate inspection results.

[0072] In order to ensure that all surface defects in the initial image 1 can be detected, in an embodiment of the present application, each sub-image is acquired by using a translation window and a translation step.

[0073] Specifically, such as Figure 3 As shown, the translation window refers to the window for acquiring sub-images. In other words, the size of each sub-image is equal to the size of the translation window. That is to say, in the embodiment of the present application, the length of the translation window needs to be less than the length of the initial image, and the height of the translation window needs to be less than the height of the initial image. The translation step size refers to the length of the translation window that moves each time a sub-image is acquired. It is easy to understand that if the translation step size is greater than the length or height of the translation window, it is impossible to sample all areas in the initial image. Therefore, in the embodiment of the present application, the translation step size of the translation window along the length direction of the initial image is less than or equal to the length of the translation window; the translation step size of the translation window along the height direction of the initial image is less than or equal to the height of the translation window.

[0074] like Figure 3 As shown, Figure 3This is a specific embodiment of obtaining each sub-image 2 from the initial image 1 by using the translation sampling method. In this embodiment, the length of the translation window is 1 / 2 of the length of the initial image 1, the height of the translation window is 1 / 2 of the height of the initial image 1, and the translation step is 1 / 4 of the length and 1 / 4 of the width of the initial image 1. Based on the above translation window size and translation step, a total of 9 sub-images 2 can be obtained, which are Figure 3 The shaded areas in a, b, c, d, e, f, g, h, and i are shown. Figure 3 The dotted boxes in a, b, c, d, e, f, g, h, and i represent the initial image 1. It should be understood that in the embodiment of the present application, the translation window size and the translation step size can be selected according to actual needs without any restrictions. In the embodiment of the present application, if the translation step size along the length direction of the initial image is equal to the length of the translation window, and the translation step size along the height direction of the initial image is equal to the height of the translation window, then the sub-images obtained are equivalent to the sub-images obtained by the direct segmentation method proposed above.

[0075] It should be noted that if the translation step length along the length of the initial image is less than the length of the translation window, and the translation step length along the height of the initial image is less than the height of the translation window, then the area where the segmentation line of each sub-image 2 is located can be fully reflected in the adjacent sub-image 2. In other words, after the initial image 1 is segmented, all defect information in the initial image 1 can be obtained through each sub-image 2, thereby improving the accuracy of the defect detection results.

[0076] It should be clear that in the embodiments of the present application, it is assumed that the length of the translation window is 1 / P of the length of the initial image; the height of the translation window is 1 / Q of the height of the initial image; the translation step along the length direction of the initial image is 1 / R of the translation window; and the translation step along the height direction of the initial image is 1 / T of the translation window. Then, based on the initial image, (P*R-1)*(Q*T-1) sub-images can be obtained. For example: in one embodiment of the present application, if the length of the translation window is 1 / 2 of the length of the initial image; the height of the translation window is 1 / 2 of the height of the initial image; the translation step along the length direction of the initial image is 1 / 2 of the translation window; and the translation step along the height direction of the initial image is 1 / 2 of the translation window, then based on the initial image, (2*2-1)*(2*2-1)=9 sub-images can be obtained. In one embodiment of the present application, if the length of the translation window is 1 / 2 of the length of the initial image; the height of the translation window is 1 / 2 of the height of the initial image; the translation step along the length direction of the initial image is 1 / 4 of the translation window; and the translation step along the height direction of the initial image is 1 / 4 of the translation window, then based on the initial image, (2*4-1)*(2*4-1)=49 sub-images can be obtained, and so on.

[0077] In an embodiment of the present application, if the size of the translation window is small, a large number of sub-images will be acquired subsequently. A large number of sub-images will result in a large amount of subsequent data processing. In order to enable the method in the embodiment of the present application to achieve a balance between the speed and accuracy of image processing, in an embodiment of the present application, the length of the translation window is less than or equal to 1 / 2 of the length of the initial image and greater than or equal to 1 / 10 of the length of the initial image. Specifically, the length of the translation window can be any one of 1 / 2, 1 / 3, 1 / 4, 1 / 5, 1 / 6, 1 / 7, 1 / 8, 1 / 9 and 1 / 10 of the length of the initial image, or any value between the two adjacent values mentioned above. Similarly, in an embodiment of the present application, the height of the translation window is less than or equal to 1 / 2 of the height of the initial image and greater than or equal to 1 / 10 of the height of the initial image. Specifically, the height of the translation window can be any one of 1 / 2, 1 / 3, 1 / 4, 1 / 5, 1 / 6, 1 / 7, 1 / 8, 1 / 9 and 1 / 10 of the initial image height, or any value between two adjacent values mentioned above.

[0078] In one embodiment of the present application, if the detection model of the deep convolutional neural network used subsequently can only recognize sub-images of a fixed size, that is, the size of the translation window needs to be fixed. In this case, we can pre-set the size of the translation window to a fixed size. Of course, in other embodiments of the present application, the detection model of the deep convolutional neural network used subsequently can recognize sub-images of a certain range of sizes, then the translation window and translation step size can be generated based on the initial image, so that the speed and accuracy of subsequent image processing can be balanced. For example: if the detection model of the deep convolutional neural network can recognize all images with a side length of 500 pixels to 1000 pixels, and the initial image has a side length greater than 1000 pixels, then the translation window can be designed to be any pixel between 500 pixels and 1000 pixels based on the data processing amount.

[0079] Step S200: Based on the multiple sub-images, a defect detection frame corresponding to each sub-image is obtained.

[0080] It should be understood that in the embodiments of this application, defect detection frames for each sub-image can be obtained using any existing method, without any limitation. Specifically, prior art primarily uses deep convolutional neural network detection models to perform defect detection on images. In other words, in the embodiments of this application, defect detection can be performed on each sub-image using any deep convolutional neural network detection model. For example, HALCON, HexSight, VisionPro, and LEADTOOLS can be used.

[0081] In a specific embodiment of the present application, a Faster RCNN detection model is used to perform defect detection on each sub-image. Specifically, the Faster RCNN detection model consists of a convolutional backbone network, a region proposal network (RPN), and a region classification network. Given a sub-image, the convolutional backbone network extracts features from it to obtain a feature map, which is then fed into the region proposal network to predict multiple defect detection boxes (bounding boxes, bboxes) that may contain defective areas. The defect detection boxes are then fed into the region classification network to obtain their defect categories and the location information of the defect detection boxes.

[0082] Step S300: Obtain defect detection results based on each defect detection frame.

[0083] Specifically, what is obtained in step S200 is the defect detection results of each sub-image. In step S300, the information of these defect detection results (i.e., each defect detection frame) needs to be fused to obtain the defect detection results of the initial image 1. It should be understood that if the above-mentioned translation sampling method is used to obtain the sub-images, the defect detection results of the initial image 1 obtained by subsequent fusion may be as follows: Figure 4 As shown, Figure 4 The initial image 1 in the figure has defect detection frames A, B, D, F, G, and H. In other words, when acquiring sub-images by using the translation sampling method, it is possible that small defects in the overlapping parts of multiple sub-images will be completely divided into multiple sub-images, thereby generating redundant detection results in the subsequent detection process. For example: Figure 3 As shown in the figure, sub-images a, b, and d share an overlapping area. If a defect is found in this overlapping area, it can be detected in sub-images a, b, and d during subsequent inspection. In other words, defects in this overlapping area can render subsequent defect detection results redundant.

[0084] In a specific embodiment of the present application, in order to reduce redundant detection results, obtaining defect detection results based on each defect detection frame includes:

[0085] Step S310: obtaining at least one defect detection group based on each defect detection frame.

[0086] Specifically, it is easy to understand that the defect labels of mutually redundant defect detection frames must be consistent. Defect detection frames with inconsistent defect labels must be non-redundant. The purpose of grouping the defect detection frames is to facilitate the determination and discovery of redundant defect detection frames for defect detection frames with the same defect labels. In other words, in the embodiment of the present application, each defect detection group obtained has at least one defect detection frame, and the defect labels of the defect detection frames in each defect detection group are the same. In the embodiment of the present application, the defect label refers to the type of surface defect detected, such as scratches, pits, bumps, discoloration or cracks, etc. Since this is a prior art, they are not listed one by one.

[0087] Of course, in other embodiments of the present application, step S310 may not be used to group the defect detection frames, but a traversal method may be used to compare two defect detection frames to find redundant defect detection frames.

[0088] Step S320: Acquire a first defect detection group.

[0089] As can be seen above, the purpose of grouping defect detection frames is to subsequently compare defect detection frames with the same defect label to identify redundant defect detection frames. The purpose of obtaining the first defect detection group is to compare the defect detection frames within the first defect detection group. In other words, the first defect detection group is the group of defect detection frames that has not undergone redundant comparison processing. Specifically, the steps from step S330 to step S340 or from step S330 to step S344 below are the redundant comparison processing steps proposed in the embodiments of this application.

[0090] Step S330: Obtain the confidence of each defect detection frame in the first defect detection group.

[0091] Specifically, in the embodiments of the present application, the confidence level of each defect detection frame refers to the reliability of the defect detection frame at its current location. This data can be obtained through a detection model using a deep convolutional neural network. Since this is a mature existing technology, we will not elaborate on it in detail.

[0092] Step S340: Based on the confidence of each defect detection frame, update the first defect detection grouping to obtain a current defect detection grouping.

[0093] Specifically, the purpose of performing redundant comparison processing is to delete defect detection frames that may be redundant. As mentioned above, if two defect detection frames are redundant with each other, then the two defect detection frames must intersect. Figure 4 As shown, if the defect detection frame A and the defect detection frame B are redundant with each other, then the defect detection frame A and the defect detection frame B must intersect, that is, an overlapping area C is formed.

[0094] In one embodiment of the present application, in order to delete redundant defect detection frames in the first defect detection group to obtain an updated first defect detection group, each defect detection frame in the first defect detection group is sorted by confidence, and each defect detection frame is compared with the defect detection frame with the highest confidence. If an overlapping area C is formed between the two, the defect detection frame with the lower confidence is deleted.

[0095] It should be noted that the risk of using the above method to update the first defect detection group is that if the overlapping area of the two defect detection frames is small, that is, it is very likely that the two defect detection frames are not redundant defect detection frames. Figure 4 As shown, the overlapping area of defect detection frame F and defect detection frame G is small, and it is very likely that these two defect detection frames are not redundant. Deleting defect detection frame F or defect detection frame G may lead to inaccurate subsequent defect detection results.

[0096] In another embodiment of the present application, in order to improve the accuracy of the defect detection result of the initial image 1, the updating of the first defect detection group based on the confidence of each defect detection frame and obtaining the current defect detection group includes:

[0097] Step S341: Based on the confidence of each defect detection frame, obtain a first defect detection frame.

[0098] Specifically, in the embodiment of the present application, the first defect detection frame refers to the defect detection frame with the highest confidence in the first defect detection group. Furthermore, the first defect detection frame is not subjected to interactive comparison processing with other defect detection frames in the first defect detection group. In the embodiment of the present application, the interactive comparison processing refers to the processing steps in steps S342 and S343.

[0099] Step S342: Based on the first defect detection frame, sequentially obtain the interaction ratios of each defect detection frame in the first defect detection group and the first defect detection frame in descending order of confidence.

[0100] Specifically, in the embodiment of the present application, it is necessary to calculate the interaction ratio between the first defect detection frame and each defect detection frame in the first defect detection group. The interaction ratio refers to the ratio of the area of the intersection of two regions to the area of the union of the two regions. Figure 4 As shown, the defect detection frame A and the defect detection frame B form an overlapping area C, and the interaction ratio of the defect detection frame A and the defect detection frame B is the ratio of the overlapping area C to the area of the defect detection frame A and the defect detection frame B minus the overlapping area C.

[0101] It is easy to understand that if the interaction ratio between defect detection frame A and defect detection frame B is larger, it means that the overlapping area C formed by defect detection frame A and defect detection frame B is larger. In other words, defect detection frame A or defect detection frame B is likely to be a redundant defect detection frame.

[0102] Step S343: if the interaction ratio of the defect detection frame is greater than the first threshold, the defect detection frame is deleted until all defect detection frames in the first defect detection group are interactively compared, then the first defect detection group is updated to obtain a second defect detection group.

[0103] Specifically, this method can effectively delete redundant defect detection frames with low confidence. In an embodiment of the present application, the first threshold can be pre-set according to actual needs. It is easy to understand that if the first threshold is set too large, it will be difficult to accurately exclude redundant defect detection frames. If it is set too small, non-redundant defect detection frames may be deleted. In an embodiment of the present application, after multiple experimental verifications by the applicant, the first threshold can be set to be greater than or equal to 0.60 and less than or equal to 0.80. In a specific embodiment of the present application, the first threshold can be any one of 0.60, 0.61, 0.62, 0.63, 0.64, 0.65, 0.66, 0.67, 0.68, 0.69, 0.70, 0.71, 0.72, 0.73, 0.74, 0.75, 0.76, 0.77, 0.78, 0.79 and 0.80, and of course it can also be any value between the above two adjacent values.

[0104] Step S344: Based on the second defect detection group, obtain the current defect detection group.

[0105] Specifically, in an embodiment of the present application, the second defect detection group can be directly used as the current defect detection group, and the surface defect detection result of the initial image is obtained based on the current defect detection group.

[0106] It should be noted that, in an embodiment of the present application, if a defect can be completely divided into one sub-image, but part of the defect is divided into other sub-images, it is necessary to delete the sufficient partial defect detection results. Therefore, in one embodiment of the present application, based on the second defect detection group, obtaining the current defect detection group includes:

[0107] Step S345: Based on the second defect detection group, obtain a second defect detection frame and a third defect detection frame.

[0108] As previously mentioned, the second defect detection group is derived based on the first defect detection group. That is, the defect labels of all defect detection frames in the second defect detection group are identical. Specifically, the second defect detection frame and the third defect detection frame are defect detection frames in the second defect detection group that have not undergone overlapping comparison processing. In other words, in this embodiment of the present application, overlapping comparison processing is required for all defect detection frames in the second defect detection group. In this embodiment of the present application, overlapping comparison processing refers to the processing steps from steps S346 to S347.

[0109] Step S346: Based on the second defect detection frame and the third defect detection frame, obtain an overlap ratio of the second defect detection frame and an overlap ratio of the third defect detection frame.

[0110] Specifically, in the embodiment of the present application, the overlap ratio refers to the ratio of the overlapping area formed between two trap detection frames to the area of the trap detection frame. Figure 4 As shown in the figure, defect detection frame A and defect detection frame B form an overlapping area C. The overlap ratio of defect detection frame A to defect detection frame B is the ratio of the area of overlapping area C to the area of defect detection frame A. Similarly, the overlap ratio of defect detection frame B to defect detection frame A is the ratio of the area of overlapping area C to the area of defect detection frame B. It is easy to understand that if the overlap ratio of defect detection frame A is high, it means that most of defect detection frame A is located in defect detection frame B. If the overlap ratio of defect detection frame B is high, it means that most of defect detection frame B is located in defect detection frame A.

[0111] Step S347: If the overlap ratio of the third defect detection frame is greater than the second threshold and the overlap ratio of the second defect detection frame is less than the second threshold, delete the third defect detection frame; if the overlap ratio of the second defect detection frame is greater than the second threshold and the overlap ratio of the third defect detection frame is less than the second threshold, delete the second defect detection frame.

[0112] As mentioned above, if the overlap ratio of the third defect detection frame is greater than the second threshold, and the overlap ratio of the second defect detection frame is less than the second threshold, it means that the majority of the third defect detection frame is located within the second defect detection frame. In other words, the defects in the third defect detection frame are likely part of the defects in the second defect detection frame. In other words, the third defect detection frame needs to be deleted. Similarly, if the overlap ratio of the second defect detection frame is greater than the second threshold, and the overlap ratio of the third defect detection frame is less than the second threshold, the second defect detection frame is deleted.

[0113] Specifically, through the above method, it is possible to effectively delete partial defect detection results that belong to the complete defect detection results, so as to improve the accuracy of subsequent defect detection results. In the embodiment of the present application, the second threshold can be pre-set according to actual needs. It is easy to understand that if the second threshold is set too large, it will be difficult to accurately exclude most of the partial defect detection frames. If it is set too small, other defect detection frames that do not belong to the partial defect detection frames may be deleted, making the detection results inaccurate. In the embodiment of the present application, after multiple experimental verifications by the applicant, the second threshold can be set to be greater than or equal to 0.60 and less than or equal to 0.80. In a specific embodiment of the present application, the second threshold value can be any one of 0.60, 0.61, 0.62, 0.63, 0.64, 0.65, 0.66, 0.67, 0.68, 0.69, 0.70, 0.71, 0.72, 0.73, 0.74, 0.75, 0.76, 0.77, 0.78, 0.79 and 0.80, and of course it can also be any value between two adjacent values mentioned above.

[0114] It is easy to understand that in the embodiment of the present application, after all defect detection frames in the second defect detection group are overlapped and compared, the second defect detection group is updated to obtain a third defect detection group.

[0115] Step S348: Based on the third defect detection group, obtain the current defect detection group.

[0116] Specifically, in one embodiment of the present application, the third defect detection group may be directly used as the current defect detection group.

[0117] As can be seen from the foregoing, if the sub-images are acquired by using the translation sampling method in the embodiment of the present application, a complete defect may be distributed among multiple sub-images. In other words, when the surface detection results of the initial image are subsequently acquired, multiple defect detection results may be obtained based on one surface defect. Figure 4 As shown, defect detection frame F, defect detection frame H and defect detection frame G are multiple defect detection frames (ie, defect detection results) formed based on the same defect.

[0118] In order to combine multiple defect detection results based on a surface defect into one defect detection result to improve the accuracy of the surface detection result of the initial image, in one embodiment of the present application, based on the third defect detection group, obtaining the current defect detection group includes:

[0119] Step S349: Based on the third defect detection group, a fourth defect detection frame and a fifth defect detection frame are obtained; the fourth defect detection frame and the fifth defect detection frame are defect detection frames in the third defect detection group that have not been merged and compared.

[0120] It's easy to understand that the defect labels of multiple defect detection frames generated based on the same defect are guaranteed to be consistent. In other words, if the defect labels of two defect detection frames are inconsistent, then these two defect detection frames are definitely not generated based on the same defect. In other words, in the embodiments of this application, only all defect detection frames with the same defect label need to be compared to determine whether they need to be merged.

[0121] As previously mentioned, the third defect detection group is obtained by updating the second defect detection group. In other words, the labels of the defect detection frames in the third defect detection group are consistent. In other words, it is sufficient to simply merge and compare the defect detection frames in the third defect detection group. Specifically, in the embodiment of the present application, this merging and comparison process refers to steps S3410 and S3411.

[0122] Step S3410: Based on the fourth defect detection frame and the fifth defect detection frame, obtain a merging ratio of the fourth defect detection frame and the fifth defect detection frame.

[0123] As mentioned above, if two defect detection frames are obtained based on a single surface defect, that is, if the two defect detection frames have the same defect label and are located close to each other (i.e., the merged image described below is larger), then it can be considered that the two defect detection frames are obtained based on a single surface defect.

[0124] Specifically, in the embodiment of the present application, the merging ratio refers to the ratio of the area formed by the two defect detection frames to the minimum circumscribed rectangle of the two defect detection frames. Figure 4 As shown in the figure, defect detection frame A and defect detection frame B form an overlapping area C, and the minimum bounding rectangle formed by defect detection frame A and defect detection frame B is rectangular frame E. Then the merging ratio of defect detection frame A and defect detection frame B is (area of defect detection frame A + area of defect detection frame B - area of overlapping area C) / area of rectangular frame E. Figure 4 As shown, the defect detection frame D and the defect detection frame M do not overlap, and the minimum circumscribed rectangle formed by the defect detection frame D and the defect detection frame M is the rectangular frame N. The merging ratio of the defect detection frame D and the defect detection frame M is (the area of the defect detection frame D + the area of the defect detection frame M) / the area of the rectangular frame N.

[0125] It should be noted that in one embodiment of the present application, to facilitate the subsequent determination of which defect detection frames need to be merged, all defect detection frames in the third defect detection group are assigned the same initial label value. Specifically, the initial label value can be any value, such as 0 or 1.

[0126] Step S3411: If the merging ratio of the fourth defect detection frame and the fifth defect detection frame is greater than the third threshold, the historical label values of the fourth defect detection frame and the fifth defect detection frame are updated, and the current label values of the fourth defect detection frame and the fifth defect detection frame are obtained.

[0127] It should be noted that, in the embodiment of the present application, all defect detection frames in the third defect detection group need to be compared with each other to determine whether two defect detection frames need to be merged. Figure 4 As shown, defect detection frame F, defect detection frame H and defect detection frame G are multiple defect detection frames formed based on the same defect, that is, defect detection frame F, defect detection frame H and defect detection frame G need to be merged into one defect detection frame.

[0128] In the embodiment of the present application, a label value is set for each defect detection frame to determine how each defect detection frame is subsequently merged. In the embodiment of the present application, if the merging ratio of the fourth defect detection frame and the fifth defect detection frame is greater than a third threshold, the historical label values of the fourth defect detection frame and the fifth defect detection frame are updated, and obtaining the current label values of the fourth defect detection frame and the fifth defect detection frame includes:

[0129] If the historical label values of the fourth defect detection frame and the fifth defect detection frame are equal to the initial label values, a fixed threshold is added to the historical label values of the fourth defect detection frame and the fifth defect detection frame, respectively. In the embodiment of the present application, the fixed threshold can be any value without any restriction, for example, the fixed threshold can be equal to 1 or 2. If there are defect detection frames in the fourth defect detection frame and the fifth defect detection frame whose historical label values are not equal to the initial label values, the current label values of the fourth defect detection frame and the fifth defect detection frame are set to be equal to the maximum of the two historical label values.

[0130] As can be seen above, each defect detection frame in the third defect detection group needs to be merged and compared with the other defect detection frames in the third defect detection group. In other words, the label value of each defect detection frame needs to be updated multiple times. In the embodiments of the present application, the current label value refers to the latest label value of the defect detection frame, while the historical label value refers to the label value of the defect detection frame before the current label value. In other words, when all defect detection frames undergo their first label value update, their historical label values are all initial label values.

[0131] In a specific embodiment of the present application, the initial tag value is defined as 0 and the fixed threshold is defined as 1. Figure 4 As shown, defect detection frame F is merged and compared with defect detection frame A. Since defect detection frame F is far away from defect detection frame A, that is, the merging ratio of defect detection frame F and defect detection frame A is less than the third threshold, the current label values of defect detection frame F and defect detection frame A are still equal to the initial label value 0. Defect detection frame F is then merged and compared with defect detection frame G. Since defect detection frame F is close to defect detection frame G, that is, the merging ratio of defect detection frame F and defect detection frame G is greater than the third threshold, and defect detection frame F and defect detection frame G are both equal to the initial label value, the current label values of defect detection frame F and defect detection frame G are equal to the initial label value + 1, that is, the current label values of defect detection frame F and defect detection frame G are both equal to 1. Defect detection frame F is then merged and compared with defect detection frame H. Since defect detection frame F is also close to defect detection frame H, that is, the merging ratio of defect detection frame F and defect detection frame H is greater than the third threshold, and the label value of defect detection frame F is not equal to the initial label value, the current label values of defect detection frame F and defect detection frame H are updated to the label value with the largest label value between defect detection frame F and defect detection frame H. In other words, the updated current label values of defect detection frame F and defect detection frame H are both 1.

[0132] It is easy to understand that after the merging process, if there is a defect detection frame whose current label value is equal to the initial label value, it means that the defect detection frame cannot be merged with any other defect detection frame. If the current label values of the defect detection frames are equal, all defect detection frames with the same current label values need to be merged into a single defect detection frame. In other words, after all the defect detection frames in the third defect detection group have been merged and compared, all defect detection frames with the same current label value are merged to obtain the fourth defect detection group. Figure 4 As shown, after the merging and comparison processing, the defect detection frame A and the defect detection frame B are merged to form a new defect detection frame E (that is, the rectangular frame E in the previous text); the defect detection frame F, the defect detection frame H and the defect detection frame G are merged to form a new defect detection frame J (that is, the rectangular frame J in the previous text).

[0133] Specifically, the above method can effectively solve the technical problem that a complete surface defect is divided into multiple defect detection results, which leads to inaccurate subsequent initial image surface defect detection results. In an embodiment of the present application, the third threshold value can be pre-set according to actual needs. It is easy to understand that if the third threshold value is set too large, it will be difficult to merge multiple surface defect detection results belonging to one surface defect into one surface defect detection result. If it is set too small, multiple surface defect detection results that do not belong to one surface defect may be merged into one surface defect detection result. In an embodiment of the present application, after multiple experimental verifications by the applicant, the second threshold value can be set to be greater than or equal to 0.70 and less than or equal to 0.80. In a specific embodiment of the present application, the second threshold value can be any one of 0.70, 0.71, 0.72, 0.73, 0.74, 0.75, 0.76, 0.77, 0.78, 0.79 and 0.80, and of course it can also be any value between the above two adjacent values.

[0134] Step S3412: Based on the fourth defect detection group, obtain the current defect detection group.

[0135] Specifically, in an embodiment of the present application, the fourth defect detection group obtained is the current defect detection group.

[0136] Step S350: Obtain defect detection results based on each defect detection frame in each current defect detection group.

[0137] The surface defect detection method proposed in this application divides a large initial image into multiple smaller sub-images, enabling existing surface defect detection methods to be applied to each of the smaller sub-images. After obtaining surface defect detection results for each sub-image, these results are fused to form the surface defect detection result for the initial image. This addresses the technical issue that existing surface defect detection methods are not applicable to surface defect detection of large initial images.

[0138] After introducing all the embodiments of the surface defect detection method of the present application, a surface defect detection device proposed in the present application is introduced below.

[0139] Specifically, such as Figure 5As shown, the surface defect detection device includes a processing module 12, which is used to obtain multiple sub-images based on an initial image, a translation window and a translation step; the length of the translation window is less than the length of the initial image, and the height of the translation window is less than the height of the initial image; the initial image is pre-acquired; the translation window and the translation step are pre-set or generated based on the initial image; the translation step of the translation window along the length direction of the initial image is less than or equal to the length of the translation window; the translation step of the translation window along the height direction of the initial image is less than or equal to the height of the translation window;

[0140] and, based on the plurality of sub-images, obtaining a defect detection frame corresponding to each sub-image;

[0141] And, based on each defect detection frame, a defect detection result is obtained.

[0142] In a specific embodiment of the present application, the length of the translation window is less than or equal to 1 / 2 of the length of the initial image and greater than or equal to 1 / 10 of the length of the initial image; the height of the translation window is less than or equal to 1 / 2 of the height of the initial image and greater than or equal to 1 / 10 of the height of the initial image.

[0143] In a specific embodiment of the present application, the processing module 12 is further configured to obtain at least one defect detection group based on each defect detection frame; each defect detection group has at least one defect detection frame, and each defect detection frame in each defect detection group has the same defect label;

[0144] The acquisition module 11 is further configured to acquire a first defect detection group; the first defect detection group is a group among the defect detection groups that has not undergone redundant comparison processing;

[0145] The acquisition module 11 is further configured to acquire the confidence level of each defect detection frame in the first defect detection group;

[0146] The processing module 12 is further configured to update the first defect detection group based on the confidence level of each defect detection frame to obtain a current defect detection group;

[0147] The processing module 12 is also used to obtain defect detection results based on each defect detection frame in each current defect detection group.

[0148] In a specific embodiment of the present application, the acquisition module 11 is further configured to acquire a first defect detection frame based on the confidence of each defect detection frame; the first defect detection frame is a defect detection frame in the first defect detection group that has not been subjected to interactive comparison processing and has the highest confidence;

[0149] The processing module 12 is further configured to obtain, based on the first defect detection frame, an interaction ratio between each defect detection frame in the first defect detection group and the first defect detection frame in descending order of confidence;

[0150] The processing module 12 is further configured to delete the defect detection frame if the interaction ratio of the defect detection frame is greater than a first threshold, until all defect detection frames in the first defect detection group are interactively compared, and then update the first defect detection group to obtain a second defect detection group, wherein the first threshold is preset;

[0151] The processing module 12 is further configured to obtain the current defect detection group based on the second defect detection group.

[0152] In a specific embodiment of the present application, the acquisition module 11 is further configured to acquire a second defect detection frame and a third defect detection frame based on the second defect detection group; the second defect detection frame and the third defect detection frame are defect detection frames in the second defect detection group that have not been subjected to overlapping comparison processing;

[0153] The processing module 12 is further configured to obtain an overlap ratio of the second defect detection frame and an overlap ratio of the third defect detection frame based on the second defect detection frame and the third defect detection frame;

[0154] The processing module 12 is further configured to delete the third defect detection frame if the overlap ratio of the third defect detection frame is greater than a second threshold and the overlap ratio of the second defect detection frame is less than the second threshold; delete the second defect detection frame if the overlap ratio of the second defect detection frame is greater than the second threshold and the overlap ratio of the third defect detection frame is less than the second threshold; and update the second defect detection group to obtain a third defect detection group after all defect detection frames in the second defect detection group have been subjected to overlap comparison processing, wherein the second threshold is preset;

[0155] The processing module 12 is further configured to obtain the current defect detection group based on the third defect detection group.

[0156] In a specific embodiment of the present application, the acquisition module 11 is further configured to acquire a fourth defect detection frame and a fifth defect detection frame based on the third defect detection group; the fourth defect detection frame and the fifth defect detection frame are defect detection frames in the third defect detection group that have not been merged and compared;

[0157] The processing module 12 is further configured to obtain a merging ratio of the fourth defect detection frame and the fifth defect detection frame based on the fourth defect detection frame and the fifth defect detection frame;

[0158] The processing module 12 is further configured to update historical label values of the fourth defect detection frame and the fifth defect detection frame if the combined ratio of the fourth defect detection frame and the fifth defect detection frame is greater than a third threshold, and obtain current label values of the fourth defect detection frame and the fifth defect detection frame; after all defect detection frames in the third defect detection group are combined and compared, combine all defect detection frames with the same current label value to obtain a fourth defect detection group, where the third threshold is preset;

[0159] The processing module 12 is further configured to obtain the current defect detection group based on the fourth defect detection group.

[0160] The surface defect detection device proposed in this application divides a large initial image into multiple smaller sub-images, enabling existing surface defect detection methods to be applied to each of the smaller sub-images. After obtaining surface defect detection results for each sub-image, these results are then fused to form the surface defect detection result for the initial image. This addresses the technical issue of existing surface defect detection methods being unsuitable for detecting surface defects in large initial images.

[0161] After introducing all the embodiments of the surface defect detection device of the present application, the following introduces a surface defect detection device proposed in the present application.

[0162] Specifically, such as Figure 6 As shown, the surface defect detection device includes a memory 21 and a processor 22, wherein the memory stores a computer program, and when the processor executes the computer program, an embodiment of the surface defect detection method as described in any one of the above is implemented.

[0163] After introducing all the embodiments of the surface defect detection device of the present application, the following introduces a computer-readable storage medium proposed in the present application.

[0164] Specifically, a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, an embodiment of the surface defect detection method as described in any one of the above is implemented.

[0165] It should be understood that computer-readable storage media in this application include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0166] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0167] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the methods, devices and equipment described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0168] In the several embodiments provided in the embodiments of the present application, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the modules is merely a logical function division. In actual implementation, there may be other division methods, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or modules, which can be electrical, mechanical or other forms.

[0169] The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules may be selected to achieve the purpose of the present embodiment according to actual needs.

[0170] In addition, the functional modules in each embodiment of the present application can be integrated into a processing module, or each module can exist physically separately, or two or more modules can be integrated into a module. The above-mentioned integrated modules can be implemented in the form of hardware or in the form of software functional modules. If the integrated modules are implemented in the form of software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium.

[0171] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware, or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0172] The computer program product includes one or more computer instructions. When the computer program is loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website, computer, server or data center to another website, computer, server or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that a computer can store or a data storage device such as a server or data center that includes one or more available media integrations. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive SolidStateDisk (SSD)).

[0173] Although the embodiments of the present application have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and variations may be made to these embodiments without departing from the principles and spirit of the present application, and the scope of the present application is defined by the appended claims and their equivalents.

Claims

1. A surface defect detection method, characterized in that: The method comprises: Based on an initial image, a translation window and a translation step, a plurality of sub-images are acquired; the length of the translation window is less than the length of the initial image, and the height of the translation window is less than the height of the initial image; the initial image is acquired in advance; the translation window and the translation step are preset or generated based on the initial image; the translation step of the translation window along the length direction of the initial image is less than or equal to the length of the translation window; the translation step of the translation window along the height direction of the initial image is less than or equal to the height of the translation window; Based on the multiple sub-images, a defect detection frame corresponding to each sub-image is obtained; Obtain defect detection results based on each defect detection frame; The obtaining of defect detection results based on each defect detection frame includes: Based on each defect detection frame, at least one defect detection group is obtained; each defect detection group has at least one defect detection frame, and each defect detection frame in each defect detection group has the same defect label; Obtaining a first defect detection group; the first defect detection group is a group that has not been subjected to redundant comparison processing among the defect detection groups; Obtaining the confidence of each defect detection frame in the first defect detection group; Based on the confidence of each defect detection frame, the first defect detection group is updated to obtain a current defect detection group; Obtaining defect detection results based on each defect detection frame in each current defect detection group; The updating of the first defect detection group based on the confidence of each defect detection frame to obtain the current defect detection group includes: Based on the confidence of each defect detection frame, a first defect detection frame is obtained; the first defect detection frame is a defect detection frame in the first defect detection group that has not been subjected to interactive comparison processing and has the highest confidence; Based on the first defect detection frame, sequentially obtaining, in descending order of confidence, an interaction ratio between each defect detection frame in the first defect detection group and the first defect detection frame; The interaction ratio refers to the ratio of the area of the intersection of two regions to the area of the union; If the interaction ratio of the defect detection frame is greater than a first threshold, the defect detection frame is deleted until all defect detection frames in the first defect detection group are interactively compared, and then the first defect detection group is updated to obtain a second defect detection group. The first threshold is preset. Based on the second defect detection group, obtaining the current defect detection group; The acquiring the current defect detection group based on the second defect detection group includes: Based on the second defect detection group, a second defect detection frame and a third defect detection frame are obtained; the second defect detection frame and the third defect detection frame are defect detection frames in the second defect detection group that have not been subjected to overlapping comparison processing; Based on the second defect detection frame and the third defect detection frame, obtaining an overlap ratio of the second defect detection frame and an overlap ratio of the third defect detection frame; The overlap ratio refers to the ratio of the overlap area formed between two defect detection frames to the area of the defect detection frames; If the overlap ratio of the third defect detection frame is greater than a second threshold and the overlap ratio of the second defect detection frame is less than the second threshold, the third defect detection frame is deleted; if the overlap ratio of the second defect detection frame is greater than the second threshold and the overlap ratio of the third defect detection frame is less than the second threshold, the second defect detection frame is deleted; after all defect detection frames in the second defect detection group have been overlapped and compared, the second defect detection group is updated to obtain a third defect detection group, and the second threshold is preset; Based on the third defect detection group, the current defect detection group is obtained.

2. The surface defect detection method according to claim 1, characterized in that: The length of the translation window is less than or equal to 1 / 2 of the length of the initial image and greater than or equal to 1 / 10 of the length of the initial image; the height of the translation window is less than or equal to 1 / 2 of the height of the initial image and greater than or equal to 1 / 10 of the height of the initial image.

3. The surface defect detection method according to claim 1, characterized in that: The acquiring the current defect detection group based on the third defect detection group includes: Based on the third defect detection group, a fourth defect detection frame and a fifth defect detection frame are obtained; the fourth defect detection frame and the fifth defect detection frame are defect detection frames in the third defect detection group that have not been merged and compared; acquiring a merging ratio of the fourth defect detection frame and the fifth defect detection frame based on the fourth defect detection frame and the fifth defect detection frame; If the merging ratio of the fourth defect detection frame and the fifth defect detection frame is greater than a third threshold, updating the historical label values of the fourth defect detection frame and the fifth defect detection frame, and obtaining the current label values of the fourth defect detection frame and the fifth defect detection frame; after all defect detection frames in the third defect detection group are merged and compared, merging all defect detection frames with the same current label value to obtain a fourth defect detection group, where the third threshold is preset; Based on the fourth defect detection group, the current defect detection group is obtained.

4. A surface defect detection device, characterized in that: The invention comprises a processing module and an acquisition module, wherein the processing module is used to acquire multiple sub-images based on an initial image, a translation window, and a translation step length; the length of the translation window is less than the length of the initial image, and the height of the translation window is less than the height of the initial image; the initial image is acquired in advance; the translation window and the translation step length are preset or generated based on the initial image; the translation step length of the translation window along the length direction of the initial image is less than or equal to the length of the translation window; The translation step length of the translation window along the height direction of the initial image is less than or equal to the height of the translation window; and, based on the plurality of sub-images, obtaining a defect detection frame corresponding to each sub-image; and, obtaining defect detection results based on each defect detection frame; The acquisition module is further configured to acquire a first defect detection group; the first defect detection group is a group among the defect detection groups that has not undergone redundant comparison processing; The acquisition module is further configured to acquire the confidence level of each defect detection frame in the first defect detection group; The processing module is further configured to update the first defect detection group based on the confidence level of each defect detection frame to obtain a current defect detection group; The processing module is further configured to obtain a defect detection result based on each defect detection frame in each current defect detection group; the obtaining module is further configured to obtain a first defect detection frame based on the confidence level of each defect detection frame; The first defect detection frame is a defect detection frame in the first defect detection group that has not been subjected to interactive comparison processing and has the highest confidence; The processing module is further configured to obtain, based on the first defect detection frame, an interaction ratio between each defect detection frame in the first defect detection group and the first defect detection frame in descending order of confidence; the interaction ratio being a ratio of an area of an intersection of two regions to an area of a union of two regions; The processing module is further configured to delete the defect detection frame if the interaction ratio of the defect detection frame is greater than a first threshold, until all defect detection frames in the first defect detection group are interactively compared, and then update the first defect detection group to obtain a second defect detection group, wherein the first threshold is preset; The processing module is further configured to obtain the current defect detection group based on the second defect detection group; the obtaining module is further configured to obtain a second defect detection frame and a third defect detection frame based on the second defect detection group; The second defect detection frame and the third defect detection frame are defect detection frames in the second defect detection group that have not been subjected to overlapping comparison processing; The processing module is further configured to obtain an overlap ratio of the second defect detection frame and an overlap ratio of the third defect detection frame based on the second defect detection frame and the third defect detection frame; the overlap ratio refers to a ratio of an overlapping area formed between two defect detection frames to an area of the defect detection frame; The processing module is further configured to delete the third defect detection frame if the overlap ratio of the third defect detection frame is greater than a second threshold and the overlap ratio of the second defect detection frame is less than the second threshold; If the overlap ratio of the second defect detection frame is greater than a second threshold, and the overlap ratio of the third defect detection frame is less than the second threshold, deleting the second defect detection frame; After all defect detection frames in the second defect detection group are overlapped and compared, the second defect detection group is updated to obtain a third defect detection group, wherein the second threshold is preset; The processing module is further configured to obtain the current defect detection group based on the third defect detection group.

5. The surface defect detection device according to claim 4, characterized in that: The length of the translation window is less than or equal to 1 / 2 of the length of the initial image and greater than or equal to 1 / 10 of the length of the initial image; the height of the translation window is less than or equal to 1 / 2 of the height of the initial image and greater than or equal to 1 / 10 of the height of the initial image.

6. A surface defect detection device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the surface defect detection method according to any one of claims 1 to 3 when executing the computer program.

7. A computer-readable storage medium, characterized in that A computer program is stored thereon, and when the computer program is executed by a processor, the surface defect detection method according to any one of claims 1 to 3 is implemented.

Images