Substrate punching quality detection method and device, electronic equipment and storage medium

Abstract

The invention discloses a substrate punching quality detection method and device, electronic equipment and a storage medium. The method comprises the steps that a to-be-detected image, placed on a detection platform, of a to-be-detected substrate is acquired; performing flat field correction processing on the to-be-detected image to obtain a target to-be-detected image; performing defect segmentation and defect type identification on the target to-be-detected image by using a preset deep learning twin segmentation model to obtain defect labeling information and defect type information; and acquiring substrate punching data of the to-be-detected substrate, and obtaining a punching quality detection result for the to-be-detected substrate based on the substrate punching data, the defect labeling information and the defect category information. In this way, the defect detection efficiency and accuracy can be improved, and then the accuracy of the punching quality detection result of the to-be-detected substrate is improved.

Description

Technical Field

[0001] This application relates to the technical field of data detection, and in particular to a method, device, electronic device and storage medium for detecting the quality of substrate punching. Background Art

[0002] In many industries with extremely high requirements for product quality and safety, such as manufacturing, aerospace, automotive, energy, etc., defect detection technology plays an important role and the application demand is increasing continuously. Taking the electronic information industry as an example, as a core component of electronic products, the quality of the glass substrate directly affects the performance and lifespan of the products, and is crucial in terminal devices such as smart phones, tablet computers, displays, etc. Therefore, there are high requirements for the glass substrate defect detection technology.

[0003] The existing methods for inspecting and classifying surface defects mainly rely on manual operation. Manual detection has low efficiency and accuracy, and completely fails to meet the usage requirements of the production line. At the same time, the manual detection method has the disadvantages of being time-consuming and laborious, low efficiency, cumbersome process, strong subjectivity, and being affected by factors such as personnel emotions, eyesight, environmental light, and inconsistent judgment criteria of different people's eyes.

[0004] Therefore, how to improve the defect detection efficiency and accuracy of the substrate punching production line is an urgent problem to be solved. Summary of the Invention

[0005] To solve the above problems, embodiments of the present application provide a method, device, electronic device, computer-readable storage medium and computer program product for detecting the quality of substrate punching.

[0006] In a first aspect, to solve the above technical problems, the present application provides a method for detecting the quality of substrate punching, including:

[0007] Obtain a to-be-detected image of the to-be-detected substrate placed on the detection platform;

[0008] Perform flat-field correction processing on the to-be-detected image to obtain a target to-be-detected image;

[0009] Use a preset deep learning Siamese segmentation model to perform defect segmentation and defect type recognition on the target to-be-detected image to obtain defect annotation information and defect category information;

[0010] Obtain the substrate punching data of the to-be-detected substrate, and obtain a punching quality detection result for the to-be-detected substrate based on the substrate punching data, the defect annotation information and the defect category information.

[0011] The beneficial effects are:

[0012] In the technical solution provided in the embodiment of the present application, after obtaining the image to be tested of the substrate to be tested placed on the detection platform, the image to be tested is subjected to flat field correction processing to obtain the target image to be tested; the target image to be tested is subjected to defect segmentation using a preset deep learning twin segmentation model to obtain defect annotation information; the substrate punching data of the substrate to be tested is obtained, and the punching quality detection result for the substrate to be tested is obtained based on the substrate punching data and the defect annotation information. In this way, the present application removes the influence of background unevenness on the detection through ordinary correction processing to obtain a uniform target image to be tested within the field of view, improves the defect detection efficiency and accuracy by improving the uniformity and clarity of the image to be tested, and uses a deep learning twin segmentation model to perform defect segmentation and type recognition on the target image to be tested, and each segmented defect is separately labeled to more accurately identify the hole defect, thereby improving the accuracy of the punching quality detection result of the substrate to be tested.

[0013] In a second aspect, the present invention provides a substrate drilling quality detection device, comprising an acquisition unit, a correction unit, a segmentation unit and a result unit;

[0014] An acquisition unit, used for acquiring an image of a substrate to be tested placed on a detection platform;

[0015] A correction unit, used for performing a flat field correction process on the image to be tested to obtain a target image to be tested;

[0016] A defect segmentation unit, used to perform defect segmentation and defect type identification on the target image to be tested by using a preset deep learning twin segmentation model to obtain defect labeling information and defect category information;

[0017] The result unit is used to obtain substrate drilling data of the substrate to be tested, and obtain a drilling quality detection result for the substrate to be tested based on the substrate drilling data, the defect marking information and the defect category information.

[0018] In a third aspect, the present application also provides an electronic device, comprising: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the electronic device implements the method for detecting the substrate punching quality as described above.

[0019] In a fourth aspect, the present application further provides a computer-readable storage medium having computer-readable instructions stored thereon, and when the computer-readable instructions are executed by a processor of a computer, the computer executes the method for detecting substrate punching quality as described above.

[0020] Fifth aspect, the present application further provides a computer program product or a computer program. The computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method for detecting the quality of substrate punching provided in the above various alternative embodiments.

[0021] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The accompanying drawings herein are incorporated into the specification and form a part of the specification, showing embodiments consistent with the present application, and are used together with the specification to explain the principles of the present application. Obviously, the accompanying drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts. In the drawings:

[0023] Figure 1 is a schematic diagram of an implementation environment related to the present application;

[0024] Figure 2 is a flowchart of a method for detecting the quality of substrate punching shown in an exemplary embodiment of the present application;

[0025] Figure 3 is a schematic diagram of the detection results of defects around holes, leakage holes, and hole contamination in an exemplary embodiment of the present application;

[0026] Figure 4 is a schematic diagram of template identifiers Mark1 and Mark2 in an exemplary embodiment of the present application;

[0027] Figure 5 is a comparison diagram of the to-be-measured image and the target to-be-measured image before and after flat-field correction processing in an exemplary embodiment of the present application;

[0028] Figure 6 is a schematic diagram of multiple grayscale images in an exemplary embodiment of the present application;

[0029] Figure 7 is the target to-be-measured image, reference image, and mask image input into the deep learning Siamese segmentation model in an exemplary embodiment of the present application;

[0030] Figure 8 is a schematic diagram of the architecture of the deep learning Siamese segmentation model in an exemplary embodiment of the present application;

[0031] Figure 9It is a block diagram of a device for detecting the quality of substrate punching shown in an exemplary embodiment of the present application;

[0032] Figure 10 It is a schematic structural diagram of a computer system of an electronic device suitable for implementing the embodiments of the present application. Detailed implementation manners

[0033] Here, the exemplary embodiments will be described in detail, and the examples are shown in the drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementation manners described in the following exemplary embodiments do not represent all implementation manners consistent with the present application. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0034] The block diagrams shown in the drawings are only functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0035] The flowcharts shown in the drawings are only for exemplary illustration, and do not necessarily include all contents and operations / steps, nor do they necessarily need to be executed in the described order. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined. Therefore, the actual execution order may be changed according to the actual situation.

[0036] As used in the present application, "a plurality of" means two or more. "And / or" describes the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. The character " / " generally represents an "or" relationship between the associated objects before and after.

[0037] In order to solve the problems of low detection efficiency and insufficient accuracy in the existing methods for inspecting and classifying surface defects due to the dependence on manual methods, the embodiments of the present application propose a method, device, electronic device, and computer-readable storage medium for detecting the quality of substrate punching, which mainly relate to the detection technology of substrate punching quality included in data detection technology. The following will describe these embodiments in detail.

[0038] First, please refer to Figure 1 , Figure 1 It is a schematic diagram of an implementation environment related to the present application. This implementation environment includes a detection platform 10 and a server 20, and the detection platform 10 and the server 20 communicate with each other through a wired or wireless network.

[0039] The detection platform 10 is used to carry the substrate to be tested, and the server 20 is used to obtain the image to be tested of the substrate to be tested placed on the detection platform; perform flat-field correction processing on the image to be tested to obtain the target image to be tested; use a preset deep learning Siamese segmentation model to perform defect segmentation on the target image to be tested to obtain defect annotation information; obtain the substrate punching data of the substrate to be tested, and obtain the punching quality detection result for the substrate to be tested based on the substrate punching data and the defect annotation information. Compared with the defect detection scheme for substrate punching in the prior art, the detection method for substrate punching quality provided in this implementation environment can improve the defect detection efficiency and accuracy of the substrate punching production line.

[0040] It should be noted that Figure 1 The server 20 in the shown implementation environment can be an independent server, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, Content Delivery Network (CDN), and big data and artificial intelligence platforms. No restrictions are imposed here.

[0041] Figure 2 is a flowchart of the detection method for substrate punching quality shown in an exemplary embodiment of the present application. This method can be applied to Figure 1 the shown implementation environment, and is specifically executed by Figure 1 the server 20 in the shown implementation environment. In other implementation environments, this method can be executed by devices in other implementation environments, and this embodiment does not limit this.

[0042] As Figure 2 shown, in an exemplary embodiment, the detection method for substrate punching quality may include steps S201 to S204, which are introduced in detail as follows:

[0043] Step S201, obtain the image to be tested of the substrate to be tested placed on the detection platform.

[0044] Step S202, perform flat-field correction processing on the image to be tested to obtain the target image to be tested.

[0045] Step S203, use a preset deep learning Siamese segmentation model to perform defect segmentation and defect type recognition on the target image to be tested to obtain defect annotation information and defect category information.

[0046] Step S204, obtain the substrate punching data of the substrate to be tested, and obtain the punching quality detection result for the substrate to be tested based on the substrate punching data, defect annotation information, and defect category information.

[0047] As described above, in the method provided in this embodiment, after obtaining the image to be measured of the substrate to be measured placed on the detection platform, the flat-field correction process is performed on the image to be measured to obtain the target image to be measured; the preset deep learning Siamese segmentation model is used to perform defect segmentation on the target image to be measured to obtain defect annotation information; the substrate drilling data of the substrate to be measured is obtained, and the drilling quality detection result for the substrate to be measured is obtained based on the substrate drilling data and the defect annotation information. In this way, in this application, the influence of uneven background on detection is removed through the flat-field correction process, and a uniform target image to be measured within the field of view is obtained. The defect detection efficiency and accuracy are improved by enhancing the uniformity and clarity of the image to be measured, and the deep learning Siamese segmentation model is used to perform defect segmentation and category recognition on the target image to be measured, and each segmented defect is respectively annotated to more accurately identify the hole defects, thereby improving the accuracy of the drilling quality detection result of the substrate to be measured.

[0048] The detection method provided in this embodiment can be used in a board-level TGV (Through Glass Via, which is a vertical electrical interconnection through a glass substrate) advanced packaging production line to quickly inspect and classify various surface defects in the glass substrate drilling, etching, developing, etching and other processes in the TGV process, meeting the high-precision and high-efficiency detection requirements of TGV packaging products. It can realize the detection and annotation of various defects such as abnormal aperture, position accuracy, roundness, missed drilling, multiple holes, pattern defects, surface defects (scratches, cracks, chipping, foreign objects, etc.) after substrate drilling, as Figure 3 shown, Figure 3 is a schematic diagram of the detection results of defects around the hole, missed holes, and hole contamination in an exemplary embodiment of this application. And the relevant substrate drilling data is output to form a drilling quality detection result.

[0049] In an exemplary embodiment provided by this application, when obtaining the image to be measured of the substrate to be measured placed on the detection platform, precise positioning is performed on the substrate to be measured, and its specific steps may include:

[0050] Obtain the actual coordinates of the to-be-measured identifier of the substrate to be measured in the platform coordinate system of the detection platform;

[0051] Obtain the offset information between the template coordinates and the actual coordinates of the template identifier of the detection platform;

[0052] Based on the offset information, perform position uniformity processing on the substrate to be measured, and move the substrate to be measured to the target position;

[0053] Obtain the image to be measured of the substrate to be measured at the target position.

[0054] In the embodiments provided by the present application, before performing hole punching quality detection on each substrate, a template identification is first constructed for precise positioning of the substrate to be measured. In this embodiment, the number of constructed template identifications is 2, which can be represented by Mark1 and Mark2. As Figure 4 shown, Figure 4 is a schematic diagram of the template identifications Mark1 and Mark2 in an exemplary embodiment of the present application. The steps for constructing the template identification are specifically as follows: First, on both sides of the wafer, two template identifications Mark1 and Mark2 are selected, and images are respectively taken with Mark1 and Mark2 as the centers. The relevant images are as Figure 4 shown. Then, using the ROI (region of interest), the coordinates and pixel position coordinates where the mark is located are marked. The relevant data is shown as follows.

[0055] {"index":0,"stageX":77.324,"stageY":106.0507,"imageX":1228,"imageY":979,"image W":99,"imageH":97,"filename":"F:\\Root\\Recipe\\Sandieji\\Alignkernel\\2.bmp","p osId":1},

[0056] {"index":1,"stageX":366.634,"stageY":106.0507,"imageX":1261,"imageY":966,"imag eW":97,"imageH":101,"filename":"F:\\Root\\Recipe\\Sandieji\\Alignkernel\\3.bmp","posId":2}]。

[0057] When the substrate to be measured is placed on the detection platform to perform hole punching quality detection, first, the actual coordinates of the to-be-measured identification of the substrate to be measured in the platform coordinate system of the detection platform are obtained. The number of to-be-measured identifications is the same as the number of template identifications. For ease of understanding, the number of to-be-measured identifications and the number of template identifications are assigned as 2 for description; then, the offset information between the template coordinates and the actual coordinates of the template identification of the detection platform is obtained; finally, based on the offset information, position unification processing is performed on the substrate to be measured, and the substrate to be measured is moved to the target position, and then the to-be-measured image of the substrate to be measured at the target position is obtained.

[0058] In another exemplary embodiment provided by the present application, the offset information includes an offset distance and an offset angle, and the specific steps of performing position unification processing on the substrate to be measured based on the offset information and moving the substrate to be measured to the target position may include:

[0059] Obtain the template angle and template distance between the preset template coordinates and the detection platform;

[0060] Based on the offset distance, offset angle, template angle, and template distance, calculate the moving distance and moving angle;

[0061] Perform position unification processing on the substrate to be measured based on the moving distance and moving angle, and move the substrate to be measured to the target position.

[0062] In this embodiment, the offset information includes an offset distance and an offset angle. By subtracting the template coordinates of the template identifier from the actual coordinates, the offset distances Delx1, Dely1 corresponding to the first identifier to be measured, the offset distances Delx2, Dely2 corresponding to the second identifier to be measured, and the offset angle delta_Theta between the actual coordinates and the template coordinates are obtained. The calculation formula is as follows.

[0063] Delx1 = X1_Test - X1_Model;

[0064] Dely1 = Y1_Test - Y1_Model;

[0065] Delx2 = X2_Test - X2_Model;

[0066] Dely2 = Y2_Test - Y2_Model;

[0067] Theta = acrtan((Dely2 - Dely1) / (Delx2 - Delx1)).

[0068] Wherein, X1_Test and Y1_Test are the actual coordinates of the first identifier to be measured, X2_Test and Y2_Test are the actual coordinates of the second identifier to be measured, X1_Model and Y1_Model are the template coordinates of the first template identifier, X2_Model and Y2_Model are the template coordinates of the second template identifier, and Theta is the offset angle between the line connecting the two identifiers to be measured and the line connecting the two template identifiers.

[0069] After obtaining the offset information between the to-be-tested identifier and the template identifier, first obtain the template angle and template distance between the preset template coordinates and the detection platform, that is, the angle and distance between the template identifier and the center of the wafer, such as the template distance R1 and template angle θ1 corresponding to one template identifier, and the template distance R2 and template angle θ2 corresponding to another template identifier.

[0070] After that, based on the offset distance, offset angle, template angle, and template distance, calculate the movement distance and movement angle. For example, for one to-be-tested identifier and the corresponding template identifier, the calculation formula is as follows:

[0071] R1cos(θ1 + Δθ1) - R1cos(θ1) = Δx1;

[0072] R1sin(θ1 + Δθ1) - R1sin(θ1) = Δy1.

[0073] Among them, Δθ1 is the movement angle, representing the angle difference between the template angle and the offset angle; Δx1 and Δy1 are the movement distances. Similarly, the movement distances and movement angles between another to-be-tested identifier and the corresponding template identifier can be obtained. Then, based on the movement distances and movement angles, perform position unification processing on the to-be-tested substrate, and move the to-be-tested substrate to the target position.

[0074] In this way, through the above embodiments of the present application, through the offset information between the preset template identifier and the to-be-tested identifier set on the to-be-tested substrate, perform position unification processing on the to-be-tested substrate, move the to-be-tested substrate to the target position, that is, move all to-be-tested substrates to the unified target position, improve the accuracy of the obtained to-be-tested images, and thus improve the accuracy of detection.

[0075] In an exemplary embodiment provided by the present application, the specific steps for performing flat-field correction processing on the to-be-tested image may include:

[0076] Obtain multiple grayscale images corresponding to the target to-be-tested image under preset multiple brightness values;

[0077] Obtain the grayscale mean value between the multiple grayscale images, and perform least squares fitting processing on the pixel points of the to-be-tested image based on the grayscale mean value to obtain a coefficient matrix;

[0078] Perform flat-field correction processing on the pixel points of the to-be-tested image based on the coefficient matrix to obtain the target to-be-tested image.

[0079] In this embodiment, after the substrate to be measured is placed on the detection platform, the server first searches for a flat area, sets different brightness values, so that the camera captures images in different grayscale modes, and obtains multiple grayscale images corresponding to the target image to be measured under a preset number of brightness values. Then, for the images in different grayscale modes, each pixel point is traversed, subtracted from the black image, the grayscale value of each grayscale image is obtained, and then the grayscale mean value of the grayscale values under different grayscales is calculated. Then, for all points in the image, least squares fitting is performed to obtain the coefficient matrices K and B. Finally, based on the coefficient matrix, flat field correction is performed on the pixel points of the image to be measured to obtain the target image to be measured, as Figure 5 shown Figure 5 Figure Figure 5 is a comparison diagram of the image to be measured and the target image to be measured before and after flat field correction in an exemplary embodiment of the present application.

[0080] Please refer to Figure 6 , Figure 6 which is a schematic diagram of multiple grayscale images in an exemplary embodiment of the present application. In this embodiment, after obtaining the grayscale image as shown in Figure 6 , the calculated coefficient matrix K of the two-dimensional matrix of 2560*2048 is: Figure 6 shown [[1.01607 1.0419 1.07846...1.07846 1.10761 1.08801] [1.02454 1.05987 1.05987...1.10761 1.06908 1.07846] [1.02454 1.0419 1.00774...1.11068 1.05081 1.08801] ... [1.05081 1.05987 1.02454...1.05987 1.07846 1.05081] [1.03315 1.05081 1.05987...1.05081 1.06908 1.08801] [1.02454 1.05987 1.09772...1.05081 1.03315 1.0419]]

[0088] The coefficient matrix B is:

[0089] [[-4.08349e-01 -8.13079e-01 -2.82376e+00...4.11615e-01 -2.21167e+00

[0090] 1.00334e+00]

[0091] [1.16655e+00 -7.41201e-01 -2.86094e+00...3.55612e-03 1.96835e+00

[0092] 1.49010e+00]

[0093] [1.16655e+00 -8.13079e-01 2.07352e+00...-1.62765e+00 5.00200e+00

[0094] -1.10268e+00] ...

[0096] [-1.30285e+00 -2.86094e+00 2.19108e+00...3.49825e+00 1.49010e+00

[0097] 7.98779e-01]

[0098] [7.01623e-01 -2.35367e+00 1.37851e+00...1.84957e+00 8.99281e-01

[0099] -1.10268e+00]

[0100] [1.16655e+00 3.18654e-01 -3.88296e+00...2.90037e+00 3.80109e+00

[0101] 2.31266e+00]]

[0102] In this way, through the above embodiments, the present application performs brightness equalization processing on the image through flat-field correction to obtain a test image with uniform brightness, improving the consistency of the test image so as to compare it with the template representation and complete the acquisition of offset information.

[0103] In an exemplary embodiment provided by the present application, the images input into the preset deep learning Siamese segmentation model include the target test image, the reference image and the mask image that match the threshold, and the specific steps for obtaining the defect annotation information and the defect category information may include:

[0104] Obtain a reference image and a mask image that match the target test image;

[0105] Input the target test image, the reference image and the mask image into the preset deep learning Siamese segmentation model to perform defect segmentation on the target test image to obtain defect annotation information, and the defect annotation information includes the defect position.

[0106] Please refer to Figure 7 , Figure 7 which are the target image to be measured, the reference image, and the mask image input into the deep learning Siamese segmentation model in an exemplary embodiment of the present application. In this embodiment, the recognition rate and accuracy of the model are improved by inputting the reference image, the target image to be measured, and the mask image.

[0107] In another exemplary embodiment provided by the present application, the deep learning Siamese segmentation model is a model improved based on the BiSeNet-V2 model, including a detail branch, a semantic branch, and an aggregation layer. The specific steps of processing the target image to be measured using the architecture of the deep learning Siamese segmentation model may include:

[0108] Using the deep learning Siamese segmentation model based on the BiSeNet-V2 model, input the target image to be measured into the detail branch and the semantic branch to obtain a first feature map and a second feature map;

[0109] Input the first feature map and the second feature map into the aggregation layer for aggregation processing to obtain an aggregated feature;

[0110] Perform defect segmentation and defect type recognition on the aggregated feature to obtain defect annotation information and defect category information. The defect annotation information includes the defect location.

[0111] In this embodiment, the deep learning Siamese segmentation model based on the BiSeNet-V2 model is adopted. By performing timeliness and accuracy tests on the DeepLabv3 and BiSeNet-V2 models, it can be known that when the batch size is 768 and the output layer has 3 layers (3-category output), the inference time of DeepLab v3 is 35ms, and that of BiSeNet-V2 is 24ms. After comparing with the ground truth, the integrity of defect segmentation of BiSeNet-V2 is also more excellent.

[0112] As Figure 8 shown, Figure 8 is a schematic diagram of the architecture of the deep learning Siamese segmentation model in an exemplary embodiment of the present application. Figure 8 In it, the deep learning Siamese segmentation model includes a detail branch, a semantic branch, and an aggregation layer. The upper half of the backbone network is the detail branch, and the lower half is the semantic branch. The detail branch utilizes a very limited number of convolutions, with a small number of convolutions but a large number of feature layers per convolution. In this way, these convolutional layers are mainly responsible for extracting high-resolution semantic information features; the semantic branch has a large number of convolutions, but a small number of feature layers per convolution, so that it can more quickly abstract deep and large-scale semantic information to obtain the first feature map and the second feature map corresponding to the target image to be measured.

[0113] After that, the aggregation layer performs operations such as mutual upsampling and downsampling on the first feature map and the second feature map, and then converges them together to perform defect segmentation and defect type recognition, obtaining defect annotation information and defect category information. The defect annotation information includes the defect location.

[0114] It should be noted that in the embodiments provided by this application, the deep learning Siamese segmentation model based on the BiSeNet-V2 model adopts a mechanism of simultaneous processing of 4 processes, that is, it can identify up to four target images to be measured simultaneously, improving the model efficiency. Further, for further optimization of the model efficiency, to avoid spending a long time performing a non-linear mapping of the activation function on the output result by the CPU after the output of the neural network, the code of the activation function is changed to the inside of the model and processed by the GPU. In this way, when the model operation outputs, the final result can be directly obtained. To change the code, only need to write the sigmoid function for the activation.fn function: net = slim.conv2d(net, num_classes, [1, 1], activation_fn = nn.sigmoid, scope = 'logits'). This slight modification directly reduces the original speed of 100ms to about 70ms. For example, when there are 16 small images to be processed in the whole image, it saves a total of 480ms of time consumption, having a very obvious optimization effect.

[0115] In addition, when performing deep learning with the deep learning Siamese segmentation model based on the BiSeNet-V2 model, the deep learning deployment parameters are as follows:

[0116]

[0117]

[0118] The deep learning training parameters are as follows:

[0119] In this way, through the above embodiments of this application, the deep learning Siamese segmentation model based on the BiSeNet-V2 model is used to extract low-contrast defects, making up for the robustness problem of traditional algorithms.

[0120] In an exemplary embodiment provided by this application, the specific steps to obtain the punching quality detection result for the substrate to be measured may include:

[0121] Detect the substrate punching data at the punching position of the substrate to be measured, and the substrate punching data includes roundness and aperture;

[0122] The punching quality detection result for the substrate to be measured is obtained based on the roundness, hole diameter, defect annotation information, and defect category information.

[0123] In this embodiment, while obtaining the defect annotation information and defect category information, the substrate punching data at the punching position of the substrate to be measured is detected. The substrate punching data includes roundness and hole diameter to form a complete punching quality detection result for subsequent processing.

[0124] Figure 9 It is a block diagram of a detection device 900 for the punching quality of a substrate shown in an exemplary embodiment of the present application. As Figure 9 shown, the device includes:

[0125] An acquisition unit 901, configured to acquire a to-be-measured image of the substrate to be measured placed on a detection platform;

[0126] A correction unit 902, configured to perform flat-field correction processing on the to-be-measured image to obtain a target to-be-measured image;

[0127] A defect segmentation unit 903, configured to perform defect segmentation and defect type recognition on the target to-be-measured image by using a preset deep learning Siamese segmentation model to obtain defect annotation information and defect category information;

[0128] A result unit 904, configured to acquire the substrate punching data of the substrate to be measured, and obtain the punching quality detection result for the substrate to be measured based on the substrate punching data, defect annotation information, and defect category information.

[0129] This device applies the detection method for the punching quality of the substrate provided by the present application. After the acquisition unit 901 acquires the to-be-measured image of the substrate to be measured placed on the detection platform, the correction unit 902 performs flat-field correction processing on the to-be-measured image to obtain a target to-be-measured image; the defect segmentation unit 903 performs defect segmentation on the target to-be-measured image by using a preset deep learning Siamese segmentation model to obtain defect annotation information; the result unit 904 acquires the substrate punching data of the substrate to be measured, and obtains the punching quality detection result for the substrate to be measured based on the substrate punching data and defect annotation information. In this way, the present application removes the influence of uneven background on detection through flat-field correction processing, obtains a uniform target to-be-measured image within the field of view, improves the defect detection efficiency and accuracy by enhancing the uniformity and clarity of the to-be-measured image, and uses a deep learning Siamese segmentation model to perform defect segmentation and type recognition on the target to-be-measured image, and labels each segmented defect respectively to more accurately identify hole defects, thereby improving the accuracy of the punching quality detection result of the substrate to be measured.

[0130] In another exemplary embodiment, the acquisition unit 901 is further configured to acquire the actual coordinates of the to-be-tested identifier of the to-be-tested substrate in the platform coordinate system of the detection platform; acquire the offset information between the template coordinates and the actual coordinates of the template identifier of the detection platform; perform position unification processing on the to-be-tested substrate based on the offset information, and move the to-be-tested substrate to the target position; acquire the to-be-tested image of the to-be-tested substrate at the target position.

[0131] In another exemplary embodiment, the offset information includes an offset distance and an offset angle; the acquisition unit 901 is further configured to acquire the template angle and the template distance between the preset template coordinates and the detection platform; calculate the movement distance and the movement angle based on the offset distance, the offset angle, the template angle, and the template distance; perform position unification processing on the to-be-tested substrate based on the movement distance and the movement angle, and move the to-be-tested substrate to the target position.

[0132] In another exemplary embodiment, the correction unit 902 is further configured to acquire a plurality of grayscale images corresponding to the target to-be-tested image under a plurality of preset brightness values; acquire the grayscale mean value between the plurality of grayscale images, and perform least squares fitting processing on the pixel points of the to-be-tested image based on the grayscale mean value to obtain a coefficient matrix; perform flat-field correction processing on the pixel points of the to-be-tested image based on the coefficient matrix to obtain the target to-be-tested image.

[0133] In another exemplary embodiment, the defect segmentation unit 903 is further configured to acquire a reference image and a mask image that match the target to-be-tested image; input the target to-be-tested image, the reference image, and the mask image into a preset deep learning Siamese segmentation model to perform defect segmentation and defect type recognition on the target to-be-tested image, and obtain defect annotation information and defect category information, where the defect annotation information includes defect positions.

[0134] In another exemplary embodiment, the deep learning Siamese segmentation model includes a detail branch, a semantic branch, and an aggregation layer; the defect segmentation unit 903 is further configured to use the deep learning Siamese segmentation model based on the BiSeNet-V2 model, input the target to-be-tested image into the detail branch and the semantic branch to obtain a first feature map and a second feature map; input the first feature map and the second feature map into the aggregation layer for aggregation processing to obtain an aggregated feature; perform defect segmentation and defect type recognition on the aggregated feature to obtain defect annotation information and defect category information, where the defect annotation information includes defect positions.

[0135] In another exemplary embodiment, the result unit 904 is further configured to detect the substrate drilling data at the drilling position of the to-be-tested substrate, where the substrate drilling data includes roundness and aperture; obtain a drilling quality detection result for the to-be-tested substrate based on the roundness, the aperture, the defect annotation information, and the defect category information.

[0136] It should be noted that the substrate punching quality detection device provided in the above embodiments and the substrate punching quality detection method provided in the above embodiments belong to the same concept. The specific ways in which each module and unit perform operations have been described in detail in the method embodiments, and will not be elaborated here. In practical applications, the substrate punching quality detection device provided in the above embodiments can, according to needs, allocate the above functions to different functional modules, that is, divide the internal structure of the device into different functional modules to complete all or part of the functions described above. This is not limited here either.

[0137] Embodiments of the present application also provide an electronic device, including: one or more processors; a storage device for storing one or more programs, which, when executed by the one or more processors, cause the electronic device to implement the substrate punching quality detection methods provided in the above various embodiments.

[0138] Figure 10 The structural schematic diagram of a computer system of an electronic device suitable for implementing the embodiments of the present application is shown. It should be noted that Figure 10 The computer system 1000 of the electronic device shown is only an example and should not impose any limitations on the functions and usage scope of the embodiments of the present application.

[0139] As Figure 10 shown, the computer system 1000 includes a central processing unit (CPU) 1001, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 1002 or the program loaded from the storage section 1008 into the random access memory (RAM) 1003, such as executing the method in the above embodiments. In the RAM 1003, various programs and data required for system operation are also stored. The CPU 1001, ROM 1002, and RAM 1003 are connected to each other through a bus 1004. The input / output (I / O) interface 1005 is also connected to the bus 1004.

[0140] The following components are connected to the I / O interface 1005: an input part 1006 including a keyboard, a mouse, etc.; an output part 1007 including, for example, a cathode ray tube (CRT), a liquid crystal display (LCD), etc. and a speaker, etc.; a storage part 1008 including a hard disk, etc.; and a communication part 1009 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication part 1009 performs communication processing via a network such as the Internet. The drive 1010 is also connected to the I / O interface 1005 as needed. A removable medium 1011, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1010 as needed so that a computer program read from it can be installed into the storage part 1008 as needed.

[0141] Specifically, according to an embodiment of the present application, the processes described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains a computer program for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 1009, and / or installed from the removable medium 1011. When the computer program is executed by the central processing unit (CPU) 1001, various functions defined in the system of the present application are executed.

[0142] It should be noted that the computer-readable medium shown in the embodiments of the present application may be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. A computer-readable storage medium may be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable computer program. Such a propagated data signal may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0143] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present application. Among them, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code, and the above module, program segment, or part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram or flowchart, and the combination of blocks in the block diagram or flowchart, may be implemented by a dedicated hardware-based system for performing the specified functions or operations, or may be implemented by a combination of dedicated hardware and computer instructions.

[0144] The units involved in the embodiments of the present application can be implemented in software or in hardware, and the described units can also be provided in a processor. Among them, the names of these units do not constitute a limitation to the unit itself in some cases.

[0145] Another aspect of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the method for detecting the quality of substrate punching as described above is implemented. The computer-readable storage medium can be included in the electronic device described in the above embodiments, or can exist alone without being assembled into the electronic device.

[0146] Another aspect of the present application further provides a computer program product or a computer program. The computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method for detecting the quality of substrate punching provided in the above embodiments.

[0147] The above are only the preferred embodiments of the present application, and are not intended to limit the present application. Any modifications, equivalent replacements, or improvements made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for detecting the quality of substrate punching, characterized in that, The method includes: Obtaining a to-be-tested image of the to-be-tested substrate placed on a detection platform; Performing flat-field correction processing on the to-be-tested image to obtain a target to-be-tested image; Using a preset deep learning siamese segmentation model to perform defect segmentation and defect type recognition on the target to-be-tested image, obtaining defect annotation information and defect category information; Obtaining the substrate punching data of the to-be-tested substrate, and obtaining a punching quality detection result for the to-be-tested substrate based on the substrate punching data, the defect annotation information, and the defect category information.

2. The method according to claim 1, characterized in that, The obtaining a to-be-tested image of the to-be-tested substrate placed on a detection platform includes: Obtaining the actual coordinates of the to-be-tested identifier of the to-be-tested substrate in the platform coordinate system of the detection platform; Obtaining the offset information between the template coordinates of the template identifier of the detection platform and the actual coordinates; Performing position unification processing on the to-be-tested substrate based on the offset information, and moving the to-be-tested substrate to a target position; Obtaining a to-be-tested image of the to-be-tested substrate at the target position.

3. The method according to claim 2, wherein The offset information includes an offset distance and an offset angle; The performing position unification processing on the to-be-tested substrate based on the offset information and moving the to-be-tested substrate to a target position includes: Obtaining a template angle and a template distance between the preset template coordinates and the detection platform; Calculating a moving distance and a moving angle based on the offset distance, the offset angle, the template angle, and the template distance; Performing position unification processing on the to-be-tested substrate based on the moving distance and the moving angle, and moving the to-be-tested substrate to a target position.

4. The method according to claim 1, characterized in that, The performing flat-field correction processing on the to-be-tested image to obtain a target to-be-tested image includes: Obtaining a plurality of grayscale images corresponding to the target to-be-tested image under a plurality of preset brightness values; Obtaining the grayscale mean value between the plurality of grayscale images, and performing least squares fitting processing on the pixel points of the to-be-tested image based on the grayscale mean value to obtain a coefficient matrix; Performing flat-field correction processing on the pixel points of the to-be-tested image based on the coefficient matrix to obtain a target to-be-tested image.

5. The method according to claim 1, wherein The using a preset deep learning siamese segmentation model to perform defect segmentation and defect type recognition on the target to-be-tested image, obtaining defect annotation information and defect category information includes: Obtaining a reference image and a mask image matching the target to-be-tested image; Inputting the target to-be-tested image, the reference image, and the mask image into a preset deep learning siamese segmentation model to perform defect segmentation and defect type recognition on the target to-be-tested image, obtaining defect annotation information and defect category information, where the defect annotation information includes defect positions.

6. The method according to any one of claims 1 or 5, characterized in that The deep learning siamese segmentation model includes a detail branch, a semantic branch, and an aggregation layer; The using a preset deep learning siamese segmentation model to perform defect segmentation and defect type recognition on the target to-be-tested image, obtaining defect annotation information and defect category information includes: Using a deep learning siamese segmentation model based on the BiSeNet-V2 model, inputting the target to-be-tested image into the detail branch and the semantic branch to obtain a first feature map and a second feature map; Input the first feature map and the second feature map into the aggregation layer for aggregation processing to obtain an aggregated feature; Perform defect segmentation and defect type recognition on the aggregated feature to obtain defect annotation information and defect category information, where the defect annotation information includes defect positions.

7. The method according to claim 1, wherein The method for obtaining the substrate drilling data of the substrate to be measured and obtaining the drilling quality detection result for the substrate to be measured based on the substrate drilling data, the defect annotation information, and the defect category information includes: Detect the substrate drilling data at the drilling position of the substrate to be measured, where the substrate drilling data includes roundness and aperture; Obtain the drilling quality detection result for the substrate to be measured based on the roundness, the aperture, the defect annotation information, and the defect category information.

8. A detection device for the quality of substrate punching, characterized in that, It includes: An acquisition unit for acquiring a to-be-measured image of the substrate to be measured placed on a detection platform; A correction unit for performing flat-field correction processing on the to-be-measured image to obtain a target to-be-measured image; A defect segmentation unit for using a preset deep learning Siamese segmentation model to perform defect segmentation and defect type recognition on the target to-be-measured image to obtain defect annotation information and defect category information; A result unit for obtaining the substrate drilling data of the substrate to be measured and obtaining the drilling quality detection result for the substrate to be measured based on the substrate drilling data, the defect annotation information, and the defect category information.

9. An electronic device, characterized in that, It includes: One or more processors; A storage device for storing one or more programs, which when executed by the one or more processors, cause the electronic device to implement the method for detecting the drilling quality of the substrate as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, A computer-readable instruction is stored thereon, which when executed by a processor of a computer, causes the computer to execute the method for detecting the drilling quality of the substrate as described in any one of claims 1 to 7.

Images