Defect detection method, device and equipment based on multi-channel scanning and storage medium

Abstract

The invention discloses a defect detection method, device and equipment based on multi-channel scanning and a storage medium, and relates to the field of AOI image detection.The method comprises the steps that a channel graph of a material plate is obtained, and PCS particle images in the channel graph are extracted through a CAD mother graph; performing alignment correction on the PCS particle image according to a PCS contour line in the CAD mother layout, and generating a PCS detection image according to a target detection area of the PCS particle image; after defect detection, performing label fusion and label mapping on defects according to types; and receiving a selection operation on a target defect, and displaying a defect channel graph on a defect display interface according to the PCS particle number and defect label mapping relationship. According to the scheme, the target detection areas are selected according to the channel types and uploaded to the corresponding detection models, the different detection models only carry out defect detection on the respective target detection areas, and the detected defect labels are fused and then displayed. Therefore, regional more accurate defect detection can be realized, and the detection precision is improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of AOI image detection, and in particular to a defect detection method, device, equipment and storage medium based on multi-channel scanning. Background Art

[0002] Wafer processing is complex and closely linked. Quality inspection is required for each step from wafer substrate design to final etching. This allows timely discovery of quality problems in wafer products, so that process adjustments can be made or wafers can be scrapped to stop losses in a timely manner. Defect detection requires advanced inspection machines, advanced inspection algorithms, and a complete testing and re-inspection process.

[0003] In the related technology, some manufacturers design machine production lines, and the inspection modules need to be equipped with various scanning devices and light source devices during defect recognition detection to complete the refined modeling of the material board. However, some defects are affected by the background of the material board and the color of the components, and will show clear features under high-brightness light sources, such as dark-colored dirt. Some defect types show clear features under low-brightness light sources, such as bright-colored scratches. Therefore, when dealing with such multi-type defect detection projects, one is to use equipment with higher computing power and algorithm models with higher detection performance for detection, and the other is to shoot different imaging effects for the same material, and perform detection and defect matching calibration separately through different detection models. The former solution requires more cost, and the detection accuracy and accuracy cannot be guaranteed. The latter solution requires repeated machine operations, especially the defect matching calibration of multiple images will reduce the efficiency of detection. Summary of the invention

[0004] The embodiments of the present application provide a defect detection method, device, equipment and storage medium based on multi-channel scanning to solve the problem.

[0005] On the one hand, the present application provides a defect detection method based on multi-channel scanning, the method comprising:

[0006] Scan the channel map of the material board under different optical channels by using the same line scan camera, and extract the PCS particle image in the channel map using the CAD master map of the material board;

[0007] Performing alignment correction on the PCS particle image according to the PCS contour line in the CAD master image, and generating a PCS detection image according to the target detection area of ​​the PCS particle image under different optical channels; the PCS detection images corresponding to different optical channels contain different target detection areas;

[0008] Push the PCS inspection image to the target detection model for defect detection, and perform label fusion and label mapping on all defects according to their types;

[0009] In response to receiving a selection operation for a target defect, a defective PCS image is displayed on a defect display interface according to the mapping relationship between the PCS particle number and the defect label; the defective PCS image is marked with the selected target defect.

[0010] Specifically, the extracting of the PCS particle image in the channel image using the CAD master image of the material board includes:

[0011] Identify a tray in the channel map, and select at least two image inflection point contours on the acupuncture point material in the tray;

[0012] Mapping the CAD master image based on the image inflection point contour, and aligning based on the mapped contour and template matching algorithm;

[0013] The PCS lines of the CAD master drawing are mapped to the channel drawing, the PCS contours of the acupoints are located and the PCS particle images are extracted, and a channel PCS drawing is obtained according to the positional relationship; the channel PCS drawing displays each PCS particle image according to position distribution.

[0014] Specifically, the extracted PCS particle image is corrected, including:

[0015] The image inflection point contours of each PCS particle image in the channel PCS map are selected one by one, and mapped to the mapping inflection point contours of the target PCS line on the CAD master map;

[0016] Based on the mapping relationship between the mapping inflection point contour and the image inflection point contour, a template matching algorithm is used to perform angle and position correction on the PCS particle image to determine the posture adjustment amount;

[0017] Based on the matching relationship between the channel PCS graph and the attitude adjustment amount, all channel graphs are subjected to batch alignment correction processing.

[0018] Specifically, the PCS detection image is pushed to the target detection model for defect detection, and all defects are labeled and mapped according to their types, including:

[0019] Determine the target area and non-target area of ​​the PCS particle image, set an opaque mask layer for the non-target area, and generate a fused area image;

[0020] Pushing the fused area image to the corresponding target detection model for defect detection, and outputting defect results and defect labels;

[0021] The detection results of the PCS particle images under all channels are labeled and summarized, and the same defect labels detected in the same target area are fused according to the confidence and defect area.

[0022] Specifically, after label fusion and label mapping are performed on all defects according to types, the method further includes:

[0023] Map the defective PCS in the CAD master drawing according to the defect label, and display the defective PCS contour on the defect display interface; wherein the normal PCS contour and the defective PCS contour are marked with different color borders respectively;

[0024] In response to receiving a click operation on a target defect PCS contour in the defect PCS contour map, a defect PCS information box is displayed, in which defect images detected under all channels are displayed.

[0025] Specifically, when the material board is inspected on both sides, the front side contour map and the back side contour map are displayed in the defective PCS contour map, and the PCS contour is framed and marked according to the side where the defect position is located.

[0026] Specifically, in response to receiving a selection operation on a defective PCS in the channel image, a defect display box is displayed; the defect display box displays a defective PCS image and a defect information display area; the defect information display area lists defect label information corresponding to all channel image detections of the defective PCS.

[0027] On the other hand, the present application provides a defect detection device based on multi-channel scanning, the device comprising:

[0028] An image extraction module is used to scan the channel map of the material board under different optical channels through the same line scan camera, and use the CAD master map of the material board to extract the PCS particle image in the channel map;

[0029] A region mapping module, used for performing alignment correction on the PCS particle image according to the PCS contour line in the CAD master image, and generating a PCS detection image according to the target detection area of ​​the PCS particle image under different optical channels; the PCS detection images corresponding to different optical channels contain different target detection areas;

[0030] A defect detection module is used to push the PCS detection image to the target detection model for defect detection, and to perform label fusion and label mapping on all defects according to their types;

[0031] The defect display module is used to respond to receiving a selection operation for a target defect and display a defective PCS image on a defect display interface according to the mapping relationship between the PCS particle number and the defect label; the defective PCS image is marked with the selected target defect.

[0032] On the other hand, the present application provides a computer device, comprising a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set are loaded and executed by the processor to implement the defect detection method based on multi-channel scanning described in the above aspects.

[0033] On the other hand, the present application provides a computer-readable storage medium, which stores at least one instruction, at least one program, a code set or an instruction set. The at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by a processor to implement the defect detection method based on multi-channel scanning described in the above aspects.

[0034] The beneficial effects brought about by the technical solution provided by the embodiment of the present application include at least: the present application uses different optical channels to shoot the channel map of the material board under the same viewing angle in view of the image clarity characteristics of different defects under different light environments, and uses the CAD master map to perform overall alignment correction for the selected channel map, and each PCS image in the image is also aligned and corrected using the CAD contour map. In the defect detection stage, the target detection area is selected according to the channel type in units of PCS images and uploaded to the corresponding detection model. Different detection models only perform defect detection on their respective target detection areas, and the defect labels after detection are then fused and displayed. In this way, more accurate defect detection in different regions can be achieved, thereby improving detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 is a flow chart of a defect detection method based on multi-channel scanning provided in an embodiment of the present application;

[0036] Figure 2 is a flow chart of a defect detection method based on multi-channel scanning provided in an embodiment of the present application;

[0037] Figure 3 It is a schematic diagram of alignment correction of PCS image by contour line;

[0038] Figure 4 This is a possible schematic diagram of dividing the target detection area;

[0039] Figure 5 A schematic diagram showing a possible defect display interface;

[0040] Figure 6 A schematic diagram of a CAD master drawing of image inflection point contour mapping is shown;

[0041] Figure 7 A schematic diagram showing the mapping of CAD contour lines according to the image inflection point contour of the PCS particle image is shown;

[0042] Figure 8 A schematic diagram of dividing a target area by multiple channels and generating a fused area image is shown;

[0043] Fig. 9 It is a schematic diagram of multi-channel and regional label fusion;

[0044] Fig.10 A possible defect display interface is a schematic diagram showing a defect PCS contour map;

[0045] Fig.11 It is a schematic diagram of the PCS contour diagram of the double-sided display defect;

[0046] Fig.12 This is a possible schematic diagram of a defective PCS information box;

[0047] Fig.13 is a schematic diagram of the defective PCS particle image;

[0048] Fig.14 A structural block diagram of a defect detection device based on multi-channel scanning is shown;

[0049] Fig.15 A structural block diagram of a computer device provided by an exemplary embodiment of the present application is shown. DETAILED DESCRIPTION

[0050] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0051] The term "multiple" as used herein refers to two or more than two. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the related objects are in an "or" relationship.

[0052] Figure 1 : is a flow chart of a defect detection method based on multi-channel scanning provided in an embodiment of the present application, comprising the following steps:

[0053] S1. Scan the channel map of the material board under different optical channels by the same line scan camera, and use the CAD master map of the material board to extract the PCS particle image in the channel map;

[0054] The scanning stroke of the line scan camera is usually fixed, so the image content of the channel image obtained after switching different light sources to scan the material board is the same, the only difference is that the light source causes different pixel differences in the image. The light source and number of scanning channels used in this step are determined according to the various parameters of the material board and the defect type and other characteristics.

[0055] Due to the accuracy of the robot, machine, and tray, the material board or PCS particles may have different degrees of positional offset when placed, and this offset will be reflected in all channel diagrams. This application mainly implements position correction through CAD master diagram.

[0056] Figure 2 It is a schematic diagram of the CAD master drawing corresponding to the material board. The CAD master contains the position and contour information of all PCS particles and trays on the material board. When the material board is tilted as a whole, it can be aligned and corrected by matching the line information of the CAD master drawing, and then the PCS particle image can be identified and extracted.

[0057] S2, performing alignment correction on the PCS particle image according to the PCS contour line in the CAD master image, and generating a PCS detection image according to the target detection area of ​​the PCS particle image under different optical channels;

[0058] Traditional defect detection can be performed by segmenting the input after a single linear scan, but this application involves multi-channel images, and the method of identifying and segmenting one by one will seriously reduce the detection efficiency. Therefore, this application does not operate in a way that occupies computing resources such as AI image recognition, but is performed by matching and extracting the master image. Because the size, position, and angle of the PCS contour lines in the CAD master image are strictly designed, when the material board is offset after production, only one of the channel images is used as the base for operation, and the correction parameters are summarized. All other channel images are executed synchronously according to this parameter to achieve batch operation.

[0059] Figure 3 This is a schematic diagram of alignment correction of PCS image with contour line. In order to extract accurate PCS particle image, after completing channel map matching, the PCS contour line in CAD master map is mapped to the channel map, and alignment is achieved by matching the CAD contour line with the image contour in the channel image. Its purpose is to identify the structure of the image PCS, such as the gold surface area, text area, head and tail, etc. If the PCS particle itself is offset, alignment correction operations such as translation and rotation can be performed, and then the PCS particle image can be extracted, because defect detection is performed on a PCS particle basis.

[0060] After the extraction is completed, the target detection area can be divided according to the optical channel attributes. Due to the influence of factors such as the material and color of PCS particles, defects in different locations can be more easily identified under the corresponding optical channel. Therefore, different target areas need to be set for the same PCS particle under different channels.

[0061] Figure 4 It is a schematic diagram of a possible division of the target detection area. It is assumed that a PCS particle image is planned into three detection areas according to the defect type and possible distribution, namely detection area A, detection area B, and detection area C. The three detection areas are specifically used to detect specific defect types. Among them, detection area A is dedicated to the extraction and detection of channel images taken in the first optical channel scene, detection area B is dedicated to the extraction and detection of channel images taken in the first optical channel scene, and detection area C is dedicated to the extraction and detection of channel images taken in the first optical channel scene. Although it is the same PCS particle, the detection areas are set up separately in three channels for inspection. Of course, for some special scenes and special defects, the corresponding PCS detection images under different optical channels may contain overlapping target detection areas.

[0062] S3: Push the PCS inspection image to the target detection model for defect detection, and perform label fusion and label mapping on all defects according to their types;

[0063] The number of defect detection models at least covers all possible channel types. The model can be located in the cloud server, and the PCS detection images of all channel types are pushed to the cloud through offline segmentation and settings. After selecting the corresponding target detection model, the detection output is performed. The model output results include all defect thumbnails and matching defect labels identified from the PCS detection image (target detection area). Further, these labels are classified and fused according to type and mapped back to the target PCS particle image of the original channel image.

[0064] S4. In response to receiving a selection operation for a target defect, a defective PCS image is displayed on a defect display interface according to a mapping relationship between a PCS particle number and a defect label.

[0065] In a possible implementation, the inspection results are displayed through a visual display interface, and the defect label content is queried and displayed through human-computer interaction. Figure 5A schematic diagram of a possible defect display interface is shown, and the defect display interface 500 includes at least a complete image display area 510 and a defect display area 520. The image display area 510 lists channel images or CAD master images 511. The CAD master image 511 displays several PCS line outlines 511. The defect display area 520 lists a defect grid image 521, and the defect grid image 521 displays a defect image 522 based on the mapping relationship between the PCS particle number and the defect label. For example, when channel 1 is selected, the defect grid image 521 displays all the defect thumbnails 522 detected and intercepted from the PCS of the channel 1 image. At the same time, the grid also lists specific label information, including defect type, confidence value, defect area, and PCS number. Figure 5 In the image, the defects listed in channel 1 are mainly CVL deviation defects, because this defect is easiest to detect under channel 1 image, and the defects displayed in the image are more intuitive to observe with the naked eye, with higher detection accuracy, and no obvious light and dark stripes. Similarly, other defects are detected and displayed using specific channel images and specific models. Further, through these interface contents, operators can perform targeted inspection, repair, re-inspection, and scrapping operations on material boards.

[0066] In summary, this application uses different optical channels to shoot channel images of material boards under the same viewing angle in view of the image clarity characteristics of different defects under different light environments, and uses the CAD master image to perform overall alignment correction for the selected channel image, and each PCS image in the image is also aligned and corrected using the CAD contour image. In the defect detection stage, the PCS image is used as a unit, the target detection area is selected according to the channel type, and uploaded to the corresponding detection model. Different detection models only perform defect detection on their respective target detection areas, and the defect labels after detection are fused and displayed. In this way, more accurate defect detection in different regions can be achieved, thereby improving detection accuracy.

[0067] In some embodiments, for S1, using the CAD master image of the material board to extract the PCS particle image in the channel image may specifically include the following steps:

[0068] S11, identifying a tray in the channel diagram, and selecting at least two image inflection point contours on the acupuncture point material in the tray;

[0069] The tray is a device for storing material plates. The corresponding contour map is also drawn in CAD. Its purpose is to place the robot in the correct position when grabbing. However, the accuracy of CAD master image matching is pixel-level. It is inevitable that there will be offsets due to machine errors, so the initial state needs to be initially calibrated using a tray. This application uses the contour information of the acupuncture point material in the tray for matching. Specifically, a matching algorithm logic based on edge direction gradient can be used, and the gradient correlation of the object edge is used as the matching standard. The principle is: to extract the edge contour features in the ROI, combine the grayscale information to create a template, and generate a multi-level image pyramid model according to the template size and clarity requirements. Then search the template image layer by layer from top to bottom in the image pyramid model until the bottom layer is searched or the matching result position is determined.

[0070] S12, mapping the CAD master image based on the image inflection point contour, and aligning based on the mapped contour and template matching algorithm;

[0071] Figure 6 A schematic diagram of the image inflection point contour mapping CAD master is shown. Figure 6 Select two contour inflection points near PCS 4 and PCS24, map them to the CAD master, and calculate the CAD inflection point lines that meet the distance similarity, contour similarity, and position similarity in CAD through the algorithm. When the result output by the template matching algorithm meets the threshold condition, the alignment is completed, and then the channel image can be rotated, translated, and other alignment operations can be performed on the entire channel image.

[0072] S13, mapping the PCS lines of the CAD master drawing to the channel drawing, locating the PCS contour of the acupoint and extracting the PCS particle image, and obtaining the channel PCS drawing according to the position relationship; and displaying each PCS particle image according to the position distribution in the channel PCS drawing.

[0073] This step is to efficiently extract the PCS particle image. Traditionally, AI image recognition is used, but the shapes and sizes of PCS particles vary greatly, and the images extracted by pixel recognition have non-PCS edges of a certain width, which interferes with subsequent defect detection. Therefore, this application also uses CAD line mapping to extract. The previous feature images have been aligned, so the CAD lines of the PCS particles and PCS have been matched. The next step is to map them directly to the channel map, and then extract the PCS particle image. This step minimizes the computing power resource usage, and there is no need to use AI recognition and matching models, and the image extraction efficiency is also higher. All PCS particle images are combined according to the actual position and angle to form a channel PCS map in the form of an array.

[0074] In some embodiments, for S2, the extracted PCS particle image is subjected to alignment correction, and the specific steps may include the following:

[0075] S21, successively selecting the image inflection point contours of each PCS particle image in the channel PCS diagram, and mapping them to the mapping inflection point contours of the target PCS line on the CAD master diagram;

[0076] Figure 7 The figure shows a schematic diagram of mapping CAD contour lines based on the image inflection point contour of the PCS particle image; one or more contour inflection points can be selected, and OpenCV or AI model algorithms can be used for training and identification matching, and then the template matching algorithm can be used for alignment. Figure 7 Taking the inflection point contour selected in as an example, the two line contours are unique and complex. Associating and mapping the two can improve the accuracy of alignment.

[0077] S22, based on the mapping relationship between the mapping inflection point contour and the image inflection point contour, use a template matching algorithm to perform angle and position correction on the PCS particle image to determine the posture adjustment amount;

[0078] Angle and position correction is mainly to take into account the position offset when the PCS particles are installed. At this time, there will inevitably be deviations from the CAD contour line. At this time, the image should be rotated to ensure that the two contour lines coincide as much as possible. After the coincidence, the rotation offset angle parameter of the image can be determined, that is, the attitude adjustment amount.

[0079] S23, based on the matching relationship between the channel PCS graph and the posture adjustment amount, batch alignment correction processing is performed on all channel graphs.

[0080] The aforementioned channel PCS map is set for batch operation, because it contains all PCS particle images, and after calculating the attitude adjustment amount of each PCS particle image for the theoretical CAD line contour in S23, the attitude adjustment amount of the entire channel PCS map is obtained, and the attitude adjustment amount is applied to other channel maps at the same time, so that the PCS particle images in all channel maps can be extracted and corrected in batches.

[0081] In some embodiments, for S3, the PCS detection image is pushed to the target detection model for defect detection, and all defects are labeled and mapped according to their types, which may specifically include the following steps:

[0082] S31, determining the target area and non-target area of ​​the PCS particle image, setting an opaque mask layer for the non-target area, and generating a fused area image;

[0083] The target area is determined relative to the channel type, because defects may exist in various areas of a complete PCS particle map, and this application adopts directional specific defect recognition, so those that are not in this target detection area should be filtered out, and in order to unify the input and output of the detection model, a mask layer is added for coverage. No detection will be performed on the mask layer area when pushing the detection later, further reducing the system computing power requirements.

[0084] Figure 8 The figure shows a schematic diagram of dividing the target area and generating a fused area image by multiple channels. The complete PCS particle image is divided into three areas. PCS channel 1 only performs defect detection on area A, PCS channel 2 only performs defect detection on area B, and PCS channel 3 performs defect detection on areas B and C. In the fused area image obtained by the PCS particle image under the three channels, mask layers of different colors can be set according to needs to enhance the contrast effect.

[0085] Furthermore, the presentation effect of specific defects in pixels can be further adjusted by setting parameters, such as configuring different types of labels in a unique channel as much as possible, but the same label may also be configured in the same two channels, and the specific setting is based on the recognition accuracy of the defect type.

[0086] S32, pushing the fused area image to the corresponding target detection model for defect detection, and outputting defect results and defect labels;

[0087] S33, labeling the detection results of the PCS particle images under all channels, and fusing the same defect labels detected in the same target area according to the confidence and defect area.

[0088] Fig. 9 This is a schematic diagram of multi-channel label fusion in the same area. Label aggregation is mainly for the case where the same defect is detected in the same area of ​​multiple channels. For example, all three channels set area A as the target detection area. Specifically, area A of channel 1 outputs label 1 and label 3, area A of channel 1 outputs label 1, label 2 and label 4, and area A of channel 3 outputs label 2 and label 5. Because area A is detected together, the recognition results can be aggregated.

[0089] In particular, if both channel 2 and channel 3 output label 2 defects, it is necessary to further perform judgment fusion based on confidence and defect area. In some embodiments, labels with high confidence or large defect area can be determined as output labels. Fig. 9 The defect area of ​​label 2 in middle channel 3 is larger, so it is output as the final fusion result.

[0090] In some embodiments, after all defects are labeled and mapped according to their types, they need to be visualized so that operators can more intuitively check and trust the subsequent process flow. Specifically, it may include the following:

[0091] A. Map the defect PCS in the CAD master drawing according to the defect label, and display the defect PCS contour map on the defect display interface;

[0092] Fig.10 This is a schematic diagram of a possible defect display interface showing a defect PCS contour map. The left side of the defect display interface 1000 shows a channel map 1010 of a material board scanned under a specific optical channel, and the right side shows a matching defect PCS contour map 1020. Different from the original CAD template map, the normal PCS contour and the defect PCS contour in the defect PCS contour map 1020 are marked with different color borders. For example Fig.10 The medium defect PCS1021 is outlined with a red frame (normal PCS is not outlined with a green frame), and the number of detected defects (labels) can also be displayed in the frame. Fig.10 PCS1 number detected 20 defects in a certain channel.

[0093] In particular, for some special material boards that require double-sided inspection, when the material board is double-sided inspected, the front and back contours are displayed in the defect PCS contour map, and the PCS contour is framed and marked according to the side where the defect location is located. Fig.11 It is a schematic diagram of the PCS contour diagram of the double-sided display defect.

[0094] B. In response to receiving a click operation on the target defect PCS contour in the defect PCS contour map, a defect PCS information box is displayed, in which defect images detected under all channels are displayed.

[0095] exist Fig.10 On this basis, the label information can be integrated on the defect PCS contour map. When the target defect PCS contour is clicked, the defect PCS information box will be displayed.

[0096] Fig.12 This is a possible schematic diagram of a defective PCS information box. For a certain defective PCS, all defect images in the defective PCS information box are displayed in a rasterized form. When the operator clicks on the defect thumbnail, a pop-up window can be displayed to enlarge the defect image for the purpose of re-inspection process.

[0097] In the above display scheme, the complete single PCS particle image is not displayed, which is not conducive to locating the defect location. Therefore, in order to facilitate observation and location of defects, a complete defect PCS image can also be displayed based on the channel image.

[0098] Fig.13 is a schematic diagram of the defective PCS particle image. Fig.10 On the basis of, in response to receiving the selection operation of the defective PCS in the channel image, a defect display box 1300 is displayed. The defect display box 1300 displays a defective PCS image 1310 and a defect information display area 1320. In the defect information display area 1320, defect label information 1330 corresponding to all channel image detections of the defective PCS is listed. The defect label information can specifically include "gold out-of-plane 100 (channel 1)", "gold surface scratch 20 (channel 2)", "gold crush 100 (channel 3)", "gold foreign matter 20 (channel 1)", "gold leak 100 (channel 2)", and "gold surface nickel leak 20 (channel 1)". The number after the text represents the pixel area, and the content in brackets notes the channel source. The defective PCS image 1310 can be displayed using any channel image, and the defect boxes of all selected labels are marked in the image to facilitate the operator to observe and determine.

[0099] Fig.14 The structure block diagram of the defect detection device based on multi-channel scanning provided by an embodiment of the present application is shown, and the device includes:

[0100] An image extraction module 1410 is used to scan the channel images of the material board under different optical channels by using the same line scan camera, and extract the PCS particle image in the channel image by using the CAD master image of the material board;

[0101] The area mapping module 1420 is used to perform alignment correction on the PCS particle image according to the PCS contour line in the CAD master image, and generate a PCS detection image according to the target detection area of ​​the PCS particle image under different optical channels; the PCS detection images corresponding to different optical channels contain different target detection areas;

[0102] The defect detection module 1430 is used to push the PCS detection image to the target detection model for defect detection, and perform label fusion and label mapping on all defects according to their types;

[0103] The defect display module 1440 is used to respond to receiving a selection operation for a target defect and display a defective PCS image on a defect display interface according to the mapping relationship between the PCS particle number and the defect label; the defective PCS image is marked with the selected target defect.

[0104] The defect detection device based on multi-channel scanning provided in the embodiment of the present application can be applied to the defect detection method based on multi-channel scanning provided in the above embodiment. For relevant details, refer to the above method embodiment. The implementation principle and technical effect are similar and will not be repeated here.

[0105] It should be noted that the defect detection device based on multi-channel scanning provided in the embodiment of the present application is only illustrated by the division of the above-mentioned functional modules / functional units. In actual applications, the above-mentioned functions can be assigned to different functional modules / functional units as needed, that is, the internal structure of the defect detection device based on multi-channel scanning is divided into different functional modules / functional units to complete all or part of the functions described above. In addition, the implementation method of the defect detection method based on multi-channel scanning provided in the above method embodiment and the implementation method of the defect detection device based on multi-channel scanning provided in this embodiment belong to the same concept. The specific implementation process of the defect detection device based on multi-channel scanning provided in this embodiment is detailed in the above method embodiment, which will not be repeated here.

[0106] Fig.15 The block diagram of a computer device provided by an exemplary embodiment of the present application is shown. It is a computer device such as a desktop computer, a laptop computer, a PDA, and a cloud server. The computer device may include, but is not limited to, a processor and a memory. Among them, the processor and the memory may be connected by a bus or in other ways. Among them, the processor may be a central processing unit (CPU). The processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, graphics processors (GPU), embedded neural network processors (NPU) or other dedicated deep learning coprocessors, discrete gates or transistor logic devices, discrete hardware components and other chips, or a combination of the above-mentioned various chips.

[0107] The processor may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content that needs to be displayed on the display screen. In some embodiments, the processor may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0108] As a non-transient computer-readable storage medium, the memory can be used to store non-transient software programs, non-transient computer executable programs and modules, such as program instructions / modules corresponding to the methods in the above-mentioned embodiments of the present application. The processor executes various functional applications and data processing of the processor by running the non-transient software programs, instructions and modules stored in the memory, that is, the method in the above-mentioned method implementation is realized. The memory may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created by the processor, etc. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network and a combination thereof.

[0109] In some embodiments, the computer device may further optionally include: a peripheral device interface and at least one peripheral device. The processor, the memory and the peripheral device interface may be connected via a bus or a signal line. Each peripheral device may be connected to the peripheral device interface via a bus, a signal line or a circuit board. Specifically, the peripheral device includes: at least one of a radio frequency circuit, a display screen and a keyboard.

[0110] The peripheral device interface can be used to connect at least one peripheral device related to I / O (Input / Output) to the processor and the memory. In some embodiments, the processor, the memory, and the peripheral device interface are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor, the memory, and the peripheral device interface can be implemented on a separate chip or circuit board, which is not limited in this embodiment.

[0111] The display screen is used to display the UI (User Interface). The UI may include graphics, text, icons, videos and any combination thereof. When the display screen is a touch screen, the display screen also has the ability to collect touch signals on the surface or above the surface of the display screen. The touch signal can be input to the processor as a control signal for processing. At this time, the display screen can also be used to provide virtual buttons and / or virtual keyboards, also known as soft buttons and / or soft keyboards. In some embodiments, the display screen can be one, arranged on the front panel of the computer device; in other embodiments, the display screen can be at least two, respectively arranged on different surfaces of the computer device or in a folding design; in other embodiments, the display screen can be a flexible display screen, arranged on a curved surface or a folding surface of the computer device. Even, the display screen can be set to a non-rectangular irregular shape, that is, a special-shaped screen. The display screen can be made of materials such as LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode) and the like.

[0112] The power supply is used to power various components in the computer device. The power supply can be AC, DC, a disposable battery, or a rechargeable battery. When the power supply includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. A wired rechargeable battery is a battery that is charged through a wired line, and a wireless rechargeable battery is a battery that is charged through a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0113] Those skilled in the art will appreciate that the structure shown in this embodiment does not constitute a limitation on the computer device, and may include more or fewer components than shown in the figure, or combine certain components, or adopt a different component arrangement.

[0114] The embodiment of the present application also discloses a computer-readable storage medium. Specifically, the computer-readable storage medium is used to store a computer program, and when the computer program is executed by the processor, the method in the above-mentioned method implementation is implemented. Those skilled in the art can understand that the implementation of all or part of the process in the above-mentioned implementation method of the present application can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the process of the implementation of the above-mentioned methods. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory (Flash Memory), a hard disk (Hard Disk Drive, abbreviated as: HDD) or a solid-state drive (SSD), etc.; the storage medium can also include a combination of the above-mentioned types of memory.

[0115] This specific embodiment is merely an explanation of the present invention and is not a limitation of the present invention. After reading this specification, those skilled in the art may make non-creative modifications to the present embodiment as needed. However, as long as they are within the scope of the claims of the present invention, they are protected by the patent law.

Claims

1. A defect detection method based on multi-channel scanning, characterized in that: The method comprises: Scan the channel map of the material board under different optical channels by using the same line scan camera, and extract the PCS particle image in the channel map using the CAD master map of the material board; Performing alignment correction on the PCS particle image according to the PCS contour line in the CAD master image, and generating a PCS detection image according to the target detection area of ​​the PCS particle image under different optical channels; the PCS detection images corresponding to different optical channels contain different target detection areas; Push the PCS inspection image to the target detection model for defect detection, and perform label fusion and label mapping on all defects according to their types; In response to receiving a selection operation for a target defect, a defective PCS image is displayed on a defect display interface according to the mapping relationship between the PCS particle number and the defect label; the defective PCS image is marked with the selected target defect.

2. The method according to claim 1, characterized in that The method of extracting the PCS particle image in the channel image using the CAD master image of the material plate includes: Identify a tray in the channel map, and select at least two image inflection point contours on the acupuncture point material in the tray; Mapping the CAD master image based on the image inflection point contour, and aligning based on the mapped contour and template matching algorithm; The PCS lines of the CAD master drawing are mapped to the channel drawing, the PCS contours of the acupoints are located and the PCS particle images are extracted, and a channel PCS drawing is obtained according to the positional relationship; the channel PCS drawing displays each PCS particle image according to position distribution.

3. The method according to claim 2, characterized in that The step of performing alignment correction on the extracted PCS particle image comprises: The image inflection point contours of each PCS particle image in the channel PCS map are selected one by one, and mapped to the mapping inflection point contours of the target PCS line on the CAD master map; Based on the mapping relationship between the mapping inflection point contour and the image inflection point contour, a template matching algorithm is used to perform angle and position correction on the PCS particle image to determine the posture adjustment amount; Based on the matching relationship between the channel PCS graph and the attitude adjustment amount, all channel graphs are subjected to batch alignment correction processing.

4. The method according to claim 3, characterized in that The PCS detection image is pushed to the target detection model for defect detection, and all defects are labeled and mapped according to their types, including: Determine the target area and non-target area of ​​the PCS particle image, set an opaque mask layer for the non-target area, and generate a fused area image; Pushing the fused area image to the corresponding target detection model for defect detection, and outputting defect results and defect labels; The detection results of the PCS particle images under all channels are labeled and summarized, and the same defect labels detected in the same target area are fused according to the confidence and defect area.

5. The method according to claim 2, characterized in that: After label fusion and label mapping are performed on all defects according to their types, the method further includes: Map the defective PCS in the CAD master drawing according to the defect label, and display the defective PCS contour on the defect display interface; wherein the normal PCS contour and the defective PCS contour are marked with different color borders respectively; In response to receiving a click operation on a target defect PCS contour in the defect PCS contour map, a defect PCS information box is displayed, in which defect images detected under all channels are displayed.

6. The method according to claim 5, characterized in that When the material board is inspected on both sides, the front side contour map and the back side contour map are displayed in the defective PCS contour map, and the PCS contour is framed and marked according to the side where the defect position is located.

7. The method according to claim 6, characterized in that In response to receiving a selection operation of a defective PCS in the channel image, a defect display frame is displayed; the defect display frame displays a defective PCS image and a defect information display area; The defect information display area lists the defect label information of all channel image detections corresponding to the defect PCS.

8. A defect detection device based on multi-channel scanning, characterized in that: The device comprises: An image extraction module is used to scan the channel map of the material board under different optical channels through the same line scan camera, and use the CAD master map of the material board to extract the PCS particle image in the channel map; A region mapping module, used for performing alignment correction on the PCS particle image according to the PCS contour line in the CAD master image, and generating a PCS detection image according to the target detection area of ​​the PCS particle image under different optical channels; the PCS detection images corresponding to different optical channels contain different target detection areas; A defect detection module is used to push the PCS detection image to the target detection model for defect detection, and to perform label fusion and label mapping on all defects according to their types; The defect display module is used to respond to receiving a selection operation for a target defect and display a defective PCS image on a defect display interface according to the mapping relationship between the PCS particle number and the defect label; the defective PCS image is marked with the selected target defect.

9. A computer device, characterized in that: The computer device includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the defect detection method based on multi-channel scanning as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The readable storage medium stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the defect detection method based on multi-channel scanning as described in any one of claims 1 to 7.

Images