Defect Detection Method, Device, Equipment and Storage Medium Based on Multi-Channel Scanning

Abstract

The present application discloses a defect detection method, device, equipment and storage medium based on multi-channel scanning, which relates to the field of AOI image detection. The channel map of the material board is obtained, and the PCS particle images therein are extracted using the CAD master layout. The PCS particle images are aligned and corrected according to the PCS contour lines in the CAD master layout, and PCS detection images are generated according to the target detection areas of the PCS particle images. After defect detection, the defects are subjected to label fusion and label mapping according to types. The selection operation of the target defect is received, and the defect channel map is displayed on the defect display interface according to the PCS particle number and the defect label mapping relationship. This solution selects the target detection area according to the channel type and uploads it to the corresponding detection model. Different detection models only perform defect detection on their respective target detection areas, and the detected defect labels are fused and then displayed. In this way, more accurate defect detection can be achieved in different regions, improving the detection accuracy.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of AOI image detection, and particularly to a defect detection method, device, equipment and storage medium based on multi-channel scanning. Background Art

[0002] The wafer processing technology is complex and interlocking. Quality inspection is required for each step from designing the wafer substrate to the final etching completion, so as to timely discover the quality problems of wafer products, adjust the process or scrap the wafer to stop losses in time. Defect detection relies on advanced inspection machines, advanced inspection algorithms, and complete testing and re-inspection processes.

[0003] In related technologies, for the machine production lines designed by some manufacturers, various scanning devices and light source devices need to be equipped for the detection module during defect identification and detection to complete the refined modeling of the material board. However, due to the influence of the material board background and the color contrast of components, some defects will present clear characteristic examples under high-brightness light sources, such as dark-colored dirt. Some defect types present clear characteristic examples under low-brightness light sources, such as bright-colored scratches. Therefore, in dealing with projects of detecting multiple types of defects, one is to use devices with higher computing power and algorithm models with higher detection performance for detection, and the other is to take different imaging effects of the same material and perform detection and defect matching calibration through different detection models respectively. The former scheme requires more investment, and the detection accuracy and accuracy cannot be guaranteed. The latter scheme requires repeated machine operations, especially the defect matching calibration of multiple images will reduce the detection efficiency. Summary of the Invention

[0004] The embodiments of the present application provide a defect detection method, device, equipment and storage medium based on multi-channel scanning, and solve the problems.

[0005] On the one hand, the present application provides a defect detection method based on multi-channel scanning, and the method includes:

[0006] Scanning the channel images of the material board under different optical channels through the same line-scan camera, and extracting the PCS particle images in the channel images by using the CAD master layout of the material board;

[0007] Performing alignment correction on the PCS particle images according to the PCS contour lines in the CAD master layout, and generating PCS detection images according to the target detection regions of the PCS particle images under different optical channels; the corresponding PCS detection images under different optical channels include different target detection regions;

[0008] Pushing the PCS detection images to a target detection model for defect detection, and performing label fusion and label mapping on all defects according to types;

[0009] In response to receiving a selection operation for a target defect, a defective PCS image is displayed on the defect display interface according to the mapping relationship between the PCS particle numbers and defect labels; the selected target defect is marked in the defective PCS image.

[0010] Specifically, the extraction of the PCS particle image in the channel map using the CAD master layout of the material board includes:

[0011] Identifying the tray in the channel map and selecting at least two image inflection point contours on the acupoint materials in the tray;

[0012] Mapping the CAD master layout based on the image inflection point contours, and aligning based on the mapped contour line and the template matching algorithm;

[0013] Mapping the PCS lines of the CAD master layout to the channel map, positioning the acupoint PCS contour and cropping the PCS particle image, and obtaining the channel PCS map according to the positional relationship; the PCS particle images are displayed in the channel PCS map according to their positions.

[0014] Specifically, the alignment correction of the extracted PCS particle image includes:

[0015] Successively selecting the image inflection point contours of each PCS particle image in the channel PCS map and mapping them to the mapped inflection point contours of the target PCS line on the CAD master layout;

[0016] Based on the mapping relationship between the mapped inflection point contours and the image inflection point contours, using the template matching algorithm to correct the angle and position of the PCS particle image, and determining the attitude adjustment amount;

[0017] Based on the matching relationship between the channel PCS map and the attitude adjustment amount, batch alignment correction processing is performed on all channel maps.

[0018] Specifically, the pushing of the PCS detection image to the target detection model for defect detection, and the label fusion and label mapping of all defects according to types include:

[0019] Determining the target area and non-target area of the PCS particle image, setting an opaque mask layer for the non-target area, and generating a fused area image;

[0020] Pushing the fused area image to the corresponding target detection model for defect detection, and outputting defect results and defect labels;

[0021] Summarizing the detection results of the PCS particle images under all channels, and fusing the same defect labels detected in the same target area according to the confidence level and defect area.

[0022] Specifically, after all defects are subjected to label fusion and label mapping according to their types, the method further includes:

[0023] Mapping the defects in the CAD master layout according to the defect labels, and displaying the defect PCS contour diagram on the defect display interface; wherein the normal PCS contour and the defect PCS contour are marked with different color borders respectively;

[0024] In response to receiving a click operation on the target defect PCS contour in the defect PCS contour diagram, a defect PCS information box is displayed, and defect images detected under all channels are displayed in the defect PCS information box.

[0025] Specifically, when the material board is double-sided detected, a front contour diagram and a back contour diagram are displayed in the defect PCS contour diagram, and the PCS contour is framed and marked according to the surface order where the defect is located.

[0026] Specifically, in response to receiving a selection operation on the defect PCS in the channel image, a defect display box is displayed; a defect PCS image and a defect information display area are displayed in the defect display box; and defect label information detected for the defect PCS corresponding to all channel images is listed in the defect information display area.

[0027] On the other hand, the present application provides a defect detection device based on multi-channel scanning, and the device includes:

[0028] An image extraction module, configured to scan channel diagrams of a material board under different optical channels through the same line scan camera, and extract PCS particle images in the channel diagrams by using the CAD master layout of the material board;

[0029] A region mapping module, configured to perform alignment correction on the PCS particle images according to the PCS contour lines in the CAD master layout, and generate PCS detection images according to the target detection regions of the PCS particle images under different optical channels; the PCS detection images corresponding to different optical channels include different target detection regions;

[0030] A defect detection module, configured to push the PCS detection images to a target detection model for defect detection, and perform label fusion and label mapping on all defects according to their types;

[0031] A defect display module, configured to, in response to receiving a selection operation on a target defect, display a defect PCS image on a defect display interface according to the PCS particle number and the defect label mapping relationship; the selected target defect is marked in the defect PCS image.

[0032] In another aspect, the present application provides a computer device, which includes a processor and a memory. At least one instruction, at least one program, a code set or an instruction set is stored in the memory, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the defect detection method based on multi-channel scanning described in the above aspect.

[0033] In another aspect, the present application provides a computer-readable storage medium, in which at least one instruction, at least one program, a code set or an instruction set is stored, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by a processor to implement the defect detection method based on multi-channel scanning described in the above aspect.

[0034] The beneficial effects brought by the technical solution provided by the embodiments of the present application at least include: In view of the image clarity characteristics presented by different defects in different light environments, the present application uses different light channels to capture the channel images of the material board from the same perspective, and for the selected channel images, overall alignment correction is performed using the CAD master layout, and the individual PCS images in the image are also aligned and corrected using the CAD contour map. In the defect detection stage, taking the PCS image as a unit, the target detection area is selected according to the channel type and uploaded to the corresponding detection model. Different detection models only perform defect detection on their respective target detection areas, and the detected defect labels are fused and then displayed. In this way, more accurate defect detection can be realized in different regions, improving the detection accuracy. Description of the Drawings

[0035] Figure 1 is a flowchart of the defect detection method based on multi-channel scanning provided by the embodiments of the present application;

[0036] Figure 2 is a flowchart of the defect detection method based on multi-channel scanning provided by the embodiments of the present application;

[0037] Figure 3 is a schematic diagram of aligning and correcting the PCS image with a contour line;

[0038] Figure 4 is a schematic diagram of a possible division of the target detection area;

[0039] Figure 5 shows a schematic diagram of a possible defect display interface;

[0040] Figure 6 shows a schematic diagram of the contour mapping of the image inflection point of the PCS image to the CAD master layout;

[0041] Figure 7 shows a schematic diagram of mapping the contour of the image inflection point of the PCS particle image to the CAD contour line;

[0042] Figure 8 Shows a schematic diagram of a multi-channel for separately dividing a target area and generating an image of a fusion area;

[0043] Figure 9 Is a schematic diagram of multi-channel same-area label fusion;

[0044] Figure 10 Is a schematic diagram of a possible defect display interface showing a PCS contour map of a defect;

[0045] Figure 11 Is a schematic diagram of a double-sided display of a PCS contour map of a defect;

[0046] Figure 12 Is a schematic diagram of a possible defect PCS information box display;

[0047] Figure 13 Is a schematic diagram of a defect PCS particle image;

[0048] Figure 14 Shows a structural block diagram of a defect detection device based on multi-channel scanning;

[0049] Figure 15 Shows a structural block diagram of a computer device provided by an exemplary embodiment of the present application. Detailed implementation manners

[0050] To make the objectives, technical solutions, and advantages of the present application clearer, the following will further describe the embodiments of the present application in detail with reference to the accompanying drawings.

[0051] As used herein, "a plurality of" means two or more. "And / or" describes the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. The character " / " generally represents an "or" relationship between the associated objects before and after.

[0052] Figure 1 Is a flowchart of a defect detection method based on multi-channel scanning provided by an embodiment of the present application, including the following steps:

[0053] S1. Scan the channel maps of the material board under different optical channels through the same line scan camera, and extract the PCS particle images in the channel maps using the CAD master layout of the material board;

[0054] The scanning range of the line-scan camera is usually fixed. Therefore, the content of the channel images obtained after scanning the material board with different light sources is the same. The only difference lies in the different pixel presentations of the images caused by the light sources. The specific light source and the number of scanning channels used in this step are determined according to various parameters of the material board and characteristics such as defect types.

[0055] Affected by the accuracy of the robotic arm, machine platform, and tray, the material board or PCS particles may have different degrees of position offsets during placement, and such offsets will be reflected in all channel images. In this application, position correction is mainly achieved through the CAD master layout.

[0056] Figure 2 It is a schematic diagram of the CAD master layout corresponding to the material board. The CAD master layout contains the position and contour information of all PCS particles and trays on the material board. When the material board is placed obliquely as a whole, it can be aligned and corrected by matching the line information of the CAD master layout, and then the PCS particle images are identified and extracted.

[0057] S2. Perform alignment correction on the PCS particle images according to the PCS contour lines in the CAD master layout, and generate PCS detection images based on the target detection areas of the PCS particle images under different light channels;

[0058] Traditional defect detection can perform input segmentation after single linear scanning. However, the images involved in this application are multi-channel images, and using the method of identifying and segmenting one by one will seriously reduce the detection efficiency. Therefore, this application does not operate in the way of AI image recognition that occupies computing resources, but performs it by matching and extracting the master layout. Because the size, position, and angle of the PCS contour lines in the CAD master layout are strictly designed, when the material board after production has an offset, as long as one of the channel images is used as the base for operation and the correction parameters are summarized, all subsequent channel images will be executed synchronously according to these parameters to achieve batch operation.

[0059] Figure 3 It is a schematic diagram of aligning and correcting the PCS image with the contour line. In order to extract accurate PCS particle images, after completing the channel image matching, the PCS contour lines in the CAD master layout are mapped to the channel image, and alignment is achieved by matching the CAD contour line and the image contour in the channel image. The purpose is to identify the structure of the PCS in the image, such as the gold surface area, text area, head, and tail, etc. If the PCS particles themselves are offset, alignment correction operations such as translation and rotation can be performed, and then the PCS particle images are extracted because defect detection is performed in units of PCS particles.

[0060] After the extraction is completed, the target detection area can be divided according to the optical channel attributes. Due to factors such as the material and color of the PCS particles, the defect recognition at different positions is easier to identify under the corresponding optical channels. Therefore, different target areas need to be set for the same PCS particle under different channels.

[0061] Figure 4 It is a schematic diagram of a possible division of the target detection area. Assume that a PCS particle image is planned with three detection areas according to the defect type and possible distribution, namely Area A detection area, Area B detection area, and Area C detection area. The three detection areas are specifically used to detect specific defect types. Among them, Area A detection area is dedicated to the extraction and detection of the channel map taken under the first optical channel scenario, Area B detection area is dedicated to the extraction and detection of the channel map taken under the first optical channel scenario, and Area C detection area is dedicated to the extraction and detection of the channel map taken under the first optical channel scenario. Although it is the same PCS particle, detection areas are set respectively under the three channels for submission for inspection. Of course, for some special scenarios and special defects, the target detection areas included in the corresponding PCS detection images under different optical channels can overlap.

[0062] S3. Push the PCS detection image to the target detection model for defect detection, and perform label fusion and label mapping on all defects according to the type;

[0063] The number of defect detection models covers at least all possible channel types. The model can be located on the cloud server. Through offline segmentation and setting, the PCS detection images of all channel types are uniformly pushed to the cloud, and the corresponding target detection model is selected for detection and output. The model output results include all defect sub-images identified from the PCS detection image (target detection area) and the matching defect labels. Further, these labels are classified and fused according to the type and mapped back to the target PCS particle image of the original channel image.

[0064] S4. In response to receiving a selection operation on the target defect, display the defective PCS image on the defect display interface according to the PCS particle number and defect label mapping relationship.

[0065] In a possible implementation manner, the detection result is displayed through a visualization display interface, and the defect label content is queried and displayed through human-computer interaction. Figure 5A schematic diagram of a possible defect display interface is shown. The defect display interface 500 at least includes a complete image display area 510 and a defect display area 520. In the image display area 510, a channel map or a CAD master layout 511 is listed. A number of PCS line profiles 511 are displayed in the CAD master layout 511. In the defect display area 520, a defect grid map 521 is listed, and the defect grid map 521 displays a defect image 522 according to the mapping relationship between the PCS particle number and the defect label. For example, when channel 1 is selected, all the defect thumbnail images 522 detected and intercepted from the PCS of the channel 1 image are displayed in the defect grid map 521, and specific label information is also listed in the grid, including the defect type, confidence value, defect area, and PCS number, etc. In Figure 5 Among them, the defects listed in the channel 1 image are mainly CVL offset defects because this defect is most easily detected in the channel 1 image, and the defects shown in the image are more intuitive to observe with the naked eye, the detection accuracy is higher, and there are no obvious bright and dark stripes. Similarly, other defects are detected and displayed using specific channel maps and specific models. Further, through these interface contents, the operator can perform operations such as inspecting, repairing, reinspecting, and scrapping the material board in a targeted manner.

[0066] In summary, in the present application, according to the image clarity characteristics presented by different defects in different light environments, channel maps of the material board at the same viewing angle are taken using different light channels, and for the selected channel map, the CAD master layout is used for overall alignment correction, and each PCS image in the image is also aligned using the CAD contour map. In the defect detection stage, taking the PCS image as a unit, the target detection area is selected according to the channel type and uploaded to the corresponding detection model. Different detection models only perform defect detection on their respective target detection areas, and the detected defect labels are fused and then displayed. In this way, more accurate defect detection in sub-regions can be achieved, and the detection accuracy can be improved.

[0067] In some embodiments, for S1, the CAD master layout of the material board is used to extract the PCS particle images in the channel map, which may specifically include the following steps:

[0068] S11. Identify the tray in the channel map and select at least two image inflection point contours on the acupoint materials in the tray;

[0069] The tray is a device for storing material plates, and a corresponding contour diagram is also depicted in CAD. The purpose is to enable the robotic arm to grasp the material plates at the correct position. However, the accuracy of matching the CAD master layout is at the pixel level, and there will inevitably be offsets under the influence of machine errors. Therefore, the tray needs to be used for initial calibration in the initial state. In this application, the contour information of the acupoint materials in the tray is selected for matching. Specifically, a matching algorithm logic based on edge direction gradient can be adopted, using the gradient correlation of the object edge as the matching criterion. The principle is as follows: extract the edge contour features in the ROI, create a template in combination with the gray information, and generate a multi-level image pyramid model according to the template size and clarity requirements. Then, search for the template image layer by layer from top to bottom in the image pyramid model until the bottom layer is searched or the position of the determined matching result is obtained.

[0070] S12. Map the CAD master layout based on the image inflection point contour, and align it based on the mapped contour line and the template matching algorithm;

[0071] Figure 6 Fig. shows a schematic diagram of mapping the CAD master layout based on the image inflection point contour. In Figure 6 select two contour inflection points near PCS 4 and PCS24, map them to the CAD master layout, and calculate the CAD inflection point lines in CAD that meet the distance similarity, contour similarity, and position similarity through the algorithm. When the result output by the template matching algorithm meets the threshold condition, the alignment is completed, and then rotation, translation, and other alignment operations can be performed on the entire channel image.

[0072] S13. Map the PCS line of the CAD master layout to the channel image, locate the acupoint PCS contour and extract the PCS particle image, and obtain the channel PCS map according to the position relationship; the PCS particle images are displayed according to the position distribution in the channel PCS map.

[0073] This step is to efficiently extract the PCS particle images. Traditionally, AI image recognition is used, but the shapes and sizes of PCS particles vary greatly, and there is a certain width of non-PCS edges in the images extracted by pixel recognition, which interferes with subsequent defect detection. Therefore, this application also uses CAD line mapping to extract. Since the previous feature images have been aligned, the PCS particles and the CAD lines of PCS have been matched. Next, they are directly mapped to the channel image, and then the PCS particle images are extracted. This step reduces the occupancy of computing resources to the lowest level, and there is no need to use an AI recognition and matching model, and the image extraction efficiency is also higher. All PCS particle images are combined in the form of an array according to the actual position and angle to form the channel PCS map.

[0074] In some embodiments, for S2, the alignment correction of the extracted PCS particle images is specifically as follows:

[0075] S21. Sequentially select the image inflection point contours of each PCS particle image in the channel PCS diagram, and map them to the mapped inflection point contours of the target PCS line on the CAD master layout;

[0076] Figure 7 The schematic diagram of mapping the CAD contour line according to the image inflection point contour of the PCS particle image is shown; one or more contour line inflection points can be selected, trained and recognized using the OpenCV or AI model algorithm, and then the template matching algorithm is used for alignment. Taking Figure 7 the selected inflection point contour as an example, the two line contours have uniqueness and complexity, and the alignment accuracy can be improved after the two are associated and mapped.

[0077] S22. Based on the mapping relationship between the mapped inflection point contour and the image inflection point contour, use the template matching algorithm to correct the angle and position of the PCS particle image, and determine the attitude adjustment amount;

[0078] The angle and position correction mainly takes into account the position offset during the installation of the PCS particle. At this time, there will inevitably be a deviation from the CAD contour line. At this time, the image should be rotated to ensure that the two contour lines coincide as much as possible. After coincidence, the rotation offset angle parameter of the image can be determined, that is, the attitude adjustment amount.

[0079] S23. Based on the matching relationship between the channel PCS diagram and the attitude adjustment amount, batch alignment correction processing is performed on all channel diagrams.

[0080] The foregoing setting of the channel PCS diagram is for batch operation because it contains all PCS particle images. After calculating the attitude adjustment amount of each PCS particle image relative to the theoretical CAD line contour in S23, the attitude adjustment amount of the entire channel PCS diagram is obtained. Applying this attitude adjustment amount to other channel diagrams at the same time can batch extract and correct the PCS particle images in all channel diagrams.

[0081] In some embodiments, for S3, the PCS detection image is pushed to the target detection model for defect detection, and all defects are subjected to label fusion and label mapping according to types, which can specifically include the following steps:

[0082] S31. Determine the target area and non-target area of the PCS particle image, set an opaque mask layer for the non-target area, and generate a fused area image;

[0083] The target area is determined relative to the channel type. Since there may be defects in various areas on a complete PCS particle map, and the present application adopts directional specific defect recognition, those not in the target detection area should be filtered. To unify the input and output of the detection model, a mask layer is selected to cover them. Subsequently, no detection will be performed on the mask layer area during push detection, further reducing the system computing power requirements.

[0084] Figure 8 A schematic diagram showing the division of the target area and the generation of the fused area image for multiple channels is presented. The complete PCS particle image is divided into three areas. PCS channel 1 only performs defect detection on area A, PCS channel 2 only performs defect detection on area B, and PCS channel 3 performs defect detection on areas B and C. In the fused area images obtained from the PCS particle images under the three channels, different colored mask layers can be specifically set according to requirements to enhance the contrast effect.

[0085] Furthermore, for the presentation effect of specific defects in pixels, parameter adjustment settings can be further made. For example, different types of labels are preferably configured in a single channel, but there may also be a situation where the same label is configured in the same two channels, which is specifically set according to the recognition accuracy of the defect type.

[0086] S32. Push the fused area image to the corresponding target detection model for defect detection, and output the defect results and defect labels;

[0087] S33. Summarize the detection results of the PCS particle images under all channels, and fuse the same defect labels detected in the same target area according to the confidence level and defect area.

[0088] Figure 9 It is a schematic diagram of multi-channel same-area label fusion. Label summarization mainly aims at the situation where the same defect is detected in the same area of multiple channels. For example, all three channels set area A as the target detection area. Specifically, area A of channel 1 outputs label 1 and label 3, area A of channel 1 outputs label 1, label 2, and label 4, and area A of channel 3 outputs label 2 and label 5. Since the detection is jointly performed on area A, the recognition results can be summarized.

[0089] Especially when both channel 2 and channel 3 output the defect label 2, it is necessary to further judge and fuse according to the confidence level and defect area. In some embodiments, the label with a high confidence level or a large defect area can be determined as the output label. Figure 9 In [reference], the defect area of label 2 in channel 3 is larger, so it is used as the final fused result for output.

[0090] In some embodiments, after fusing and mapping the labels of all defects by type, it is also necessary to perform visual display so that the operator can view and confirm the subsequent process flow more intuitively. Specifically, it may include the following:

[0091] A. According to the defect labels, map the defect PCS in the CAD master layout, and display the defect PCS contour diagram on the defect display interface;

[0092] Figure 10 is a schematic diagram of a possible defect display interface showing the defect PCS contour diagram. On the left side of the defect display interface 1000 is the channel diagram 1010 scanned by the material board under a specific optical channel, and on the right side is the matching defect PCS contour diagram 1020. Different from the original CAD template diagram, in this defect PCS contour diagram 1020, the normal PCS contour and the defect PCS contour are marked with different color borders respectively. For example Figure 10 in the defect PCS 1021 is outlined with a red frame (the normal PCS is not drawn with a green frame), and the number of detected defects (labels) can also be displayed in the frame. Figure 10 In PCS1, 20 defect points are detected under a certain channel.

[0093] Particularly, for some special material boards that need to be detected on both sides, when the material board is detected on both sides, the front contour diagram and the back contour diagram are displayed in the defect PCS contour diagram, and the PCS contour is framed and marked according to the surface order where the defect is located. Figure 11 is a schematic diagram of a double-sided display of the defect PCS contour diagram.

[0094] B. In response to receiving a click operation on the target defect PCS contour in the defect PCS contour diagram, display a defect PCS information box, and all defect images detected under all channels are displayed in the defect PCS information box.

[0095] In Figure 10 on the basis of this, label information can be fused on the defect PCS contour diagram, and when the target defect PCS contour is clicked, a defect PCS information box will be jumped to and displayed.

[0096] Figure 12 is a schematic diagram of a possible display of the defect PCS information box. For a certain determined defect PCS, all defect pictures in the defect PCS information box are displayed in a rasterized manner. When the operator clicks on the small defect picture, the defect picture can be further popped up and enlarged for display for the re-inspection process.

[0097] In the above display scheme, it is not displayed as a complete single PCS particle image, which is not conducive to locating the position of the defect. Therefore, in order to facilitate the observation and location of the defect, a complete defect PCS image can also be displayed on the basis of the channel image.

[0098] Figure 13 is a schematic diagram of defective PCS particle images. Based on Figure 10 , in response to receiving a selection operation on the defective PCS in the channel image, a defect display box 1300 is displayed. The defect display box 1300 displays a defective PCS image 1310 and a defect information display area 1320. In the defect information display area 1320, defect label information 1330 detected for all channel images corresponding to the defective PCS is listed. The defect label information may specifically include "golden different surface 100 (channel 1)", "gold surface scratch 20 (channel 2)", "golden indentation 100 (channel 3)", "golden foreign object 20 (channel 1)", "golden hole 100 (channel 2)", and "gold surface nickel leakage 20 (channel 1)". The number after the text represents the pixel area, and the content in the parentheses remarks the channel source. The defective PCS image 1310 can be selected from any channel image for display, and defect boxes of all selected labels are marked in the image for the operator to easily observe and determine.

[0099] Figure 14 shows a structural block diagram of a defect detection device based on multi-channel scanning provided by an embodiment of the present application. The device includes:

[0100] An image extraction module 1410, configured to scan channel images of a material plate under different optical channels through the same line-scan camera, and extract PCS particle images in the channel images using the CAD master layout of the material plate;

[0101] An area mapping module 1420, configured to perform alignment correction on the PCS particle images according to the PCS contour lines in the CAD master layout, and generate PCS detection images according to the target detection areas of the PCS particle images under different optical channels; the PCS detection images corresponding to different optical channels include different target detection areas;

[0102] A defect detection module 1430, configured to push the PCS detection images to a target detection model for defect detection, and perform label fusion and label mapping on all defects according to types;

[0103] A defect display module 1440, configured to, in response to receiving a selection operation on a target defect, display a defective PCS image on a defect display interface according to the PCS particle number and defect label mapping relationship; the defective PCS image is marked with the selected target defect.

[0104] The defect detection device based on multi-channel scanning provided by an embodiment of the present application can be applied to the defect detection method based on multi-channel scanning provided in the above embodiment. For related details, refer to the above method embodiment. The implementation principle and technical effect are similar and will not be elaborated here.

[0105] It should be noted that the defect detection device based on multi-channel scanning provided in the embodiments of the present application is only illustrated by the above division of each functional module / functional unit. In actual applications, the above functions can be allocated to different functional modules / functional units according to needs, that is, the internal structure of the defect detection device based on multi-channel scanning is divided into different functional modules / functional units to complete all or part of the functions described above. In addition, the implementation manner of the defect detection method based on multi-channel scanning provided in the above method embodiment and the implementation manner of the defect detection device based on multi-channel scanning provided in this embodiment belong to the same concept. The specific implementation process of the defect detection device based on multi-channel scanning provided in this embodiment can be found in the above method embodiment and will not be elaborated here.

[0106] Figure 15 The block diagram of the structure of a computer device provided by an exemplary embodiment of the present application is shown. It is a computer device such as a desktop computer, a laptop computer, a handheld computer, and a cloud server. The computer device may include, but is not limited to, a processor and a memory. Among them, the processor and the memory can be connected through a bus or other means. Among them, the processor can be a central processing unit (CPU). The processor can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, graphics processing units (GPUs), embedded neural network processors (NPUs) or other dedicated deep learning coprocessors, discrete gate or transistor logic devices, discrete hardware components, etc. chips, or combinations of the above types of chips.

[0107] The processor may include one or more processing cores, such as a quad-core processor, an octa-core processor, etc. The processor may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), or PLA (Programmable Logic Array). The processor may also include a main processor and a coprocessor. The main processor is a processor used to process data in the wake state, also known as the CPU (Central Processing Unit); the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, the processor may be integrated with a GPU (Graphics Processing Unit), and the GPU is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor may further include an AI (Artificial Intelligence) processor, and the AI processor is used to process computational operations related to machine learning.

[0108] The memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the methods in the above embodiments of the present application. The processor executes various functional applications and data processing of the processor by running the non-transitory software programs, instructions, and modules stored in the memory, that is, implements the methods in the above method embodiments. The memory may include a program storage area and a data storage area. Among them, the program storage area can store an operating system and application programs required for at least one function; the data storage area can store data created by the processor, etc. In addition, the memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state storage devices. In some embodiments, the memory may optionally include a memory remotely set relative to the processor, and these remote memories can be connected to the processor through a network. Examples of the above networks include, but are not limited to, the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0109] In some embodiments, the computer device may also optionally include: a peripheral device interface and at least one peripheral device. The processor, the memory, and the peripheral device interface may be connected through a bus or signal lines. Each peripheral device may be connected to the peripheral device interface through a bus, signal lines, or a circuit board. Specifically, the peripheral devices include: at least one of a radio frequency circuit, a display screen, and a keyboard.

[0110] The peripheral device interface can be used to connect at least one peripheral device related to I / O (Input / Output) to the processor and the memory. In some embodiments, the processor, the memory, and the peripheral device interface are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor, the memory, and the peripheral device interface can be implemented on a separate chip or circuit board, and this embodiment does not limit this.

[0111] The display screen is used to display the UI (User Interface). The UI may include graphics, text, icons, videos, and any combination thereof. When the display screen is a touch display screen, the display screen also has the ability to collect touch signals on or above the surface of the display screen. The touch signal can be input to the processor as a control signal for processing. At this time, the display screen can also be used to provide virtual buttons and / or a virtual keyboard, also known as soft buttons and / or a soft keyboard. In some embodiments, there can be one display screen, which is set on the front panel of the computer device; in some other embodiments, there can be at least two display screens, which are respectively set on different surfaces of the computer device or are in a folding design; in some other embodiments, the display screen can be a flexible display screen, which is set on the curved surface or the folding surface of the computer device. Even, the display screen can be set to an irregular non-rectangular shape, that is, a special-shaped screen. The display screen can be prepared from materials such as LCD (Liquid Crystal Display) and OLED (Organic Light-Emitting Diode).

[0112] The power supply is used to supply power to each component in the computer device. The power supply can be alternating current, direct current, a disposable battery, or a rechargeable battery. When the power supply includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. A wired rechargeable battery is a battery charged through a wired line, and a wireless rechargeable battery is a battery charged through a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0113] Those skilled in the art can understand that the structure shown in this embodiment does not constitute a limitation on the computer device, and it may include more or fewer components than shown in the figure, or combine certain components, or adopt different component arrangements.

[0114] An embodiment of the present application also discloses a computer-readable storage medium. Specifically, the computer-readable storage medium is used to store a computer program, and when the computer program is executed by a processor, the method in the above method embodiment is implemented. Those skilled in the art can understand that all or part of the processes of implementing the methods in the above embodiments of the present application can be completed by instructing relevant hardware through a computer program. The program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the above method embodiments. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory, a hard disk drive (HDD), or a solid-state drive (SSD), etc.; the storage medium can also include a combination of the above types of memories.

[0115] This specific embodiment is only an interpretation of the present invention and does not limit the present invention. After reading this specification, those skilled in the art can make modifications to this embodiment that do not contribute creatively according to needs, but as long as they are within the scope of the claims of the present invention, they are protected by the patent law.

Claims

1. A defect detection method based on multi-channel scanning, characterized in that, The method includes: Scanning the channel images of the material board under different optical channels through the same line-scanning camera, and extracting the PCS particle images in the channel images using the CAD master layout of the material board; Performing alignment correction on the PCS particle images according to the PCS contour lines in the CAD master layout, and generating PCS detection images according to the target detection areas of the PCS particle images under different optical channels; the PCS detection images corresponding to different optical channels include different target detection areas; Pushing the PCS detection images to a target detection model for defect detection, and performing label fusion and label mapping on all defects according to types; specifically determining the target area and non-target area of the PCS particle images, setting an opaque mask layer for the non-target area, and generating a fused area image; Pushing the fused area image to the corresponding target detection model for defect detection, and outputting defect results and defect labels; Summarizing the detection results of the PCS particle images under all channels, and fusing the same defect labels detected in the same target area according to confidence and defect area; In response to receiving a selection operation on a target defect, displaying a defective PCS image on a defect display interface according to the mapping relationship between the PCS particle number and the defect label; the defective PCS image is marked with the selected target defect.

2. The method according to claim 1, characterized in that, The extracting the PCS particle images in the channel images using the CAD master layout of the material board includes: Identifying the tray in the channel image, and selecting at least two image inflection point contours on the acupoint materials in the tray; Mapping the CAD master layout based on the image inflection point contours, and performing alignment based on the mapped contour lines and the template matching algorithm; Mapping the PCS lines of the CAD master layout to the channel image, positioning the acupoint PCS contour and cropping the PCS particle images, and obtaining a channel PCS image according to the positional relationship; the PCS particle images are displayed in the channel PCS image according to their positions.

3. The method according to claim 2, wherein The performing alignment correction on the extracted PCS particle images includes: Successively selecting the image inflection point contours of each PCS particle image in the channel PCS image, and mapping them to the mapped inflection point contours of the target PCS lines on the CAD master layout; Based on the mapping relationship between the mapped inflection point contours and the image inflection point contours, using the template matching algorithm to correct the angle and position of the PCS particle images, and determining the attitude adjustment amount; Based on the matching relationship between the channel PCS image and the attitude adjustment amount, performing batch alignment correction processing on all channel images.

4. The method according to claim 2, wherein After performing label fusion and label mapping on all defects according to types, the method further includes: Mapping the defective PCS in the CAD master layout according to the defect label, and displaying the defective PCS contour map on the defect display interface; where the normal PCS contour and the defective PCS contour are marked with different color borders respectively; In response to receiving a click operation on the target defective PCS contour in the defective PCS contour map, displaying a defective PCS information box, and the defective PCS information box displays the defect images detected under all channels.

5. The method according to claim 4, wherein When the material board is for double-sided inspection, a front contour map and a back contour map are displayed in the defective PCS contour map, and the PCS contour is framed and marked according to the surface order where the defect location is located.

6. The method according to claim 5, wherein In response to receiving a selection operation on the defective PCS in the channel map, a defect display frame is displayed; the defect display frame displays a defective PCS image and a defect information display area; The defect information display area lists the defect label information detected for all channel maps corresponding to the defective PCS.

7. A defect detection device based on multi-channel scanning, characterized in that, The device includes: An image extraction module, configured to scan the channel map of the material board under different optical channels through the same line scan camera, and extract the PCS particle image in the channel map using the CAD master layout of the material board; A region mapping module, configured to perform alignment correction on the PCS particle image according to the PCS contour line in the CAD master layout, and generate a PCS detection image according to the target detection region of the PCS particle image under different optical channels; the PCS detection images corresponding to different optical channels include different target detection regions; A defect detection module, configured to push the PCS detection image to a target detection model for defect detection, perform label fusion and label mapping on all defects according to types; specifically determine the target region and non-target region of the PCS particle image, set an opaque mask layer for the non-target region, and generate a fused region image; Push the fused region image to the corresponding target detection model for defect detection, and output defect results and defect labels; Summarize the detection results of the PCS particle images under all channels, and fuse the same defect labels detected in the same target region according to the confidence level and defect area; A defect display module, configured to, in response to receiving a selection operation on a target defect, display a defective PCS image on the defect display interface according to the mapping relationship between the PCS particle number and the defect label; the defective PCS image is marked with the selected target defect.

8. A computer device, characterized in that, The computer device includes a processor and a memory, and the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the multi-channel scanning-based defect detection method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The readable storage medium stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by a processor to implement the multi-channel scanning-based defect detection method according to any one of claims 1 to 6.

Images