Defect detection method of single board, electronic equipment, storage medium and program product

Abstract

The embodiment of the invention provides a single board defect detection method, electronic equipment, a storage medium and a program product. The method comprises the following steps: shooting a to-be-detected single board from multiple angles through automatic X-ray detection equipment to obtain two-dimensional slice image groups corresponding to the multiple angles respectively; performing transmission scanning on the to-be-detected single plate to obtain a two-dimensional grayscale image; performing defect detection based on the plurality of two-dimensional slice image groups to obtain a defect area; performing defect detection on the two-dimensional grayscale image to obtain a region of interest; performing defect classification based on the defect region and the region of interest to obtain a defect type of the to-be-detected single board; the method is used for achieving the effect of improving the defect detection accuracy.

Description

Technical Field

[0001] This application relates to the technical fields of defect detection and artificial intelligence, and particularly relates to a method for defect detection of a single board, an electronic device, a storage medium, and a program product. Background Art

[0002] In the production stage of a single board, the link of defect detection of the single board is crucial. If the defects of the single board cannot be accurately detected, it will cause quality problems in the single board and the equipment using the single board.

[0003] In the related art, on the production line of a single board, the defects of the single board are detected manually. Such a method is prone to missed detection, resulting in a low accuracy rate of defect detection of the single board. Summary of the Invention

[0004] Embodiments of this application provide a method for defect detection of a single board, an electronic device, a storage medium, and a program product, so as to achieve the effect of improving the accuracy of defect detection.

[0005] In a first aspect, an embodiment of this application provides a method for defect detection of a single board, including:

[0006] Taking pictures of a single board to be tested from multiple angles through an automatic X-ray detection device to obtain a group of two-dimensional slice images corresponding to each of the multiple angles;

[0007] Performing transmission scanning on the single board to be tested to obtain a two-dimensional grayscale image;

[0008] Performing defect detection based on the group of two-dimensional slice images to obtain a defect area;

[0009] Performing defect detection on the two-dimensional grayscale image to obtain a region of interest;

[0010] Classifying defects based on the defect area and the region of interest to obtain the defect type of the single board to be tested.

[0011] In a second aspect, an embodiment of this application provides a device for defect detection of a single board, including:

[0012] A first image acquisition module, configured to take pictures of a single board to be tested from multiple angles through an automatic X-ray detection device to obtain a group of two-dimensional slice images corresponding to each of the multiple angles;

[0013] A second image acquisition module, configured to perform transmission scanning on the single board to be tested to obtain a two-dimensional grayscale image;

[0014] A first defect detection module, configured to perform defect detection based on the group of two-dimensional slice images to obtain a defect area;

[0015] The second defect detection module is used to detect defects in the two-dimensional grayscale image to obtain the region of interest;

[0016] The defect classification module is used to classify defects based on the defect region and the region of interest to obtain the defect type of the single board to be tested.

[0017] In a third aspect, an embodiment of the present application provides an electronic device, including: a memory and a processor; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory, so that the processor executes the above first aspect and / or various possible implementation manners of the first aspect.

[0018] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored, and when the computer-executable instructions are executed by a processor, they are used to implement the above first aspect and / or various possible implementation manners of the first aspect.

[0019] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, and when the computer program is executed by a processor, it implements the above first aspect and / or various possible implementation manners of the first aspect.

[0020] The defect detection method, electronic device, storage medium and program product of the single board provided by the embodiments of the present application are as follows: The automatic X-ray detection device takes pictures of the single board to be tested from multiple angles to obtain multiple groups of two-dimensional slice images, performs transmission scanning on the single board to be tested to obtain a two-dimensional grayscale image, performs defect detection based on multiple groups of two-dimensional slice images to obtain a defect region, performs defect detection based on the two-dimensional grayscale image to obtain a region of interest, and performs defect classification based on the defect region and the region of interest to obtain the defect type of the single board to be tested; Since multiple groups of two-dimensional slice images provide the internal situation of the single board to be tested from multiple angles, the situation where internal components of the single board to be tested block each other can be avoided. The defect region has the characteristics of complex defects in the single board to be tested, and the region of interest is determined according to the two-dimensional grayscale image, reflecting the global density distribution of the single board to be tested. The region of interest has the characteristics of simple defects in the single board to be tested, and the region of interest can be quickly determined according to the two-dimensional grayscale image. Therefore, combining the defect region and the region of interest for defect detection can improve the comprehensiveness, accuracy and efficiency of defect detection. Description of the Drawings

[0021] The drawings here are incorporated into the description and form a part of this description, showing embodiments consistent with the present application, and are used together with the description to explain the principles of the present application.

[0022] Figure 1 It is a schematic flow chart of the defect detection method of the single board provided by the present application Figure One ;

[0023] Figure 2 Flow schematic of the defect detection method for the single board provided by this application Figure Two ;

[0024] Figure 3 Schematic diagram of photographing the single board to be measured from three angles provided by this application;

[0025] Figure 4 Flow schematic diagram provided by this application for defect detection of three groups of two-dimensional slice images to obtain defect regions;

[0026] Figure 5 Schematic diagram of the structure of the first feature extraction module provided by this application;

[0027] Figure 6 Schematic diagram of the structure of the residual block provided by this application;

[0028] Figure 7 Flow schematic diagram provided by this application for extracting the first intersection feature map and the second intersection feature map;

[0029] Figure 8 Schematic diagram of the structure of the defect detection device for the single board provided by this application;

[0030] Figure 9 Schematic diagram of the structure of the electronic device provided by this application.

[0031] Through the above-mentioned drawings, the specific embodiments of this application have been shown, and there will be more detailed descriptions hereinafter. These drawings and textual descriptions are not intended to limit the scope of the concept of this application in any way, but to illustrate the concept of this application to those skilled in the art by referring to specific embodiments. Specific Embodiments

[0032] Here, the exemplary embodiments will be described in detail, and the examples are shown in the drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. On the contrary, they are merely examples of devices and methods consistent with some aspects of this application as detailed in the appended claims.

[0033] Figure 1 Flow schematic of the defect detection method for the single board provided by this application Figure One , the defect detection method for the single board can be applied to a defect detection device, and the defect detection device can be an electronic device; as shown in the figure, the defect detection method for the single board includes:

[0034] S101. Use an automated X-ray inspection device to take pictures of the single board to be tested from multiple angles, and obtain a group of two-dimensional slice images corresponding to each of the multiple angles.

[0035] Among them, the single board to be tested is the single board to be inspected. A single board is the core component of a computer. A single board can integrate multiple core components of a computer. For example, a processor, memory, and input / output interfaces can be integrated into a single board.

[0036] An automated X-ray inspection device (AXI) uses X-rays to penetrate an object and generate slice images.

[0037] The multiple angles refer to at least two angles. By taking pictures of the single board to be tested from multiple angles, a group of two-dimensional slice images from different perspectives can be obtained. Based on the group of two-dimensional slice images from multiple angles, the three-dimensional information of the single board to be tested can be constructed.

[0038] Optionally, the multiple angles can be two orthogonal angles; the multiple angles can also be three angles, and any two of the three angles are orthogonal to each other. For example, based on the coordinate system in which the single board to be tested is located, the three angles can include: the angle in the first horizontal direction (the positive direction of the x-axis) in the coordinate system in which the single board to be tested is located, the angle in the second horizontal direction (the positive direction of the y-axis), and the height direction (the positive direction of the z-axis). Among them, the coordinate system in which the single board to be tested is located can be a Cartesian coordinate system constructed with a certain corner point or the center of the single board to be tested as the origin.

[0039] The three angles can also be such that two of them are orthogonal to each other and the other angle is between the two orthogonal angles; the three angles can also be three angles that divide 360 degrees evenly.

[0040] The multiple angles can also be determined at a preset step size between 0 degrees and 360 degrees. For example, if the preset step size is 30 degrees, there are 12 multiple angles.

[0041] Optionally, obtain the complexity identifier of the single board to be tested, and determine the multiple angles according to the complexity identifier. The complexity identifier is used to characterize the complexity of the component structure in the single board to be tested. When the complexity identifier is the first identifier, it means that the component structure in the single board to be tested is simple, and the multiple angles can be two orthogonal angles. When the complexity identifier is the second identifier, it means that the component structure in the single board to be tested is relatively complex, and the multiple angles can be three angles, and any two of the three angles are orthogonal to each other. When the complexity identifier is the third identifier, it means the complexity of the component structure in the single board to be tested, and the multiple angles can be determined at a preset step size between 0 degrees and 360 degrees. The value of the preset step size can be set according to actual needs, and this application embodiment does not limit this.

[0042] It should be noted that when the AXI takes pictures of the single board under test from each angle, a two-dimensional slice image group at that angle will be obtained. The two-dimensional slice image group includes multiple two-dimensional slice images taken at that angle, and the multiple two-dimensional slice images correspond to multiple different X-ray frequencies; that is, at each angle, the AXI sequentially takes pictures of the single board under test according to a preset multiple X-ray frequencies, obtains multiple two-dimensional slice images of the single board under test at that angle, and then obtains the two-dimensional slice image group at that angle based on the multiple two-dimensional slice images at that angle. Among them, the preset multiple X-ray frequencies can be set according to actual needs, and the embodiments of the present application do not limit this.

[0043] Specifically, when the single board under test is in the detection area, the detection device can determine multiple angles according to the complexity identification of the single board under test, and sequentially control the rotation of the AXI according to the multiple angles. Whenever it rotates to any one of the multiple angles, the AXI uses a preset multiple X-ray frequencies to sequentially take pictures of the single board under test, and obtains the two-dimensional slice image group at the current angle.

[0044] S102. Perform transmission scanning on the single board under test to obtain a two-dimensional grayscale image.

[0045] Among them, the AXI can perform transmission scanning on the single board under test at a preset angle; the number of two-dimensional grayscale images is 1.

[0046] Specifically, the AXI uses an X-ray beam with a preset fixed frequency to penetrate the single board under test through a preset angle to obtain a two-dimensional grayscale image; among them, the preset angle can be an angle perpendicular to the target plane of the single board under test, and the target plane is the plane with the largest area in the single board under test. By performing transmission scanning on the single board under test through this preset angle, more information about the single board under test can be obtained.

[0047] S103. Perform defect detection based on multiple two-dimensional slice image groups to obtain a defect area.

[0048] Among them, the defect area is a partial area in the single board under test, and is an area that may have defects determined based on multiple two-dimensional slice image groups.

[0049] Optionally, the feature maps of multiple two-dimensional slice image groups are respectively extracted through a first defect detection model, the feature maps of the multiple two-dimensional slice image groups are fused to obtain a three-dimensional feature map, and defect detection is performed on the three-dimensional feature map to obtain a defect area.

[0050] Optionally, a three-dimensional image of the single board under test is constructed according to multiple two-dimensional slice image groups, and defect detection is performed on the three-dimensional image through a second defect detection model to obtain a defect area.

[0051] In a possible implementation, defect detection is performed based on multiple groups of two-dimensional slice images to obtain a defect area, including: respectively performing feature extraction on multiple groups of two-dimensional slice images to obtain multiple candidate feature maps; fusing the multiple candidate feature maps to obtain a three-dimensional feature map; and performing defect detection on the three-dimensional feature map to obtain the defect area.

[0052] Among them, the multiple candidate feature maps respectively correspond one-to-one to the multiple groups of two-dimensional slice images, that is, the multiple candidate feature maps are candidate feature maps at multiple angles.

[0053] Specifically, the defect area is obtained by processing the multiple groups of two-dimensional slice images through a first defect detection model; the first defect detection model includes a first feature extraction module, a first feature fusion module, and a first classification module.

[0054] For each group of two-dimensional slice images at an angle, input the group of two-dimensional slice images into the first feature extraction module to obtain the candidate feature map at that angle; input the candidate feature maps at multiple angles into the first feature fusion module to obtain a three-dimensional feature map; and input the three-dimensional feature map into the first classification module to obtain the defect area.

[0055] Among them, the first feature fusion module can be used to project the candidate feature map at each angle into the three-dimensional space where the single board to be measured is located to obtain the three-dimensional feature voxels at each angle, and then perform weighted average fusion or maximum value fusion on the three-dimensional feature voxels at multiple angles to obtain a three-dimensional feature map.

[0056] In a possible implementation, defect detection is performed based on multiple groups of two-dimensional slice images to obtain a defect area, including: constructing a three-dimensional image of the single board to be measured according to the multiple groups of two-dimensional slice images; performing feature extraction on the three-dimensional image to obtain a three-dimensional feature map, and performing defect detection on the three-dimensional feature map to obtain the defect area.

[0057] Specifically, a three-dimensional image of the single board to be measured is constructed according to the multiple groups of two-dimensional slice images, and the defect area is obtained by processing the three-dimensional image through a second defect detection model; the second defect detection model includes a second feature extraction module and a second classification module. Input the three-dimensional image into the second feature extraction module (such as a three-dimensional convolutional layer) to obtain a three-dimensional feature map, and input the three-dimensional feature map into the second classification module to obtain the defect area.

[0058] In the above embodiments, defect detection is performed through multiple groups of two-dimensional slice images. The multiple groups of two-dimensional slice images provide the internal situation of the single board to be measured from multiple angles, reduce the mutual occlusion inside the single board to be measured, and improve the defect detection accuracy.

[0059] S104. Perform defect detection on the two-dimensional grayscale image to obtain the region of interest.

[0060] Among them, the region of interest is a partial region in the two-dimensional grayscale image, and is a region that may have defects determined based on the two-dimensional grayscale image.

[0061] Optionally, perform noise reduction processing on the two-dimensional grayscale image to obtain a denoised image; the noise reduction processing may include, but is not limited to, median filtering and anisotropic diffusion. Median filtering can eliminate salt-and-pepper noise in the two-dimensional grayscale image, and anisotropic diffusion can suppress Gaussian noise while preserving the edges of the two-dimensional grayscale image; perform binary segmentation on the denoised image to obtain a binary image, and determine the region of interest according to the binary image.

[0062] Among them, determining the region of interest according to the binary image may be to extract the area feature, morphological feature, texture feature, and position feature of the binary image, and analyze the area feature, morphological feature, texture feature, and position feature using a preset decision tree structure to obtain the region of interest.

[0063] Optionally, the two-dimensional grayscale image can be subjected to defect detection through a third defect detection model to obtain the region of interest; the third defect detection model includes a third feature extraction module and a third classification module. Input the two-dimensional grayscale image into the third feature extraction module to obtain a grayscale feature map, and input the grayscale feature map into the third classification module to obtain the region of interest.

[0064] S105. Perform defect classification based on the defect region and the region of interest to obtain the defect type of the single board to be tested.

[0065] Among them, the defect type of the single board to be tested is one of multiple preset types, and the multiple preset types include, but are not limited to, cracks, holes, delamination, pores, and bridging.

[0066] It should be noted that the defect region is determined according to multiple groups of two-dimensional images. Since multiple groups of two-dimensional images can provide the internal situation of the single board to be tested from multiple angles, the defect region can be used to detect complex defect types such as cracks, holes, and delamination; the region of interest is determined according to the two-dimensional grayscale image, which reflects the global density distribution of the single board to be tested and can be used to detect simple defect types such as pores and bridging. In addition, the region of interest can be quickly determined through the two-dimensional grayscale image; therefore, by combining the defect region and the region of interest, the comprehensiveness, accuracy, and efficiency of defect detection can be improved.

[0067] Optionally, according to the position information of the defect region and the region of interest, determine a first intersection region in the defect region and a second intersection region in the region of interest; the first intersection region and the second intersection region correspond to the same part of the single board to be tested.

[0068] Extract the feature map of the first intersection region, extract the feature map of the second intersection region, fuse the feature map of the first intersection region and the feature map of the second intersection region to obtain a fused feature map, and perform defect classification on the fused feature map to obtain the defect type of the single board to be tested. Defect classification is performed based on the information of the intersection of the defect region and the region of interest, integrating the defect features included in the defect region and the defect features included in the region of interest, and the region of interest can be quickly determined according to the two-dimensional grayscale image, which can improve the comprehensiveness, accuracy and efficiency of defect detection.

[0069] Optionally, perform defect classification on the defect region to obtain the first defect probabilities of each first preset type, perform defect classification on the region of interest to obtain the second defect probabilities of each second preset type, and determine the defect type according to the first defect probabilities of each first preset type and the second defect probabilities of each second preset type.

[0070] Among them, the first preset type may include but is not limited to cracks, holes and delamination; the second preset type may include but is not limited to air holes and bridging; determining the defect type according to the first defect probabilities of each first preset type and the second defect probabilities of each second preset type may be to determine the target defect probabilities exceeding the probability threshold among the first defect probabilities and the second defect probabilities, and the number of target defect probabilities may be one or more. For each target defect probability, the first preset type or the second preset type corresponding to the target defect probability is used as the defect type. In the above manner, complex type defects such as cracks, holes and delamination can be detected through the defect region, and simple type defects such as air holes and bridging can be detected through the region of interest, which can improve the comprehensiveness and accuracy of defect detection while improving the defect detection efficiency.

[0071] The defect detection method for a single board provided by the embodiments of the present application takes pictures of the single board to be tested from multiple angles through an automatic X-ray detection device to obtain a plurality of two-dimensional slice image groups, performs transmission scanning on the single board to be tested to obtain a two-dimensional grayscale image, performs defect detection based on the plurality of two-dimensional slice image groups to obtain a defect region, performs defect detection based on the two-dimensional grayscale image to obtain a region of interest, and performs defect classification based on the defect region and the region of interest to obtain the defect type of the single board to be tested; since the plurality of two-dimensional slice image groups provide the internal situation of the single board to be tested from multiple angles, the situation where the internal components of the single board to be tested block each other can be avoided. The defect region has the characteristics of complex defects in the single board to be tested, the region of interest is determined according to the two-dimensional grayscale image, reflecting the global density distribution of the single board to be tested, the region of interest has the characteristics of simple defects in the single board to be tested, and the region of interest can be quickly determined according to the two-dimensional grayscale image. Therefore, combining the defect region and the region of interest for defect detection can improve the comprehensiveness, accuracy and efficiency of defect detection.

[0072] In a possible implementation, defect classification is performed based on a defect area and an area of interest to obtain the defect type of the single board to be tested, including: determining the target layer to which the defect area belongs based on the coordinate values of the defect area; and performing defect classification on the defect area and the area of interest through a target classification model corresponding to the target layer to obtain the defect type of the single board to be tested.

[0073] Among them, the coordinate values of the defect area can reflect the position of the defect area in the single board to be tested and can be determined based on the coordinate system in which the single board to be tested is located.

[0074] The target layer is the layer in which the defect area is located in the single board to be tested. The target layer can be the surface layer, the transition layer, or the deep layer; when the target layer is the surface layer, it indicates that there may be surface defects in the single board to be tested. When the target layer is the transition layer, it indicates that there may be defects in the shallow layer of the single board to be tested. When the target layer is the deep layer, it indicates that there may be defects in the deep layer of the single board to be tested.

[0075] Optionally, determining the target layer to which the defect area belongs based on the coordinate values of the defect area includes: determining the target depth value based on the coordinate values of the defect area, the X-ray absorption rate, and the thermal distribution characteristics; and determining the target layer to which the defect area belongs according to the preset interval to which the target depth value belongs.

[0076] Among them, the coordinate value can be the height coordinate value; the X-ray absorption rate is the absorption rate of the defect area to X-rays; and the thermal distribution characteristics can be determined by the thermal imaging of the defect area.

[0077] Specifically, obtain the coordinate values, X-ray absorption rate, and thermal distribution characteristics of the defect area; determine the average height value of the defect area based on the height coordinate value of the defect area, and determine the first depth value of the defect area according to the average height value and the reference height coordinate values of each layer in the single board to be tested; determine the second depth value of the defect area according to the X-ray absorption rate of the defect area and the absorption rates of each layer in the single board to be tested; process the thermal distribution characteristics using a heat conduction model to obtain the third depth value of the defect area; perform weighted summation on the first depth value, the second depth value, and the third depth value to obtain the target depth value; among them, the weights of the first depth value, the second depth value, and the third depth used for weighted summation can be set according to actual requirements.

[0078] The preset intervals include a first preset interval, a second preset interval, and a third preset interval; the depth corresponding to the first preset interval is less than the depth corresponding to the second preset interval, and the depth corresponding to the second preset interval is less than the depth corresponding to the third preset interval.

[0079] When the target depth value belongs to the first preset interval, the target level is determined to be the surface layer; when the target depth value belongs to the second preset interval, the target level is determined to be the transition layer; when the target depth value belongs to the third preset interval, the target level is determined to be the deep layer. The first preset interval, the second preset interval, and the third preset interval can be set according to the actual situation.

[0080] In the above embodiment, the target level of the defect area is determined by integrating the coordinate value, X-ray absorption rate, and thermal distribution characteristics of the defect area, improving the accuracy of the target level.

[0081] Optionally, based on the height coordinate value of the defect area, the average height value of the defect area is determined, and the average height value is used as the target depth value. According to the preset interval to which the target depth value belongs, the target level to which the defect area belongs is determined.

[0082] In the above embodiment, using the average height value as the target depth value can quickly determine the target depth value, and then quickly determine the target level, improving the efficiency of determining the target level.

[0083] Optionally, through the target classification model corresponding to the target level, the defect areas and the areas of interest are classified for defects to obtain the defect types of the single board to be tested, including: determining a first intersection area that intersects the area of interest in the defect area; determining a second intersection area that intersects the defect area in the area of interest; based on the target classification model corresponding to the target level, classifying the defects of the first intersection area and the second intersection area to obtain the defect types of the single board to be tested;

[0084] Through the target feature extraction module corresponding to the target level, feature extraction is performed on the first intersection area to obtain a first intersection feature map; the second intersection feature map of the second intersection area is extracted; based on the first intersection feature map and the second intersection feature map, defect classification is performed to obtain the defect types of the single board to be tested. Based on the target classification model corresponding to the target level, the defects of the first intersection area and the second intersection area are classified to obtain the defect types of the single board to be tested.

[0085] Among them, the first intersection area is a partial area of the defect area, and the second intersection area is a partial area of the area of interest.

[0086] Specifically, according to the coordinate values of each first pixel point in the defect area and the coordinate values of each second pixel point in the area of interest, a first intersection area that intersects the area of interest is determined in the defect area, and a second intersection area that intersects the defect area is determined in the area of interest. The horizontal plane coordinate value of the first pixel point in the first intersection area is the same as the horizontal plane coordinate value of the second pixel point in the second intersection area.

[0087] Optionally, the target classification model includes: a target feature extraction module corresponding to the target level, an initial feature extraction module, a target fusion module, and a target classification module; based on the target classification model corresponding to the target level, defect classification is performed on the first intersection region and the second intersection region to obtain the defect type of the single board to be tested, including:

[0088] Obtain the target feature extraction module corresponding to the target level; when the target level is the surface layer, determine the target feature extraction module as the surface layer feature extraction module, and use the surface layer feature extraction module to extract the first intersection feature map of the first intersection region; when the target level is the transition layer, determine the target feature extraction module as the transition layer feature extraction module, and use the transition layer feature extraction module to extract the first intersection feature map of the first intersection region; when the target level is the deep layer, determine the target feature extraction module as the deep layer feature extraction module, and use the deep layer feature extraction module to extract the first intersection feature map of the first intersection region;

[0089] The surface layer feature extraction module includes: three residual modules; the transition layer feature extraction module includes: five residual modules, and the transition layer feature extraction module also introduces an attention mechanism; the deep layer feature extraction module includes: seven residual modules, and the deep layer feature extraction module also introduces a bidirectional long short-term memory network. For deep defects, the bidirectional long short-term memory network can play a better role.

[0090] Extract the second intersection feature map of the second intersection region through the initial feature extraction module, splice the first intersection feature map and the second intersection feature map using the target fusion module to obtain the first target feature map, and perform defect classification on the first target feature map through the target classification module to obtain the defect type of the single board to be tested.

[0091] In the above embodiment, according to the defect region and the region of interest, the first intersection region and the second intersection region are determined, the defect region and the region of interest can be combined, and further defect localization can be performed. Through the target classification model corresponding to the target level, the defect type of the single board to be tested can be detected specifically, and the accuracy of defect detection can be improved.

[0092] Optionally, the target classification model includes: a target feature extraction module corresponding to the target level, an initial feature extraction module, a first target classification module, and a second target classification module; based on the target classification model corresponding to the target level, defect classification is performed on the defect area and the area of interest to obtain the defect type of the single board to be tested, including: extracting a first feature map of the defect area through the target feature extraction module corresponding to the target level, performing defect classification on the first feature map through the first target classification module to obtain the third defect probability of each first preset type; extracting a second feature map of the area of interest through the initial feature extraction module, performing defect classification on the first feature map through the first target classification module to obtain the fourth defect probability of each second preset type; determining the defect type of the single board to be tested from each first preset type and each second preset type according to each third defect probability and each fourth defect probability.

[0093] Among them, the first preset type may include, but is not limited to, cracks, holes, and delamination; the second preset type may include, but is not limited to, air holes and bridging.

[0094] Determining the defect type of the single board to be tested from each first preset type and each second preset type according to each third defect probability and each fourth defect probability may be to determine the target defect probabilities exceeding the probability threshold among each third defect probability and each fourth defect probability. The number of target defect probabilities may be one or more. For each target defect probability, the first preset type or the second preset type corresponding to the target defect probability is used as the defect type.

[0095] Through the above method, it is possible to detect whether there are defects of complex types such as cracks, holes, and delamination through the defect area, and detect whether there are defects of simple types such as air holes and bridging through the area of interest, which can improve the comprehensiveness and accuracy of defect detection while improving the defect detection efficiency.

[0096] In some embodiments, defect classification is performed based on the defect area and the area of interest to obtain the defect type of the single board to be tested, including: performing defect classification based on the defect area to obtain the first defect probability corresponding to each first preset type; performing defect classification based on the area of interest to obtain the second defect probability corresponding to each second preset type; determining the defect type of the single board to be tested from each first preset type and each second preset type according to each first defect probability and each second defect probability.

[0097] Specifically, performing defect classification based on the defect area to obtain the first defect probability corresponding to each first preset type includes: extracting the feature map of the defect area and performing defect classification on the feature map of the defect area to obtain the first defect probability corresponding to each first preset type.

[0098] Among them, the feature map of the defective area can be obtained by extracting the features of the defective area through a preset feature extraction module, or by determining the target level to which the defective area belongs based on the coordinate values of the defective area, and extracting the feature map of the defective area through the target feature extraction module corresponding to the target level.

[0099] Based on the region of interest, defect classification is performed to obtain the second defect probabilities corresponding to the second preset types respectively, including: extracting the feature map of the region of interest, and performing defect classification on the feature map of the region of interest to obtain the second defect probabilities corresponding to the second preset types respectively.

[0100] According to the first defect probabilities and the second defect probabilities, among the first preset types and the second preset types, the defect type of the single board to be tested is determined. It can be to determine the target defect probabilities exceeding the probability threshold among the first defect probabilities and the second defect probabilities. The number of target defect probabilities can be one or more. For each target defect probability, the first preset type or the second preset type corresponding to the target defect probability is used as the defect type.

[0101] In the above embodiment, complex type defects such as cracks, holes, and delamination can be detected through the defective area, and simple type defects such as air holes and bridging can be detected through the region of interest. While improving the comprehensiveness and accuracy of defect detection, the defect detection efficiency can be improved.

[0102] In a specific example, such as Figure 2 shown, the defect detection of the single board includes:

[0103] (1). Place the single board to be tested into the detection mold, and the defect detection device controls the detection mold to place the single board to be tested into the spiral AXI.

[0104] (2). According to the positions of the key components on the single board to be tested, place the single board to be tested at the preset position; the preset position is the position where the single board to be tested is located during defect detection and is preset.

[0105] (3). The defect detection device uses the spiral AXI to sequentially take pictures of the single board to be tested at three angles with a preset number of frequencies, and obtains three groups of two-dimensional slice images.

[0106] Among them, taking pictures of the single board to be tested with a preset number of frequencies makes the sampling accuracy of each group of two-dimensional slice images be M0.

[0107] For example, take pictures of the single board to be tested from three angles with different frequencies; such as Figure 3As shown in the figure, the single board to be measured in the Cartesian coordinate system is represented by a cube, and the vertex coordinates of the single board to be measured are respectively: (x0, y0, z0), (x1, y1, z1), (x2, y2, z2), (x3, y3, z3), (x4, y4, z4), (x5, y5, z5), (x6, y6, z6) and (x7, y7, z7); the three angles can include: the angle corresponding to the direction vector (a0, b0, c0), the angle corresponding to the direction vector (a1, b1, c1), and the angle corresponding to the direction vector (a2, b2, c2).

[0108] The single board to be measured is photographed at the angle of the direction vector (a0, b0, c0) to obtain the first group of two-dimensional slice image groups. The first group of two-dimensional slice image groups is parallel to the plane A0, and the plane A0 is the plane formed by the vertices: (x0, y0, z0), (x1, y1, z1), (x2, y2, z2), (x3, y3, z3).

[0109] The single board to be measured is photographed at the angle of the direction vector (a1, b1, c1) to obtain the second group of two-dimensional slice image groups. The second group of two-dimensional slice image groups is parallel to the plane B0, and the plane B0 is the plane formed by the vertices: (x0, y0, z0), (x3, y3, z3), (x4, y4, z4), (x7, y7, z7).

[0110] The single board to be measured is photographed at the angle of the direction vector (a2, b2, c2) to obtain the third group of two-dimensional slice image groups. The third group of two-dimensional slice image groups is parallel to the plane C0, and the plane C0 is the plane formed by the vertices: (x0, y0, z0), (x1, y1, z1), (x4, y4, z4), (x5, y5, z5).

[0111] (4) Defect detection is performed on the three two-dimensional slice image groups to obtain the defect area.

[0112] Specifically, as Figure 4 shown, through the first feature extraction module, the candidate feature maps of the three two-dimensional slice image groups are extracted respectively. The three candidate feature maps are input into the first feature fusion module to obtain a three-dimensional feature map, and the three-dimensional feature map is input into the first classification module to obtain the defect area.

[0113] Among them, as Figure 5 shown, the first feature extraction module includes: a two-dimensional convolutional layer (Conv2D), a first max pooling layer (MaxPooling2D-1), a first residual block (Residual Block-1), a second max pooling layer (MaxPooling2D-2), and a second residual block (Residual Block-2).

[0114] The two-dimensional convolutional layer uses the ReLU activation function, and the max pooling layer is used for dimensionality reduction and extraction of main features; the residual block alleviates the vanishing gradient problem in the deep network through skip connections, improving the training effect of the model; the structure of the residual block (the first residual block or the second residual block) is as Figure 6 shown. The residual block includes two two-dimensional convolutional layers, two activation layers, and a batch normalization layer. There is a skip connection between the input item of the residual block and the second convolutional layer; the activation layer uses the ReLU activation function.

[0115] (5) Perform a transmission scan on the single board under test to obtain a two-dimensional grayscale image, and perform defect detection on the two-dimensional grayscale image to obtain the region of interest.

[0116] (6) Determine the first intersection region that intersects with the region of interest in the defect region; determine the second intersection region that intersects with the defect region in the region of interest.

[0117] (7) According to the coordinate values of the defect region, determine the target level to which the defect region belongs; through the target feature extraction module corresponding to the target level, extract features from the first intersection region to obtain the first intersection feature map; through the initial feature extraction module, extract features from the second intersection region to obtain the second intersection feature map.

[0118] Taking the target level as the transition layer as an example; as Figure 7 shown, defect detection is performed based on three groups of two-dimensional slice images to obtain the defect region, defect detection is performed on the two-dimensional grayscale image to obtain the region of interest; determine the first intersection region that intersects with the region of interest in the defect region; determine the second intersection region that intersects with the defect region in the region of interest; determine that the defect region belongs to the transition layer; input the first intersection region into the transition layer feature extraction module to obtain the first intersection feature map; through the initial feature extraction module, extract features from the second intersection region to obtain the second intersection feature map.

[0119] The transition layer feature extraction module includes: five residual blocks, a self-attention unit, and a fully connected layer; each residual block includes: a first three-dimensional convolutional layer, a batch normalization layer, a second three-dimensional convolutional layer, a batch normalization layer, and there is a skip connection between the input of the residual block and the second three-dimensional convolutional layer; among them, the three-dimensional convolutional layer uses the ReLU activation function; through the self-attention unit, the model can better focus on important feature regions, improving the accuracy of defect detection.

[0120] (8) Use the target fusion module to splice the first intersection feature map and the second intersection feature map to obtain the first target feature map, and perform defect classification on the first target feature map through the target classification module to obtain the defect type of the single board under test.

[0121] (9) When a defect is detected in the single board to be tested, the single board to be tested is photographed again at three candidate angles, and the defect detection of the single board to be tested is carried out again according to the above steps.

[0122] (10) When the defect detection is carried out again and a defect is detected in the single board to be tested, it is determined that the test of the single board to be tested fails, and the maintenance work station is controlled to receive the single board to be tested.

[0123] (11) When it is detected that the single board to be tested has no defect, or when the defect detection is carried out again and it is detected that the single board to be tested has no defect, it is determined that the test of the single board to be tested passes, and the subsequent work station is controlled to receive the single board to be tested.

[0124] (12) Control the detection mold to eject the single board to be tested.

[0125] The defect detection method for a single board provided by the embodiment of the present application photographs the single board to be tested from multiple angles through an automatic X-ray detection device to obtain multiple two-dimensional slice image groups, performs transmission scanning on the single board to be tested to obtain a two-dimensional grayscale image, performs defect detection based on the multiple two-dimensional slice image groups to obtain a defect area, performs defect detection based on the two-dimensional grayscale image to obtain a region of interest, and performs defect classification based on the defect area and the region of interest to obtain the defect type of the single board to be tested; since the multiple two-dimensional slice image groups provide the internal situation of the single board to be tested from multiple angles, the situation where the internal components of the single board to be tested block each other can be avoided, the defect area has the characteristics of complex defects in the single board to be tested, the region of interest is determined according to the two-dimensional grayscale image, reflecting the global density distribution of the single board to be tested, the region of interest has the characteristics of simple defects in the single board to be tested, and the region of interest can be quickly determined according to the two-dimensional grayscale image. Therefore, combining the defect area and the region of interest for defect detection can improve the comprehensiveness, accuracy and efficiency of defect detection.

[0126] It should be understood that although the steps in the flowcharts involved in the above embodiments are shown in sequence according to the arrows, these steps do not necessarily have to be executed in the order indicated by the arrows. Unless there is a clear indication in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages. These steps or stages do not necessarily have to be executed at the same time, but can be executed at different times. The execution order of these steps or stages does not necessarily have to be sequential, but can be executed alternately or alternately with at least a part of other steps or steps in other steps.

[0127] Figure 8 It is a schematic structural diagram of the defect detection device for a single board provided by the present application, as Figure 8As shown in the figure, the defect detection device 80 for a single board provided in this embodiment includes:

[0128] A first image acquisition module 801, configured to capture the single board to be measured from multiple angles through an automatic X-ray detection device, and obtain a group of two-dimensional slice images corresponding to each of the multiple angles;

[0129] A second image acquisition module 802, configured to perform transmission scanning on the single board to be measured, and obtain a two-dimensional grayscale image;

[0130] A first defect detection module 803, configured to perform defect detection based on the group of two-dimensional slice images, and obtain a defect area;

[0131] A second defect detection module 804, configured to perform defect detection on the two-dimensional grayscale image, and obtain a region of interest;

[0132] A defect classification module 805, configured to perform defect classification based on the defect area and the region of interest, and obtain the defect type of the single board to be measured.

[0133] In a possible implementation manner, the defect classification module 805 is further configured to determine the target level to which the defect area belongs based on the coordinate values of the defect area; and perform defect classification on the defect area and the region of interest based on the target classification model corresponding to the target level, so as to obtain the defect type of the single board to be measured.

[0134] In a possible implementation manner, the defect classification module 805 is further configured to determine a first intersection area that intersects with the region of interest in the defect area; determine a second intersection area that intersects with the defect area in the region of interest; and perform defect classification on the first intersection area and the second intersection area based on the target classification model corresponding to the target level, so as to obtain the defect type of the single board to be measured.

[0135] In a possible implementation manner, the defect classification module 805 is further configured to determine a target depth value based on the coordinate values, X-ray absorption rate, and thermal distribution characteristics of the defect area; and determine the target level to which the defect area belongs according to the preset interval to which the target depth value belongs.

[0136] In a possible implementation manner, the defect classification module 805 is further configured to perform defect classification based on the defect area, and obtain first defect probabilities corresponding to each of the first preset types; perform defect classification based on the region of interest, and obtain second defect probabilities corresponding to each of the second preset types; and determine the defect type of the single board to be measured among each of the first preset types and each of the second preset types according to each of the first defect probabilities and each of the second defect probabilities.

[0137] In a possible implementation, the first defect detection module 803 is further configured to extract features from multiple two-dimensional slice image groups respectively to obtain multiple candidate feature maps; fuse the multiple candidate feature maps to obtain a three-dimensional feature map; and perform defect detection on the three-dimensional feature map to obtain a defect region.

[0138] The defect detection device for a single board provided in this embodiment can execute the defect detection method for a single board provided in the above method embodiment, and its implementation principle and technical effects are similar, which will not be elaborated here in this embodiment.

[0139] Figure 9 It is a schematic structural diagram of an electronic device provided in this application. As Figure 9 shown, the electronic device 90 provided in this embodiment includes: at least one processor 901 and a memory 902. Optionally, the device 90 further includes a communication component 903. Among them, the processor 901, the memory 902, and the communication component 903 are connected through a bus.

[0140] In a specific implementation process, at least one processor 901 executes computer-executable instructions stored in the memory 902, so that at least one processor 901 executes the above method.

[0141] For the specific implementation process of the processor 901, reference can be made to the above method embodiment, and its implementation principle and technical effects are similar, which will not be elaborated here in this embodiment.

[0142] In the above embodiment, it should be understood that the processor may be a central processing unit (English: Central Processing Unit, abbreviated as: CPU), or may also be other general-purpose processors, digital signal processors (English: Digital Signal Processor, abbreviated as: DSP), application specific integrated circuits (English: Application Specific Integrated Circuit, abbreviated as: ASIC), etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in combination with the invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules in the processor.

[0143] The memory may include a high-speed memory (Random Access Memory, RAM), and may also include a non-volatile memory (Non-volatile Memory, NVM), such as at least one disk memory.

[0144] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the buses in the drawings of this application are not limited to only one bus or one type of bus.

[0145] This application also provides a computer program product, including a computer program, which implements the above method when executed by a processor.

[0146] This application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When the processor executes the computer-executable instructions, the above method is implemented.

[0147] The above-readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, a magnetic disk or an optical disc. The readable storage medium can be any available medium accessible by a general-purpose or special-purpose computer.

[0148] An exemplary readable storage medium is coupled to the processor, enabling the processor to read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an Application Specific Integrated Circuits (ASIC). Of course, the processor and the readable storage medium can also exist as discrete components in a device.

[0149] The division of units is only a logical function division. In actual implementation, there can be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection to each other can be through some interfaces. The indirect coupling or communication connection of devices or units can be in an electrical, mechanical or other form.

[0150] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0151] In addition, in each embodiment of the present invention, the various functional units may be integrated in a processing unit, may exist separately as individual physical units, or two or more units may be integrated in one unit.

[0152] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present invention. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.

[0153] Those of ordinary skill in the art can understand that all or part of the steps of implementing the above method embodiments can be completed by hardware related to program instructions. The foregoing program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps including the above method embodiments; and the foregoing storage medium includes: various media such as ROM, RAM, magnetic disks, or optical discs that can store program codes.

[0154] Finally, it should be noted that: after considering the specification and practicing the invention disclosed herein, those skilled in the art will easily think of other implementation schemes of the present invention. The present invention is intended to cover any variations, uses, or adaptive changes of the present invention, which follow the general principles of the present invention and include common general knowledge or conventional technical means in the technical field not disclosed in the present invention. It is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present invention is only limited by the appended claims.

Claims

1. A method for defect detection of a single board, characterized in that, Including: Taking pictures of the single board to be tested from multiple angles by an automatic X-ray detection device to obtain a group of two-dimensional slice images corresponding to the multiple angles respectively; Performing transmission scanning on the single board to be tested to obtain a two-dimensional grayscale image; Performing defect detection based on a group of multiple two-dimensional slice images to obtain a defect area; Performing defect detection on the two-dimensional grayscale image to obtain a region of interest; Performing defect classification based on the defect area and the region of interest to obtain the defect type of the single board to be tested.

2. The method according to claim 1, wherein The performing defect classification based on the defect area and the region of interest to obtain the defect type of the single board to be tested includes: Determining the target level to which the defect area belongs based on the coordinate values of the defect area; Performing defect classification on the defect area and the region of interest based on the target classification model corresponding to the target level to obtain the defect type of the single board to be tested.

3. The method according to claim 2, wherein The performing defect classification on the defect area and the region of interest based on the target classification model corresponding to the target level to obtain the defect type of the single board to be tested includes: Determining a first intersection area intersecting with the region of interest in the defect area; Determining a second intersection area intersecting with the defect area in the region of interest; Performing defect classification on the first intersection area and the second intersection area based on the target classification model corresponding to the target level to obtain the defect type of the single board to be tested.

4. The method according to claim 2, wherein The determining the target level to which the defect area belongs based on the coordinate values of the defect area includes: Determining a target depth value based on the coordinate values of the defect area, the X-ray absorption rate, and the thermal distribution characteristics; Determining the target level to which the defect area belongs according to the preset interval to which the target depth value belongs.

5. The method according to claim 1, characterized in that The performing defect classification based on the defect area and the region of interest to obtain the defect type of the single board to be tested includes: Performing defect classification based on the defect area to obtain first defect probabilities corresponding to respective first preset types; Performing defect classification based on the region of interest to obtain second defect probabilities corresponding to respective second preset types; Determining the defect type of the single board to be tested among the respective first preset types and the respective second preset types according to the respective first defect probabilities and the respective second defect probabilities.

6. The method according to any one of claims 1 to 5, characterized in that, The performing defect detection based on a group of multiple two-dimensional slice images to obtain a defect area includes: Respectively performing feature extraction on a group of multiple two-dimensional slice images to obtain a group of multiple candidate feature maps; Fusing the group of multiple candidate feature maps to obtain a three-dimensional feature map; Performing defect detection on the three-dimensional feature map to obtain a defect area.

7. A defect detection device for a single board, characterized in that, The device includes: A first image acquisition module, configured to take pictures of the single board to be tested from multiple angles by an automatic X-ray detection device to obtain a group of two-dimensional slice images corresponding to the multiple angles respectively; A second image acquisition module, configured to perform transmission scanning on the single board to be tested to obtain a two-dimensional grayscale image; A first defect detection module, configured to perform defect detection based on a group of multiple two-dimensional slice images to obtain a defect area; A second defect detection module, configured to perform defect detection on the two-dimensional grayscale image to obtain a region of interest; A defect classification module, configured to perform defect classification based on the defect area and the region of interest to obtain the defect type of the single board under test.

8. An electronic device, characterized in that, It includes: A processor and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, Computer-executable instructions are stored in the computer-readable storage medium, and when the computer-executable instructions are executed by a processor, they are used to implement the method according to any one of claims 1 to 6.

10. A computer program product, characterized in that, It includes computer-executable instructions, and when the computer-executable instructions are executed by a processor, they implement the method according to any one of claims 1 to 6.

Images