AI vision-based computing card welding quality real-time detection system

Abstract

The application relates to the technical field of image processing, in particular to an AI vision-based computing card welding quality real-time detection system, which comprises a processor and a memory, and the processor executes the computer program of the memory to realize the following steps: acquiring suspected welding defect areas in an X-ray image of a target computing card, acquiring high-suspected overlapping areas according to the centroid offset degree and the gray distribution uniformity of each suspected welding defect area, obtaining a comprehensive defect score of each high-suspected overlapping area according to the difference between the spectral image of each high-suspected overlapping area and the adjacent area and the gray distribution in the sub-image of different scales of each high-suspected overlapping area; and acquiring a score image according to the comprehensive defect score of each high-suspected overlapping area, performing feature fusion on the score image and the X-ray image, and identifying the welding defect area, so that the accuracy of detecting the welding defect area in the computing card is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and in particular to an AI vision-based real-time detection system for welding quality of a computing card. BACKGROUND

[0002] Welding quality is a key determinant of the reliability of high-density electronic devices such as computing cards. The problem lies in abnormal welding processes, such as uneven solder paste printing, inaccurate reflow soldering temperature curves, or poor coplanarity of component pins. Hidden solder joints under BGA, CSP and other packages, as a typical defect, are usually manifested as significant differences in solder joint morphology, gray scale or texture from standard solder joints. If efficient and accurate quality detection is not implemented, it may lead to early product failure, customer complaints and huge after-sales costs.

[0003] Currently, the industry usually uses X-ray imaging technology and image processing algorithms to detect hidden solder joints. By penetrating the rays to obtain the internal image of the computing card, the image is preprocessed and compared with the template or the defect is identified by a deep learning model. However, in the actual detection process, the extremely high component density in the computing card may cause the overlapping of solder joints and wires of different heights in the two-dimensional projection, resulting in complex mottled images. These shadows formed by the superposition of normal structures are very similar to the real solder defect area in terms of gray scale, texture and other features, so that the deep learning model learns ambiguous features, which seriously interferes with the accuracy and efficiency of the detection of the solder defect area.

[0004] Therefore, how to improve the accuracy of detecting the solder defect area in the computing card has become a problem to be solved. SUMMARY

[0005] Therefore, the embodiments of the present application provide an AI vision-based real-time detection system for welding quality of a computing card to solve the problem of how to improve the accuracy of detecting the solder defect area in the computing card.

[0006] The embodiments of the present application provide an AI vision-based real-time detection system for welding quality of a computing card, which includes a memory, a processor and a computer program stored in the memory and running on the processor. The processor implements the following steps when executing the computer program:

[0007] An X-ray image of a target computing card is obtained to obtain at least one suspected solder defect area in the X-ray image. Whether each suspected solder defect area is a height suspected overlapping area is determined according to the centroid offset degree and the gray scale distribution uniformity of each suspected solder defect area.

[0008] If there is a highly suspected overlapping area, for any highly suspected overlapping area, the any highly suspected overlapping area is converted into a frequency spectrum image, and an initial defect score of the any highly suspected overlapping area is obtained according to a difference between a frequency spectrum image of the any highly suspected overlapping area and a neighborhood area thereof in the X-ray image;

[0009] The any highly suspected overlapping area is up-sampled and down-sampled to obtain a preset number of sub-images of different scales of the any highly suspected overlapping area, and an initial defect score of the any highly suspected overlapping area is optimized according to a gray scale distribution in each sub-image to obtain a comprehensive defect score of the any highly suspected overlapping area;

[0010] According to the comprehensive defect score of each highly suspected overlapping area in the X-ray image, the X-ray image is converted into a score image, feature fusion is performed on the score image and the X-ray image, and a welding defect area in the target algorithm card is identified according to a result of the feature fusion, so as to realize real-time detection of the welding quality of the target algorithm card.

[0011] Preferably, the judging whether each suspected welding defect area is a highly suspected overlapping area according to the centroid offset degree and the gray scale distribution uniformity of each suspected welding defect area comprises:

[0012] An PCB design drawing of the target algorithm card is obtained, and a two-dimensional rectangular coordinate system of the X-ray image and the PCB design drawing is obtained respectively by using an image registration technology;

[0013] For any suspected welding defect area, a centroid coordinate of the any suspected welding defect area is obtained according to a two-dimensional rectangular coordinate system of the X-ray image, and is recorded as an actual centroid coordinate, a centroid coordinate of a corresponding pad of the any suspected welding defect area is obtained according to a two-dimensional rectangular coordinate system of the PCB design drawing, and is recorded as a standard centroid coordinate, an Euclidean distance between the actual centroid coordinate and the standard centroid coordinate is calculated to obtain a centroid offset degree of the any suspected welding defect area, and an inverse number of the centroid offset degree is taken as an independent variable of an exponential function with a natural constant as a base number to obtain a geometric position coordination degree of the any suspected welding defect area;

[0014] A standard deviation of gray scale values of all pixel points in the any suspected welding defect area is obtained, and an inverse number of the standard deviation is taken as an independent variable of an exponential function with a natural constant as a base number to obtain a gray scale coordination degree of the any suspected welding defect area;

[0015] The geometric position coordination degree and the gray scale coordination degree are weighted and summed to obtain a projection overlap degree of the any suspected welding defect area;

[0016] Obtaining a projection overlap degree of each suspected welding defect region, judging whether each suspected welding defect region is a highly suspected overlap region according to the projection overlap degree of each suspected welding defect region.

[0017] Preferably, judging whether each suspected welding defect region is a highly suspected overlap region according to the projection overlap degree of each suspected welding defect region comprises:

[0018] For any suspected welding defect region, if the projection overlap degree of the any suspected welding defect region is greater than a preset projection overlap degree threshold, the any suspected welding defect region is recorded as a highly suspected overlap region.

[0019] Preferably, obtaining an initial defect score of the any highly suspected overlap region according to the difference between the spectral image of the any highly suspected overlap region and the spectral image of the neighborhood region of the any highly suspected overlap region comprises:

[0020] In the X-ray image, obtaining a suspected welding defect region adjacent to the any highly suspected overlap region in eight neighborhood directions of the any highly suspected overlap region, and recording the suspected welding defect region as a neighborhood region of the any highly suspected overlap region.

[0021] For any neighborhood region, obtaining a log power spectrum of the any highly suspected overlap region and a log power spectrum of the any neighborhood region according to the spectral image of the any highly suspected overlap region and the spectral image of the any neighborhood region, respectively, calculating a cosine similarity of the log power spectrum of the any highly suspected overlap region and the log power spectrum of the any neighborhood region, and recording the cosine similarity as a spectral similarity between the any highly suspected overlap region and the any neighborhood region.

[0022] Obtaining a spectral similarity between the any highly suspected overlap region and each of the neighborhood regions, respectively, subtracting a constant 1 from an average value of all spectral similarities, and obtaining an initial defect score of the any highly suspected overlap region.

[0023] Preferably, optimizing the initial defect score of the any highly suspected overlap region according to the gray scale distribution in each sub-image to obtain a comprehensive defect score of the any highly suspected overlap region comprises:

[0024] Calculating a standard deviation of the gray scale values of all pixel points in the any highly suspected overlap region, taking an inverse of the standard deviation as an independent variable of an exponential function with a natural constant as a base number, and obtaining a gray scale distribution uniformity of the any highly suspected overlap region.

[0025] respectively according to the standard deviation of the gray scale values of the pixel points in each sub-image, obtaining the gray scale distribution uniformity of each sub-image, calculating the average value of the gray scale distribution uniformity of all sub-images and the any highly suspected overlapping area, and obtaining the multi-scale gray scale uniformity of the any highly suspected overlapping area;

[0026] taking the inverse of the product of the multi-scale gray scale uniformity and the optimal attenuation coefficient as the independent variable of the exponential function with the natural constant as the base, obtaining the optimization coefficient of the initial defect score of the any highly suspected overlapping area;

[0027] calculating the product of the optimization coefficient and the initial defect score of the any highly suspected overlapping area, and obtaining the comprehensive defect score of the any highly suspected overlapping area.

[0028] Preferably, the obtaining of the optimal attenuation coefficient comprises:

[0029] obtaining other algorithm cards of the same welding batch as the target algorithm card, denoted as experimental algorithm cards, wherein the experimental algorithm cards at least contain an actual welding defect area, obtaining the multi-scale gray scale uniformity and the initial defect score of each experimental highly suspected overlapping area in the experimental X-ray image of the experimental algorithm cards;

[0030] according to a preset initial attenuation coefficient, the multi-scale gray scale uniformity and the initial defect score of each of the experimental highly suspected overlapping areas, obtaining the comprehensive defect score of each of the experimental highly suspected overlapping areas;

[0031] according to the comprehensive defect score of each of the experimental highly suspected overlapping areas in the experimental X-ray image, converting the experimental X-ray image into an experimental score image, performing feature fusion on the experimental score image and the experimental X-ray image, identifying the welding defect area in the experimental algorithm cards according to the result of the feature fusion, and obtaining the false positive rate and the false negative rate of identifying the welding defect area in the experimental algorithm cards;

[0032] if the false positive rate meets the preset industry false positive standard range, and the false negative rates all meet the preset industry false negative standard range, then taking the preset initial attenuation coefficient as the optimal attenuation coefficient;

[0033] if the false positive rate is greater than the preset industry false positive standard range, and the false negative rates meet the preset industry false negative standard range, then calculating the sum between the preset initial attenuation coefficient and a preset adjustment coefficient to obtain a new attenuation coefficient;

[0034] if the false negative rates are greater than the preset industry false negative standard range, then calculating the difference between the preset initial attenuation coefficient and a preset adjustment coefficient to obtain a new attenuation coefficient;

[0035] According to the new attenuation coefficient, and the multi-scale gray uniformity and initial defect score of each experimental high-suspected overlapping area, a new false positive rate and a new false negative rate for identifying the welding defect area in the experimental computing card are obtained;

[0036] According to the new false positive rate and the new false negative rate, the new attenuation coefficient is adjusted to obtain an adjusted new attenuation coefficient, the adjusted new attenuation coefficient is taken as the new attenuation coefficient, the method for obtaining the new false positive rate and the new false negative rate is repeated until the new false positive rate and the new false negative rate meet preset conditions, a new attenuation coefficient meeting the preset conditions is taken as an optimal attenuation coefficient, and the preset conditions include that the new false positive rate and the new false negative rate meet preset industry false positive standard ranges and preset industry false negative standard ranges respectively, or the false negative rate meets the preset industry false negative standard range, and a new false positive rate corresponding to the adjusted new attenuation coefficient is equal to a new false positive rate corresponding to a last round of adjusted new attenuation coefficient.

[0037] Preferably, if there is no high-suspected overlapping area, the X-ray image of the target computing card is input into a preset welding defect detection model to obtain the welding defect area in the target computing card.

[0038] Compared with the prior art, the embodiment of the present application has the following beneficial effects:

[0039] The application obtains an X-ray image of a target computing power card, obtains at least one suspected welding defect area in the X-ray image, judges whether each suspected welding defect area is a high suspected overlapping area according to a centroid offset degree and a gray scale distribution uniformity of each suspected welding defect area, if there is a high suspected overlapping area, for any high suspected overlapping area, the any high suspected overlapping area is converted into a frequency spectrum image, in the X-ray image, an initial defect score of the any high suspected overlapping area is obtained according to a difference between the frequency spectrum image of the any high suspected overlapping area and a neighborhood area thereof, upsampling and downsampling are performed on the any high suspected overlapping area, a preset number of sub-images of different scales of the any high suspected overlapping area are obtained, the initial defect score of the any high suspected overlapping area is optimized according to a gray scale distribution in each sub-image, and a comprehensive defect score of the any high suspected overlapping area is obtained, the X-ray image is converted into a score image according to the comprehensive defect score of each high suspected overlapping area in the X-ray image, feature fusion is performed on the score image and the X-ray image, and a welding defect area in the target computing power card is recognized according to a result of the feature fusion, so as to realize real-time detection of the welding quality of the target computing power card. Wherein, the comprehensive defect score of each high suspected overlapping area is obtained according to a frequency spectrum and a gray scale distribution of each high suspected overlapping area, and is used to represent a credibility of each high suspected overlapping area as a real welding defect, so as to reduce the cost of recognizing normal projection overlapping areas and real welding defect areas by using a three-dimensional X-ray image, and improve the accuracy of detecting the welding defect area in the computing power card by using a deep learning model. BRIEF DESCRIPTION OF DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0041] Figure 1 is a flow chart of an AI vision-based computing power card welding quality real-time detection method provided by the embodiment one of the present application;

[0042] Figure 2 is a schematic diagram of an X-ray image of a computing power card provided by the embodiment one of the present application. DETAILED DESCRIPTION

[0043] The embodiments of the present application will be described in detail below, and examples of the embodiments are shown in the drawings. The embodiments described below by referring to the drawings are exemplary and are intended to explain the present application, and cannot be understood as a limitation of the present application.

[0044] It should be noted that the terms "first", "second", etc. in the specification of the present disclosure and the above drawings are used to distinguish similar objects, and do not necessarily describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present disclosure described herein can be implemented in an order other than that illustrated or described herein. The implementation described in the following exemplary embodiments does not represent all implementations consistent with the present disclosure. Rather, they are merely examples of devices and methods consistent with some aspects of the present disclosure.

[0045] In order to illustrate the technical solutions of the present application, the following will be described by specific embodiments.

[0046] The specific scenario targeted by the present application is that: the industry usually uses X-ray imaging technology and image processing algorithms to detect hidden weld points, but the extremely high component density in the power card can cause weld points and wires of different heights to overlap in two-dimensional projection, producing complex mottled images. These shadows formed by the superposition of normal structures are very similar to real welding defect areas in terms of gray scale, texture and other features, so that the deep learning model learns ambiguous features, which seriously interferes with the accuracy and efficiency of the detection of the welding defect area. Therefore, according to the position of the highly suspected overlapping area which is extremely similar to the real welding defect area in morphology, and further combining the features such as frequency spectrum and gray scale uniformity to comprehensively obtain the comprehensive defect score of the highly suspected overlapping area, the deep learning model learns to optimize the features, and the accuracy of detecting the welding defect area in the power card is improved.

[0047] The embodiment of the present application provides a kind of based on AI vision's power card welding quality real-time detection system, including processor and memory, the processor executes the computer program stored in the memory to realize a kind of based on AI vision's power card welding quality real-time detection method, as shown in Figure 1 The method comprises the following steps:

[0048] Step S101, obtain the X-ray image of the target power card, obtain at least one suspected welding defect area in the X-ray image, and judge whether each suspected welding defect area is a highly suspected overlapping area according to the centroid offset degree and the gray scale distribution uniformity of each suspected welding defect area.

[0049] Any to be detected computing power card is recorded as a target computing power card, under constant X-ray source key parameters, using a high-resolution area array X-ray camera to collect two-dimensional projection images of the target computing power card, considering that the actually collected two-dimensional projection images are often affected by noise, scattering, uneven equipment response and other factors, so in the embodiment of the present application, the collected two-dimensional projection images of the target computing power card are preprocessed to obtain the X-ray image of the target computing power card, referring to Figure 2 , which is a schematic diagram of an X-ray image of a computing power card, Figure 2 , the black circle area is the welding area. The image preprocessing includes but is not limited to illumination correction, blur and focus correction, filtering and denoising, etc., and the image preprocessing technology is prior art, which will not be described here.

[0050] Further, a significant detection algorithm based on frequency domain residual is used to detect the welding abnormal area in the X-ray image of the target computing power card, and the significant detection algorithm based on frequency domain residual is prior art, which will not be described here. Due to the high component density in the computing power card, different height solder joints and wires may overlap in the two-dimensional projection, resulting in complex mottling images. These shadows formed by the superposition of normal structures are very similar to the real welding defect area in gray scale, texture and other characteristics. For example: two normally independent and circular solder balls overlap due to layout and angle, forming a continuous, dumbbell-shaped or "8" shaped dark area on the two-dimensional plane, which is similar to the defect form of "solder bridge" or "excess solder"; two solder balls with different heights but partially overlapping in the horizontal plane, due to the influence of X-ray attenuation, the solder ball with lower position is similar in form to the defect form of "hole" or "crack". In the traditional way, the normal projection overlapping area and the real welding defect area can be identified through three-dimensional X-ray images, but the cost of identifying them through three-dimensional X-ray images is high, and the efficiency is very low, so in the embodiment of the present application, the normal projection overlapping area and the real welding defect area are identified according to the two-dimensional X-ray image.

[0051] In the welding abnormal area detected by the significant detection algorithm based on frequency domain residual, there may be both projection overlapping area and real welding defect area, so the welding abnormal area detected by the significant detection algorithm based on frequency domain residual is marked as a suspected welding defect, which is used for analyzing each suspected welding defect area to detect the real welding defect area and improve the accuracy of detecting the welding defect area in the computing power card.

[0052] For real defect areas, the centroid of the solder joint is shown to be seriously deviated from the geometric center of the pad, or the gray value distribution is extremely uneven; while for the projection area similar in shape to the real defect, the image of two independent solder joints is essentially superimposed, the gray value is not from one entity, is affected by X-ray attenuation, has a clear brightness gradient in the two-dimensional plane, that is, the gray value distribution is relatively uniform, and the deviation degree of the centroid is small, so in the embodiment of the application, first, the projection overlap degree of each suspected solder defect area is obtained according to the above characteristics, to screen out the height suspected overlap area similar in height to the projection overlap, so as to facilitate further analysis and obtain the possibility of each height suspected overlap area being a real solder defect area, and improve the accuracy of detecting the solder defect area in the algorithm card.

[0053] Taking the i-th suspected solder defect area as an example, the specific way of obtaining the projection overlap degree of the i-th suspected solder defect area is as follows:

[0054] The PCB design diagram of the target algorithm card is obtained, and the two-dimensional rectangular coordinate systems of the X-ray image and the PCB design diagram are obtained respectively by using image registration technology, so as to ensure that the origin of the two-dimensional rectangular coordinate system of the X-ray image is aligned with the origin of the two-dimensional rectangular coordinate system of the PCB design diagram. The image registration technology is prior art, which will not be described here.

[0055] The centroid coordinates of the i-th suspected solder defect area are obtained according to the two-dimensional rectangular coordinate system of the X-ray image, denoted as actual centroid coordinates, the centroid coordinates of the pad corresponding to the i-th suspected solder defect area are obtained according to the two-dimensional rectangular coordinate system of the PCB design diagram, denoted as standard centroid coordinates, the Euclidean distance between the actual centroid coordinates and the standard centroid coordinates is calculated, and the centroid deviation degree of the i-th suspected solder defect area is obtained. The reciprocal of the centroid deviation degree is taken as the independent variable of the exponential function with the natural constant as the base number, and the geometric position coordination degree of the i-th suspected solder defect area is obtained.

[0056] The standard deviation of the gray value of all pixel points in the i-th suspected solder defect area is obtained, the reciprocal of the standard deviation is taken as the independent variable of the exponential function with the natural constant as the base number, and the gray coordination degree of the i-th suspected solder defect area is obtained.

[0057] The geometric position coordination degree and the gray coordination degree are weighted and summed to obtain the projection overlap degree of the i-th suspected solder defect area.

[0058] In an embodiment, the calculation formula of the projection overlap degree of the i-th suspected solder defect area is as follows:

[0059]

[0060] wherein, a projection overlap degree representing the i-th suspected solder defect region, a centroid offset degree representing the i-th suspected solder defect region, a standard deviation of gray values of all pixel points in the i-th suspected solder defect region, a first weight, a second weight, in the embodiment of the present application, the second weight is set as , herein, the implementer can set according to the scene, an exponential function with a natural constant e as the base.

[0061] It should be noted that, the greater the value is, the more the position of the solder ball in the i-th suspected solder defect region deviates from the position of the actual pad, and further the smaller the value is, the more the i-th suspected solder defect region conforms to the characteristics of the real solder defect region; the greater the value is, the more uneven the gray value distribution of the i-th suspected solder defect region is, and further the smaller the value is, the more the i-th suspected solder defect region conforms to the characteristics of the real solder defect region.

[0062] Further, according to statistical standards, at least 3000 samples of known projection overlap regions are collected, and the projection overlap degree is calculated. Herein, the implementer can set the number of samples according to the scene, and the value of the 5th percentile of the projection overlap degree of all samples is obtained as a threshold value, which is denoted as a preset projection overlap degree threshold value.

[0063] The projection overlap degree of each suspected solder defect region in the X-ray image is obtained, and the suspected solder defect region greater than the preset projection overlap degree threshold value is recorded as a highly suspected overlap region.

[0064] In step S102, if there is a highly suspected overlap region, for any highly suspected overlap region, the any highly suspected overlap region is converted into a frequency spectrum image, and according to the difference between the frequency spectrum image of the any highly suspected overlap region and its neighborhood region in the X-ray image, an initial defect score of the any highly suspected overlap region is obtained.

[0065] If there is no highly suspected overlap region, the X-ray image of the target computing power card is input into a preset solder defect detection model to obtain a solder defect region in the target computing power card, wherein the preset solder defect detection model is trained by YOLOv8. Herein, the implementer can set according to the scene.

[0066] If there are highly suspected overlapping regions, it means that the preset welding defect detection model is likely to mistake these regions as real welding defect regions and cannot distinguish the normal projection overlapping regions therein, which may cause inaccurate detection of the welding defect regions in the graphics card. Therefore, further analysis is needed to find out the regions that are likely to be real welding defects.

[0067] Considering that the spectrum of the projection overlapping region is continuous in the local region, and the real welding defect will destroy this continuity, taking the kth highly suspected overlapping region as an example, in the X-ray image, the suspected welding defect region adjacent to the kth highly suspected overlapping region is obtained in the eight-neighbor direction of the kth highly suspected overlapping region, denoted as the neighbor region of the kth highly suspected overlapping region, and the kth highly suspected overlapping region and its neighbor region are respectively converted into a spectrum image by using two-dimensional discrete Fourier transform. According to the difference between the spectrum images of the kth highly suspected overlapping region and its neighbor region, the initial defect score of the kth highly suspected overlapping region is obtained to preliminarily judge the possibility of the kth highly suspected overlapping region being a real welding defect region. The two-dimensional discrete Fourier transform is a prior art and will not be described here.

[0068] The specific way of obtaining the initial defect score of the kth highly suspected overlapping region is: for any neighbor region, according to the spectrum images of the kth highly suspected overlapping region and the any neighbor region, the logarithmic power spectrum of the kth highly suspected overlapping region and the any neighbor region is respectively obtained, and the cosine similarity of the logarithmic power spectrum of the kth highly suspected overlapping region and the any neighbor region is calculated, denoted as the spectrum similarity of the kth highly suspected overlapping region and the any neighbor region. The acquisition of the logarithmic power spectrum is a prior art and will not be described here.

[0069] The spectrum similarity of the kth highly suspected overlapping region and each of the neighbor regions is obtained respectively, and the average value of all spectrum similarities is subtracted by a constant 1 to obtain the initial defect score of the kth highly suspected overlapping region.

[0070] In an embodiment, the calculation formula of the initial defect score of the kth highly suspected overlapping region is:

[0071]

[0072] wherein, represents the initial defect score of the kth highly suspected overlapping region, N represents the number of neighbor regions of the kth highly suspected overlapping region (in the embodiment of the present application, the maximum value of N is 8), represents the logarithmic power spectrum of the kth highly suspected overlapping region, represents the logarithmic power spectrum of the jth neighbor region, a cosine similarity between a log power spectrum of the kth highly suspected overlapping region and a log power spectrum of the jth neighborhood region.

[0073] It should be noted that, The smaller, the greater the difference between the kth highly suspected overlapping region and the jth neighborhood region in the frequency domain structure, that is, the more discontinuous the spectrum of the kth highly suspected overlapping region in the local region, and thus The greater, the greater the likelihood that the kth highly suspected overlapping region is a real welding defect region.

[0074] In step S103, the any highly suspected overlapping region is up-sampled and down-sampled to obtain a preset number of sub-images of different scales of the any highly suspected overlapping region, and the initial defect score of the any highly suspected overlapping region is optimized according to the gray scale distribution in each sub-image to obtain a comprehensive defect score of the any highly suspected overlapping region.

[0075] Considering that a normal projection overlap may exhibit accidental spectral differences at a single scale, for example, a coincidence of local textures causes spectral differences, but after scale enlargement or reduction, the overall spectrum conforms to the normal spectral characteristics, which may cause a normal projection overlap to be misjudged as a real welding defect region, that is, the initial defect score of the kth highly suspected overlapping region obtained based on the spectral difference in step S102 may be inaccurate. Since the gray scale values of a real welding defect exhibit unevenness under single-scale or multi-scale observation, in the embodiment of the present application, the kth highly suspected overlapping region is up-sampled and down-sampled to obtain a preset number of sub-images of different scales of the kth highly suspected overlapping region. Up-sampling and down-sampling are prior art and will not be described here. In the embodiment of the present application, the preset number is set to 3, the down-sampling scale is 0.5 times the original scale, and the up-sampling scale is 2 times the original scale. The preset number, up-sampling scale and down-sampling scale are not limited here and can be set according to specific scenarios by the implementer. The initial defect score of the kth highly suspected overlapping region is optimized according to the gray scale uniformity at different scales to obtain a comprehensive defect score of the kth highly suspected overlapping region. Specifically,

[0076] The standard deviation of the gray scale values of all pixel points in the kth highly suspected overlapping region is calculated, and the inverse of the standard deviation is taken as the argument of an exponential function with a natural constant as the base number to obtain the gray scale distribution uniformity of the kth highly suspected overlapping region.

[0077] According to a standard deviation of the gray scale values of the pixels in each sub-image, a gray scale distribution uniformity of each sub-image is obtained, and an average of the gray scale distribution uniformities of all the sub-images and the kth highly suspected overlapping region is calculated to obtain a multi-scale gray scale uniformity of the kth highly suspected overlapping region, denoted as U, that is, wherein M represents the number of sub-images, and in the embodiment of the present application, M = 2, represents a standard deviation of the gray scale values of all the pixels in the mth sub-image, that is, a gray scale distribution uniformity of the mth sub-image, represents a gray scale distribution uniformity of the kth highly suspected overlapping region, represents an exponential function with a natural constant e as a base number, The greater U is, the more uniform the distribution of the gray scale values in the kth highly suspected overlapping region is, and the smaller U is, the less uniform the distribution of the gray scale values in the kth highly suspected overlapping region is. The greater U is, the more uniform the distribution of the gray scale values in the kth highly suspected overlapping region is, and the smaller U is, the less uniform the distribution of the gray scale values in the kth highly suspected overlapping region is.

[0078] An inverse of a product of the multi-scale gray scale uniformity and the optimal attenuation coefficient is taken as an independent variable of an exponential function with a natural constant as a base number to obtain an optimization coefficient of the initial defect score of the kth highly suspected overlapping region.

[0079] A product of the optimization coefficient and the initial defect score of the kth highly suspected overlapping region is calculated to obtain a comprehensive defect score of the kth highly suspected overlapping region.

[0080] In an embodiment, a calculation formula of the comprehensive defect score of the kth highly suspected overlapping region is:

[0081]

[0082] wherein, represents the comprehensive defect score of the kth highly suspected overlapping region, represents the initial defect score of the kth highly suspected overlapping region, and U represents the multi-scale gray scale uniformity of the kth highly suspected overlapping region, represents the optimal attenuation coefficient, represents an exponential function with a natural constant e as a base number.

[0083] It should be noted that the greater U is, the more uniform the distribution of the gray scale values in the kth highly suspected overlapping region is, and the higher the possibility that the kth highly suspected overlapping region is misjudged as a real welding defect region is, and the smaller U is, the less uniform the distribution of the gray scale values in the kth highly suspected overlapping region is, and the lower the possibility that the kth highly suspected overlapping region is misjudged as a real welding defect region is. The smaller the U is, the smaller the possibility that the kth highly suspected overlapping region is a real welding defect region is; on the contrary, the smaller the U is, the more uneven the gray value distribution of the kth highly suspected overlapping region is, the smaller the possibility that the kth highly suspected overlapping region is misjudged as a real welding defect region is, and further tends to the initial defect score of the kth highly suspected overlapping region.

[0084] wherein the optimal attenuation coefficient is obtained in the following manner:

[0085] An other power card with the same welding batch as the target power card is obtained, denoted as an experimental power card, the experimental power card containing at least one actual welding defect region, and the multi-scale gray uniformity and initial defect score of each experimental highly suspected overlapping region in an experimental X-ray image of the experimental power card are obtained.

[0086] A preset initial attenuation coefficient is set to 1, which is not limited here, and the implementer can set it according to the scene, and according to the preset initial attenuation coefficient, the multi-scale gray uniformity and initial defect score of each experimental highly suspected overlapping region are used to obtain the comprehensive defect score of each experimental highly suspected overlapping region.

[0087] According to the comprehensive defect score of each experimental highly suspected overlapping region in the experimental X-ray image, the experimental X-ray image is converted into an experimental score image, the experimental score image and the experimental X-ray image are feature fused, the welding defect region in the experimental power card is identified according to the feature fusion result, and the false positive rate and the false negative rate of identifying the welding defect region in the experimental power card are obtained.

[0088] If the false positive rate meets the preset industry false positive standard range, the false positive rate is less than 3% in the electronic industry scene, so the preset industry false positive standard range is set to [0, 3%], which is not limited here, and the implementer can set it according to the scene, and the false negative rate meets the preset industry false negative standard range, the preset initial attenuation coefficient is taken as the optimal attenuation coefficient;

[0089] If the false positive rate is greater than the preset industry false positive standard range, and the false negative rate meets the preset industry false negative standard range, the sum of the preset initial attenuation coefficient and a preset adjustment coefficient is calculated to obtain a new attenuation coefficient, wherein in the electronic industry scene, the false negative rate is less than 1%, so the preset industry false negative standard range is set to [0, 1%], and the preset adjustment coefficient y is set to 0.2, which is not limited here, and the implementer can set it according to the scene.

[0090] If the false negative rate is greater than the preset industry false negative standard range, a difference between the preset initial attenuation coefficient and a preset adjustment coefficient is calculated to obtain a new attenuation coefficient. It should be noted that, in an ideal case, both the false positive rate and the false negative rate need to be controlled within the standard range, but it is extremely difficult to achieve in reality, and there is a "trade-off" relationship between the two. The usual strategy is to prioritize ensuring that the false negative rate is within the standard range, and then do everything possible to reduce the false positive rate. Therefore, when the false negative rate is greater than the preset industry false negative standard range, the false negative rate is first ensured to be within the standard range.

[0091] According to the new attenuation coefficient, and the multi-scale gray uniformity and initial defect score of each of the experimental highly suspected overlapping regions, a new false positive rate and a new false negative rate for identifying the welding defect regions in the experimental algorithm card are obtained.

[0092] According to the new false positive rate and the new false negative rate, the new attenuation coefficient is adjusted to obtain an adjusted new attenuation coefficient. The adjusted new attenuation coefficient is taken as the new attenuation coefficient, and the method of obtaining the new false positive rate and the new false negative rate is repeated until the new false positive rate and the new false negative rate meet a preset condition. The new attenuation coefficient corresponding to the preset condition is taken as an optimal attenuation coefficient. The preset condition includes that the new false positive rate and the new false negative rate meet a preset industry false positive standard range and a preset industry false negative standard range, respectively, or the false negative rate meets the preset industry false negative standard range, and the new false positive rate corresponding to the adjusted new attenuation coefficient is equal to the new false positive rate corresponding to the last round of adjusted new attenuation coefficient.

[0093] At this point, the comprehensive defect score of the kth highly suspected overlapping region is obtained.

[0094] In step S104, according to the comprehensive defect score of each highly suspected overlapping region in the X-ray image, the X-ray image is converted into a score image, the score image and the X-ray image are feature fused, and the welding defect region in the target algorithm card is identified according to the feature fusion result, which is used for real-time detection of the welding quality of the target algorithm card.

[0095] According to the steps of obtaining the comprehensive defect score of the kth highly suspected overlapping region in step S103, the comprehensive defect score of each highly suspected overlapping region in the X-ray image is obtained.

[0096] Further, according to the comprehensive defect score of each highly suspected overlapping area in the X-ray image, the X-ray image is converted into a score image to encode abstract'score' information into an image format that can be processed by a convolutional network, which is aligned with the original image space size. The steps are roughly as follows: (1) create a full zero matrix with only one channel and the same size as the X-ray image as the initial score image; (2) the comprehensive defect score of each highly suspected overlapping area in the X-ray image, i.e. the comprehensive defect score of each pixel in the highly suspected overlapping area in the X-ray image, is mapped to the initial score image and smoothed and normalized using Gaussian filtering and other techniques to obtain the score image. Converting the X-ray image into a score image is prior art and will not be described here.

[0097] Further, the score image is feature-fused with the feature map of the X-ray image. Feature fusion can be performed at the pixel level or the feature level, such as feature-level attention fusion: the feature map of the X-ray image is extracted by a target detection neural network such as YOLOv8, ResNet, etc., and the score image is multiplied with the feature map as a spatial attention mask using PyTorch and a broadcast mechanism to obtain a weighted feature map. According to the weighted feature map, the position of the welding defect area in the target GPU is obtained using the trained welding defect detection model. Feature fusion and training of the welding defect detection model are prior art and will not be described here. The welding defect detection model is trained by YOLOv8, which is not limited here and can be set by the implementer according to the scene.

[0098] It is worth noting that the focus of the present application is how to obtain the credibility of suspected defect areas as real welding defect areas in two-dimensional X-ray images, i.e. the comprehensive defect score of each highly suspected overlapping area, for identifying normal projection areas that are highly similar to real welding defect areas. Feature fusion and detection of welding defect area positions are prior art and will not be described here.

[0099] In summary, the X-ray image of the target computing power card is obtained, at least one suspected welding defect area in the X-ray image is obtained, whether each suspected welding defect area is a highly suspected overlapping area is judged according to the centroid offset degree and the gray distribution uniformity of each suspected welding defect area, if there is a highly suspected overlapping area, for any highly suspected overlapping area, the any highly suspected overlapping area is converted into a spectrum image, in the X-ray image, according to the difference between the spectrum image of the any highly suspected overlapping area and its neighborhood area, the initial defect score of the any highly suspected overlapping area is obtained, upsampling and downsampling are performed on the any highly suspected overlapping area, a preset number of sub-images of different scales of the any highly suspected overlapping area are obtained, the initial defect score of the any highly suspected overlapping area is optimized according to the gray distribution in each sub-image, and the comprehensive defect score of the any highly suspected overlapping area is obtained, according to the comprehensive defect score of each highly suspected overlapping area in the X-ray image, the X-ray image is converted into a score image, feature fusion is performed on the score image and the X-ray image, and the welding defect area in the target computing power card is recognized according to the result of the feature fusion, so that the welding quality of the target computing power card is detected in real time. Wherein, according to the spectrum and the gray distribution of each highly suspected overlapping area, the comprehensive defect score of each highly suspected overlapping area is obtained, which is used to represent the credibility of each highly suspected overlapping area as a real welding defect, while reducing the cost of identifying normal projection overlapping areas and real welding defect areas by using three-dimensional X-ray images, the accuracy of detecting the welding defect area in the computing power card by using the deep learning model is improved.

[0100] The above examples are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing examples, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing examples, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A system for real-time detection of welding quality of an AI vision-based computing card, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, The processor implements the following steps when executing the computer program: An X-ray image of the target computing power card is acquired, at least one suspected welding defect area in the X-ray image is obtained, whether each suspected welding defect area is a highly suspected overlapping area is judged according to a centroid offset degree and a gray distribution uniformity of each suspected welding defect area; If there is a highly suspected overlapping area, for any highly suspected overlapping area, the any highly suspected overlapping area is converted into a frequency spectrum image, and an initial defect score of the any highly suspected overlapping area is obtained according to a difference between the frequency spectrum image of the any highly suspected overlapping area and a neighborhood area thereof in the X-ray image; The any highly suspected overlapping area is up-sampled and down-sampled to obtain a preset number of sub-images of different scales of the any highly suspected overlapping area, the initial defect score of the any highly suspected overlapping area is optimized according to a gray distribution in each sub-image, and a comprehensive defect score of the any highly suspected overlapping area is obtained; The X-ray image is converted into a score image according to the comprehensive defect score of each highly suspected overlapping area in the X-ray image, feature fusion is performed on the score image and the X-ray image, and a welding defect area in the target computing power card is recognized according to a result of the feature fusion, so as to realize real-time detection of a welding quality of the target computing power card; The judgment of whether each suspected welding defect area is a highly suspected overlapping area according to the centroid offset degree and the gray distribution uniformity of each suspected welding defect area comprises: A PCB design drawing of the target computing power card is acquired, and a two-dimensional rectangular coordinate system of the X-ray image and the PCB design drawing is respectively acquired by using an image registration technology; For any suspected welding defect area, a centroid coordinate of the any suspected welding defect area is acquired according to the two-dimensional rectangular coordinate system of the X-ray image, and is recorded as an actual centroid coordinate, a centroid coordinate of a pad corresponding to the any suspected welding defect area is acquired according to the two-dimensional rectangular coordinate system of the PCB design drawing, and is recorded as a standard centroid coordinate, an Euclidean distance between the actual centroid coordinate and the standard centroid coordinate is calculated to obtain a centroid offset degree of the any suspected welding defect area, and a reciprocal of the centroid offset degree is taken as an argument of an exponential function with a natural constant as a base to obtain a geometric position coordination degree of the any suspected welding defect area; A standard deviation of gray values of all pixel points in the any suspected welding defect area is acquired, and a reciprocal of the standard deviation is taken as an argument of an exponential function with a natural constant as a base to obtain a gray coordination degree of the any suspected welding defect area; The geometric position coordination degree and the gray coordination degree are weighted and summed to obtain a projection overlapping degree of the any suspected welding defect area, the projection overlapping degrees of each suspected welding defect area are acquired, and whether each suspected welding defect area is a highly suspected overlapping area is judged according to the projection overlapping degrees of each suspected welding defect area.

2. The AI vision-based real-time detection system for power card welding quality according to claim 1, characterized in that, The judgment of whether each suspected welding defect area is a highly suspected overlapping area according to the projection overlapping degrees of each suspected welding defect area comprises: If the projection overlap degree of any suspected welding defect region is greater than the preset projection overlap degree threshold, the any suspected welding defect region is recorded as a highly suspected overlap region.

3. The AI vision-based real-time detection system for power card welding quality according to claim 1, wherein, The initial defect score of the any highly suspected overlap region is obtained according to the difference between the spectral image of the any highly suspected overlap region and the spectral image of the neighborhood region of the any highly suspected overlap region, and the initial defect score of the any highly suspected overlap region is obtained according to the difference between the spectral image of the any highly suspected overlap region and the spectral image of the neighborhood region of the any highly suspected overlap region. In the X-ray image, the suspected welding defect regions adjacent to the any highly suspected overlap region are obtained in the eight neighborhood directions of the any highly suspected overlap region, and are recorded as the neighborhood regions of the any highly suspected overlap region. For any neighborhood region, the log power spectrum of the any highly suspected overlap region and the log power spectrum of the any neighborhood region are obtained according to the spectral image of the any highly suspected overlap region and the spectral image of the any neighborhood region, respectively, the cosine similarity of the log power spectrum of the any highly suspected overlap region and the log power spectrum of the any neighborhood region is calculated, and is recorded as the spectral similarity between the any highly suspected overlap region and the any neighborhood region. The spectral similarity between the any highly suspected overlap region and each of the neighborhood regions is obtained, and the average value of all spectral similarities is subtracted by a constant 1 to obtain the initial defect score of the any highly suspected overlap region.

4. The AI vision-based real-time detection system for power card welding quality according to claim 1, wherein, The initial defect score of the any highly suspected overlap region is optimized according to the gray distribution in each sub-image to obtain the comprehensive defect score of the any highly suspected overlap region, and the initial defect score of the any highly suspected overlap region is optimized according to the gray distribution in each sub-image to obtain the comprehensive defect score of the any highly suspected overlap region. The standard deviation of the gray values of all pixel points in the any highly suspected overlap region is calculated, and the inverse of the standard deviation is taken as the argument of the exponential function with the natural constant as the base to obtain the gray distribution uniformity of the any highly suspected overlap region. The gray distribution uniformity of each sub-image is obtained according to the standard deviation of the gray values of the pixel points in each sub-image, and the average value of the gray distribution uniformity of all sub-images and the any highly suspected overlap region is calculated to obtain the multi-scale gray uniformity of the any highly suspected overlap region. The inverse of the product between the multi-scale gray uniformity and the optimal attenuation coefficient is taken as the argument of the exponential function with the natural constant as the base to obtain the optimization coefficient of the initial defect score of the any highly suspected overlap region. The product between the optimization coefficient and the initial defect score of the any highly suspected overlap region is calculated to obtain the comprehensive defect score of the any highly suspected overlap region.

5. The AI vision-based real-time detection system for power card welding quality according to claim 4, characterized in that, The optimal attenuation coefficient is obtained by: An other computing power card with the same welding batch as the target computing power card is obtained, which is recorded as an experimental computing power card, and the experimental computing power card contains at least one actual welding defect region, and the multi-scale gray uniformity and the initial defect score of each experimental highly suspected overlap region in the experimental X-ray image of the experimental computing power card are obtained. According to the preset initial attenuation coefficient, the multi-scale gray uniformity and the initial defect score of each experimental highly suspected overlap region, the comprehensive defect score of each experimental highly suspected overlap region is obtained. According to the comprehensive defect score of each experimental highly suspected overlapping area in the experimental X-ray image, the experimental X-ray image is converted into an experimental score image, the experimental score image and the experimental X-ray image are feature fused, a welding defect area in the experimental algorithm card is identified according to a result of the feature fusion, and a false positive rate and a false negative rate of identifying the welding defect area in the experimental algorithm card are obtained; If the false positive rate meets a preset industry false positive standard range, and the false negative rate meets a preset industry false negative standard range, the preset initial attenuation coefficient is taken as an optimal attenuation coefficient; If the false positive rate is greater than the preset industry false positive standard range, and the false negative rate meets the preset industry false negative standard range, a sum between the preset initial attenuation coefficient and a preset adjustment coefficient is calculated to obtain a new attenuation coefficient; If the false negative rate is greater than the preset industry false negative standard range, a difference between the preset initial attenuation coefficient and the preset adjustment coefficient is calculated to obtain a new attenuation coefficient; According to the new attenuation coefficient, and a multi-scale gray uniformity degree and an initial defect score of each experimental highly suspected overlapping area, a new false positive rate and a new false negative rate of identifying the welding defect area in the experimental algorithm card are obtained; According to the new false positive rate and the new false negative rate, the new attenuation coefficient is adjusted to obtain an adjusted new attenuation coefficient, the adjusted new attenuation coefficient is taken as the new attenuation coefficient, the method of obtaining the new false positive rate and the new false negative rate is repeated until the new false positive rate and the new false negative rate meet a preset condition, a new attenuation coefficient corresponding to the preset condition is taken as an optimal attenuation coefficient, and the preset condition includes that the new false positive rate and the new false negative rate meet a preset industry false positive standard range and a preset industry false negative standard range respectively, or the false negative rate meets the preset industry false negative standard range, and a new false positive rate corresponding to the adjusted new attenuation coefficient is equal to a new false positive rate corresponding to a last round of adjusted new attenuation coefficient.

6. The AI vision-based real-time detection system for power card welding quality according to claim 1, wherein, If there is no highly suspected overlapping area, an X-ray image of a target algorithm card is input into a preset welding defect detection model to obtain a welding defect area in the target algorithm card.

Images