SMT mounting defect detection method and system based on image recognition

Abstract

The invention relates to the field of image defect detection, in particular to an SMT defect detection method and system based on image recognition, and the method comprises the steps: shooting a to-be-detected circuit board through an optical detection device to obtain an original circuit image, carrying out the illumination compensation of the original circuit image to obtain a target circuit image, carrying out the two-dimensional wavelet decomposition of the target circuit image, and carrying out the detection of the SMT defect. The method comprises the steps of obtaining a horizontal component floating-point diagram and a vertical component floating-point diagram, performing edge identification on a target circuit image according to the horizontal component floating-point diagram and the vertical component floating-point diagram to obtain an edge pixel point set, clustering the edge pixel point set by using the number of elements to obtain a plurality of element edge point sets, and constructing a plurality of surface-mounted component areas based on the plurality of component edge point sets, determining a defect component area group in the plurality of surface-mounted component areas, and transmitting the defect component area group to a display device. According to the invention, the accuracy of SMT mounting defect detection can be improved, and the defect response speed of a production line is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image defect detection, and in particular to an SMT mounting defect detection method and system based on image recognition. BACKGROUND

[0002] With the rapid evolution of electronic manufacturing towards high density and miniaturization, SMT mounting has become the core link of PCBA production. If small defects such as virtual soldering of solder joints, component offset, and less tin are not discovered in time, they will cause functional failure, reliability drop, and even batch recall in terminal products. Therefore, high-precision and full-detection of SMT mounting defects has become an industry demand.

[0003] Traditional technology usually uses threshold segmentation and template matching to distinguish defects in the entire circuit board image. Although this method can cover all components, it is prone to produce a large number of false positive errors due to uneven illumination leading to threshold drift. In addition, since the component level contour is not distinguished, the defect positioning accuracy is low, which makes the subsequent manual re-determination time-consuming. SUMMARY

[0004] The present application provides an SMT mounting defect detection method and system based on image recognition, which aims to improve the accuracy of SMT mounting defect detection and improve the defect response speed of the production line.

[0005] To achieve the above purpose, the present application provides an SMT mounting defect detection method based on image recognition, which comprises: Receiving a defect detection instruction, determining a mounted circuit board based on the defect detection instruction, moving the mounted circuit board to a pre-constructed optical detection device, and obtaining a to-be-detected circuit board, wherein the optical detection device comprises a high-resolution camera, a ring light source, and an optical filter; Using the optical detection device to take pictures of the to-be-detected circuit board, obtaining an original circuit image, and performing illumination compensation on the original circuit image to obtain a target circuit image; Performing two-dimensional wavelet decomposition on the target circuit image to obtain a horizontal component floating point image and a vertical component floating point image; According to the horizontal component floating point image and the vertical component floating point image, performing edge recognition on the target circuit image to obtain a strong edge pixel point set and a weak edge pixel point set; Using the weak edge pixel point set to supplement the strong edge pixel point set to obtain an edge pixel point set; Obtaining the number of components of the mounted circuit board, clustering the edge pixel point set using the number of components to obtain a plurality of component edge point sets; Constructing a plurality of mounted component regions based on a plurality of element edge point sets, comparing a plurality of standard component regions with the plurality of mounted component regions in similarity, obtaining a plurality of mounted defect probability values, identifying a high defect value group from the plurality of mounted defect probability values according to a preset high probability value; Determining a defect component region group corresponding to the high defect value group in the plurality of mounted component regions, and transmitting the defect component region group to a pre-constructed display device to complete SMT mounted defect detection based on image recognition.

[0006] Optionally, the illuminance compensation on the original circuit image to obtain a target circuit image comprises: Confirming an original pixel point set in the original circuit image, wherein the original pixel point set comprises a plurality of original pixel points; Selecting an initial compensation coefficient based on a plurality of historical circuit images; Extracting original pixel points in the original pixel point set in sequence, and performing compensation calculation on the original pixel points by using the initial compensation coefficient to obtain compensated pixel points; Summarizing the compensated pixel points to obtain a compensated pixel point set, updating the original circuit image based on the compensated pixel point set to obtain a compensated circuit image; Performing pixel average difference calculation on the original circuit image and the compensated circuit image to obtain original pixel average difference and target pixel average difference; Calculating a compensation amount based on the original pixel average difference, the target pixel average difference, and a preset compensation interval; If the compensation amount is greater than a preset minimum compensation amount, then calculating an updated compensation coefficient according to the compensation interval and the initial compensation coefficient, wherein the updated compensation coefficient is a value obtained by subtracting the compensation interval from the initial compensation coefficient; Taking the updated compensation coefficient and the compensated circuit image as the initial compensation coefficient and the original circuit image respectively, and returning to the step of confirming the original pixel point set in the original circuit image until the compensation amount is not greater than the minimum compensation amount; If the compensation amount is not greater than the minimum compensation amount, then recording the compensated circuit image as the target circuit image.

[0007] Optionally, the selecting of the initial compensation coefficient based on the plurality of historical circuit images comprises: Extracting historical circuit images in the plurality of historical circuit images in sequence, and calculating historical pixel average difference of the historical circuit images; Setting a compensation coefficient range, and selecting a plurality of historical compensation coefficients in the compensation coefficient range according to the compensation interval; Extracting historical compensation coefficients in the plurality of historical compensation coefficients in sequence; Compensating the historical circuit images according to the historical compensation coefficients to obtain historical compensated images, and performing pixel average difference calculation on the historical compensated images to obtain compensated pixel average difference; calculating a difference ratio based on the historical pixel average difference and the compensated pixel average difference; combining the difference ratio with the historical compensation coefficient to obtain historical compensation data; aggregating the historical compensation data corresponding to the historical compensation coefficient to obtain a historical compensation data group, and merging the historical compensation data groups corresponding to the historical circuit images to obtain a historical compensation data set; performing curve fitting based on the historical compensation data set to obtain a compensation curve, wherein the horizontal axis of the compensation curve is the historical compensation coefficient in the historical compensation data, and the vertical axis of the compensation curve is the difference ratio in the historical compensation data; identifying an inflection point in the compensation curve, and determining an initial compensation coefficient based on the inflection point, wherein the initial compensation coefficient is the horizontal coordinate of the inflection point.

[0008] Optionally, the compensation calculation on the original pixel point based on the initial compensation coefficient to obtain a compensated pixel point comprises: selecting a rectangular region in the original circuit image according to a preset rectangular size and the original pixel point, wherein the rectangular region includes a plurality of rectangular pixel points, and the original pixel point is the geometric center of the rectangular region; determining a plurality of separated pixel points in the plurality of rectangular pixel points in the rectangular region according to a preset background ratio, and calculating a separated pixel value based on the plurality of separated pixel points; compensating the original pixel point based on the separated pixel value and the initial compensation coefficient to obtain the compensated pixel point.

[0009] Optionally, the compensation calculation on the original pixel point based on the separated pixel value and the initial compensation coefficient to obtain the compensated pixel point comprises: obtaining an original pixel value of the original pixel point; calculating a compensated pixel value based on the original pixel value, the separated pixel value and the initial compensation coefficient, and using the following formula: ; wherein, represents the compensated pixel value, represents the initial compensation coefficient, represents the original pixel value, represents the separated pixel value; updating the compensated pixel point using the compensated pixel value to obtain the compensated pixel point.

[0010] Optionally, the edge recognition on the target circuit image based on the horizontal component floating point graph and the vertical component floating point graph to obtain a strong edge pixel point set and a weak edge pixel point set comprises: determining a target pixel point set in the target circuit image, wherein the target pixel point set includes a plurality of target pixel points; Extract the target pixel points in turn in the target pixel point set, and perform maximum value suppression on the target pixel points based on the horizontal component floating point graph and the vertical component floating point graph to obtain updated pixel points; Update the pixel points to obtain an updated pixel point set, and update the target circuit image using the updated pixel point set to obtain an updated circuit image, wherein the updated circuit image is composed of the updated pixel point set; Construct a gray level histogram of the updated circuit image, and select a high gray level threshold and a low gray level threshold based on the gray level histogram; According to the high gray level threshold and the low gray level threshold, the updated pixel point set in the updated circuit image is filtered to obtain a strong edge pixel point set and a weak edge pixel point set.

[0011] Optionally, the maximum value suppression on the target pixel points based on the horizontal component floating point graph and the vertical component floating point graph to obtain the updated pixel points comprises: Extract the horizontal pixel points and the vertical pixel points corresponding to the target pixel points in the horizontal component floating point graph and the vertical component floating point graph, respectively; According to the horizontal pixel points and the vertical pixel points, calculate the edge strength and the edge direction; Based on the edge direction, the target pixel point and the preset number of adjacent pixels, identify a plurality of adjacent pixels in the target circuit image; Obtain the adjacent edge strength of each adjacent pixel in the plurality of adjacent pixels to obtain a plurality of adjacent edge strengths; Based on the plurality of adjacent edge strengths, update the target pixel point to obtain the updated pixel point.

[0012] Optionally, the edge point supplementing of the strong edge pixel point set using the weak edge pixel point set to obtain the edge pixel point set comprises: Extract the weak edge pixel points in the weak edge pixel point set in turn, and select the near neighbors of the weak edge pixel points according to a preset local window to obtain a plurality of local pixel points; Based on the strong edge pixel point set, obtain a local strong edge point group in the plurality of local pixel points, and count the number of edge points in the local strong edge point group; If the number of edge points is greater than a preset minimum number, the weak edge pixel point is recorded as a qualified edge pixel point, and the qualified edge pixel point is added to the strong edge pixel point set to obtain an updated strong edge point set; The updated strong edge point set is taken as the strong edge pixel point set, and the step of extracting the weak edge pixel points in the weak edge pixel point set in turn is returned until the weak edge pixel point set is empty; When the weak edge pixel point set is empty, the updated strong edge point set is recorded as the edge pixel point set.

[0013] Optionally, the edge pixel point set is clustered based on the element quantity to obtain a plurality of element edge point sets, comprising: An initial center coordinate set is generated based on the element quantity and a preset pixel coordinate range, wherein the number of initial center coordinates in the initial center coordinate set is the same as the element quantity; An initial clustering distribution set is generated based on the initial center coordinate set; Edge pixel points in the edge pixel point set are sequentially extracted, and a center distance set is obtained by respectively calculating the center distance of each edge pixel point to each initial center coordinate in the initial center coordinate set; The edge pixel points are assigned to the initial clustering distribution set according to the center distance set, and the assignment of the edge pixel points is completed; When each edge pixel point in the edge pixel point set is assigned, the initial clustering distribution set is recorded as a target clustering distribution set; A target center coordinate set is calculated based on the target clustering distribution set, and a clustering change rate is calculated according to the initial center coordinate set and the target center coordinate set; If the clustering change rate is greater than a preset standard change rate, the target center coordinate set and the target clustering distribution set are taken as the initial center coordinate set and the initial clustering distribution set respectively, and the step of sequentially extracting edge pixel points in the edge pixel point set is returned until the clustering change rate is not greater than the standard change rate; If the clustering change rate is not greater than the standard change rate, a plurality of element edge point sets are confirmed based on the target clustering distribution set, wherein each element edge point set in the plurality of element edge point sets corresponds to a target clustering distribution in the target clustering distribution set.

[0014] To achieve the above object, the application further provides an SMT mounting defect detection system based on image recognition, comprising: A target image acquisition module is configured to receive a defect detection instruction, determine a mounting circuit board based on the defect detection instruction, move the mounting circuit board to a pre-constructed optical detection device, and obtain a to-be-detected circuit board, wherein the optical detection device comprises a high-resolution camera, a ring-shaped light source and an optical filter, the to-be-detected circuit board is photographed by using the optical detection device to obtain an original circuit image, and the original circuit image is subjected to illumination compensation to obtain a target circuit image; An edge pixel recognition module is configured to perform two-dimensional wavelet decomposition on the target circuit image to obtain a horizontal component floating point image and a vertical component floating point image, perform edge recognition on the target circuit image based on the horizontal component floating point image and the vertical component floating point image, and obtain a strong edge pixel point set and a weak edge pixel point set; An edge pixel clustering module is configured to supplement the set of strong edge pixels with the set of weak edge pixels to obtain a set of edge pixels, obtain a number of components of the circuit board, and cluster the set of edge pixels according to the number of components to obtain a plurality of sets of component edge points. A defect component display module is configured to construct a plurality of component mounting areas based on the plurality of sets of component edge points, compare a plurality of standard component areas with the plurality of component mounting areas to obtain a plurality of component mounting defect probability values, identify a high defect value group from the plurality of component mounting defect probability values according to a preset high probability value, determine a defect component area group corresponding to the high defect value group from the plurality of component mounting areas, and transmit the defect component area group to a pre-constructed display device.

[0015] To solve the above problems, the present application further provides an electronic device, which comprises: a memory configured to store at least one instruction; and a processor configured to execute the instruction stored in the memory to implement the SMT mounting defect detection method based on image recognition.

[0016] To solve the above problems, the present application further provides a computer readable storage medium, which stores at least one instruction, and the at least one instruction is executed by a processor in an electronic device to implement the SMT mounting defect detection method based on image recognition.

[0017] The present application is to solve the problems described in the background art, first, the original circuit image is subjected to illumination compensation to obtain a target circuit image, this step avoids the misjudgment problem caused by uneven illumination in the traditional method through the iterative illumination compensation algorithm, and in this process, the historical optimal compensation coefficient is combined as the initial value to significantly speed up the compensation convergence process, at the same time, through the dynamic comparison mechanism of compensation amount and minimum compensation amount, it is ensured that the compensation process improves the gray scale uniformity without introducing additional noise, thereby greatly improving the accuracy of subsequent image analysis, then the target circuit image is subjected to two-dimensional wavelet decomposition to obtain a horizontal component floating point image and a vertical component floating point image, this step separates the horizontal and vertical feature information of the component edge through two-dimensional wavelet decomposition of the target circuit image, which can more accurately capture the subtle changes of the component outline compared with the traditional edge detection method, providing higher distinguishability feature input for subsequent edge recognition, then the target circuit image is subjected to edge recognition according to the horizontal component floating point image and the vertical component floating point image to obtain a strong edge pixel point set and a weak edge pixel point set, this step significantly reduces the interference of false edges through non-maximum suppression and a double-threshold screening mechanism, making the division of the strong edge pixel point set and the weak edge pixel point set more accurate, further, the present application uses the weak edge pixel point set to supplement the strong edge pixel point set to obtain an edge pixel point set, this step supplements the weak edge pixel point to the strong edge pixel point set through local window connectivity analysis, effectively repairing the edge breakage problem caused by illumination or noise, significantly improving the integrity of the component outline, then the edge pixel point set is clustered using the number of components to obtain a plurality of component edge point sets, this step divides the edge pixel point set into a plurality of component edge point sets by using the number of components as prior knowledge to guide clustering, significantly reducing the misclassification problem caused by the randomness of the initial center in the traditional clustering algorithm, at the same time, the dynamic clustering update mechanism based on the Euclidean distance ensures a high matching degree between each component edge point set and the actual component outline, finally, the defect component region group is transmitted to the display device in real time and highlighted to realize the visualization and rapid positioning of defects, significantly shortening the response time from detection to repair. Therefore, the present application can improve the accuracy of SMT mounting defect detection and improve the defect response speed of the production line. BRIEF DESCRIPTION OF DRAWINGS

[0018] Figure 1 The flowchart of the SMT mounting defect detection method based on image recognition provided by an embodiment of the present application is shown in the figure. Figure 2 The functional module diagram of the SMT mounting defect detection system based on image recognition provided by an embodiment of the present application is shown in the figure. Figure 3A structural schematic diagram of an electronic device for implementing the image recognition-based SMT mounting defect detection method according to an embodiment of the present application is provided.

[0019] Legend of reference signs: 1, electronic device; 10, processor; 11, memory; 12, bus.

[0020] The implementation, functional features and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0021] It should be understood that the specific embodiments described herein are merely intended to explain the present application and not to limit the present application.

[0022] An image recognition-based SMT mounting defect detection method is provided in the embodiments of the present application. The execution subject of the image recognition-based SMT mounting defect detection method includes, but is not limited to, at least one of the electronic devices that can be configured to execute the method provided in the embodiments of the present application, such as a server and a terminal. In other words, the image recognition-based SMT mounting defect detection method can be executed by software or hardware installed in a terminal device or a server device, and the software can be a blockchain platform. The server includes, but is not limited to, a single server, a server cluster, a cloud server or a cloud server cluster, etc.

[0023] Reference Figure 1 Fig. 1 shows a flowchart of the image recognition-based SMT mounting defect detection method according to an embodiment of the present application. In the embodiment, the image recognition-based SMT mounting defect detection method includes the following steps. S1, receiving a defect detection instruction, determining a mounting circuit board based on the defect detection instruction, moving the mounting circuit board to a pre-constructed optical detection device, and obtaining a to-be-detected circuit board, wherein the optical detection device includes a high-resolution camera, a ring-shaped light source and an optical filter.

[0024] It can be understood that the defect detection instruction refers to an instruction for artificially initiating surface defect detection of a specific circuit board. The mounting circuit board refers to the specific circuit board indicated by the defect detection instruction. The mounting circuit board is a printed circuit board that has been processed by printing, component mounting, welding and the like. The mounting circuit board is covered with a plurality of components, such as capacitors, resistors, etc., on the surface. Through subsequent defect detection, defects of the components on the surface of the mounting circuit board, such as open circuit, less tin, etc., can be detected. The to-be-detected circuit board refers to the mounting circuit board after being moved to the optical detection device.

[0025] Further, the optical detection device refers to a device for optical shooting of the surface mount circuit board, wherein the high-resolution camera refers to an industrial camera with micron-level resolution, which is used to capture the fine texture of the elements on the surface of the circuit board. The ring-shaped light source refers to an LED array light source arranged around the camera, which provides uniform shadow-free illumination and eliminates shadows caused by the height difference of the elements. The optical filter refers to a narrow-band pass filter, which is used to suppress environmental stray light interference and enhance the image contrast at a specific wavelength (such as red light). In addition to the above, the optical detection device also includes an omnidirectional moving stage for carrying the surface mount circuit board, which can automatically adjust the position and angle so that the surface mount circuit board can be located directly below the high-resolution camera.

[0026] S2, shooting the circuit board to be detected by using the optical detection device to obtain an original circuit image, and performing light compensation on the original circuit image to obtain a target circuit image.

[0027] It should be explained that the original circuit image refers to the overhead image of the circuit board to be detected after shooting. The target circuit image refers to the original circuit image after light compensation, wherein the purpose of light compensation is to eliminate the gray scale distribution difference caused by uneven light source and improve the reliability of image analysis.

[0028] It should be noted that the original circuit image has undergone preprocessing steps such as filtering and gray scale conversion.

[0029] In detail, the light compensation on the original circuit image to obtain the target circuit image includes: confirming an original pixel point set in the original circuit image, wherein the original pixel point set includes a plurality of original pixel points; selecting an initial compensation coefficient based on a plurality of historical circuit images; extracting the original pixel points in the original pixel point set in turn, and performing compensation calculation on the original pixel points by using the initial compensation coefficient to obtain compensated pixel points; aggregating the compensated pixel points to obtain a compensated pixel point set, updating the original circuit image based on the compensated pixel point set to obtain a compensated circuit image; performing pixel mean difference calculation on the original circuit image and the compensated circuit image to obtain original pixel mean difference and target pixel mean difference; calculating the compensation amount based on the original pixel mean difference, the target pixel mean difference, and a preset compensation interval; if the compensation amount is greater than a preset minimum compensation amount, then calculating an updated compensation coefficient according to the compensation interval and the initial compensation coefficient, wherein the updated compensation coefficient is a value obtained by subtracting the compensation interval from the initial compensation coefficient; The updated compensation coefficient and the compensation circuit image are taken as an initial compensation coefficient and an original circuit image respectively, and the step of confirming the original pixel set in the original circuit image is returned until the compensation amount is not greater than the minimum compensation amount; If the compensation amount is not greater than the minimum compensation amount, the compensation circuit image is recorded as a target circuit image.

[0030] It can be understood that the original pixel set refers to a set of multiple original pixels, wherein the original pixel refers to a pixel in the original circuit image. The multiple historical circuit images refer to multiple images of the printed circuit board obtained in the past period under the optical detection device. The initial compensation coefficient refers to the historical optimal illumination compensation intensity parameter, and the role of the initial compensation coefficient is to serve as a starting point for iterative compensation calculation and accelerate the convergence of subsequent compensation. The compensated pixel refers to the original pixel after compensation calculation. The compensation circuit image refers to the original circuit image after updating, wherein the updating of the original circuit image based on the compensated pixel set refers to replacing the pixel at the corresponding position of the original image with the compensated pixel (i.e. the compensated pixel set).

[0031] Further, the original pixel mean difference refers to a numerical value quantifying the uniformity of the gray value distribution in the original circuit image. The larger the original pixel mean difference, the more uneven the gray value distribution in the original circuit image. The target pixel mean difference refers to a numerical value quantifying the uniformity of the gray value distribution in the target circuit image. The compensation interval refers to a human-set compensation coefficient adjustment step (such as 0.05). The compensation amount refers to a numerical value quantifying the compensation effect. The larger the compensation amount, the more significant the improvement in image gray uniformity. The minimum compensation amount refers to a human-set compensation amount when the compensation process reaches convergence. Optionally, the minimum compensation amount can be set to 0.01. When the compensation amount is greater than the minimum compensation amount, it indicates that the current compensation effect is significant, and there is still room for further compensation. At this time, the compensation circuit image needs to be compensated again. When the compensation amount is not greater than the minimum compensation amount, it indicates that the compensation of the original circuit image has reached saturation. Even if the compensation circuit image is compensated again, the image quality cannot be significantly improved, and even noise may be introduced.

[0032] It can be understood that the calculation method of the original pixel mean difference is as follows: ; Wherein, represents the original pixel mean difference, represents the number of original pixels in the original pixel set, represents the gray value of the original pixel in the original pixel set, ​represents an average value of the gray scale values of all original pixels in the original pixel set. The calculation method of the target pixel average difference is the same as that of the original pixel average difference. The calculation method of the compensation amount is as follows: wherein, represents the compensation amount, and respectively represent the original pixel average difference and the target pixel average difference, represents taking an absolute value, represents a compensation interval.

[0033] In detail, the initial compensation coefficient is selected based on a plurality of historical circuit images, including: extracting historical circuit images in sequence from the plurality of historical circuit images, and calculating a historical pixel average difference of the historical circuit images; setting a compensation coefficient range, and selecting a plurality of historical compensation coefficients in the compensation coefficient range according to the compensation interval; extracting historical compensation coefficients in sequence from the plurality of historical compensation coefficients; compensating the historical circuit images according to the historical compensation coefficients to obtain historical compensation images, and calculating a compensation pixel average difference by performing pixel average difference calculation on the historical compensation images; calculating a difference ratio based on the historical pixel average difference and the compensation pixel average difference; combining the difference ratio with the historical compensation coefficients to obtain historical compensation data; summarizing the historical compensation data corresponding to the historical compensation coefficients to obtain a historical compensation data group, and merging the historical compensation data groups corresponding to the historical circuit images to obtain a historical compensation data set; performing curve fitting on the historical compensation data set to obtain a compensation curve, wherein the horizontal axis of the compensation curve is the historical compensation coefficient in the historical compensation data, and the vertical axis of the compensation curve is the difference ratio in the historical compensation data; identifying an inflection point in the compensation curve, and confirming the initial compensation coefficient based on the inflection point, wherein the initial compensation coefficient is the horizontal coordinate of the inflection point.

[0034] It can be understood that the historical pixel mean difference refers to the pixel mean difference of the historical circuit image, and the calculation method of the historical pixel mean difference is the same as that of the original pixel mean difference. The compensation coefficient range refers to the range of the compensation coefficient set artificially, and the compensation coefficient range can be set to be greater than 0 and less than 2. The historical compensation coefficients selected according to the compensation interval in the compensation coefficient range refer to the compensation coefficients selected in sequence according to the compensation interval. The historical compensation coefficient refers to a certain compensation coefficient in the compensation coefficient range. For example, if the compensation interval is 0.5, the historical compensation coefficients selected in the range greater than 0 and less than 2 are 0.5, 1 and 1.5 (because the range of the compensation interval is greater than 0 and less than 2, the historical compensation coefficient cannot be equal to 0 and cannot be equal to 2).

[0035] Further, the historical compensation image refers to the historical circuit image after compensation, and the steps of compensating the historical circuit image according to the historical compensation coefficient and compensating the original circuit image to obtain the target circuit image are the same. For details, refer to the subsequent steps of obtaining the target circuit image. The mean difference ratio refers to the ratio of the historical pixel mean difference to the compensation pixel mean difference. The mean difference ratio reflects the degree of improvement of the gray scale uniformity after compensation. The greater the mean difference ratio, the higher the degree of improvement of the gray scale uniformity after compensation. The historical compensation data refers to the combination of the mean difference ratio and the historical compensation coefficient. The reason for taking the abscissa of the inflection point as the initial compensation coefficient is that the inflection point of the compensation curve represents the critical point at which the slope of the gray scale uniformity gain curve significantly decreases when the compensation coefficient is further increased. Therefore, taking the abscissa of the inflection point as the initial compensation coefficient can effectively compress the subsequent convergence path and improve the efficiency of illumination compensation.

[0036] In detail, the compensation of the original pixel point by using the initial compensation coefficient to obtain the compensation pixel point comprises: selecting a rectangular region in the original circuit image according to the preset rectangular size and the original pixel point, wherein the rectangular region comprises a plurality of rectangular pixel points, and the original pixel point is the geometric center of the rectangular region; determining a plurality of separated pixel points in the plurality of rectangular pixel points in the rectangular region according to a preset background ratio, and calculating a separated pixel value based on the plurality of separated pixel points; compensating the original pixel point based on the separated pixel value and the initial compensation coefficient to obtain the compensation pixel point.

[0037] It can be understood that the rectangular size is an artificially set neighborhood window size (such as 5x5 pixels). The rectangular region refers to a local image block with the original pixel point as the center and the size of the rectangular size. The background ratio refers to an artificially set constant, which represents the proportion of the pixel points regarded as background in the rectangular region. The greater the background ratio, the more the number of background pixel points.

[0038] It needs to be explained that determining multiple separate pixels from multiple rectangular pixels in a rectangular area according to a preset background ratio refers to: counting the number of original pixels in the original pixel set (denoted as the original number); calculating the number of background pixels based on the original number and the background ratio, where the number of background pixels is the product of the original number and the background ratio; obtaining the rectangular pixel value of each rectangular pixel in the multiple rectangular pixels to obtain multiple rectangular pixel values; identifying the largest 'a' rectangular pixel values ​​from the multiple rectangular pixel values, where 'a' represents the original number minus the background pixel number; and recording the 'a' rectangular pixel values ​​corresponding to these 'a' rectangular pixel values ​​as multiple separate pixels. The separate pixel value refers to the average of the multiple grayscale values ​​corresponding to the multiple separate pixels.

[0039] In detail, the step of compensating the original pixels based on the separated pixel values ​​and the initial compensation coefficient to obtain compensated pixels includes: Get the original pixel value of the original pixel; Based on the original pixel values, separated pixel values, and initial compensation coefficients, the compensated pixel values ​​are calculated using the following formula: ; in, Indicates the compensation pixel value. Indicates the initial compensation coefficient. Represents the original pixel value. Indicates the separated pixel values; The compensation pixel is obtained by updating the compensation pixel value.

[0040] It needs to be explained that the original pixel value refers to the grayscale value of the original pixel. In the above formula for calculating the compensated pixel value, when This indicates that the original pixel value is darker than the surrounding background (i.e., within the rectangular area). The method for calculating the compensation pixel value is: subtract a factor equal to the square of the darkness difference from the original pixel value. The correction amount is proportional to the item, thus making the dark areas brighter (i.e., the gray value becomes smaller), while also avoiding over-brightening of the dark areas with a single compensation. This indicates that the original pixel value is already brighter than the background pixel value, and no additional brightness enhancement is needed.

[0041] S3. Perform two-dimensional wavelet decomposition on the target circuit image to obtain the horizontal component floating-point image and the vertical component floating-point image.

[0042] It can be understood that the horizontal component floating point image refers to a gray image in which only the horizontal change information is retained after wavelet decomposition, and the pixel points in the horizontal component floating point image represent the horizontal edge intensity values of the corresponding pixel points in the target circuit image. The vertical component floating point image refers to a gray image in which only the vertical change information is retained after wavelet decomposition. The above-mentioned two-dimensional wavelet decomposition of the target circuit image refers to high-pass filtering in the horizontal and vertical directions of the target circuit image, so as to obtain the horizontal component floating point image and the vertical component floating point image.

[0043] S4, edge recognition is performed on the target circuit image according to the horizontal component floating point image and the vertical component floating point image, to obtain a strong edge pixel point set and a weak edge pixel point set.

[0044] It should be explained that the strong edge pixel point set refers to a collection of multiple strong edge pixel points, wherein the strong edge pixel point refers to a pixel point representing an edge in the target circuit image. The weak edge pixel point set refers to a collection of multiple weak edge pixel points, wherein the weak edge pixel point refers to a pixel point that may represent an edge in the target circuit image, and whether the weak edge pixel point is an edge also needs to be determined in combination with the strong edge pixel point set.

[0045] In detail, the edge recognition of the target circuit image according to the horizontal component floating point image and the vertical component floating point image to obtain the strong edge pixel point set and the weak edge pixel point set includes: determining a target pixel point set in the target circuit image, wherein the target pixel point set includes multiple target pixel points; extracting the target pixel points in the target pixel point set in sequence, performing maximum value suppression on the target pixel points based on the horizontal component floating point image and the vertical component floating point image, to obtain updated pixel points; updating the target circuit image by using the updated pixel point set, to obtain an updated circuit image, wherein the updated circuit image is composed of the updated pixel point set; constructing a gray histogram of the updated circuit image, and selecting a high gray threshold and a low gray threshold based on the gray histogram; screening the updated pixel point set in the updated circuit image according to the high gray threshold and the low gray threshold, to obtain the strong edge pixel point set and the weak edge pixel point set.

[0046] Need to explain, the target pixel point refers to the pixel point in the target circuit image. The updated pixel point refers to the target pixel point after maximum value suppression, wherein the purpose of maximum value suppression is to refine the edge through non-maximum suppression (NMS), and the detailed steps of the maximum value suppression will be given later. The updated circuit image refers to the target circuit image after updating. The gray histogram refers to the pixel distribution statistics chart of each gray level of the updated circuit image. The high gray threshold refers to the valley value between the two peaks of the histogram, which can be automatically calculated by Otsu algorithm. The low gray threshold refers to 40% of the high threshold (to ensure the continuity of the weak edge).

[0047] In detail, the maximum value suppression is performed on the target pixel point based on the horizontal component floating point graph and the vertical component floating point graph, and the updated pixel point is obtained, comprising: extracting the horizontal pixel point and the vertical pixel point corresponding to the target pixel point in the horizontal component floating point graph and the vertical component floating point graph respectively; calculating the edge strength and the edge direction according to the horizontal pixel point and the vertical pixel point; identifying a plurality of adjacent pixel points in the target circuit image based on the edge direction, the target pixel point and the preset number of adjacent pixels; obtaining the adjacent edge strength of each adjacent pixel point in the plurality of adjacent pixel points, and obtaining a plurality of adjacent edge strengths; updating the target pixel point based on the plurality of adjacent edge strengths to obtain the updated pixel point.

[0048] Need to explain, the horizontal pixel point refers to the pixel point corresponding to the target pixel point in the horizontal component floating point graph, and the horizontal pixel point is obtained by first obtaining the coordinates of the target pixel point in the target circuit image, and then determining the pixel point corresponding to the coordinates in the horizontal component floating point graph, which is the horizontal pixel point. The vertical pixel point refers to the pixel point corresponding to the target pixel point in the vertical component floating point graph, and the acquisition method of the vertical pixel point is the same as that of the horizontal pixel point.

[0049] Further, the edge strength refers to the gradient amplitude at the target pixel point, and the greater the edge strength, the greater the possibility that the target pixel point belongs to the edge. The calculation method of the edge strength is: wherein, indicates the edge strength, indicates the pixel value corresponding to the horizontal pixel point, indicates the pixel corresponding to the vertical pixel. The edge direction refers to the angle of the gradient vector of the target pixel point, and the calculation method of the edge direction is: wherein, indicates the edge direction, indicates the inverse tangent function.

[0050] It can be understood that the adjacent number is a constant set by a person, and the plurality of adjacent pixel points refer to a plurality of pixel points adjacent to the target pixel point in the edge direction, wherein the number of adjacent pixel points in the plurality of adjacent pixel points is the same as the adjacent number. The adjacent edge intensity refers to the edge intensity of the adjacent pixel point.

[0051] It can be understood that the above updating the target pixel point based on the plurality of adjacent edge intensities refers to judging whether the edge intensity of the target pixel point is the maximum edge intensity in the plurality of adjacent edge intensities, if the edge intensity of the target pixel point is the maximum edge intensity in the plurality of adjacent edge intensities, the target pixel point is recorded as an updated pixel point, otherwise, the pixel value of the target pixel point is 0, and the target pixel point with the pixel value of 0 is recorded as an updated pixel point. Through the above operation, the coarse edge can be refined into a single-pixel-width true contour line, so as to remove the false edge and make the subsequent detection more accurate.

[0052] S5, using the weak edge pixel point set to supplement the strong edge pixel point set to obtain an edge pixel point set.

[0053] It needs to be explained that the edge pixel point set refers to a collection of a plurality of edge pixel points, wherein the edge pixel point represents the edge contour in the target circuit image. Since there are pixel points that can represent edges in the weak edge pixel point set, it is necessary to add the weak edge pixel points in this part to the strong edge pixel point set, so as to obtain the edge pixel point set.

[0054] In detail, the use of the weak edge pixel point set to supplement the strong edge pixel point set to obtain the edge pixel point set comprises: extracting the weak edge pixel point in the weak edge pixel point set in turn, and performing near neighbor selection on the weak edge pixel point according to a preset local window to obtain a plurality of local pixel points; based on the strong edge pixel point set, obtaining a local strong edge point group in the plurality of local pixel points, and counting the edge point number of the local strong edge point group; if the edge point number is greater than a preset minimum number, the weak edge pixel point is recorded as a qualified edge pixel point, and the qualified edge pixel point is added to the strong edge pixel point set to obtain an updated strong edge point set; the updated strong edge point set is used as the strong edge pixel point set, and the step of extracting the weak edge pixel point in the weak edge pixel point set in turn is returned until the weak edge pixel point set is empty; the updated strong edge point set when the weak edge pixel point set is empty is recorded as the edge pixel point set.

[0055] The local window refers to a human-set connectivity detection range (e.g., a 3x3 or 5x5 pixel grid). The plurality of local pixels refers to a plurality of pixels with the weak edge pixel as a geometric center and within the local window. The local strong edge point group includes a plurality of local strong edge points, wherein a local strong edge point refers to a local pixel in the plurality of local pixels and belonging to the strong edge pixel set. The minimum number refers to a human-set constant. When the number of edge points is greater than the minimum number, it indicates that there are many strong edge pixels near the weak edge pixel (within the local window), and thus the weak edge pixel can be regarded as a strong edge pixel (i.e., the weak edge pixel is recorded as a qualified edge pixel). The updated strong edge pixel set refers to the strong edge pixel set after adding the qualified edge pixel.

[0056] S6, obtaining the number of components mounted on the circuit board, clustering the edge pixel set using the number of components to obtain a plurality of component edge point sets.

[0057] It should be explained that the number of components refers to the number of components to be installed on the circuit board, which is specified by relevant standards of the circuit board mounting process. The plurality of component edge point sets refers to a plurality of pixel sets obtained by clustering the edge pixel set, wherein each component edge point set represents a component contour on the circuit board.

[0058] In detail, the clustering of the edge pixel set using the number of components to obtain a plurality of component edge point sets includes: generating an initial center coordinate set based on the number of components and a preset pixel coordinate range, wherein the number of initial center coordinates in the initial center coordinate set is the same as the number of components; generating an initial clustering distribution set based on the initial center coordinate set; extracting edge pixels in the edge pixel set in turn, respectively calculating the center distance of each edge pixel to each initial center coordinate in the initial center coordinate set to obtain a center distance set; assigning the edge pixels to the initial clustering distribution set according to the center distance set to complete the assignment of the edge pixels; When each edge pixel in the edge pixel set is assigned, the initial clustering distribution set is recorded as a target clustering distribution set; calculating a target center coordinate set based on the target clustering distribution set, and calculating a clustering change rate according to the initial center coordinate set and the target center coordinate set; If the clustering change rate is greater than a preset standard change rate, the target center coordinate set and the target clustering distribution set are taken as the initial center coordinate set and the initial clustering distribution set, respectively, and the step of extracting edge pixels in the edge pixel set in turn is returned until the clustering change rate is not greater than the standard change rate; If the cluster change rate is not greater than the standard change rate, then multiple element edge point sets are identified based on the target cluster distribution set. Among them, the element edge point sets in the multiple element edge point sets correspond one-to-one with the target cluster distribution in the target cluster distribution set.

[0059] It is clear that the pixel coordinate range refers to the constraint range of the initial center coordinates set manually. This pixel coordinate range can be set according to the size of the mounting circuit board. For example, if the length and width of the rectangular mounting circuit board are obtained through the same process standard as b and c respectively, and the origin of the coordinates in the target circuit image is at the four vertices of the mounting circuit board, then the pixel coordinate range can be set to (0, 0) to (b, c). The initial center coordinate set refers to a set of multiple initial center coordinates, where the initial center coordinates refer to the center point of the corresponding initial cluster distribution in the subsequent clustering process. The initial center coordinates are generated randomly within the pixel coordinate range. The initial cluster distribution set refers to a set of multiple initial cluster distributions, where the initial cluster distribution refers to a set of multiple edge pixels distributed in the same component outline as the corresponding initial center coordinates. At the beginning of clustering, the initial cluster distribution is an empty set, that is, all initial cluster distributions in the initial cluster distribution set generated by the initial center coordinate set are empty sets. The center distance refers to the Euclidean distance between the edge pixel and the initial center coordinate. The process of assigning edge pixels to the initial cluster distribution set based on the center distance set refers to: identifying the minimum center distance in the center distance set, recognizing the initial cluster distribution corresponding to the minimum center distance, and adding the edge pixels to the initial cluster distribution corresponding to the minimum center distance, thus completing the assignment of edge pixels. The clustering change rate refers to a numerical value that quantifies the similarity between the initial center coordinate set and the target center coordinate set. The larger the clustering change rate, the lower the similarity between the initial center coordinate set and the target center coordinate set. The standard change rate refers to a manually set constant. When the clustering change rate is greater than the standard change rate, it indicates that clustering still needs to be performed; when the clustering change rate is not greater than the standard change rate, it indicates that the clustering has converged. The above-mentioned clustering change rate is calculated as follows: ; in, This represents the clustering change rate, where N represents the number of elements. Represents the initial center coordinate set of the first The x-coordinate of the initial center coordinates, Represents the first in the target center coordinate set The x-coordinate of the center coordinates of each target. Represents the initial center coordinate set of the first The ordinate of the initial center coordinates, Represents the first in the target center coordinate set The ordinate of the center coordinates of each target.

[0060] S7, constructing a plurality of mounted component regions based on the plurality of element edge point sets, comparing the plurality of preset standard component regions with the plurality of mounted component regions in similarity, obtaining a plurality of mounted defect probability values, and identifying a high defect value group from the plurality of mounted defect probability values according to a preset high probability value.

[0061] It can be understood that the mounted component region refers to a closed contour region surrounded by a corresponding element edge point set, wherein one element edge point set corresponds to one mounted component region. The standard component region refers to a pre-acquired defect-free same-type component template contour region, which represents a component region with surface defects such as open circuit and little tin, wherein one standard component region corresponds to one mounted component region. The standard component region is obtained by extracting the contours of the same-type components (in the same position as the mounted component region) from a large number of qualified (qualified means no defects) same-model (same model as the mounted circuit board) circuit boards, obtaining a plurality of same-type component contours, wherein one same-type component contour corresponds to one same-model circuit board, and then averaging the plurality of same-type component contours to obtain the standard component region of the mounted component region, wherein averaging refers to averaging the pixel values of a plurality of pixel points with the same position in the plurality of same-type component contours. The mounted defect probability value refers to the similarity between the standard component region and the corresponding mounted component region. The lower the similarity between the standard component region and the corresponding mounted component region, the higher the probability of surface defects in the mounted component region, i.e. the higher the mounted defect probability value. The above comparison of the preset plurality of standard component regions with the plurality of mounted component regions in similarity refers to calculating the structural similarity index of the standard component region and the corresponding mounted component region, and taking the inverse of the structural similarity index as the mounted defect probability value. The high probability value refers to a constant artificially set. When the mounted defect probability value is greater than the high probability value, it means that the corresponding mounted component region has defects. Optionally, the high probability value can be set to 0.85. The high defect value group refers to a combination of a plurality of high defect values, and the high defect value refers to a mounted defect probability value greater than the high probability value.

[0062] S8, determining a defect component region group corresponding to the high defect value group from the plurality of mounted component regions, and transmitting the defect component region group to a pre-constructed display device to complete the SMT mounted defect detection based on image recognition.

[0063] The defect element region group includes a plurality of defect element regions, and the defect element region refers to a mounting element region corresponding to a high defect value, and the defect element region in the defect element region group has defects. The display device refers to an industrial computer or an HMI human-machine interface, and supports highlighting of the defect region. When the defect element region group is obtained, the defect element region group needs to be transmitted to the display device, and the display device will mark the defect element region in the image of the mounted circuit board, and the related personnel will manually repair the defect element region.

[0064] To solve the problems in the background art, first, the original circuit image is subjected to illumination compensation to obtain a target circuit image. This step avoids misjudgment caused by uneven illumination in traditional methods through an iterative illumination compensation algorithm. In this process, the historical optimal compensation coefficient is used as the initial value, which significantly speeds up the compensation convergence process. At the same time, through a dynamic comparison mechanism of the compensation amount and the minimum compensation amount, the compensation process ensures the improvement of the gray scale uniformity without introducing additional noise, thereby significantly improving the accuracy of subsequent image analysis. Next, the target circuit image is subjected to two-dimensional wavelet decomposition to obtain a horizontal component floating point image and a vertical component floating point image. This step decomposes the target circuit image into horizontal and vertical component floating point images through two-dimensional wavelet decomposition, effectively separating the horizontal and vertical feature information of the element edge. Compared with traditional edge detection methods, this decomposition method can more accurately capture the subtle changes in the element contour, providing higher-differentiation feature input for subsequent edge recognition. Then, the target circuit image is subjected to edge recognition based on the horizontal component floating point image and the vertical component floating point image to obtain a strong edge pixel point set and a weak edge pixel point set. This step significantly reduces the interference of false edges through non-maximum suppression and a double-threshold screening mechanism, making the division of the strong edge pixel point set and the weak edge pixel point set more accurate. Further, the scheme uses the weak edge pixel point set to supplement the strong edge pixel point set to obtain an edge pixel point set. This step supplements the weak edge pixel point to the strong edge pixel point set through local window connectivity analysis, effectively repairing the edge breakage caused by illumination or noise, and significantly improving the integrity of the element contour. Then, the edge pixel point set is clustered using the number of elements to obtain a plurality of element edge point sets. This step divides the edge pixel point set into a plurality of element edge point sets by using the number of elements as prior knowledge to guide clustering, significantly reducing the misclassification problem caused by the randomness of the initial center in traditional clustering algorithms. At the same time, a dynamic clustering update mechanism based on the Euclidean distance ensures a high matching degree between each element edge point set and the actual element contour. Finally, the defect element region group is transmitted to the display device in real time and highlighted to realize the visualization and rapid positioning of defects, significantly shortening the response time from detection to repair. Therefore, the present application can improve the accuracy of SMT mounting defect detection and improve the defect response speed of the production line.

[0065] As Figure 2 shown, it is an embodiment provided by the application based on image recognition SMT mounting defect detection system function module diagram.

[0066] The SMT mounting defect detection system based on image recognition 100 can be installed in an electronic device. According to the function realized, the SMT mounting defect detection system based on image recognition 100 can include a target image acquisition module 101, an edge pixel identification module 102, an edge pixel identification module 103, and a defect element display module 104. The modules of the application can also be called units, which refer to a series of computer program segments that can be executed by an electronic device processor and can complete a fixed function, which are stored in the memory of the electronic device.

[0067] The target image acquisition module 101 is used to receive a defect detection instruction, determine a mounting circuit board based on the defect detection instruction, move the mounting circuit board to a pre-constructed optical detection device, and obtain a circuit board to be detected, wherein the optical detection device includes a high-resolution camera, a ring light source, and an optical filter. The optical detection device is used to take pictures of the circuit board to be detected to obtain an original circuit image, and the original circuit image is subjected to light compensation to obtain a target circuit image. The edge pixel identification module 102 is used to perform two-dimensional wavelet decomposition on the target circuit image to obtain a horizontal component floating point image and a vertical component floating point image, perform edge identification on the target circuit image according to the horizontal component floating point image and the vertical component floating point image, and obtain a strong edge pixel point set and a weak edge pixel point set. The edge pixel clustering module 103 is used to supplement the strong edge pixel point set with the weak edge pixel point set to obtain an edge pixel point set, obtain the number of elements of the mounting circuit board, and cluster the edge pixel point set according to the number of elements to obtain a plurality of element edge point sets. The defect element display module 104 is used to construct a plurality of mounting element regions based on the plurality of element edge point sets, compare a plurality of standard element regions with the plurality of mounting element regions according to a preset similarity, obtain a plurality of mounting defect probability values, identify a high defect value group in the plurality of mounting defect probability values according to a preset high probability value, determine a defect element region group corresponding to the high defect value group in the plurality of mounting element regions, and transmit the defect element region group to a pre-constructed display device.

[0068] In detail, the modules in the SMT mounting defect detection system based on image recognition 100 in the embodiment use the same technical means as the SMT mounting defect detection method based on image recognition in the above Figure 1 , and can produce the same technical effects, which will not be described here.

[0069] As Figure 3 shown is a structural schematic diagram of an electronic device for implementing the image recognition-based SMT mounting defect detection method according to an embodiment of the present application.

[0070] The electronic device 1 can include a processor 10, a memory 11 and a bus 12, and can further include a computer program stored in the memory 11 and executable on the processor 10, such as the image recognition-based SMT mounting defect detection method program.

[0071] The memory 11 includes at least one type of readable storage medium, such as flash memory, mobile hard disk, multimedia card, card-type memory (e.g., SD or DX memory, etc.), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 11 can be an internal storage unit of the electronic device 1, such as a mobile hard disk of the electronic device 1. In other embodiments, the memory 11 can also be an external storage device of the electronic device 1, such as a plug-in mobile hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc. Further, the memory 11 includes both the internal storage unit and the external storage device of the electronic device 1. The memory 11 can be used not only to store application software and various data installed in the electronic device 1, such as the code of the image recognition-based SMT mounting defect detection method program, but also to temporarily store data that has been output or will be output.

[0072] The processor 10 can be composed of integrated circuits in some embodiments, such as a single packaged integrated circuit or multiple packaged integrated circuits with the same or different functions, including one or more combinations of central processing unit (CPU), microprocessor, digital processing chip, graphics processor and various control chips, etc. The processor 10 is the control unit of the electronic device, which connects various components of the entire electronic device through various interfaces and lines, executes or runs programs or modules stored in the memory 11 (such as the image recognition-based SMT mounting defect detection method program, etc.), and calls data stored in the memory 11 to perform various functions and process data of the electronic device 1.

[0073] The bus 12 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus 12 can be divided into an address bus, a data bus, a control bus, etc. The bus 12 is configured to enable connection and communication between the memory 11, the at least one processor 10, and the like.

[0074] Figure 3 Only the electronic device with components is shown, and those skilled in the art can understand that, Figure 3 The structure shown does not constitute a limitation on the electronic device 1, and can include fewer or more components than shown, or combine certain components, or different component arrangements.

[0075] For example, although not shown, the electronic device 1 can also include a power supply (such as a battery) to power the various components. Preferably, the power supply can be logically connected to the at least one processor 10 through a power management system, so that the power management system can implement functions such as charge management, discharge management, and power consumption management. The power supply can also include one or more direct current or alternating current power sources, a recharging system, a power failure detection circuit, a power converter or inverter, a power status indicator, and the like. The electronic device 1 can also include various sensors, Bluetooth modules, Wi-Fi modules, and the like, which are not described here.

[0076] Further, the electronic device 1 can also include a network interface, which can optionally include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), and is typically used to establish a communication connection between the electronic device 1 and other electronic devices.

[0077] Optionally, the electronic device 1 can also include a user interface, which can be a display (Display), an input unit (such as a keyboard (Keyboard)), and optionally a standard wired interface, a wireless interface. Optionally, in some embodiments, the display can be an LED display, a liquid crystal display, a touch liquid crystal display, an OLED (Organic Light-Emitting Diode) touch, and the like. The display can also be appropriately referred to as a display screen or a display unit, and is used to display information processed in the electronic device 1 and to display a visualized user interface.

[0078] The image recognition-based SMT mounting defect detection method program stored in the memory 11 in the electronic device 1 is a combination of multiple instructions, which, when running in the processor 10, can realize: receiving a defect detection instruction, determining a mounting circuit board based on the defect detection instruction, moving the mounting circuit board to a pre-constructed optical detection device, and obtaining a to-be-detected circuit board, wherein the optical detection device comprises a high-resolution camera, a ring-shaped light source, and an optical filter; photographing the to-be-detected circuit board by using the optical detection device to obtain an original circuit image, and performing light compensation on the original circuit image to obtain a target circuit image; performing two-dimensional wavelet decomposition on the target circuit image to obtain a horizontal component floating point image and a vertical component floating point image; performing edge recognition on the target circuit image according to the horizontal component floating point image and the vertical component floating point image to obtain a strong edge pixel point set and a weak edge pixel point set; supplementing the strong edge pixel point set by using the weak edge pixel point set to obtain an edge pixel point set; obtaining the number of elements of the mounting circuit board, clustering the edge pixel point set by using the number of elements to obtain a plurality of element edge point sets; constructing a plurality of mounting element regions based on the plurality of element edge point sets, comparing a plurality of standard element regions with the plurality of mounting element regions according to a preset similarity, obtaining a plurality of mounting defect probability values, and identifying a high defect value group from the plurality of mounting defect probability values according to a preset high probability value; determining a defect element region group corresponding to the high defect value group from the plurality of mounting element regions, and transmitting the defect element region group to a pre-constructed display device to complete the image recognition-based SMT mounting defect detection.

[0079] Specifically, the specific implementation method of the processor 10 to the above instructions can refer to Figures 1 to 3 The descriptions of related steps in the corresponding embodiments are not repeated here.

[0080] Further, the modules / units integrated in the electronic device 1 are implemented in the form of software function units and sold or used as independent products, which can be stored in a computer readable storage medium. The computer readable storage medium can be volatile or non-volatile. For example, the computer readable medium can include any entity or system capable of carrying the computer program code, a recording medium, a U disk, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM, Read-Only Memory).

[0081] The application further provides a computer readable storage medium, which stores a computer program, and the computer program can realize the following steps when executed by a processor of an electronic device: receiving a defect detection instruction, determining the mounted circuit board based on the defect detection instruction, moving the mounted circuit board to a pre-constructed optical detection device to obtain a to-be-detected circuit board, wherein the optical detection device comprises a high-resolution camera, a ring-shaped light source and an optical filter; photographing the to-be-detected circuit board by using the optical detection device to obtain an original circuit image, and performing light compensation on the original circuit image to obtain a target circuit image; performing two-dimensional wavelet decomposition on the target circuit image to obtain a horizontal component floating point image and a vertical component floating point image; performing edge recognition on the target circuit image according to the horizontal component floating point image and the vertical component floating point image to obtain a strong edge pixel point set and a weak edge pixel point set; supplementing the strong edge pixel point set by using the weak edge pixel point set to obtain an edge pixel point set; obtaining the number of elements of the mounted circuit board, clustering the edge pixel point set by using the number of elements to obtain a plurality of element edge point sets; constructing a plurality of mounted element regions based on the plurality of element edge point sets, comparing a plurality of standard element regions with the plurality of mounted element regions according to a preset similarity to obtain a plurality of mounted defect probability values, and identifying a high defect value group from the plurality of mounted defect probability values according to a preset high probability value; determining a defect element region group corresponding to the high defect value group from the plurality of mounted element regions, and transmitting the defect element region group to a pre-constructed display device to complete SMT mounted defect detection based on image recognition.

[0082] In the several embodiments provided in the present application, it should be understood that the disclosed devices, systems and methods can be implemented in other ways. For example, the system embodiments described above are only schematic. Actual implementation can have another division way.

[0083] The modules described as separate components can or can not be physically separated, and the components shown as modules can or can not be physical units, i.e. they can be located in one place or distributed on multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the present embodiment.

[0084] In addition, each function module in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of hardware plus software function module.

[0085] It is obvious for those skilled in the art that the present application is not limited to the details of the above exemplary embodiments, and the present application can be realized in other specific forms without departing from the spirit or essential characteristics of the present application.

[0086] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application and are not limiting. Although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or replaced equivalently without departing from the spirit and scope of the technical solutions of the present application.

Claims

1. A method for detecting SMT assembly defects based on image recognition, characterized in that, The method includes: Receive a defect detection command, determine the mounting circuit board based on the defect detection command, move the mounting circuit board to a pre-built optical inspection device to obtain the circuit board to be inspected, wherein the optical inspection device includes: a high-resolution camera, a ring light source and an optical filter; An optical inspection device is used to photograph the circuit board to be inspected to obtain the original circuit image. Illumination compensation is then performed on the original circuit image to obtain the target circuit image. Two-dimensional wavelet decomposition is performed on the target circuit image to obtain the horizontal component floating-point image and the vertical component floating-point image; Based on the horizontal and vertical floating-point graphs, edge recognition is performed on the target circuit image to obtain the set of strong edge pixels and the set of weak edge pixels. By supplementing the set of strong edge pixels with the set of weak edge pixels, the set of edge pixels is obtained; Obtain the number of components on the mounted circuit board, and use the number of components to cluster the edge pixel set to obtain multiple component edge point sets; Multiple mounting component regions are constructed based on multiple component edge point sets. The similarity of multiple preset standard component regions with multiple mounting component regions is compared to obtain multiple mounting defect probability values. High defect value groups are identified among the multiple mounting defect probability values ​​based on preset high probability values. In multiple component mounting regions, a defective component region group corresponding to a high defect value group is identified, and the defective component region group is transmitted to a pre-built display device to complete the SMT mounting defect detection based on image recognition.

2. The SMT assembly defect detection method based on image recognition as described in claim 1, characterized in that, The process of performing illumination compensation on the original circuit image to obtain the target circuit image includes: Identify the original pixel set in the original circuit image, wherein the original pixel set includes multiple original pixels; Initial compensation coefficients are selected based on multiple preset historical circuit images; The original pixels are extracted sequentially from the original pixel set, and the compensation is calculated using the initial compensation coefficient to obtain the compensated pixels. The compensated pixels are summarized to obtain a set of compensated pixels. The original circuit image is then updated based on the set of compensated pixels to obtain a compensated circuit image. Pixel mean difference is calculated for both the original circuit image and the compensation circuit image to obtain the original pixel mean difference and the target pixel mean difference. The compensation amount is calculated based on the original pixel mean difference, the target pixel mean difference, and the preset compensation interval. If the compensation amount is greater than the preset minimum compensation amount, the updated compensation coefficient is calculated based on the compensation interval and the initial compensation coefficient. The updated compensation coefficient is the value obtained by subtracting the compensation interval from the initial compensation coefficient. The updated compensation coefficient and the compensation circuit image are used as the initial compensation coefficient and the original circuit image, respectively, and the step of confirming the original pixel set in the original circuit image is returned until the compensation amount is no greater than the minimum compensation amount. If the compensation amount is not greater than the minimum compensation amount, then the compensation circuit image is recorded as the target circuit image.

3. The SMT assembly defect detection method based on image recognition as described in claim 2, characterized in that, The initial compensation coefficients are selected based on a set of preset historical circuit images, including: Historical circuit images are extracted sequentially from multiple historical circuit images, and the historical pixel mean difference of the historical circuit images is calculated. Set a compensation coefficient range, and select multiple historical compensation coefficients within the compensation coefficient range according to the compensation interval; Extract the historical compensation coefficients sequentially from multiple historical compensation coefficients; The historical circuit image is compensated according to the historical compensation coefficient to obtain the historical compensated image. The pixel average difference is calculated on the historical compensated image to obtain the compensated pixel average difference. The average difference ratio is calculated based on the historical average pixel difference and the compensated average pixel difference. The average difference ratio is combined with the historical compensation coefficient to obtain the historical compensation data; Summarize the historical compensation data corresponding to the historical compensation coefficients to obtain the historical compensation data set, and merge the historical compensation data sets corresponding to the historical circuit images to obtain the historical compensation dataset. Curve fitting is performed based on the historical compensation dataset to obtain the compensation curve, where the horizontal axis of the compensation curve is the historical compensation coefficient in the historical compensation data, and the vertical axis of the compensation curve is the mean difference ratio in the historical compensation data. Identify the inflection point in the compensation curve and determine the initial compensation coefficient based on the inflection point, wherein the initial compensation coefficient is the x-coordinate of the inflection point.

4. The SMT assembly defect detection method based on image recognition as described in claim 3, characterized in that, The step of calculating compensation for the original pixels using initial compensation coefficients to obtain compensated pixels includes: A rectangular region is selected in the original circuit image based on the preset rectangle size and the original pixels. The rectangular region includes multiple rectangular pixels, and the original pixels are the geometric center of the rectangular region. Based on a preset background ratio, multiple separation pixels are determined from multiple rectangular pixels in a rectangular area, and the separation pixel value is calculated based on the multiple separation pixels. The original pixels are compensated based on the separated pixel values ​​and the initial compensation coefficients to obtain compensated pixels.

5. The SMT assembly defect detection method based on image recognition as described in claim 4, characterized in that, The process of compensating the original pixels based on the separated pixel values ​​and initial compensation coefficients to obtain compensated pixels includes: Get the original pixel value of the original pixel; Based on the original pixel values, separated pixel values, and initial compensation coefficients, the compensated pixel values ​​are calculated using the following formula: ； in, Indicates the compensation pixel value. Indicates the initial compensation coefficient. Represents the original pixel value. Indicates the separated pixel values; The compensation pixel is obtained by updating the compensation pixel value.

6. The SMT assembly defect detection method based on image recognition as described in claim 5, characterized in that, The step of performing edge recognition on the target circuit image based on the horizontal and vertical floating-point image components to obtain a set of strong edge pixels and a set of weak edge pixels includes: Determine the target pixel set in the target circuit image, wherein the target pixel set includes multiple target pixels; Target pixels are extracted sequentially from the target pixel set, and maximum suppression is performed on the target pixels based on the horizontal component floating-point image and the vertical component floating-point image to obtain updated pixels; The updated pixels are summarized to obtain an updated pixel set. The target circuit image is updated using the updated pixel set to obtain an updated circuit image, which is composed of the updated pixel set. Construct a grayscale histogram of the updated circuit image, and select high grayscale thresholds and low grayscale thresholds based on the grayscale histogram; The updated pixel set in the updated circuit image is filtered based on the high grayscale threshold and the low grayscale threshold to obtain the strong edge pixel set and the weak edge pixel set.

7. The SMT assembly defect detection method based on image recognition as described in claim 6, characterized in that, The step of performing maximum suppression on the target pixel based on the horizontal and vertical component floating-point images to obtain updated pixels includes: Extract the horizontal and vertical pixels corresponding to the target pixel from the horizontal component floating-point image and the vertical component floating-point image, respectively. Calculate edge strength and edge direction based on horizontal and vertical pixels; Based on edge direction, target pixel, and preset number of adjacent pixels, multiple adjacent pixels are identified in the target circuit image; Obtain the adjacent edge intensity of each adjacent pixel in a plurality of adjacent pixels to obtain multiple adjacent edge intensities; The target pixel is updated based on the intensity of multiple adjacent edges to obtain the updated pixel.

8. The SMT assembly defect detection method based on image recognition as described in claim 7, characterized in that, The process of supplementing the strong edge pixel set with weak edge pixel set to obtain an edge pixel set includes: Weak edge pixels are extracted sequentially from the set of weak edge pixels, and neighbor selection is performed on the weak edge pixels according to the preset local window to obtain multiple local pixels; Based on the set of strong edge pixels, obtain local strong edge point groups from multiple local pixels and count the number of edge points in the local strong edge point groups. If the number of edge points is greater than the preset minimum number, then the weak edge pixels are recorded as qualified edge pixels, and the qualified edge pixels are added to the strong edge pixel set to obtain the updated strong edge pixel set. Update the set of strong edge points as the set of strong edge pixels, and return to the step of sequentially extracting weak edge pixels from the set of weak edge pixels until the set of weak edge pixels is empty. The set of strong edge points updated when the set of weak edge pixels is empty is denoted as the edge pixel set.

9. The SMT assembly defect detection method based on image recognition as described in claim 8, characterized in that, The method of clustering edge pixel sets based on the number of components yields multiple component edge point sets, including: An initial center coordinate set is generated based on the number of components and a preset pixel coordinate range, wherein the number of initial center coordinates in the initial center coordinate set is the same as the number of components; Generate an initial cluster distribution set based on the initial center coordinate set; Extract edge pixels sequentially from the edge pixel set, and calculate the center distance from each edge pixel to each initial center coordinate in the initial center coordinate set to obtain the center distance set; The edge pixels are assigned to the initial cluster distribution set based on the center distance set, thus completing the assignment of edge pixels; When every edge pixel in the edge pixel set has been assigned, the initial clustering distribution set is denoted as the target clustering distribution set; Calculate the target center coordinate set based on the target cluster distribution set, and calculate the cluster change rate based on the initial center coordinate set and the target center coordinate set; If the cluster change rate is greater than the preset standard change rate, then the target center coordinate set and the target cluster distribution set are used as the initial center coordinate set and the initial cluster distribution set, respectively, and the step of sequentially extracting edge pixels in the edge pixel set is returned until the cluster change rate is not greater than the standard change rate. If the cluster change rate is not greater than the standard change rate, then multiple element edge point sets are identified based on the target cluster distribution set. Among them, the element edge point sets in the multiple element edge point sets correspond one-to-one with the target cluster distribution in the target cluster distribution set.

10. An SMT assembly defect detection system based on image recognition, characterized in that, The system includes: The target image acquisition module is used to receive defect detection instructions, determine the mounting circuit board based on the defect detection instructions, move the mounting circuit board to the pre-built optical inspection device to obtain the circuit board to be inspected. The optical inspection device includes a high-resolution camera, a ring light source and an optical filter. The optical inspection device is used to take pictures of the circuit board to be inspected to obtain the original circuit image. The original circuit image is then subjected to illumination compensation to obtain the target circuit image. The edge pixel recognition module is used to perform two-dimensional wavelet decomposition on the target circuit image to obtain a horizontal component floating-point image and a vertical component floating-point image. Based on the horizontal component floating-point image and the vertical component floating-point image, the edge recognition of the target circuit image is performed to obtain a set of strong edge pixels and a set of weak edge pixels. The edge pixel clustering module is used to supplement the strong edge pixel set with the weak edge pixel set to obtain the edge pixel set, obtain the number of components on the mounted circuit board, and use the number of components to cluster the edge pixel set to obtain multiple component edge point sets. The defective component display module is used to construct multiple mounting component regions based on multiple component edge point sets, compare the similarity of multiple preset standard component regions with multiple mounting component regions to obtain multiple mounting defect probability values, identify high defect value groups among the multiple mounting defect probability values ​​based on preset high probability values, determine the defective component region group corresponding to the high defect value group among the multiple mounting component regions, and transmit the defective component region group to the pre-constructed display device.

Images