Electronic component defect vision inspection system and method

Abstract

The present application relates to the technical field of defect detection, in particular to an electronic component defect visual detection system and method, the system comprising an image illumination control module, a defect boundary identification module, a confidence level calculation module, a result level classification module and a structure repetition elimination module.In the present application, the edge gray value and the inside pixel gray value of the packaging pin and the soldering point area are extracted, and the brightness gradient change trend is calculated, to realize dynamic adaptation of image illumination control, improve the clear identification of defect contour, build a defect intensity index sequence, and combine historical fluctuation interval difference to complete dynamic adjustment of the confidence interval, so that the defect level evaluation has stable distinguishing ability, compare the defect intensity value with the upper and lower limits of the confidence interval, combine the label attribution logic to form a clear distribution layer, match the boundary contour with the gray texture feature to realize structure repetition elimination between defects of the same level, and improve the independence and annotation accuracy of the detection layer.

Description

Technical Field

[0001] This invention relates to the field of defect detection technology, and in particular to a visual inspection system and method for defects in electronic components. Background Technology

[0002] The field of defect detection technology primarily refers to the use of machine vision, image processing, artificial intelligence, and sensor detection methods to automatically identify and analyze the surfaces of parts, products, or materials in industrial production processes to discover appearance defects, structural anomalies, or quality problems. The core objective of this field is to replace or assist manual inspection through non-contact, high-precision, and high-efficiency detection methods, thereby improving product quality, reducing production costs, and enhancing the automation and intelligence of production lines. Its applications are wide-ranging, covering multiple industries including semiconductor manufacturing, electronic assembly, metal processing, automotive parts, textiles, and food and pharmaceuticals.

[0003] Among them, the electronic component defect visual inspection system mainly targets the detection of appearance or functional defects in electronic components that occur during production and use, such as cracks, missing corners, bubbles, scratches, and abnormal solder joints. The system utilizes high-speed industrial cameras, light source design, and image processing to achieve rapid, high-precision, automated inspection of electronic components. This significantly improves inspection efficiency, reduces human error and missed detections, ensures stable product quality, and is widely used in electronic manufacturing and quality control processes.

[0004] Traditional inspection systems rely on fixed light source parameters for image acquisition, which cannot adapt to uneven image brightness caused by differences in the reflectivity of electronic component surface materials. In the edge recognition stage, image enhancement processing is not performed for low-contrast defects, which can easily lead to the inability to accurately extract closed boundaries of minor scratches or small cracks. In addition, the numerical fluctuation relationship in continuous frame information is not considered in defect recognition, resulting in unstable confidence judgment results. This can lead to blurred defect level classification boundaries and difficulty in consistently falling within the preset level range for defect intensity numerical judgment results. Furthermore, the lack of structural similarity comparison after multi-region recognition can lead to repeated labeling of the same defect. For example, the shadow of the same defect formed under multiple lighting angles may be repeatedly marked as an independent region, reducing overall detection efficiency and labeling effectiveness. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of the existing technology and to propose a visual inspection system and method for defects in electronic components.

[0006] To achieve the above objectives, the present invention employs the following technical solution: a visual inspection system for defects in electronic components, the system comprising:

[0007] The image illumination control module acquires a grayscale image of the electronic components on the inspection platform after being illuminated by an array of light sources. Based on the comparison between the brightness gradient amplitude of the pixel and the average brightness of the surrounding units, the module performs intensity adjustment of the light source units to generate an enhanced image of the defect edges.

[0008] Based on the enhanced image of the defect edge, the defect boundary recognition module extracts the ratio of the mean gray-level gradient between the edge and non-edge regions of the target area, performs gray-level difference enhancement processing on local image blocks, calibrates the closed path of the visible boundary in the enhanced image, and generates an electronic component boundary extraction map.

[0009] The confidence level calculation module extracts the boundary map of the electronic components, locates the suspected defect area, calculates the defect intensity index, determines whether it exceeds the predetermined defect confidence stability benchmark value, adjusts the upper and lower limits of the current level confidence interval, and generates the defect level confidence interval adjustment result.

[0010] The result classification module calls the defect level confidence interval adjustment result, extracts the defect intensity index value of each defect region in the current image frame, compares the defect intensity index value with the adjusted confidence interval upper and lower limits, performs level classification, and generates a defect region level label layer.

[0011] As a further aspect of the present invention, the defect edge enhancement image includes an edge brightness distribution structure, a light source adjustment region index, and an image frame after intensity adjustment; the electronic component boundary extraction map includes a set of packaging boundary paths, a crack contour linear structure, and a boundary closure region identifier; the defect level confidence interval adjustment result includes the interval boundary adjustment amplitude, the defect score fluctuation sequence, and the offset ratio mapping parameter; and the defect region level label layer includes defect level marking information, score interval attribution relationship, and level boundary positioning data.

[0012] As a further aspect of the present invention, the image illumination control module includes:

[0013] The image grayscale construction submodule acquires the grayscale image formed by the electronic components on the detection platform after being illuminated by an array of light sources. It collects the set of pixel grayscale values ​​corresponding to the package pin area and solder joint area in the image, constructs a two-dimensional grayscale matrix structure based on image coordinate mapping, and serializes and organizes the pixel grayscale values ​​according to their spatial positions to generate image grayscale distribution information.

[0014] The edge gradient extraction submodule calls the image grayscale distribution information to locate the set of pixels of the boundary contour of the package pin and solder joint area, obtains the corresponding grayscale value pairs of the boundary pixels and the inner neighbor pixels in the grayscale distribution, calculates the set of brightness gradient amplitudes based on the difference relationship between the grayscale value pairs, and extracts the coordinate point corresponding to the maximum value in the brightness gradient amplitude to generate the edge brightness gradient peak coordinates.

[0015] The array light source adjustment submodule calls the light source number index of the corresponding position in the array light source according to the peak coordinate of the edge brightness gradient, obtains the current average output intensity of the light source units around the unit, compares whether the difference between the brightness gradient amplitude and the average intensity exceeds the set brightness comparison benchmark value, if it exceeds, sets the current light source unit output intensity to be linearly adjusted between the upper and lower limits and reconstructs the image, obtains the adjusted image output, and generates a defect edge comparison image.

[0016] As a further aspect of the present invention, the defect boundary identification module includes:

[0017] The edge gradient extraction submodule acquires the defect edge enhancement image, extracts the gray values ​​of edge pixels in the solder joint area and package pin area of ​​the image and the corresponding gray values ​​of non-edge areas, calculates the average gray gradient value of the edge pixel set and the non-edge pixel set, establishes the proportional relationship between the average gray gradient values ​​of the two types of pixels, and generates a gray-scale dataset of edge and non-edge gray ratio.

[0018] The gradient ratio judgment submodule calls the edge and non-edge grayscale ratio dataset, judges the ratio data in each image block against the preset edge extraction contrast threshold, filters image regions with ratios lower than the threshold, obtains the image block number index with the largest grayscale gradient change in each region, and generates a low-contrast defect region index set.

[0019] The image boundary enhancement submodule obtains the pixel grayscale distribution within the corresponding image block region based on the low-contrast defect region index set, calculates the difference between the maximum and minimum grayscale values ​​within the region and constructs a local enhancement factor, performs grayscale adjustment processing on the pixels, re-annotates the boundary structure, extracts complete boundary contour lines and closes the boundary path, and generates an electronic component boundary extraction map.

[0020] As a further aspect of the present invention, the confidence level calculation module includes:

[0021] The defect index extraction submodule obtains the boundary extraction map of the electronic components, locates the suspected defect areas marked in the map, collects the edge gray-scale gradient amplitude, the pixel ratio of the area and the texture direction angle change value of the area in continuous image frames, performs normalization processing on the three indicators and performs weighted merging, calculates the defect intensity index of the corresponding defect area in each frame, and generates a defect intensity index sequence.

[0022] The confidence fluctuation assessment submodule calls the defect strength index sequence, performs difference calculation on the maximum and minimum values ​​in the sequence to obtain the interval amplitude value, selects the difference between the index value of the current frame and the index value of the previous frame to perform averaging, and determines whether the average change value exceeds the preset defect confidence stability benchmark value. If the condition is met, the difference between the index value of the current frame and the mean of the sequence is constructed as the fluctuation difference parameter to generate a confidence change assessment parameter set.

[0023] The grade interval adjustment submodule extracts the mean and fluctuation range of each defect region based on the confidence change assessment parameter set, performs mapping matching of the two values, compares the upper and lower limit boundary values ​​of the current grade interval, determines whether the mapping value needs to drive boundary adjustment, and if so, updates the confidence interval boundary position data, obtains the adjusted grade assessment interval structure, and generates the defect grade confidence interval adjustment result.

[0024] As a further aspect of the present invention, the result classification module includes:

[0025] The index value determination submodule calls the defect level confidence interval adjustment result, extracts the defect intensity index value of each defect region in the current image frame, compares the defect intensity index value with the upper and lower limits of the confidence interval corresponding to the region, and determines whether the value is within the closed interval between the upper and lower limits. If the condition is met, the status flag of the region entering the interval is recorded, and a confidence interval attribution determination result is generated.

[0026] The grade label assignment submodule assigns grade labels corresponding to the confidence interval to the defect areas marked as falling into the confidence interval based on the confidence interval attribution determination result. For areas not falling into the interval, it extracts the difference between the defect intensity index value and the upper and lower limits of the interval, identifies the boundary that the difference is close to, and performs attribution processing based on the grade label corresponding to the smallest difference, generating a defect grade label distribution list.

[0027] The classification layer generation submodule calls the defect level label distribution list to obtain the pixel position data of each defect area in the image coordinate system and the corresponding level label content. It maps the label data to the image plane according to the area number and constructs a multi-area level classification layer structure for electronic component images to generate a defect area level label layer.

[0028] As a further aspect of the present invention, the system further includes:

[0029] The structural duplication resolution module identifies defect regions assigned the same level label based on the defect region level label layer, calculates the overall similarity score between each pair of defect regions, and if the score is lower than the set duplication structure similarity threshold, the region is marked as a duplication defect, and a duplication defect annotation layer is generated.

[0030] The eliminated duplicate defect annotation layer includes duplicate defect exclusion areas, independent defect identification numbers, and structural difference scoring indexes.

[0031] As a further aspect of the present invention, the structure repetition elimination module includes:

[0032] The region feature extraction submodule obtains the defect region level label layer, identifies defect regions assigned the same level label, extracts the boundary contour coordinate point set of the region, the pixel area value within the region, the texture direction vector sequence and the geometric center coordinate position, constructs a structural feature combination including multi-dimensional feature quantities, and generates a multi-region structural feature set.

[0033] The similarity score calculation submodule calls the multi-region structural feature set, and for each pair of defective regions, calculates the similarity of the cosine angle between the contour vectors, the area overlap ratio, and the inverse geometric distance factor, and calculates and obtains the structural similarity score between the region pairs.

[0034] The duplicate defect marking submodule compares the structural similarity score between the regions with the duplicate structural similarity threshold, extracts the region number whose structural similarity score is higher than the duplicate structural similarity threshold, marks the corresponding region as a structural duplicate target, and marks the exclusion status in the original image. After removing the region, the remaining label structure is reconstructed to generate a layer of eliminated duplicate defects.

[0035] As a further aspect of the present invention, the formula for obtaining the structural similarity score between region pairs is specifically as follows:

[0036] ;

[0037] in, Indicates the first Texture direction vectors of each defect region Indicates the first Texture direction vectors of each defect region Indicates the first The magnitude of the texture direction vector of each region. Indicates the first The magnitude of the texture direction vector of each region. Indicates the first The pixel area value of each defect region Indicates the first The pixel area value of each defect region Indicates the first The region and the first The Euclidean distance between the geometric centers of the regions This represents the maximum geometric center distance between defect areas within the same label level. Indicates the first The defect area and the first Structural similarity score between defective regions.

[0038] A visual inspection method for defects in electronic components, the method being used to implement the aforementioned visual inspection system for defects in electronic components, includes the following steps:

[0039] S1: Acquire a grayscale image of an electronic component on a testing platform after being illuminated by an array of light sources. Based on the comparison between the brightness gradient amplitude of the pixel and the average brightness of the surrounding units, adjust the intensity of the light source units to generate an enhanced image of the defect edge.

[0040] S2: Based on the defect edge enhancement image, extract the ratio of the mean gray level gradient between the target area edge and the non-edge area, perform gray level difference enhancement processing on the local image block, calibrate the closed path of the visible boundary in the enhanced image, and generate an electronic component boundary extraction map.

[0041] S3: Based on the boundary extraction map of the electronic components, locate the suspected defect area, calculate the defect intensity index, determine whether it exceeds the predetermined defect confidence stability benchmark value, adjust the upper and lower limits of the current level confidence interval, and generate the defect level confidence interval adjustment result.

[0042] S4: Call the defect level confidence interval adjustment result, extract the defect intensity index value of each defect region in the current image frame, compare the defect intensity index value with the adjusted confidence interval upper and lower limits, classify the level, and generate a defect region level label layer.

[0043] S5: Based on the defect area level label layer, identify defect areas that have been assigned the same level label, calculate the overall similarity score between each pair of defect areas, and if the score is lower than the set repetitive structure similarity threshold, then the marked area is a repetitive defect, and generate a resolving repetitive defect label layer.

[0044] Compared with the prior art, the advantages and positive effects of the present invention are as follows:

[0045] In this invention, dynamic adaptation of image illumination is achieved by extracting the edge grayscale values ​​and inner pixel grayscale values ​​of the package pins and solder joint areas and calculating the brightness gradient change trend. Linear adjustment of light source intensity is completed based on the comparison between the brightness gradient amplitude and the average brightness of surrounding light sources, improving the clarity of defect contours. The ratio of the mean grayscale gradient is used as the judgment criterion, combined with grayscale difference enhancement operations on local image blocks, to mark the closed boundary path of defects in the image, ensuring the continuity and accuracy of boundary extraction. Subsequently, a defect intensity index sequence is constructed and the confidence interval is dynamically adjusted by combining historical fluctuation interval differences, enabling stable differentiation in defect level assessment. A clear distribution layer is formed by comparing the defect intensity values ​​with the upper and lower limits of the confidence interval and combining labeling logic. The structural repetition between defect areas of the same level is eliminated by matching the boundary contour and grayscale texture features, improving the independence and labeling accuracy of the detection layer. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1 This is a system flowchart of the present invention;

[0048] Figure 2 This is a schematic diagram of the system framework of the present invention;

[0049] Figure 3 This is a flowchart of the image illumination control module of the present invention;

[0050] Figure 4 This is a flowchart of the defect boundary identification module of the present invention;

[0051] Figure 5 This is a flowchart of the confidence level calculation module of the present invention;

[0052] Figure 6 This is a flowchart of the result classification module of the present invention;

[0053] Figure 7 This is a flowchart of the repeating elimination module of the present invention. Detailed Implementation

[0054] The technical solution of the present invention will now be described with reference to the accompanying drawings.

[0055] In embodiments of the present invention, words such as "exemplarily," "for example," etc., are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" in the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present the concept in a concrete manner. Furthermore, in embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one.

[0056] In the embodiments of this invention, the terms "image" and "picture" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning. Similarly, the terms "of," "corresponding (relevant)," and "corresponding" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning.

[0057] In this embodiment of the invention, sometimes a subscript such as W1 may be written in a non-subscript form such as W1. When the difference is not emphasized, the meaning they express is the same.

[0058] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0059] Please see Figure 1 The present invention provides a technical solution: a visual inspection system for defects in electronic components, the system comprising an image illumination control module, a defect boundary recognition module, a confidence level calculation module, a result level classification module, and a structural repetition resolution module;

[0060] The image illumination control module acquires a grayscale image of an electronic component on a testing platform illuminated by an array of light sources. It extracts the edge grayscale values ​​and inner pixel grayscale values ​​of the package pins and solder joints in the image, calculates the brightness gradient change trend between the two, locates the pixel with the largest brightness gradient change, calls the corresponding unit number of the pixel in the array of light sources, compares the brightness gradient amplitude of the pixel with the average brightness comparison value of the surrounding units, performs linear adjustment of the light source unit intensity within a preset upper and lower limit range, acquires the adjusted image output, and generates a defect edge enhancement image.

[0061] The brightness gradient magnitude refers to the intensity of the gray-level change between adjacent pixels at the boundary of a unit pixel in an image. It is often used in edge detection operations. Its calculation is based on the gradient map obtained after processing by the Sobel operator or the Prewitt operator. The average brightness contrast value is the average difference between the gray-level values ​​of each pixel in the selected image block and the overall gray-level mean. It can be measured by the standard deviation.

[0062] The defect boundary recognition module is based on the defect edge enhancement image. It detects the edge gray-level gradient information of the regions that constitute dissimilar color points and fracture cracks in the image, extracts the ratio of the mean gray-level gradient of the target region edge to the mean gray-level gradient of the non-edge region, and determines whether the ratio is lower than the set edge extraction contrast threshold. If it is lower, it selects the region with the largest gray-level change, performs gray-level difference enhancement processing on the local image block, calibrates the closed path of the visible boundary in the enhanced image, and generates an electronic component boundary extraction map.

[0063] The gray-level gradient mean ratio, which is the proportional relationship between the average gray-level change of edge pixels and the average gray-level change of their adjacent regions, is a common indicator for judging boundary strength. The edge extraction contrast threshold is a set judgment benchmark used to judge the distinguishability between the edge and the background, and is often set empirically from the image gradient histogram.

[0064] The confidence level calculation module is based on the boundary extraction map of electronic components to locate each suspected defect area. It extracts the edge gray-scale gradient amplitude, defect area proportion and texture direction variation rate of the region in consecutive image frames, calculates the defect intensity index, and constructs a defect intensity index sequence for the region under each frame image. It uses the sequence to construct the difference relationship between the numerical fluctuation range and the current frame value, calculates the difference between the maximum and minimum values ​​as the interval amplitude value, and combines the average change level formed by the difference between the current frame value and the previous frame value to determine whether it exceeds the predetermined defect confidence stability benchmark value. If it exceeds, the average value and fluctuation amplitude of the numerical sequence are mapped to the confidence interval boundary correction parameters, and the upper and lower limits of the current level confidence interval are adjusted to generate the defect level confidence interval adjustment result.

[0065] The interval amplitude value refers to the difference between the maximum and minimum scores in a set of scoring data, reflecting the fluctuation range; the inter-frame score change amplitude is the average absolute value of the difference between scores of adjacent frames, which is often used to measure the stability of the scoring sequence; the scoring stability threshold can be set by the system during the calibration stage, and is adjusted based on the empirical value of the surface reflection stability of electronic components.

[0066] The result classification module calls the defect level confidence interval adjustment result, extracts the defect intensity index value of each defect region in the current image frame, compares the defect intensity index value with the upper and lower limits of the adjusted confidence interval, and determines whether it falls inside the confidence interval. If it is inside the interval, it is marked as the corresponding category of the level. If it is not inside the interval, it is determined to be biased region based on the distance difference between the value and the upper and lower limits. The label is assigned with reference to the nearest neighbor level boundary belonging relationship, and a classification distribution map of defect regions in the image is established to generate a defect region level label layer.

[0067] The grade label is defined according to the process standard, such as the IPC-A-610 standard, which classifies the acceptable defect level of components into grades of acceptable, repairable, and unacceptable. The distance between the score value and the boundary of the interval is used to evaluate which grade it is more consistent with.

[0068] The structural duplication resolution module identifies defect regions assigned the same level label based on the defect region level label layer. It extracts the set of boundary contour coordinate points, area size, grayscale texture variation range and Euclidean distance between the geometric center of the region, calculates the overall similarity score between each pair of defect regions, and if the score is lower than the set duplication structure similarity threshold, the region is marked as a duplication defect and a resolution duplication defect annotation layer is generated.

[0069] The repetitive structure similarity threshold is used to determine whether two defects are repetitive structures. It is usually set empirically based on the actual false detection rate of repetitive patterns in batches of devices.

[0070] The defect edge enhancement image includes the edge brightness distribution structure, the light source adjustment region index, and the image frame after intensity adjustment. The electronic component boundary extraction map includes the package boundary path set, the crack contour linear structure, and the boundary closure region identifier. The defect level confidence interval adjustment result includes the interval boundary adjustment amplitude, the defect score fluctuation sequence, and the offset ratio mapping parameter. The defect region level label layer includes defect level marking information, the score interval belonging relationship, and the level boundary positioning data. The eliminated duplicate defect annotation layer includes the duplicate defect exclusion region, the independent defect identification number, and the structural difference score index.

[0071] Please see Figure 2 and Figure 3 The image illumination control module includes:

[0072] The image grayscale construction submodule acquires the grayscale image formed by the electronic components on the detection platform after being illuminated by an array of light sources. It collects the set of pixel grayscale values ​​corresponding to the package pin area and solder joint area in the image, constructs a two-dimensional grayscale matrix structure based on image coordinate mapping, and serializes and organizes the pixel grayscale values ​​according to their spatial positions to generate image grayscale distribution information.

[0073] To acquire grayscale images of electronic components formed by illumination from an array of light sources on a testing platform, specifically, using an area array... The camera is focused on the stage. The packaged chip is photographed vertically to obtain an image with a resolution of [resolution value missing]. A pixel-based digital image that includes the chip's package body, pins, and related components. The coordinate range of the solder joints connecting the boards in the acquired image is from... arrive A rectangular area, covering a complete package pin and its corresponding solder joint, containing a total of [number missing]... Each pixel is analyzed, and its brightness information is read one by one to obtain a set of grayscale values, for example, coordinate points. The pixel grayscale value coordinates The pixel grayscale value , will this Each gray value is determined by its two-dimensional spatial coordinates in the original image. Arrange them to construct a A two-dimensional grayscale matrix, where the elements in the first row and first column correspond to coordinates. The pixel grayscale value, the coordinates of the element in the last row and last column. The pixel grayscale values ​​are then read. This two-dimensional matrix is ​​then expanded in row-major order; that is, starting from the first row, pixel grayscale values ​​are read sequentially from left to right until the end of the row, then moving to the beginning of the next row to continue reading, until all rows have been read, forming a matrix containing... A one-dimensional grayscale value sequence of elements, the sequence completely records the correspondence between pixel grayscale values ​​and their spatial locations within a specified area, generating image grayscale distribution information.

[0074] The edge gradient extraction submodule calls the image grayscale distribution information to locate the set of pixels of the boundary contour of the package pin and solder joint area, obtains the corresponding grayscale value pairs of the boundary pixels and their inner neighbor pixels in the grayscale distribution, calculates the set of brightness gradient amplitudes based on the difference relationship between the grayscale value pairs, and extracts the coordinate point corresponding to the maximum value in the brightness gradient amplitude to generate the peak coordinates of the edge brightness gradient.

[0075] Retrieve image grayscale distribution information; the information is a list containing... A one-dimensional grayscale sequence of elements is first analyzed to locate the boundary contour pixel set of the package pin and solder joint area by analyzing the grayscale value jumps in the sequence. Specifically, for each grayscale value in the sequence, the absolute value of its grayscale difference with its four adjacent pixels (top, bottom, left, and right) in the two-dimensional matrix structure is calculated, and these four differences are summed to obtain a gradient sum value. For example, for coordinates... Grayscale value The pixel has gray values ​​of its four adjacent pixels (top, bottom, left, and right) as follows: , , , Then its gradient sum is Set a gradient and a threshold The threshold is set based on the following: Statistical analysis was performed on sample image regions containing clear boundaries, and the gradient and values ​​of all pixels were calculated. This revealed that... The boundary pixel gradient and value are both greater than The gradient and value of non-boundary pixels are both less than Therefore, choose As a threshold to distinguish between boundary and non-boundary areas, all gradients with values ​​greater than [a certain threshold] are [selected]. The pixels are identified as boundary contour pixels and grouped together. Then, for each identified boundary pixel, its corresponding grayscale value pair with its internal neighboring pixels in the grayscale distribution is obtained. Internal neighboring pixels are pixels that are adjacent to the boundary pixel but do not belong to the boundary contour pixel set. For example, a boundary pixel with coordinates of... grayscale value Its gradient sum is Its upper neighbor pixel grayscale value The gradient sum is It also belongs to the boundary pixels, while its right neighboring pixels grayscale value The gradient sum is If it is not a boundary pixel, then select... As a set of grayscale value pairs, the brightness gradient magnitude set is calculated based on the absolute value of the difference between each grayscale value pair, such as for... The calculated brightness gradient magnitude is The grayscale value pair obtained from the other boundary pixel The calculated brightness gradient magnitude is After traversing all boundary pixels, a set of brightness gradient magnitudes is obtained. The maximum value is extracted from this set, for example, . And record the boundary pixel coordinates corresponding to the maximum value, i.e. Generate the peak coordinates of the edge brightness gradient.

[0076] The array light source adjustment submodule calls the light source number index of the corresponding position in the array light source according to the peak coordinate of the edge brightness gradient, obtains the current average output intensity of the light source units around the unit, compares whether the difference between the brightness gradient amplitude and the average intensity exceeds the set brightness comparison benchmark value, if it exceeds, sets the current light source unit output intensity to be linearly adjusted between the upper and lower limits and reconstructs the image, obtains the adjusted image output, and generates a defect edge comparison image.

[0077] Based on the peak coordinates of the edge brightness gradient This coordinate is directly mapped to the physical layout of the array light source, which consists of a... of The unit matrix is ​​composed of each A cell corresponds to a specific region in an image, with coordinates... It falls on the numbered of The light source unit is primarily responsible for illuminating the area, and then the corresponding light source number index in the array light source is called. To obtain the eight light source units surrounding this unit, i.e. , , , , , , , Read the current output intensity values ​​of these units, and the output intensity of each unit is... to The integer representation, with the read intensity values ​​set as follows: Calculate the arithmetic mean of these values ​​to obtain the mean of the current output intensity. The maximum value of the brightness gradient obtained in the previous step is used. With this average intensity The comparison is performed to determine whether the difference between the two exceeds a set brightness comparison reference value, which is set as follows: The setup process is as follows: data collection For a group of clear images obtained under optimal lighting conditions, the difference between the maximum value of the brightness gradient amplitude and the average value of the light source intensity in the corresponding area is calculated. The average value of these differences is... The standard deviation is To ensure the sensitivity of the adjustment, the benchmark value is set as the mean plus three standard deviations, i.e. The difference currently calculated is This value exceeds the set baseline value. Therefore, a linear adjustment is performed, setting the upper limit of the output intensity of the light source unit to be [value missing]. The lower limit is According to the difference The size is increased proportionally to the current light source unit. The formula for adjusting the output intensity is: New Intensity = Original Intensity + (Difference / Reference Value) Adjust the step size, assuming the original strength is Adjust the step size to The new strength is Rounded down to , will be numbered The output intensity of the light source unit is updated to The image was then re-captured using this new light source configuration, and the adjusted image output was obtained to generate a defect edge comparison image.

[0078] Please see Figure 2 and Figure 4 The defect boundary identification module includes:

[0079] The edge gradient extraction submodule acquires the defect edge enhancement image, extracts the gray values ​​of edge pixels in the solder joint area and package pin area of ​​the image and the corresponding gray values ​​of non-edge areas, calculates the average gray gradient value of the edge pixel set and the non-edge pixel set, establishes the proportional relationship between the average gray gradient values ​​of the two types of pixels, and generates a gray-scale ratio dataset of edge and non-edge pixels.

[0080] Obtain a defect edge comparison image, which is a re-acquired image after linear adjustment of the light source intensity. Within the same coordinate region of this image, re-extract the grayscale values ​​of edge pixels in the solder joint area and package pin area and compare them with the corresponding grayscale values ​​of non-edge areas. The method for determining edge pixels is the same as before, that is, calculate the gradient sum of each pixel and compare it with a threshold. By comparing the edges, this method yields an updated set of edge pixels, for example, a set containing... Read this pixel The grayscale value of each pixel is set to its grayscale value data as... At the same time, other sets not in this collection will be included. Each pixel is divided into a set of non-edge region pixels, and its grayscale value is read. The data is... Next, the average grayscale gradient values ​​of the edge pixel set and the non-edge pixel set are calculated separately. For the edge pixel set, the absolute value of the average grayscale difference between each edge pixel and its four neighboring pixels is calculated. For example, for a grayscale value of The edge pixel has four neighboring pixels with gray values ​​of . Then its gradient value is For all Perform this calculation on each edge pixel to obtain We calculate the arithmetic mean of the gradient values ​​to obtain the average gray-level gradient value of the edge pixel set, let it be... The average gray-level gradient value of the non-edge pixel set is calculated using the same method. Since the gray-level change of pixels in non-edge regions is gradual, their gradient values ​​are generally small. The calculated average value is set to... Finally, the proportional relationship between the mean gray-level gradients of these two types of pixels is established, that is, the calculation is performed. This process is applied to each preset image patch in the image, for example, the entire The image is divided into multiple For each image patch, a ratio value is calculated to generate a dataset of edge and non-edge grayscale ratios.

[0081] Call the edge-to-non-edge grayscale ratio dataset, which contains each of the image partitions. A grayscale ratio value calculated for an image patch, for example, for the numbered patch. Image patch (corresponding to original image coordinates) to The calculated ratio is The ratio data in each image block is compared with a preset edge extraction contrast threshold, which is set to... Its setting is based on, through the analysis of Analyze each sample image patch, among which These are low-contrast defect areas that have been manually identified. By considering each region as either background or normal, and calculating the ratios for all samples, it was found that the ratios for low-contrast defective regions were all distributed within a certain range. arrive The ratios between these values ​​were greater than those in the normal region. Therefore, As a threshold for differentiation, those with a ratio lower than [a certain value] are filtered out. The image region, if the image block The ratio is If the ratio is 1, then the region is selected, and the other ratio is 2. Image patches that are not filtered are instead identified. For all filtered low-contrast regions, the index of the image patch with the largest grayscale gradient change in each region is obtained. This process involves, within each filtered region, for example, a patch composed of... A larger region composed of image patches, compared to this The average gray-level gradient value of the edge pixel set of each image patch, rather than a ratio, defines the image patch within the region. The marginal average gradient is Image Patch The for Image Patch The for After comparison, image patches were found If the marginal average gradient value is the largest, then... This low-contrast region is indexed and recorded as a representative of the low-contrast region. This process is repeated to generate an index set of low-contrast defect regions.

[0082] The gradient ratio judgment submodule calls the edge and non-edge grayscale ratio dataset, judges the ratio data in each image block against the preset edge extraction contrast threshold, filters image regions with ratios lower than the threshold, obtains the image block number index with the largest grayscale gradient change in each region, and generates a low-contrast defect region index set.

[0083] The image boundary enhancement submodule obtains the pixel grayscale distribution within the corresponding image block region based on the low-contrast defect region index set, calculates the difference between the maximum and minimum grayscale values ​​within the region and constructs a local enhancement factor, re-annotates the boundary structure after grayscale adjustment of the pixels, extracts the complete boundary contour lines and closes the boundary path, and generates an electronic component boundary extraction map.

[0084] According to the low-contrast defect region index set, which contains, for example... Image patch number, obtain the number as The pixel grayscale distribution within an image block region, i.e., a The grayscale matrix, read all the values ​​in the matrix. Given the grayscale values ​​of each pixel, find the maximum and minimum grayscale values. For example, the maximum value is... The minimum value is Calculate the difference between the maximum and minimum gray values ​​within the region to obtain And construct a local enhancement factor, which is calculated as follows: ,in For grayscale range, and These are the maximum and minimum gray values ​​within the region, respectively. Substituting these values ​​into the equation yields the local enhancement factor. Each pixel within the image block undergoes grayscale adjustment using the following formula: ,in These are the adjusted pixel grayscale values. It is the original grayscale value, for example, a grayscale value of The pixels, whose adjusted value is Rounded down to the nearest integer This method allows the original distribution in The grayscale values ​​of the interval are linearly stretched to The image block is processed to capture its full dynamic range and then re-detected for boundary. Due to the significantly enhanced contrast, previously difficult-to-identify boundaries become clear, allowing for the extraction of complete boundary contours. Morphological closure operations are then used to close the boundary paths, ultimately forming a clear and closed defect contour on the processed image block. The contours of all processed low-contrast areas are integrated to generate an electronic component boundary extraction map.

[0085] Please see Figure 2 and Figure 5 The confidence level calculation module includes:

[0086] The defect index extraction submodule obtains the boundary extraction map of electronic components, locates the suspected defect areas marked in the map, collects the edge gray-scale gradient magnitude, the pixel ratio of the area and the texture direction angle change value of the area in continuous image frames, performs normalization processing on the three indicators and performs weighted merging, calculates the defect intensity index of the corresponding defect area in each frame, and generates a defect intensity index sequence.

[0087] Obtain the boundary extraction map of electronic components. This map marks the closed contours of multiple suspected defect areas. For example, locate a region marked "Defect A" in the map. The set of its contour coordinate points is known. Next, collect data on the continuous contours of this region. The edge grayscale gradient magnitude, region pixel ratio, and texture direction angle change value under the frame image, for the first index, in the ... In the frame image, calculate the brightness gradient magnitude of all pixels on the contour of the "defect A" region, and then calculate their average value to obtain... For the second metric, calculate the number of pixels in the "defect A" region, for example... Each pixel occupies the entire standard area of ​​the solder joint (set to...). The ratio of pixels is That is, the area pixel ratio value For the third indicator, first calculate the... Construct a gradient direction histogram for all pixels within the "defect A" region of the frame image, find the principal gradient direction angle, and set it as... Then in the The principal gradient direction angle is calculated in the same way in the frame image, and is set as follows: The change value of the texture direction angle is... For continuous Perform this operation on all frame images to obtain three sets of index sequences, as shown in Table 1 below:

[0088] Table 1. Feature indicators of defect A across 5 consecutive frames:

[0089] ;

[0090] As shown in Table 1, this table lists the three key feature index values ​​of defect A over five consecutive frames. The three indices are then normalized to the values ​​in the first frame. Taking a frame as an example, the normal range for setting the gradient magnitude is: The percentage range is The range of angle change values ​​is The normalized value is: gradient , percentage Angle change Then, a weighted merging process is performed. The weights are set based on the analysis of a large number of known defect samples, which revealed that gradient magnitude contributes the most to judging the severity of defects, followed by percentage, while angle change contributes the least. Therefore, their weights are set as follows: , , These weight values ​​are set based on the The values ​​were derived from logistic regression analysis of categorized defect samples. The coefficients of each feature directly reflect its importance. After normalizing the coefficients, the weights were obtained to ensure... Calculate the first The defect intensity index of the defect area corresponding to the frame is This calculation is performed for each frame to generate a sequence of defect strength indices. .

[0091] The confidence fluctuation assessment submodule calls the defect strength index sequence, performs difference calculation on the maximum and minimum values ​​in the sequence to obtain the interval amplitude value, selects the difference between the index value of the current frame and the index value of the previous frame to perform averaging, and determines whether the average change value exceeds the preset defect confidence stability benchmark value. If the condition is met, the difference between the index value of the current frame and the mean of the sequence is constructed as the fluctuation difference parameter to generate the confidence change assessment parameter set.

[0092] Call the defect strength index sequence First, perform a difference operation on the maximum and minimum values ​​in the sequence. The maximum value of the sequence is... The minimum value is The calculated interval amplitude value is Next, select the current frame (the first frame). The frame index value is compared with the previous frame (the first frame). The difference between the frame (index value) and the difference between all previous adjacent frames, i.e. , , , The difference sequence is obtained. Perform averaging and calculate the average change value. Determine whether the average change value exceeds the preset defect confidence stability benchmark value, which is set as follows: Its setup process is as follows: analysis A stable defect and The defect intensity index sequence of each pseudo-defect (such as instantaneous shadows caused by changes in illumination) was analyzed, and the average change value of each was calculated. It was found that the average change value of stable defects was all within a certain range. The following are examples of pseudo-defects, which are generally higher than those of pseudo-defects. Therefore, choose As a dividing line for judging stability, the currently calculated average change value Not exceeding The stability condition is met, therefore the current frame (the first frame) is set to... (frame) index value The difference between the value and the series mean is used to construct the volatility difference parameter, where the series mean is 1. The calculated fluctuation difference parameter is This parameter and the range value These parameters are stored together in a set to generate a confidence change assessment parameter set.

[0093] The grade interval adjustment submodule extracts the mean and fluctuation range of each defect region based on the confidence change assessment parameter set, performs mapping and matching of the two values, compares the boundary values ​​of the upper and lower limits of the current grade interval, determines whether the mapping value needs to drive boundary adjustment, and if so, updates the confidence interval boundary position data, obtains the adjusted grade assessment interval structure, and generates the defect grade confidence interval adjustment result.

[0094] The confidence change assessment parameter set includes the mean term calculated for the "Defect A" region. The fluctuation range term here uses the interval range value calculated in the previous step. The process involves mapping and matching these two values. This matching process is based on a pre-established two-dimensional lookup table, which defines the confidence interval adjustment strategies corresponding to different combinations of mean and fluctuation amplitude. Then, it compares the upper and lower limits of the current defect level's range, and sets "Defect A" to be temporarily classified as "Minor," with its corresponding confidence interval being... Determine whether the mapped value needs boundary adjustment. The adjustment rule is: if the mean term... Very close to the lower limit of the interval And the fluctuation range item The amplitude is relatively large, exceeding the benchmark value for the average fluctuation range of this level (set as follows). By statistical analysis of this level If the range of each sample is calculated, then the lower limit of the current interval may be set too high and needs to be adjusted. In this example, the mean... In the range At a relatively low level, but with a high degree of fluctuation. Greater than the benchmark value If the adjustment conditions are met, the confidence interval boundary data will be updated. The calculation method for the lower limit is: New lower limit = Original lower limit - Fluctuation amplitude term. Adjustment factor, the adjustment factor is set to This coefficient is derived from regression analysis of historical adjustment data to ensure the smoothness of the adjustment and avoid excessive oscillations. The new lower limit is calculated as follows: The upper limit remains unchanged, and the adjusted rating range structure is obtained, that is, the new "mild" rating range becomes The defect level confidence interval adjustment result is generated.

[0095] Please see Figure 2 and Figure 6 The result classification module includes:

[0096] The index value determination submodule calls the defect level confidence interval adjustment result, extracts the defect intensity index value of each defect region in the current image frame, compares the defect intensity index value with the upper and lower limits of the confidence interval corresponding to the region, and determines whether the value is within the closed interval between the upper and lower limits. If the condition is met, the status flag of the region entering the interval is recorded, and the confidence interval attribution determination result is generated.

[0097] The defect level confidence interval adjustment results have been retrieved, and the assessment interval for the "minor" level corresponding to the "Defect A" area has been updated to... Extract "Defect A" in the current image frame (the first frame). The defect intensity index value under (frame) is, this value is Compare this defect strength index value with the upper limit of the confidence interval corresponding to the region. and lower limit value By comparing the values, we can determine whether the value lies within the closed interval between the upper and lower limits, i.e., we can determine the inequality. Whether it is valid or not, due to Indeed greater than or equal to and less than or equal to Therefore, the condition is met, and the status flag of the "Defect A" region entering this interval is recorded. The flag is in the form of a Boolean value or a status code. For example, the attribution status of "Defect A" is set to "Confirmed". For all other detected defect regions, such as "Defect B", the current frame defect intensity index is... The corresponding "significant" level range is Similarly, it meets the interval assignment criteria and is also marked as "confirmed," while for "defect C," its index value is... Its original "minor" level range has been adjusted to , If it is not within this interval, its status is marked as "pending". After traversing all defect areas, a confidence interval attribution determination result is generated.

[0098] The grade label assignment submodule assigns grade labels corresponding to the confidence interval to defect areas marked as falling within the confidence interval based on the confidence interval attribution result. For areas not falling within the interval, it extracts the difference between the defect intensity index value and the upper and lower limits of the interval, identifies the boundary that the difference is close to, and performs attribution processing based on the grade label corresponding to the smallest difference, generating a defect grade label distribution list.

[0099] Based on the confidence interval assignment results, the status of "Defect A" is recorded as "Confirmed," and it belongs to the adjusted "Minor" level confidence interval. For defect areas marked as falling within the confidence interval, such as "Defect A," a level label corresponding to that interval, i.e., "Minor," is directly assigned. For areas not falling within the interval, such as "Defect C" with a status label of "Pending," its defect intensity index value is... Extract the upper and lower limits of this value relative to the original "minor" level range to which "defect C" belongs. The difference between the two is calculated, and its difference from the lower limit is... Calculate the difference between it and the upper limit. At the same time, it also calculates its interval with the adjacent next-level "negligible" level and sets it as... The difference between the upper and lower limits, and the difference between the upper limit of the interval, is... Identify which of all these differences is the smallest, and find the smallest difference among these differences that is closest to the upper limit of the "negligible" grading range. Since the difference is the smallest, the level label corresponding to the smallest difference is used for classification. The level label of "Defect C" is reassigned to "ignorable". After the level labels of all defect areas are assigned, a list is obtained, in which each element contains the unique identifier of the defect area and its final level label, generating a defect level label distribution list.

[0100] The classification layer generation submodule calls the defect level label distribution list to obtain the pixel position data of each defect area in the image coordinate system and the corresponding level label content. It maps the label data to the image plane according to the area number and constructs a multi-area level classification layer structure for electronic component images to generate a defect area level label layer.

[0101] The defect level label distribution list is invoked to obtain the pixel position data of the "Defect A" region in the image coordinate system. This data is the set of contour points saved in the initial boundary extraction step, such as a sequence of coordinate points [(210, 340), (211, 340), ..., (210, 340)]. Simultaneously, the corresponding level label "Slight" is obtained. This "Slight" label data is associated with all pixel positions of "Defect A." That is, on an additional layer of the image data, all pixel values ​​within the region enclosed by the contour of "Defect A" are assigned a specific code representing the "Slight" level, such as a number. Perform the same operation on the "Defect B" region, assigning it a code representing the "significant" level. The defect "C" is assigned a code representing a "negligible" level. This process essentially maps label data to the image plane according to region numbering, and constructs a multi-region hierarchical classification layer structure with the exact same size as the original electronic component image. This layer is a two-dimensional matrix, where the value of each element represents the defect level of the corresponding pixel, where the value is... Indicates no defects or negligible defects, with a value of [value missing]. Indicates a minor defect, with a value of [value missing]. This identifies significant defects and ultimately generates a visual defect area level label layer.

[0102] Please see Figure 2 and Figure 7 The structural repetition resolution module includes:

[0103] The region feature extraction submodule obtains the defect region level label layer, identifies defect regions assigned the same level label, extracts the boundary contour coordinate point set of the region, the pixel area value within the region, the texture direction vector sequence and the geometric center coordinate position, constructs a structural feature combination including multi-dimensional feature quantities, and generates a multi-region structural feature set.

[0104] Obtain the defect area level label layer, which is a two-dimensional matrix marked with different defect level codes. First, identify defect areas assigned the same level label in this layer. For example, by scanning the layer, find all areas coded as follows: (Representing a "minor" level) Connected regions were identified, and two such regions were found, denoted as "Defect A" and "Defect D" respectively. For "Defect A", its boundary contour coordinate point set [(210, 340), ..., (210, 340)] was extracted, and the number of pixels contained within its region was counted to obtain the pixel area value. For each pixel, calculate the gradient vector of all pixels within the region and average them to obtain a two-dimensional texture direction vector sequence. Further averaging yields the main texture direction vector. Finally, the geometric center coordinates of the region are calculated by analyzing all pixels within the region. coordinates and The coordinates are obtained by averaging them, for example, as follows: The same operation is performed on another "minor" level "defect D" region to obtain its boundary contour coordinate point set and pixel area value. Pixels, Main Texture Direction Vector Geometric center coordinates These extracted multidimensional features, including contour point sets, areas, texture vectors, and center coordinates, are constructed into a structured feature set. After completing this process for all defect regions of the same level, a multi-region structural feature set is generated.

[0105] The similarity scoring submodule calls a multi-region structural feature set. For each pair of defect regions, it calculates the similarity of the cosine angle between the contour vectors, the area overlap ratio, and the inverse geometric distance factor, using the following formula:

[0106] ;

[0107] The calculation yields a structural similarity score between pairs of regions;

[0108] in, Indicates the first Texture direction vectors of each defect region Indicates the first Texture direction vectors of each defect region Indicates the first The region and the first The dot product between the texture direction vectors of each region Indicates the first The magnitude of the texture direction vector of each region. Indicates the first The magnitude of the texture direction vector of each region. Indicates the first The pixel area value of each defect region is obtained by counting the number of pixels included within the closed boundary. Indicates the first The pixel area value of each defect region Indicates the region With the region The smaller of the area values, Indicates the first The region and the first The Euclidean distance between the geometric centers of the regions This represents the maximum geometric center distance between defect areas within the same label level. Indicates the first The defect area and the first The structural similarity score between defect regions is set by calculating the similarity score of all real duplicate defect region pairs in a batch of manually labeled defect images, extracting the mean and standard deviation of the distribution of these scores, and setting the threshold as the mean of the distribution minus a standard deviation interval value to exclude non-duplicate but high similarity occasional regions, while retaining the vast majority of real duplicate structure samples.

[0109] The system invokes a multi-region structural feature set containing structural feature combinations of "Defect A" and "Defect D". For this pair of defect regions both belonging to the "minor" level, a structural similarity score is calculated between them. This calculation relies on the formula:

[0110] ;

[0111] In this formula, Representing the The and the first Structural similarity score between defective regions Corresponding to "Defect A", Corresponding to "Defect D", and It is the texture direction vector of the region, representing the main direction of the texture inside the defect. It is the dot product of vectors, used to measure the consistency of direction. It represents the magnitude of the vector. The first term is the cosine similarity, calculated by the cosine of the angle between two texture directions, with a range of [value missing]. , and The first term is the pixel area value of the region. The second term calculates the overlap ratio of the areas, which is measured by dividing the smaller area by twice the total area to determine the similarity in size. The value range is [value range missing]. , It is the Euclidean distance between the geometric centers of two regions, representing the degree of proximity in spatial locations. The third term is the maximum center distance observed among all defects of the same grade, used as a normalization factor. The fourth term is the inverse geometric distance factor; the closer the distance, the closer the value of this term. The greater the distance, the closer the values. The entire formula is a weighted average of three dimensions: directional similarity, area similarity, and spatial proximity (where the weights are all equal). This formula comprehensively evaluates the structural similarity of two defects. Its advantage lies in combining three features—shape (indirectly reflected by area), orientation (texture vector), and location (geometric distance)—to provide a more comprehensive and robust similarity measure than comparing a single feature. In particular, the introduction of the inverse geometric distance factor allows spatially proximate similar defects to receive higher scores, consistent with the tendency for repetitive defects to be physically clustered. Now, let's input the feature data of "Defect A" and "Defect D" for calculation. , , , The center coordinates are respectively and The maximum center distance is set among all defects in the "minor" category. Calculated as For each pixel, the parameters are first calculated:

[0112] ;

[0113] ;

[0114] ;

[0115] ;

[0116] ;

[0117] Substitute these values ​​into the formula:

[0118] ;

[0119] The calculation yielded a structural similarity score between "Defect A" and "Defect D". .

[0120] The duplicate defect labeling submodule compares the structural similarity score between region pairs with the duplicate structural similarity threshold, extracts the region number with the structural similarity score higher than the duplicate structural similarity threshold, labels the corresponding region as the structural duplicate target, and marks the exclusion status in the original image. After removing the region, the remaining label structure is reconstructed to generate a layer of eliminated duplicate defects.

[0121] Based on the structural similarity score between region pairs This value is then compared with a preset threshold for the similarity of repeated structures. This threshold is set by selecting a structure that contains... Zhang has already manually labeled a batch of images containing real, repetitive defects. Calculate all pairs of real, repetitive defect regions within these batches (total...). The similarity score of ( ) is used to obtain a result containing A sample set of rating values ​​was analyzed, and its distribution mean was calculated. The standard deviation is The threshold is then set to the mean of the distribution minus one standard deviation interval, i.e. The calculated structural similarity score will be used to... With this threshold Comparison, because Below Therefore, "Defect A" and "Defect D" are not considered structurally repetitive targets. If there is another pair of defects, "Defect E" and "Defect F", the calculated score is... This value is higher than the threshold. Then, extract the region numbers "E" and "F" of the pair of defects, mark the two corresponding regions as structural repetition targets, mark one of them (e.g., "defect F") as excluded in the visualization result of the original image, remove the entry about "defect F" in the defect level label list, and reconstruct the remaining label structure after removing the region, such as recalculating the number and distribution statistics of various defects, and finally generate a label layer that has eliminated redundant information and resolved the repetitive defects.

[0122] A visual inspection method for defects in electronic components, used to implement the aforementioned visual inspection system for defects in electronic components, includes the following steps:

[0123] S1: Acquire a grayscale image of an electronic component on a testing platform after being illuminated by an array of light sources. Based on the comparison between the brightness gradient amplitude of the pixel and the average brightness of the surrounding units, adjust the intensity of the light source units to generate an enhanced image of the defect edge.

[0124] S2: Based on the defect edge enhancement image, extract the ratio of the mean gray-level gradient between the target area edge and the non-edge area, perform gray-level difference enhancement processing on the local image block, calibrate the closed path of the visible boundary in the enhanced image, and generate the electronic component boundary extraction map.

[0125] S3: Based on the boundary extraction map of electronic components, locate the suspected defect area, calculate the defect intensity index, determine whether it exceeds the predetermined defect confidence stability benchmark value, adjust the upper and lower limits of the current level confidence interval, and generate the defect level confidence interval adjustment result;

[0126] S4: Call the defect level confidence interval adjustment result, extract the defect intensity index value of each defect region in the current image frame, compare the defect intensity index value with the adjusted confidence interval upper and lower limits, classify the level, and generate a defect region level label layer.

[0127] S5: Based on the defect area level label layer, identify defect areas that have been assigned the same level label, calculate the overall similarity score between each pair of defect areas, and if the score is lower than the set repetitive structure similarity threshold, the area is marked as a repetitive defect, and a resolving repetitive defect label layer is generated.

[0128] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.

[0129] In this invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of a single item or a plurality of items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be a single item or multiple items.

[0130] It should be understood that, in various embodiments of the present invention, the order of the above-mentioned process numbers does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0131] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0132] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices, apparatuses, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0133] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0134] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0135] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0136] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0137] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. An electronic component defect vision inspection system, characterized by, The system comprises: The image light illumination regulation module acquires the gray scale image of the electronic component formed by the array light source on the detection platform, compares the brightness gradient amplitude of the pixel points with the average brightness of the surrounding units, executes the light source unit intensity adjustment, and generates a defect edge enhanced image; The defect boundary recognition module extracts the gray scale gradient mean ratio of the target region edge and the non-edge region based on the defect edge enhanced image, executes the gray scale difference enhancement processing of the local image block, marks the closed path of the visible boundary in the enhanced image, and generates an electronic component boundary extraction atlas; The confidence level calculation module positions the suspected defect region based on the electronic component boundary extraction atlas, calculates the defect intensity index, judges whether it exceeds the established defect confidence stability reference value, adjusts the upper and lower limits of the current level confidence interval, and generates a defect level confidence interval adjustment result; The confidence level calculation module comprises: The defect index extraction submodule acquires the electronic component boundary extraction atlas, positions the suspected defect region marked in the atlas, collects the edge gray scale gradient amplitude, the region pixel proportion value and the texture direction angle change value under the continuous image frame, executes the normalization processing and weighted combination of the three indexes, calculates the defect intensity index of each frame corresponding defect region, and generates a defect intensity index sequence; The confidence fluctuation evaluation submodule calls the defect intensity index sequence, performs difference operation on the maximum value and the minimum value in the sequence, acquires the interval amplitude value, selects the difference value between the current frame index value and the previous frame index value to execute average processing, judges whether the average change value exceeds the preset defect confidence stability reference value, and if the condition is met, the difference value between the current frame index value and the sequence mean value is constructed as a fluctuation difference parameter to generate a confidence change evaluation parameter set; The level interval adjustment submodule extracts the mean value and the fluctuation amplitude of each defect region according to the confidence change evaluation parameter set, executes the mapping matching of the two numerical values, compares the upper and lower limit range boundary values of the current level interval, judges whether the mapping value needs to drive the boundary adjustment, updates the confidence interval boundary position data if it is true, acquires the adjusted level evaluation interval structure, and generates a defect level confidence interval adjustment result; The result level classification module calls the defect level confidence interval adjustment result, extracts the defect intensity index value of each defect region under the current image frame, compares the defect intensity index value with the upper and lower limit values of the adjusted confidence interval, performs level division, and generates a defect region level label layer.

2. The electronic component defect vision inspection system of claim 1, wherein The defect edge enhanced image comprises an edge brightness distribution structure, a light source adjustment region index and an intensity adjusted image frame, the electronic component boundary extraction atlas comprises a package boundary path set, a crack contour line structure and a boundary closed region identification, the defect level confidence interval adjustment result comprises an interval boundary adjustment amplitude, a defect score fluctuation sequence and a bias ratio mapping parameter, and the defect region level label layer comprises defect level marking information, score interval attribution relationship and level boundary positioning data.

3. The electronic component defect vision inspection system of claim 2, wherein The image light illumination regulation module comprises: The image gray scale construction submodule obtains a gray scale image of the electronic component formed by irradiation of the array light source on the detection platform, collects a set of pixel gray scale values of the packaging pin area and the solder point area in the image, constructs a two-dimensional gray scale matrix structure based on image coordinate mapping, serializes and arranges the pixel gray scale values according to spatial positions, and generates image gray scale distribution information; The edge gradient extraction submodule calls the image gray scale distribution information, locates a boundary contour pixel set of the packaging pin and the solder point area, obtains a corresponding gray scale value pair of the boundary pixel and the inner adjacent pixel in the gray scale distribution, calculates a set of brightness gradient amplitude values according to a difference value relationship between the gray scale value pairs, extracts a maximum value corresponding coordinate point in the brightness gradient amplitude values, and generates an edge brightness gradient peak value coordinate; The array light source adjustment submodule obtains a current output intensity mean value of a unit peripheral light source unit according to the edge brightness gradient peak value coordinate, calls a light source number index of a corresponding position in the array light source, compares whether a difference value between the brightness gradient amplitude value and the intensity mean value exceeds a set brightness contrast reference value, sets the current light source unit output intensity to be linearly adjusted between an upper limit and a lower limit and reconstructs an image if the difference value exceeds the set brightness contrast reference value, obtains an adjusted image output, and generates a defect edge contrast image.

4. The electronic component defect vision inspection system of claim 3, wherein The defect boundary recognition module comprises: The edge gradient extraction submodule obtains the defect edge enhanced image, extracts edge pixel gray scale values and corresponding gray scale values of non-edge regions of the solder point area and the packaging pin area in the image, calculates average gray scale gradient values of the edge pixel set and the non-edge pixel set, establishes a proportional relationship between the average gray scale gradient values of the two types of pixels, and generates an edge and non-edge gray scale ratio data set; The gradient ratio judgment submodule calls the edge and non-edge gray scale ratio data set, judges the ratio data in each image block and a preset edge extraction contrast threshold value, filters image regions with a ratio lower than the threshold value, obtains an image block number index with the largest gray scale gradient change in each region, and generates a low-contrast defect region index set; The image boundary enhancement submodule obtains pixel gray scale distribution in a corresponding image block region according to the low-contrast defect region index set, calculates a difference value between the maximum and minimum gray scale values in the region and constructs a local enhancement factor, performs gray scale adjustment processing on the pixels, re-labels the boundary structure, extracts complete boundary contour lines and closes the boundary path, and generates an electronic component boundary extraction map.

5. The electronic component defect vision inspection system of claim 1, wherein The result level classification module comprises: The index value judgment submodule calls the defect level confidence interval adjustment result, extracts a defect intensity index value under a current image frame of each defect region, compares the defect intensity index value with an upper limit value and a lower limit value corresponding to the region, judges whether the value is located in a closed interval between the upper and lower limit intervals, records a state mark of the region entering the interval if the condition is met, and generates a confidence interval attribution judgment result; The grade label assignment submodule directly assigns a grade label corresponding to the interval to a defect region marked as falling into the confidence interval according to the confidence interval attribution determination result, extracts a difference value between a defect intensity index value and an upper or lower limit value of the interval for a region not falling into the interval, identifies a boundary close to the difference value, and performs attribution processing according to a grade label corresponding to a minimum difference value to generate a defect grade label distribution list; The classification layer generation submodule calls the defect grade label distribution list, obtains pixel position data and corresponding grade label content of each defect region in an image coordinate system, maps the label data to an image plane according to region numbers, and constructs a multi-region grade classification layer structure facing an electronic component image to generate a defect region grade label layer.

6. The electronic component defect vision inspection system of claim 5, wherein, The system further comprises: The structure repetition resolution module identifies defect regions assigned with the same grade label based on the defect region grade label layer, calculates an overall similarity score between each pair of defect regions, and marks the regions as repeated defects if the score is lower than a set repetition structure similarity threshold to generate a resolved repeated defect annotation layer; The resolved repeated defect annotation layer includes a repeated defect exclusion region, an independent defect identification number, and a structure difference score index.

7. The electronic component defect vision inspection system of claim 6, wherein, The structure repetition resolution module comprises: The region feature extraction submodule obtains the defect region grade label layer, identifies defect regions assigned with the same grade label, extracts a boundary contour coordinate point set, a region pixel area value, a texture direction vector sequence, and a geometric center coordinate position of the region, constructs a structure feature combination including multi-dimensional feature quantities, and generates a multi-region structure feature set; The similarity score calculation submodule calls the multi-region structure feature set, calculates a contour vector cosine angle similarity, an area overlap ratio, and a reverse geometric distance factor for each pair of defect regions, and obtains a structure similarity score value between the region pair through operation; The repeated defect marking submodule compares the structure similarity score value between the region pair with a repetition structure similarity threshold, extracts region numbers with a structure similarity score value higher than the repetition structure similarity threshold, marks the corresponding regions as structure repetition targets, and annotates an exclusion state in the original image, removes the regions, reconstructs the remaining label structure, and generates a resolved repeated defect annotation layer.

8. The electronic component defect vision inspection system of claim 7, wherein, The formula for obtaining the structure similarity score value between the region pair is specifically: ； wherein, denotes the texture direction vector of the i-th defect region, denotes the texture direction vector of the j-th defect region, denotes the length of the i-th region texture direction vector, denotes the length of the j-th region texture direction vector, denotes the pixel area value of the i-th defect region, denotes the pixel area value of the j-th defect region, denotes the Euclidean distance between the geometric centers of the i-th region and the j-th region, denotes the maximum value of the geometric center distance between defect regions in the same grade label, denotes the structural similarity score value between the i-th defect region and the j-th defect region.

9. A method for visual inspection of defects in electronic components, characterized in that The method is used for implementing the electronic component defect visual detection system of any one of claims 1-8, and comprises the following steps: S1: Obtain a gray-scale image of an electronic component formed by irradiation of an array light source on a detection platform, compare a brightness gradient amplitude of a pixel point with a peripheral unit average brightness contrast value, perform light source unit intensity adjustment, and generate a defect edge enhanced image; S2: Based on the defect edge enhanced image, extract a gray-scale gradient mean value ratio of a target region edge and a non-edge region, perform gray-scale difference enhancement processing on a local image block, calibrate a closed path of a visible boundary in the enhanced image, and generate an electronic component boundary extraction map. S3: based on the electronic component boundary extraction atlas, positioning the suspected defect area, calculating the defect intensity index, judging whether it exceeds the established defect confidence stability benchmark value, adjusting the upper and lower limits of the current grade confidence interval, and generating the defect grade confidence interval adjustment result; S4: calling the defect grade confidence interval adjustment result, extracting the defect intensity index value of each defect area under the current image frame, comparing the defect intensity index value with the adjusted upper and lower limit value of the confidence interval, performing grade division, and generating a defect area grade label layer; S5: based on the defect area grade label layer, identifying the defect areas assigned with the same grade label, calculating the overall similarity score between each pair of defect areas, and if the score is lower than the set repetitive structure similarity threshold, marking the area as a repetitive defect, and generating an eliminated repetitive defect annotation layer.

Images