An electronic component surface defect recognition method and system based on machine vision
By using machine vision technology to identify and classify surface defects in electronic components, the problem of poor flexibility in existing automatic optical inspection equipment is solved, enabling efficient defect identification and production parameter adjustment, thereby improving product quality and production efficiency.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- HEFEI YINGYAN TECHNOLOGY CO LTD
- Filing Date
- 2026-06-05
- Publication Date
- 2026-07-14
Smart Images
Figure CN122391756A_ABST
Abstract
Description
Technical Field
[0001] This application belongs to the field of defect identification and relates to electronic component defect identification technology, specifically a method and system for identifying surface defects of electronic components based on machine vision. Background Technology
[0002] Electronic components are the smallest units with independent electrical functions, usually made of a single material or with a simple structure, capable of performing specific electronic tasks. They are the foundation of electronic circuits, enabling the flow, control, or energy conversion of electrons through physical or chemical means. Surface defect identification of electronic components is a key link in ensuring product quality and production efficiency. By accurately detecting surface defects such as scratches, cracks, and oxidation, premature failure of components in extreme environments or long-term use can be effectively prevented, significantly improving product reliability and meeting the stringent standards of high-requirement fields such as aerospace and automotive electronics. At the same time, defect identification can significantly reduce rework and scrap costs, avoiding legal disputes and brand reputation losses caused by quality issues. Combining AI and machine vision technologies, defect detection can automate and intelligentize production lines, improve inspection efficiency and reduce human intervention, while optimizing process flows through data traceability and locating weak links in the supply chain.
[0003] In the existing technology, when identifying defects in electronic components, automatic optical inspection equipment is generally used to identify defects on the surface of electronic components. Automatic optical inspection equipment mostly adopts rule-based detection algorithms, which have advantages such as logical transparency and fast execution speed. However, existing automatic optical inspection equipment has poor flexibility and is difficult to adapt to complex textures, minor flaws and new defect types that have not been preset, which easily leads to false detection and missed detection.
[0004] This application provides a machine vision-based method and system for identifying surface defects in electronic components to solve the above-mentioned technical problems. Summary of the Invention
[0005] This application aims to solve at least one of the technical problems existing in the prior art; to this end, this application proposes a method and system for identifying surface defects of electronic components based on machine vision, which is used to solve the technical problem that existing automatic optical inspection equipment has poor flexibility, is difficult to adapt to complex textures, small flaws and new defect types that have not been preset, and is prone to false detection and missed detection.
[0006] To achieve the above objectives, the first aspect of this application provides a machine vision-based method for identifying surface defects in electronic components, comprising: Acquire surface image data of electronic components; Defects in electronic components are identified based on surface image data, resulting in defect identification images; Based on the defect recognition images, the defects of electronic components are classified to obtain the defect types; The production parameters of electronic components are adjusted based on the defect type of the electronic components.
[0007] Preferably, the identification of defects in electronic components based on surface image data includes: Retrieve surface image data of electronic components; acquire standard template images; preprocess the surface image data; adjust the size of the preprocessed surface image data according to the standard template images; The defect identification model is invoked by inputting the adjusted surface image data into the defect identification model to obtain the extracted defect identification image; the defect identification model is built based on an artificial intelligence model.
[0008] Preferably, the preprocessing of the surface image data includes: Retrieve surface image data of electronic components; convert the surface image data to grayscale; perform two-dimensional discrete wavelet transform on the grayscale surface image data to obtain high-frequency coefficients; The expression for the wavelet transform is:
[0009]
[0010] in, , Image size; It is a scaling function; It is a wavelet function; The number of decomposition levels; As the starting scale; These are translation parameters; A soft thresholding function is used to denoise the high-frequency coefficients; the denoised high-frequency coefficients are then subjected to inverse wavelet transform to obtain the preprocessed surface image data. The expression for the soft threshold function is:
[0011] in, The threshold is used; the expression is: ; The standard deviation of noise. This represents the total number of pixels in the surface image.
[0012] Preferably, the defect identification model is built based on an artificial intelligence model, including: Obtain a standard dataset; wherein the standard dataset includes standard input data consistent with the content attributes of the adjusted surface image data; and standard output data consistent with the content attributes of the defect image. Model frameworks and deep learning algorithms are selected from an artificial intelligence model library; artificial intelligence models are constructed based on the model frameworks and deep learning algorithms to obtain the constructed models; The standard dataset is divided into a training set, a validation set, and a test set according to a set ratio; the model is trained using the training set; the internal parameters of the model are adjusted using the validation set; and the trained model is tested using the test set to obtain test metrics. Obtain the indicator threshold; compare the test indicators with the indicator threshold. If all test indicators are greater than the indicator threshold, mark the constructed model as a defect identification model; otherwise, reconstruct and train the constructed model.
[0013] Preferably, the step of classifying defects in electronic components based on defect recognition images includes: Retrieve defect recognition images; obtain a defect type library; analyze the similarity between the defect recognition images and each defect type in the defect type library; The defect type with the highest similarity is selected as the type of defect recognition image; the defect types corresponding to the recognized defect recognition images are integrated to obtain the defect type of electronic components.
[0014] Preferably, the analysis of the similarity between the defect recognition image and each defect type in the defect type library includes: Obtain the morphological feature information of defects in the defect recognition image; wherein, the morphological feature information includes: length, width and texture feature vector; The similarity between defects in the defect recognition image and each defect type in the defect type library is calculated based on the constructed similarity analysis function. The expression for the similarity analysis function is: ;in, This represents the similarity between the defect and the i-th defect type; Indicates the length of the defect; This represents the average length of the i-th defect type; Indicates the width of the missing item; This represents the average width of the i-th defect type; This represents the texture similarity between the defect and the i-th defect type.
[0015] Preferably, the method for obtaining the texture similarity includes: Retrieve the texture feature vectors of the defects and the texture feature vectors of each defect type; calculate the texture similarity between the defects and each defect type based on the constructed texture similarity analysis function; The expression for the texture similarity analysis function is: ;in, Represents the texture feature value of the j-th dimension of the defect; The dimension of the texture feature vector; This represents the average value of the texture features in the j-th dimension for the i-th defect type; Let represent the standard deviation of the j-th dimension of the i-th defect type.
[0016] Preferably, adjusting the production parameters of electronic components based on the defect type of the electronic components includes: Retrieve the defect type; obtain the parameter adjustment library; match the defect type with the parameter adjustment library to obtain the corresponding adjustment equipment and corresponding adjustment value; Acquire real-time production parameters of the equipment during the production process; adjust the corresponding production parameters of the equipment based on the adjustment values and real-time production parameters; The system produces a preset number of electronic components using the adjusted production parameters and then tests them. Defects are identified in the tested electronic components. If no defects are identified in the tested electronic components, the adjustment of the production parameters is considered complete; otherwise, the production parameters are readjusted.
[0017] The second aspect of this application provides a machine vision-based electronic component surface defect recognition system, including: an image acquisition module, a defect recognition module, and a parameter adjustment module; The image acquisition module is used to acquire surface image data of electronic components; The defect identification module is used to identify defects in electronic components based on surface image data to obtain defect identification images; and to classify the defects in electronic components according to the defect identification images to obtain defect types. The parameter adjustment module is used to adjust the production parameters of electronic components based on the defect type of the electronic components.
[0018] A third aspect of this application provides a computer-readable storage medium storing instructions that, when executed on a machine vision-based electronic component surface defect identification system, cause the machine vision-based electronic component surface defect identification system to perform the methods described in the first aspect and any possible implementation thereof.
[0019] Compared with the prior art, the beneficial effects of this application are: 1. This application first acquires surface image data of electronic components and uses advanced image recognition technology to accurately identify defects, enabling rapid location of quality problems in the production process, effective interception of defective products, and prevention of their entry into the market, thereby significantly improving overall product quality and protecting consumer rights. Second, this process automates defect identification and classification, greatly reducing the time and labor costs required for manual inspection and significantly improving the operating efficiency of the production line. At the same time, rapid defect identification allows for timely adjustment of production parameters, reducing production interruptions and rework caused by defects, and further improving production efficiency.
[0020] 2. This application allows for targeted adjustments to production parameters based on defect types, enabling effective measures to be taken to address specific problems, optimize production processes, reduce the recurrence rate of similar defects, and promote continuous improvement in the production process. This data-driven feedback mechanism not only helps improve product quality but also reduces waste of raw materials and energy, thereby effectively lowering production costs. Furthermore, high-quality products are key to improving customer satisfaction and loyalty. By implementing this process, companies can better meet customer needs, enhance brand image and reputation, and strengthen market competitiveness. Finally, the large amount of image data accumulated by this process provides rich decision support for production management. Through in-depth analysis of this data, companies can identify bottlenecks and problems in the production process, providing management with scientific and accurate decision-making basis and promoting intelligent and refined management of the production process. Attached Figure Description
[0021] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0022] Figure 1 This is a schematic diagram illustrating the overall steps of the method described in this application; Figure 2 This is a schematic diagram of the defect image recognition steps in this application; Figure 3 This is a schematic diagram illustrating the analysis of defect types and parameter adjustment steps in this application; Figure 4 This is a connection diagram of the system module structure of this application. Detailed Implementation
[0023] The technical solutions of this application will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.
[0024] Please see Figure 1 The first aspect of this application provides a machine vision-based method for identifying surface defects in electronic components, including: S101. Obtain surface image data of electronic components.
[0025] It should be noted that this step aims to convert the surface state of physical components into digital visual signals using optical sensing devices. The term "electronic component" in this application is a broad, overarching concept, encompassing not only common integrated circuit chips, resistors, and capacitors, but also the smallest units with independent electrical functions, such as BGA packaged balls, connector terminals, and inductor coils. Correspondingly, the methods for acquiring "surface image data" are diverse. For example, it can be acquired through two-dimensional planar imaging using a high-resolution CCD or CMOS industrial camera, or through point cloud image data containing depth information acquired using a three-dimensional structured light scanning device, as long as it reflects the physical morphological characteristics of the component surface.
[0026] S102. Based on surface image data, identify defects in electronic components to obtain defect identification images.
[0027] It should be noted that the purpose of this step is to isolate the truly abnormal defect areas from the complex background image. In the field of machine vision, the recognition process is essentially a process of feature extraction and pattern determination, transforming the original image containing potential defects into a defect recognition image that only contains the defect outline, location, and pixel distribution. It should be understood that although this embodiment does not elaborate on the specific details of the recognition algorithm, this step is not a simple image comparison, but requires a keen perception of complex textured backgrounds and minute imperfections to ensure that the input source for subsequent classification has a high signal-to-noise ratio and high fidelity.
[0028] S103. Classify the defects of electronic components based on the defect recognition images to obtain the defect types.
[0029] It should be noted that this step involves semantic-level qualitative labeling of the identified defects. The "defect type" referred to in this application is a classification of the physical causes and apparent characteristics of defects, such as cracks, scratches, oxidation, dirt, missing solder balls, and deformed leads. This classification process maps disordered visual features into ordered process problem labels; this step is a crucial bridge connecting visual perception and physical control.
[0030] S104. Adjust the production parameters of electronic components based on the defect type of the electronic components.
[0031] It should be noted that this step converts the identification results of the virtual domain into device actions in the physical domain. The adjustment of production parameters covers various core equipment in the manufacturing process, such as the placement pressure of the pick-and-place machine, the peak temperature of the reflow oven, and the squeegee speed of the printer. The adjustment can be a fine-tuning of a single parameter or a coupled correction of multiple parameters.
[0032] In one possible implementation of the embodiments of this application, combined with Figure 1 ,like Figure 2 As shown, the above S102 can be specifically implemented through the following S201-S203, which are explained in detail below: S201. Retrieve surface image data of electronic components; convert the surface image data to grayscale; perform two-dimensional discrete wavelet transform on the grayscale surface image data to obtain high-frequency coefficients.
[0033] The expression for the wavelet transform is:
[0034]
[0035] in, , Image size; It is a scaling function; It is a wavelet function; The number of decomposition levels; As the starting scale; These are the translation parameters.
[0036] It should be noted that converting a color image to a grayscale image can eliminate redundant illumination interference in the color channels, while reducing the amount of data computation required for subsequent transformations, allowing the algorithm to focus on reflecting the brightness gradient changes that reflect the defect morphology. The use of two-dimensional discrete wavelet transform instead of traditional Gaussian or mean filtering is based on the following core mechanism: traditional filters, while smoothing noise, often indiscriminately smooth out high-frequency details in the image, causing minute defects (such as micrometer-level missing solder ball edges) to be filtered out as noise as well; while wavelet transform, through multi-scale decomposition, breaks down the image signal into sub-signals of different frequency bands, allowing noise and minute defect features to be separated in scale space, making accurate denoising possible.
[0037] In this embodiment, it is preferred to set 2 or 3 layers. If the number of layers is too small, the noise and details will not be separated sufficiently. If the number of layers is too large, the energy of small defects will be excessively dispersed and the computational cost will increase dramatically. 2 or 3 layers is the optimal balance between computational efficiency and noise reduction fidelity. As the starting scale; The translation parameter determines the sliding position of the wavelet basis function in the image spatial domain; These represent the feature components in the horizontal, vertical, and diagonal directions, respectively. These three directional components can capture the edge gradient information of defects such as cracks and scratches in different directions from all angles.
[0038] S202. Use a soft thresholding function to denoise the high-frequency coefficients; perform inverse wavelet transform on the denoised high-frequency coefficients to obtain the preprocessed surface image data.
[0039] The expression for the soft threshold function is:
[0040] in, The threshold is used; the expression is: ; The standard deviation of noise. This represents the total number of pixels in the surface image.
[0041] It should be noted that after obtaining high-frequency coefficients from multiple directions, thresholding must be used to remove coefficients dominated by random noise. This embodiment resolutely abandons the hard thresholding function and adopts the soft thresholding function instead. The fundamental reason is that the hard thresholding function directly retains the original value when the absolute value of the coefficient is greater than the threshold. This truncation process will produce a serious pseudo-Gibbs phenomenon in the reconstructed image, causing unnatural oscillations and truncation at the edges of minor defects, making the originally continuous weak crack edges discontinuous, which is very easy for the model to misclassify as non-defect areas in subsequent recognition. In contrast, the soft thresholding function continuously shrinks and smooths coefficients greater than the threshold, avoiding abrupt truncation of the signal, thus perfectly preserving the continuity and transition characteristics of the edges of minor defects while suppressing noise.
[0042] It should be noted that, in this embodiment, the threshold... It is not a fixed constant set by humans, but rather a constant related to the noise standard deviation. and pixel count It's dynamically linked. The mechanism of this dynamic calculation logic is that the larger the variance of the image noise, the more severe the noise pollution, and the required denoising threshold should be increased accordingly for strong denoising; while the number of pixels... The increase in the number of pixels implies an expansion of the signal sample size. According to the law of large numbers and logarithmic relationships, the threshold needs to be fine-tuned as the number of pixels increases logarithmically to prevent the false negative of real signals within a large sample size. This dynamic hooking mechanism ensures that the threshold adaptively finds the optimal boundary between noise and signal under different lighting conditions and camera resolutions. As an example of a numerical endpoint, in images of electronic components acquired by a conventional industrial camera, the noise standard deviation... The typical value range is usually between 10 and 30, when When it is in the millions, The calculation results can effectively cover most of the noise figure distribution range. It should be understood that the above... The range of values is interpretive, not restrictive. In actual production lines, Real-time estimation can be performed by extracting pixel fluctuations in flat areas of an image.
[0043] S203. Obtain the standard template image; adjust the size of the preprocessed surface image data according to the standard template image.
[0044] It should be noted that because the pixel area occupied by different batches or models of electronic components on the production line may vary in the camera's field of view, directly inputting the original image into the subsequent model would lead to chaotic input dimensions and increase the difficulty of feature extraction. A standard template image serves as a unified reference benchmark. By scaling, cropping, or affine transforming the preprocessed image to the same size as the standard template, the standardization of input data is ensured. It should be understood that the size adjustment method is not limited to simple proportional scaling. When the component has a slight tilt, edge detection can be combined with rotation correction before size alignment, as long as the final output image matches the size of the standard template.
[0045] S204. Call the defect recognition model and input the adjusted surface image data into the defect recognition model to obtain the extracted defect recognition image.
[0046] In one possible implementation, the defect identification model is built upon an artificial intelligence model, including: Obtain a standard dataset; wherein the standard dataset includes standard input data consistent with the content attributes of the adjusted surface image data; and standard output data consistent with the content attributes of the defect image. Model frameworks and deep learning algorithms are selected from an artificial intelligence model library; artificial intelligence models are constructed based on the model frameworks and deep learning algorithms to obtain the constructed models; The standard dataset is divided into a training set, a validation set, and a test set according to a set ratio; the model is trained using the training set; the internal parameters of the model are adjusted using the validation set; and the trained model is tested using the test set to obtain test metrics. Obtain the indicator threshold; compare the test indicators with the indicator threshold. If all test indicators are greater than the indicator threshold, mark the constructed model as a defect identification model; otherwise, reconstruct and train the constructed model.
[0047] It should be noted that the standard dataset serves as a prior knowledge base for the AI model to learn defect features. The "consistency of content attributes" referred to in this application means that the standard input data maintains a high degree of isomorphism with the surface image data collected from the actual production line and preprocessed and resized, in terms of physical characteristics such as image dimension, resolution, and grayscale distribution. This eliminates model transfer decay caused by data domain differences. The standard output data refers to pixel-level labeled images or bounding box coordinates that precisely correspond to the defect location, category, and contour in the standard input data. This strict correspondence between input and output attributes ensures that the model can establish an accurate mapping from the normal background to the defect area during training, rather than a fuzzy statistical association.
[0048] The AI model library contains a variety of model frameworks and algorithms with different feature extraction tendencies. As examples of what else could be included, model frameworks could be not only classic Convolutional Neural Networks (CNNs) to efficiently extract spatial topological features using their local perception and weight sharing mechanisms, but also Residual Networks (ResNets) to address the gradient vanishing problem during deep network training, or Feature Pyramid Networks (FPNs) to adapt to the identification needs of multi-scale micro-defects; corresponding deep learning algorithms could cover Stochastic Gradient Descent (SGD) and its variant Adam, among other optimization algorithms. The selection logic is not random, but rather based on matching the scale distribution and texture complexity of defects in the standard dataset: when defects are mainly micro-cracks, the ResNet framework with deep feature extraction capabilities is prioritized; when the defect scale span is large, the FPN framework with multi-scale fusion capabilities is prioritized. It should be understood that the above examples are for illustrative purposes only and not restrictive; any network architecture with non-linear mapping capabilities for image features can be considered as a candidate.
[0049] The ratio of the dataset split directly determines the sufficiency of model learning and the objectivity of evaluation. In this embodiment, the preferred ratio is 7:1.5:1.5, meaning 70% of the data is used for the training set to drive model weight updates, 15% is used for the validation set to monitor overfitting and adjust hyperparameters in real time during training, and 15% is used for the test set for blind testing after training is complete. This ratio avoids the problem of excessive evaluation variance caused by insufficient test set data in the traditional 8:1:1 split, and also avoids the problem of insufficient feature learning caused by excessive loss of training data in the 6:2:2 split. The mechanism of using the validation set to adjust internal parameters is as follows: when the loss function on the validation set starts to rise while the loss on the training set is still decreasing, the system determines that the model is overfitting. At this point, by introducing internal parameter adjustment strategies such as increasing the Dropout ratio, increasing the L2 regularization coefficient, or decreasing the learning rate, the model is forced to abandon the memory of noise specific to the training set and instead learn more generalizable, essential features.
[0050] The metric thresholds are the hard thresholds for model admission to the production line, typically including accuracy thresholds, recall thresholds, and F1-score thresholds. When all test metrics exceed the metric thresholds, it proves that the constructed model has stable defect capture and anti-interference capabilities, and can then be marked as a formal defect identification model and deployed. Importantly, when test metrics fail to meet the standards, this embodiment resolutely abandons the crude approach of directly abandoning the current model and starting from scratch to collect data for reconstruction. Instead, it adopts an iterative logic of rebuilding and training the model again. The underlying mechanism of this iterative logic is that failing to meet test metrics is often not due to fundamental defects in the model framework itself, but rather because hyperparameter configuration, data augmentation strategies, or training epochs have not reached an optimal balance. In this case, the system backtracks and adjusts internal parameters such as the number of network layers, modifies the weight ratio of the loss function, or increases the random perturbation amplitude of data augmentation, restarting the training process. This allows the system to gradually approach and eventually surpass the metric thresholds without increasing additional data collection costs. This iterative mechanism based on fine-tuning of internal parameters not only significantly shortens the model optimization cycle but also forces the model to uncover deeper defect feature representations through continuous parameter space search. This ensures that the final deployed defect identification model possesses excellent generalization ability and robustness against unknown variant defects. The above description is merely a preferred embodiment and aims to demonstrate that this functional limitation has sufficient feasibility and rigorous engineering logic, and is not an exhaustive restriction on specific types of artificial intelligence models.
[0051] In one possible implementation of the embodiments of this application, combined with Figure 1 ,like Figure 3 As shown, the above S104 can be implemented through the following S301-S305, which are explained in detail below: S301. Obtain the morphological feature information of defects in the defect recognition image; calculate the similarity between the defects in the defect recognition image and each defect type in the defect type library based on the constructed similarity analysis function.
[0052] The morphological feature information includes: length, width, and texture feature vector; the expression for the similarity analysis function is: ;in, This represents the similarity between the defect and the i-th defect type; Indicates the length of the defect; This represents the average length of the i-th defect type; Indicates the width of the missing item; This represents the average width of the i-th defect type; This represents the texture similarity between the defect and the i-th defect type.
[0053] It should be noted that length L and width W are quantitative descriptions of the macroscopic geometric contour of the defect. They can be obtained by performing connected component analysis on the defect recognition image to extract the minimum bounding rectangle of the defect region. The number of pixels on the long and short sides of this rectangle corresponds to the values of L and W. Texture feature vectors, on the other hand, provide a deep depiction of the microscopic pixel distribution patterns of the defect. They are typically extracted using algorithms such as Local Binary Pattern (LBP) or Gray-Level Co-occurrence Matrix (GLCM) and can reflect the roughness, directionality, and periodicity within the defect. It should be understood that although this embodiment preferably uses the bounding rectangle method to obtain L and W, in other embodiments, for irregularly curved cracks, the skeletonization extraction method can also be used to obtain its true extension length as L, as long as it can objectively reflect the macroscopic scale of the defect.
[0054] This embodiment abandons the traditional approach of simply using Euclidean distance to calculate geometric similarity, and instead creatively adopts a function design that integrates exponential morphology and geometric morphology. The underlying mechanism is as follows: Euclidean distance penalizes size differences linearly; when defect size fluctuates slightly, the Euclidean distance is not sensitive enough, easily leading to similar-sized scratches and microcracks failing to differentiate in distance scores. In contrast, the exponential function has a steep decay characteristic; as long as the current defect size deviates slightly from the average size of the type library, its morphological score drops dramatically at an exponential rate, thus greatly amplifying the distinguishability of small size differences and forcing candidate types with mismatched sizes to be quickly eliminated. Simultaneously, coupling the macroscopic geometric exponential term with the microscopic texture similarity means that only when both macroscopic size and microscopic texture are highly consistent can a high score be obtained. This stringent joint gating mechanism effectively prevents defects with similar morphology but different textures from being misclassified into the same category.
[0055] In one possible implementation, texture similarity can be obtained in the following ways: Retrieve the texture feature vectors of the defects and the texture feature vectors of each defect type; calculate the texture similarity between the defects and each defect type based on the constructed texture similarity analysis function; The expression for the texture similarity analysis function is: ;in, Represents the texture feature value of the j-th dimension of the defect; The dimension of the texture feature vector; This represents the average value of the texture features in the j-th dimension for the i-th defect type; Let represent the standard deviation of the j-th dimension of the i-th defect type.
[0056] It should be noted that this embodiment employs a texture similarity analysis function based on the Gaussian kernel function. Its physical significance and technical advantages lie in the fact that the Gaussian kernel function uses the standard deviation of the i-th defect type in the j-th dimension as an adaptive scaling factor to smooth out similar fluctuations. In real production line data, due to subtle changes in tool wear and lighting angle, the texture feature values of the same type of defect (such as scratches) inevitably fluctuate around the mean. If simply summing the absolute differences is used, this normal similar fluctuation will be incorrectly penalized as a decrease in similarity. However, the Gaussian kernel function normalizes and absorbs these fluctuations. When the fluctuation amplitude is within the standard deviation range, the exponential term decays extremely smoothly, giving similar defects a very high tolerance and robustness. Conversely, when encountering different types of defects (such as cracks), the texture feature values deviate from the scratch mean by much more than the standard deviation, and the exponential term rapidly decays to an extremely low value. This achieves a sharp amplification of inter-class differences and effective smoothing of intra-class fluctuations at the texture level. Regarding the typical value of dimension d, in order to achieve a balance between feature richness and computational efficiency, d is preferably set to a 128-dimensional or 256-dimensional feature vector. This high-dimensional mapping gives the defect texture sufficient degrees of freedom in the feature space, which greatly reduces the probability of feature overlap and collision between different types of defects in the low-dimensional space.
[0057] Example: A detailed calculation is given using common surface cracks in electronic components as an example. Suppose a real crack defect appears on the surface of a component, with a length L = 50 pixels and a width W = 2 pixels. The key 5th dimension value T_5 = 0.85 is extracted from its 256-dimensional texture feature vector. The system then compares this crack with the "scratch" and "crack" types in the type library for similarity.
[0058] First, comparing the "scratches" type: the average length L_1 = 48 pixels, and the average width W_1 = 3 pixels. The morphological index is calculated to be 0.607. The average texture value of scratches in the 5th dimension is 0.60, with a standard deviation of 0.15. The texture similarity component is calculated to be 0.249. The final overall similarity between cracks and scratches is 0.388. Next, comparing the "cracks" type: the average length L_2 = 52 pixels, and the average width W_2 = 1.5 pixels. The morphological index is calculated to be 2.117. The average texture value of cracks in the 5th dimension is 0.82, with a standard deviation of 0.05. The texture similarity component is calculated to be 0.835. The final overall similarity between cracks is 1.336.
[0059] The above calculation example clearly demonstrates that although cracks and scratches are extremely similar in macroscopic length (50 vs 48), making them difficult to distinguish based solely on length, the texture similarity 1 is severely penalized to 0.249 by the Gaussian kernel function because the crack's texture feature value of 0.85 deviates from the scratch's mean of 0.60 by 0.25 (0.25), far exceeding the scratch's own standard deviation of 0.15. Conversely, the same deviation of 0.85 from the crack's mean of 0.82 by 0.03 is entirely within the crack's standard deviation of 0.05, resulting in a high score of 0.835 for texture similarity 2. It is precisely this huge difference in texture similarity (0.249 vs 0.835), through a product coupling mechanism, that amplifies the difference in overall similarity from a minor morphological difference to a significant score of 0.388 vs 1.336, thus accurately classifying the defect as a crack rather than a scratch. This fully demonstrates the irreplaceable role of the similarity function morphological design in resisting avoidance misclassification. The above description is merely a preferred embodiment of this invention, intended to explain the underlying physical mechanism and defense logic of the similarity function, rather than an exhaustive limitation on specific values or formulas.
[0060] S302. Select the defect type with the highest similarity as the type of defect recognition image; integrate the defect types corresponding to the recognized defect recognition images to obtain the defect types of electronic components.
[0061] It should be noted that this step follows the maximum likelihood principle, using the type with the highest similarity score as the final judgment result. When multiple defective areas exist on the surface of the same component, the system aggregates and integrates the judgment results of all areas to form a complete list of defect types for the component, providing a comprehensive causal traceability basis for subsequent parameter adjustments.
[0062] S303. Retrieve the defect type; obtain the parameter adjustment library; match the defect type with the parameter adjustment library to obtain the corresponding adjustment equipment and the corresponding adjustment value.
[0063] It's important to note that the parameter adjustment library is a pre-built expert knowledge base that stores causal mappings between various defect types and production line physical equipment adjustment strategies. This mapping is not arbitrary parameter aggregation, but rather based on profound process mechanisms: for example, when the defect type is a crack, its physical cause is often related to excessive mechanical stress during the mounting or encapsulation process. Therefore, the system maps it to the pressure adjustment value of pressure equipment (such as a pick-and-place machine or laminator) to alleviate stress concentration by reducing pressure. When the defect type is oxidation, its physical cause is usually related to abnormal thermodynamic environments during reflow soldering or curing. Therefore, the system maps it to the temperature adjustment value of temperature equipment (such as a reflow oven or baking oven) to improve the oxidation atmosphere by adjusting peak temperature or holding time. It should be understood that the above examples are illustrative, not restrictive. As examples of what else could be, when the defect type is missing solder balls, it can also be mapped to the squeegee speed or flux spraying amount of the printing equipment, as long as the mapping accurately points to the equipment causing the defect's physical cause.
[0064] S304. Obtain the real-time production parameters of the equipment during the production process; adjust the corresponding production parameters of the equipment based on the adjustment values and the real-time production parameters.
[0065] It should be noted that this step converts virtual adjustment values into physical equipment actions. The system reads the current operating parameters of the adjusting equipment in real time through the industrial bus or PLC interface, and adds the adjustment values as incremental or absolute values to the real-time production parameters, issuing commands to drive the physical equipment to perform the adjustment. This dynamic overlay mechanism based on real-time parameters avoids the risk of equipment exceeding limits or going out of control that may result from blindly issuing absolute commands without knowing the current equipment status.
[0066] S305. Produce a preset number of electronic components using the adjusted production parameters and test them; identify defects in the tested electronic components. If no defects are identified in the tested electronic components, the adjustment of the production parameters is considered complete; otherwise, the production parameters are readjusted.
[0067] It's important to note that this step is the core of building a closed-loop feedback mechanism. Its technical mechanism lies in the fact that any theoretically based parameter adjustment carries the risk of over-adjustment or under-adjustment. Over-adjustment means that while the original defect is eliminated, drastic parameter changes may trigger new derivative defects (e.g., low pressure causing component loosening or displacement). Under-adjustment means that the cause of the original defect has not been completely eliminated, and the defect will still recur with a certain probability. Therefore, physical trial production is necessary to verify the true effectiveness of the adjustment assumptions. Why must a predetermined quantity be produced for re-identification? Because the occasional absence of defects in a small sample (e.g., 1-2 pieces) cannot represent statistically significant process stability, while trial production of a large sample (e.g., thousands of pieces) would lead to serious waste of materials and time. As an example of a numerical endpoint, a reasonable range for the predetermined quantity is preferably set to 50 to 100 pieces. This range can cover random fluctuations under a normal distribution, ensuring that the statistical confidence level reaches an engineering-acceptable level, without causing excessively long production line downtime. Only when all the pre-set number of test components pass the defect identification without any target defects or derivative defects being identified will the system determine that the adjustment is successful and lock the new parameters; otherwise, it indicates that the current adjustment strategy is flawed and the production parameters must be readjusted.
[0068] It should be noted that this embodiment adds anti-dead-loop logic to the process of readjusting production parameters. In real industrial scenarios, if the cause of a defect is misjudged or there is an irreversible hardware failure, relying solely on software-level parameter retries will lead to an infinite deadlock, causing the production line to permanently stop. Therefore, this embodiment clearly specifies the exit conditions for readjustment or the maximum number of retries. As an example, the system sets the maximum number of retries to 3. When three consecutive trial production verifications fail, the system will automatically exit the closed-loop adjustment process, trigger a hardware alarm, and freeze the current parameters, awaiting manual intervention to troubleshoot hardware failures or recalibrate the mapping relationship of the defect type library. The introduction of this anti-dead-loop logic not only ensures the operational safety of the system but also defines a clear boundary between automation and manual intervention in extreme abnormal situations, ensuring that the closed-loop adjustment mechanism, while possessing self-correction capabilities, does not deviate from the scope of engineering control. The above description is only a preferred embodiment of this example, intended to explain the internal logic of closed-loop trial production verification and anti-dead loop. The specific setting of the maximum number of retries can be adaptively adjusted according to the production line's fault tolerance rate, and is not an exhaustive limitation on the value.
[0069] Please see Figure 4 The second aspect of this application provides a machine vision-based surface defect recognition system for electronic components, including: an image acquisition module, a defect recognition module, and a parameter adjustment module; The image acquisition module is used to acquire surface image data of electronic components.
[0070] It should be noted that the image acquisition module is the system's eye for perceiving the external physical world. Its hardware is not limited to a single imaging device. For example, it could be a high-resolution CCD industrial camera to capture high-precision static images, a CMOS high-speed camera to adapt to dynamic continuous shooting on an assembly line, or even include a 3D structured light scanner to obtain point cloud data containing depth information. This module is typically also equipped with auxiliary optical hardware such as coaxial light sources and polarizing filters to eliminate reflections and highlight the contrast of minor imperfections. The image acquisition module continuously streams the acquired surface image data to the downstream defect recognition module at high frame rate and low latency via a gigabit Ethernet port using the GigE Vision protocol or a dedicated data cable using the Camera Link protocol.
[0071] The defect identification module is used to identify defects in electronic components based on surface image data to obtain defect identification images; and to classify the defects in electronic components according to the defect identification images to obtain defect types.
[0072] It should be noted that the defect identification module is the computing power center and intelligent brain of the system, undertaking the most intensive tasks of image preprocessing, feature extraction, and similarity comparison. In terms of hardware, this module is not an ordinary office computer, but a dedicated vision processor equipped with a high-performance GPU processor (such as an NVIDIA Jetson series edge computing board or a discrete graphics card in an industrial control computer). The GPU's massively parallel computing architecture can perfectly accelerate the matrix operations of wavelet transform and artificial intelligence model inference. In terms of software, this module internally implements all the algorithm logic described in Examples 2 to 4, including the wavelet soft thresholding denoising engine, the AI model inference engine, and the similarity classification engine. Its data flow sequence is as follows: first, it receives the raw image data pushed by the image acquisition module through the internal DMA channel; then, it sequentially calls the preprocessing unit, the identification unit, and the classification unit; finally, it converts the pixel features of the virtual domain into semantic-level defect type label instructions.
[0073] The parameter adjustment module is used to adjust the production parameters of electronic components based on the defect type of the electronic components.
[0074] It should be noted that the parameter adjustment module is the arm of the system that intervenes in the physical production process, responsible for converting the abstract label instructions output by the defect identification module into specific equipment control signals. In terms of hardware mapping, the core entity of this module is typically a PLC controller or a high-performance industrial computer, equipped with rich industrial bus interfaces (such as Profinet, EtherCAT, or Modbus TCP), enabling direct communication with the servo drives or temperature control units of various physical adjustment devices on the production line (such as pick-and-place machines, reflow ovens, and printers). In terms of software mapping, this module internally hosts the parameter adjustment library mapping table and the closed-loop trial production verification logic engine described in Example 5.
[0075] The key focus is on explaining the data interfaces and call sequences between the three modules, which are crucial for ensuring the system architecture's practical applicability. The entire data flow follows a strict asynchronous decoupling and callback confirmation mechanism: the image acquisition module, as the data source, packages surface image data into shared memory or a message queue in a triggered or continuous manner; the defect identification module, as the consumer, subscribes to this queue and immediately starts the algorithm pipeline. After classification, it encapsulates the instruction data, including defect type, confidence level, and coordinate location, into a JSON or binary structure and sends it to the parameter adjustment module via TCP / IP socket or industrial bus; upon receiving the instruction, the parameter adjustment module parses the defect type and matches it to the adjustment library, then sends a parameter modification message to the corresponding physical adjustment device. After the device performs the adjustment, it sends a trial production trigger signal back to the image acquisition module, requesting it to conduct a focused re-inspection of the next batch of a preset number of components. The re-inspection results are then passed back to the defect identification module for judgment, until the closed-loop verification is passed or the maximum retry limit is triggered, exiting the loop. It should be understood that although this embodiment describes a workflow architecture based on message queues and bus communication, in other embodiments, for production lines with higher integration, the three modules can also be uniformly deployed in the same multi-core industrial control computer, directly passing data pointers through inter-process communication (IPC) or shared memory, as long as the timing logic of data flow and the reliability of closed-loop verification are met. The above description is only a preferred solution of this embodiment, intended to explain the functional limitations on the entity mapping relationship at the hardware and software levels, rather than an exhaustive limitation on specific communication protocols or hardware models.
[0076] A third aspect of this application provides a computer-readable storage medium storing instructions that, when executed on a machine vision-based electronic component surface defect identification system, cause the machine vision-based electronic component surface defect identification system to perform the methods described in the first aspect and any possible implementation thereof.
[0077] It should be noted that the term "computer-readable storage medium" in this embodiment is a broad, overarching concept. It does not refer to only a specific type of storage hardware, but rather encompasses any physical medium capable of being read and from which instruction data can be extracted by an instruction execution system. For example, this medium can be a read-only memory (ROM) used to store unalterable core identification and classification algorithm instructions; it can also be random access memory (RAM) used to dynamically load and cache closed-loop adjustment instructions for trial production verification during system runtime; it can also be a magnetic disk, hard disk, optical disk (such as CD-ROM, DVD-ROM), or flash drive (USB Flash Drive) used for batch migration and deployment of the entire defect identification program between industrial control computers on different production lines; and even in some distributed architecture production lines, this medium can manifest as a storage array on a cloud server, with the system remotely reading and executing instructions within it via network protocols. It should be understood that as long as the medium possesses data persistence or temporary storage capabilities and can establish a data reading channel with the processor, it falls within the scope of computer-readable storage medium as defined in this invention.
[0078] Some of the data in the above formula are calculated by removing dimensions and taking their numerical values. The formula is the closest to the real situation obtained by software simulation of a large amount of collected data. The preset parameters and preset thresholds in the formula are set by those skilled in the art according to the actual situation or obtained through simulation of a large amount of data.
[0079] The working principle of this application is as follows: This application acquires surface image data of electronic components; identifies defects in electronic components based on the surface image data to obtain defect identification images; classifies the defects of electronic components according to the defect identification images to obtain defect types; and adjusts the production parameters of electronic components based on the defect types.
[0080] The above embodiments are only used to illustrate the technical methods of this application and are not intended to limit it. Although this application has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical methods of this application without departing from the spirit and scope of the technical methods of this application.
Claims
1. A method for identifying surface defects in electronic components based on machine vision, characterized in that, include: Acquire surface image data of electronic components; Defects in electronic components are identified based on surface image data, resulting in defect identification images; Based on the defect recognition images, the defects of electronic components are classified to obtain the defect types; The production parameters of electronic components are adjusted based on the defect type of the electronic components.
2. The method for identifying surface defects of electronic components based on machine vision according to claim 1, characterized in that, The identification of defects in electronic components based on surface image data includes: Retrieve surface image data of electronic components; acquire standard template images; preprocess the surface image data; adjust the size of the preprocessed surface image data according to the standard template images; The defect identification model is invoked by inputting the adjusted surface image data into the defect identification model to obtain the extracted defect identification image; the defect identification model is built based on an artificial intelligence model.
3. The method for identifying surface defects of electronic components based on machine vision according to claim 2, characterized in that, The preprocessing of the surface image data includes: Retrieve surface image data of electronic components; convert the surface image data to grayscale; perform two-dimensional discrete wavelet transform on the grayscale surface image data to obtain high-frequency coefficients; The expression for the wavelet transform is: in, , Image size; It is a scaling function; It is a wavelet function; The number of decomposition levels; As the starting scale; These are translation parameters; A soft thresholding function is used to denoise the high-frequency coefficients; the denoised high-frequency coefficients are then subjected to inverse wavelet transform to obtain the preprocessed surface image data. The expression for the soft threshold function is: in, The threshold is used; the expression is: ; The standard deviation of noise. This represents the total number of pixels in the surface image.
4. The method for identifying surface defects of electronic components based on machine vision according to claim 2, characterized in that, The defect identification model is built based on an artificial intelligence model and includes: Obtain a standard dataset; wherein the standard dataset includes standard input data consistent with the content attributes of the adjusted surface image data; and standard output data consistent with the content attributes of the defect image. Model frameworks and deep learning algorithms are selected from an artificial intelligence model library; artificial intelligence models are constructed based on the model frameworks and deep learning algorithms to obtain the constructed models; The standard dataset is divided into a training set, a validation set, and a test set according to a set ratio; the model is trained using the training set; the internal parameters of the model are adjusted using the validation set; and the trained model is tested using the test set to obtain test metrics. Obtain the indicator threshold; compare the test indicators with the indicator threshold. If all test indicators are greater than the indicator threshold, mark the constructed model as a defect identification model; otherwise, reconstruct and train the constructed model.
5. The method for identifying surface defects of electronic components based on machine vision according to claim 1, characterized in that, The classification of defects in electronic components based on defect recognition images includes: Retrieve defect recognition images; obtain a defect type library; analyze the similarity between the defect recognition images and each defect type in the defect type library; The defect type with the highest similarity is selected as the type of defect recognition image; the defect types corresponding to the recognized defect recognition images are integrated to obtain the defect type of electronic components.
6. The method for identifying surface defects of electronic components based on machine vision according to claim 5, characterized in that, The analysis of the similarity between the defect recognition image and each defect type in the defect type library includes: Obtain the morphological feature information of defects in the defect recognition image; wherein, the morphological feature information includes: length, width and texture feature vector; The similarity between defects in the defect recognition image and each defect type in the defect type library is calculated based on the constructed similarity analysis function. The expression for the similarity analysis function is: ;in, This represents the similarity between the defect and the i-th defect type; Indicates the length of the defect; This represents the average length of the i-th defect type; Indicates the width of the missing item; This represents the average width of the i-th defect type; This represents the texture similarity between the defect and the i-th defect type.
7. The method for identifying surface defects of electronic components based on machine vision according to claim 6, characterized in that, The methods for obtaining the texture similarity include: Retrieve the texture feature vectors of the defects and the texture feature vectors of each defect type; calculate the texture similarity between the defects and each defect type based on the constructed texture similarity analysis function; The expression for the texture similarity analysis function is: ;in, Represents the texture feature value of the j-th dimension of the defect; The dimension of the texture feature vector; This represents the average value of the texture features in the j-th dimension for the i-th defect type; Let represent the standard deviation of the j-th dimension of the i-th defect type.
8. The method for identifying surface defects of electronic components based on machine vision according to claim 1, characterized in that, The adjustment of production parameters for electronic components based on defect types includes: Retrieve the defect type; obtain the parameter adjustment library; match the defect type with the parameter adjustment library to obtain the corresponding adjustment equipment and corresponding adjustment value; Acquire real-time production parameters of the equipment during the production process; adjust the corresponding production parameters of the equipment based on the adjustment values and real-time production parameters; The system produces a preset number of electronic components using the adjusted production parameters and then tests them. Defects are identified in the tested electronic components. If no defects are identified in the tested electronic components, the adjustment of the production parameters is considered complete; otherwise, the production parameters are readjusted.
9. A machine vision-based electronic component surface defect identification system, applied to the machine vision-based electronic component surface defect identification method according to any one of claims 1-8, characterized in that, include: Image acquisition module, defect identification module, and parameter adjustment module; The image acquisition module is used to acquire surface image data of electronic components; The defect identification module is used to identify defects in electronic components based on surface image data, and obtain defect identification images; Based on the defect recognition images, the defects of electronic components are classified to obtain the defect types; The parameter adjustment module is used to adjust the production parameters of electronic components based on the defect type of the electronic components.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed on the machine vision-based electronic component surface defect identification system, cause the machine vision-based electronic component surface defect identification system to perform the steps of the method described in any one of claims 1-8.