An electronic component quality detection method and system based on deep learning

Abstract

The application discloses a kind of based on deep learning electronic component quality detection method and system, it is related to electronic component quality detection technical field.The present application is in view of the problems that existing detection scheme relies on composite defect annotation sample, composite defect recognition accuracy is low and cannot identify unknown composite defect combination outside training set, constructs the defect attribute splitting system of orthogonal shape attribute and material attribute, adopts the architecture of shared backbone network combined with double independent detection branch to extract image multi-scale basic features, synchronously outputs the attribute detection result of two types of defect regions, generates final defect detection result after spatial position alignment matching and electronic manufacturing process failure priori rule legality verification.The present application can realize stable identification of composite defect under the condition of lacking composite defect sample, reduce system deployment cost and adaptation cycle, and be suitable for appearance quality detection of various electronic component production line.

Description

Technical Field

[0001] This invention relates to the field of electronic component quality inspection technology, specifically to a method and system for electronic component quality inspection based on deep learning. Background Technology

[0002] Electronic components are the core foundational parts of the electronic information industry, and their appearance quality directly determines the electrical performance and operational reliability of terminal electronic devices. With the rapid development of the electronics manufacturing industry, electronic components continue to evolve towards miniaturization and high integration, and production line speeds are constantly increasing. Traditional manual visual inspection methods suffer from drawbacks such as low inspection efficiency, poor consistency of judgment standards, and easy visual fatigue among personnel, and can no longer meet the inspection needs of large-scale mass production. Machine vision inspection technology based on deep learning possesses the technical characteristics of high inspection accuracy, stable operation, and adaptability to high-speed production line speeds, and is currently being gradually applied to the appearance quality inspection of electronic components.

[0003] Chinese invention patent application CN111932511A discloses a method and system for quality inspection of electronic components based on deep learning. This method acquires images of the appearance of electronic components, constructs a convolutional neural network detection model, and labels and trains different types of defects as independent categories, ultimately outputting the location information and type determination results of the defects. Existing similar deep learning-based electronic component defect detection schemes generally adopt a single-branch detection network architecture, setting detection categories based on a single defect type. For composite defects formed by the superposition of multiple defects, sufficient composite defect samples must be collected in advance and their categories labeled. Only after the composite defects are added as independent categories to the training dataset can the corresponding composite defects be identified.

[0004] In actual electronic component manufacturing processes, the probability of compound defects is much lower than that of single defects. The number of effective labeled samples that can be obtained is very limited. The detection model trained using existing technical solutions has a significantly lower accuracy rate in identifying compound defects than in identifying single defects. Furthermore, it cannot identify novel combinations of compound defects not covered in the training dataset, directly leading to a persistently high rate of missed detections of compound defects on the production line. At the same time, the collection, labeling, and iterative updates of compound defect samples also significantly increase the deployment cost and adaptation cycle of the detection system. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of the prior art and provide a method and system for quality inspection of electronic components based on deep learning, so as to solve the problems of existing deep learning-based electronic component quality inspection schemes relying on composite defect labeled samples, low accuracy of composite defect identification, and inability to identify unknown composite defect combinations outside the training set.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution: On one hand, a method for quality inspection of electronic components based on deep learning, comprising acquiring appearance inspection images of electronic components, inputting the appearance inspection images into a deep learning detection network to obtain defect detection results, including the following steps: Step 1: Construct a defect attribute decomposition system in which morphological attributes and material attributes are orthogonal. The morphological attributes correspond to the geometric shape and spatial distribution characteristics of the defect, and the material attributes correspond to the surface material and optical properties of the defect. Step 2: A network architecture combining a shared backbone network and two independent detection branches is adopted. The multi-scale basic features of the input image are extracted through the shared backbone network and input into the morphology detection branch and the material detection branch respectively. The morphology attribute detection results and material attribute detection results of the defect area are output simultaneously. Step 3: Spatial alignment and matching of the two types of test results, and legality verification of the matching results based on the prior rules of electronic manufacturing process failure; Step 4: Generate the final defect detection result based on the verified attribute combinations.

[0007] Furthermore, during the joint fine-tuning of the network, an attribute orthogonal constraint loss function is used to constrain the feature independence of the two branches. The expression for the attribute orthogonal constraint loss function is: In the formula, This represents the loss value due to orthogonal constraint on the attribute. This represents the total number of input samples in a single training iteration. Indicates the sample number. This represents the single-sample feature vector output by the morphological detection branch. This represents the feature vector of the same sample output by the material detection branch. The numerator is the result of the dot product of the two feature vectors, and the denominator is the product of the L2 norms of the two feature vectors.

[0008] Furthermore, spatial alignment matching uses a spatial matching degree calculation function to determine the corresponding association between the two types of detection boxes. The expression of the spatial matching degree calculation function is: In the formula, This represents the spatial matching degree between the shape detection box and the material detection box. This represents the area of ​​the intersection region between the two detection boxes. This represents the area of ​​the defect detection box output by the morphology detection branch. This represents the area of ​​the defect detection box output by the material inspection branch. The corresponding attribute output is for two detection boxes with a spatial matching degree higher than a preset threshold, which are determined to be in the same defect region.

[0009] Furthermore, a morphology feature enhancement unit is set at the input of the morphology detection branch. This unit strengthens the edge contour features of defects by embedding edge operator convolution kernels and introduces deformable convolutions to adapt to geometric defects with different orientations. A material feature enhancement unit is set at the input of the material detection branch. This unit enhances the color deviation and texture features of defects through color space transformation and local texture extraction operators. The two detection branches are trained independently via backpropagation, with gradients not propagating between them, ensuring the decoupling of the two attribute dimensions.

[0010] Furthermore, the network training employs a three-stage execution strategy. The first stage uses a large-scale industrial defect dataset to pre-train a shared backbone network, acquiring general industrial image feature extraction capabilities. The second stage freezes all parameters of the shared backbone network and independently trains two detection branches using single-morphology defect samples and single-material defect samples, respectively. The third stage unfreezes the top-level parameters of the shared backbone network and performs joint fine-tuning using mixed single-defect samples, simultaneously optimizing feature extraction accuracy and attribute recognition accuracy.

[0011] Furthermore, the prior rules for electronic manufacturing process failures are constructed based on the physical failure mechanisms of components, and the legality of the matched attribute combinations is verified. The verification process compares each attribute combination with the preset rules, eliminating incorrect combinations that do not conform to the physical failure laws, and retaining valid attribute combinations that conform to the rules to generate the final defect detection results. The rule base supports targeted adjustments based on the process characteristics of different component categories.

[0012] On the other hand, a deep learning-based electronic component quality inspection system, applicable to a deep learning-based electronic component quality inspection method, includes an image acquisition module, a detection calculation module, a result output module, an attribute splitting configuration module, a dual-branch detection module, a spatial matching verification module, and a defect generation module. The attribute splitting configuration module is used to store defect attribute splitting rules that are orthogonal to morphological attributes and material attributes. The dual-branch detection module has a built-in shared backbone network, a morphological detection branch, and a material detection branch, used to extract corresponding features of the input image and output two types of attribute detection results. The spatial matching verification module is used to complete the spatial alignment of the two types of detection results and the legality verification of process rules. The defect generation module is used to generate the final defect detection result based on the attribute combination that has passed the verification.

[0013] Furthermore, the dual-branch detection module internally includes a shared feature extraction unit, a morphological attribute detection unit, and a material attribute detection unit. The shared feature extraction unit employs a multi-scale convolutional architecture, responsible for extracting common basic features of the input image while simultaneously feeding feature data to both detection units. Each of the two detection units has a built-in corresponding feature enhancement structure, independently completing defect identification and localization output for its respective dimension, without interfering with the detection process and ensuring the decoupling characteristics of the attribute dimensions.

[0014] Furthermore, the spatial matching verification module incorporates a built-in process prior rule base and a spatial matching calculation unit. The spatial matching calculation unit is responsible for calculating the spatial matching degree of the two types of detection boxes and establishing a one-to-one correspondence between defect areas. The process prior rule base stores verification rules constructed based on the failure mechanism of electronic manufacturing. It performs legality screening on the matched attribute combinations, eliminates incorrect combinations that do not conform to physical laws, and outputs valid attribute combinations to the defect generation module.

[0015] Furthermore, a training configuration module is included, which incorporates phased training logic and a confidence fusion calculation function. This confidence fusion calculation function generates the final comprehensive confidence score of the defect, and its expression is: In the formula, The overall confidence level value representing the defect. This represents the confidence weight of the morphological detection branch. This indicates the confidence weight of the material testing branch. This represents the defect confidence score output by the morphological detection branch. This represents the defect confidence value output by the material inspection branch. The training configuration module supports iterative updates of network parameters and dynamic adjustments of the rule base to adapt to the inspection needs of different types of components.

[0016] Compared with existing technologies, this deep learning-based method and system for quality inspection of electronic components has the following advantages: I. This invention constructs a defect attribute decomposition system with orthogonal morphological and material attributes, combined with a network architecture featuring a shared backbone network and dual independent detection branches. This transforms traditional single-dimensional defect category recognition into parallel recognition with dual independent attribute dimensions. It eliminates the need to label and train composite defects as independent categories and achieves effective recognition of composite defects without relying on a large number of labeled composite defect samples. At the same time, it can identify legitimate composite defect combinations not covered by the training dataset, effectively reducing the sample collection and labeling costs of the detection system, shortening the system's adaptation cycle on different production lines, and improving the problem of low accuracy in composite defect recognition.

[0017] Second, this invention strengthens the feature decoupling characteristics of the two detection branches by using an attribute orthogonal constraint loss function and a three-stage training strategy. Combined with the electronic manufacturing process failure prior rules to verify the legality of the matching results, it can effectively reduce false detection results caused by attribute confusion and improve the accuracy and stability of defect attribute identification. At the same time, the process rule library supports targeted adjustment according to different component categories. Combined with the feature reuse design of the shared backbone network, it improves the system's scenario adaptability while ensuring detection operation efficiency, and can flexibly adapt to the appearance inspection needs of multiple categories of electronic components.

[0018] Other advantages, objectives and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination or study, or may be learned from the practice of the invention. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0020] Figure 1 This is a flowchart of the method of the present invention; Figure 2 This is a schematic diagram of the dual-branch detection network architecture of the present invention; Figure 3 This is a schematic diagram illustrating the spatial alignment matching and rule validity verification of the present invention. Detailed Implementation

[0021] To further illustrate the technical means and effects of the present invention in achieving its intended purpose, the following detailed description of the specific implementation methods, structures, features, and effects of the present invention, in conjunction with the accompanying drawings and preferred embodiments, is provided below.

[0022] like Figure 1-3 As shown, this invention discloses a method for quality inspection of electronic components based on deep learning, comprising the following steps: Step 1: Construct a defect attribute splitting system A defect attribute decomposition system is constructed that is orthogonal between morphological attributes and material attributes. Morphological attributes correspond to the geometric shape and spatial distribution characteristics of defects, while material attributes correspond to the surface material and optical properties of defects.

[0023] Specifically, morphological attributes are hierarchically classified according to the geometric dimensions and spatial distribution patterns of defects. The basic categories include point defects, linear defects, and planar defects, each further subdivided into specific morphological features. Point defects can include small, discrete forms such as pits, protrusions, and pinholes; linear defects can include extended forms such as scratches and cracks; and planar defects can include large-area deformation areas and irregular defects. Material attributes are hierarchically classified according to the optical performance and material variation patterns of defects. The basic categories include color anomalies, texture anomalies, and reflectivity anomalies, each further subdivided into specific material features. Color anomalies can include features such as oxidation, color difference, and discoloration; texture anomalies can include features such as stains, coating peeling, and roughening; and reflectivity anomalies can include features such as uneven reflection and lack of reflection.

[0024] In some optional implementations, the two attribute dimensions remain completely orthogonal; that is, the determination of morphological attributes does not depend on material features, and the determination of material attributes does not depend on morphological features. The same morphological attribute can correspond to multiple different material attributes, and the same material attribute can also correspond to multiple different morphological attributes. All defect types can be decomposed into combinations of morphological and material attributes. Composite defects are essentially the superposition of a single morphological attribute and a single material attribute, and do not require separate labeling and training as independent categories. The role of this decomposition system is to transform traditional single-dimensional defect category recognition into two-dimensional attribute recognition, fundamentally reducing the dependence of composite defect recognition on composite labeled samples.

[0025] Step 2: Dual-branch network detection and inference A network architecture combining a shared backbone network and two independent detection branches is adopted. The shared backbone network extracts multi-scale basic features of the input image and inputs them into the morphology detection branch and the material detection branch respectively, and outputs the morphology attribute detection results and material attribute detection results of the defect area simultaneously.

[0026] Specifically, the shared backbone network employs a multi-scale convolutional architecture, containing multiple levels of convolutional pooling units, enabling the extraction of multi-scale features from the input image, ranging from low-level texture to high-level semantics. The multi-scale feature maps output by the shared backbone network are simultaneously fed to both the morphology detection branch and the material detection branch. Both branches perform attribute recognition in their respective dimensions based on the same fundamental features. The role of the shared backbone network is to reuse the fundamental feature extraction capability, reducing the overall network's parameter count and computational load, thus ensuring the efficiency of detection and inference operations.

[0027] A morphological feature enhancement unit is set at the input of the morphological detection branch. This unit strengthens the edge contour features of defects by embedding edge operator convolution kernels, and introduces deformable convolution to adapt to geometric defects with different orientations. Edge operator convolution kernels can extract grayscale abrupt change regions in the image, enhancing the boundary information of defects and helping the network more accurately identify their geometric contours. Deformable convolution can adjust the sampling point position of the convolution kernel according to the actual shape of the defect, adapting to irregular geometric defects with different curvatures and orientations, thus improving the accuracy and generalization ability of morphological recognition.

[0028] The material detection branch input end is equipped with a material feature enhancement unit, which enhances the color deviation and texture features of defects through color space conversion and local texture extraction operators. Color space conversion transforms the input image from the original RGB space to a color space more closely aligned with human color perception, separating the luminance and chrominance components for more accurate color deviation capture. The local texture extraction operator statistically analyzes the texture distribution features of local image regions, amplifying the texture differences between normal and defective areas and improving the sensitivity of material attribute recognition.

[0029] The two detection branches are trained independently via backpropagation, with gradients not propagated between them, ensuring the decoupling of the two attribute dimensions. During training, each branch calculates its own detection loss, and gradients are only propagated back to the corresponding branch's network layer, not flowing into the other branch, thus avoiding mutual interference and redundancy in the features of the two branches.

[0030] During the network joint fine-tuning process, an attribute orthogonality constraint loss function is used to constrain the feature independence of the two branches, further enhancing the decoupling effect between the two attribute dimensions. The expression for the attribute orthogonality constraint loss function is: In the formula, This represents the attribute orthogonality constraint loss value, used to quantify the degree of orthogonality between the output features of two detection branches. The smaller the loss value, the stronger the feature independence between the two branches and the better the attribute decoupling effect. This represents the total number of samples input in a single training iteration, i.e., the number of samples included in a single batch of training. This represents the sample sequence number, with a value ranging from 1 to... This corresponds to each training sample within the batch. This represents the single-sample feature vector output by the morphology detection branch. It is taken from the global pooling output of the second-to-last layer of the morphology detection branch and contains all the morphological attribute feature information corresponding to the sample. This represents the feature vector of the same sample output by the material detection branch. It is taken from the global pooling output of the second-to-last layer of the material detection branch and contains all material attribute feature information corresponding to that sample. (Molecular part) The dot product of two feature vectors represents the degree of linear correlation between them; a larger dot product value indicates a higher degree of overlap between the two features. The denominator part... This is the L2 norm product of two eigenvectors, used to normalize the eigenvectors and eliminate the influence of the eigenvector's own magnitude on the correlation calculation results.

[0031] This loss function constrains the cosine similarity of the feature vectors of the two branches to approach zero, making the features extracted by the two branches as orthogonal as possible, avoiding the two branches learning duplicate feature information, and ensuring that the recognition of morphological attributes and material attributes is based entirely on their respective independent feature dimensions.

[0032] The network training adopts a three-stage execution strategy, which optimizes the network parameters in stages, taking into account both the universality of feature extraction and the accuracy of attribute recognition.

[0033] The first stage uses a large-scale industrial defect dataset to pre-train a shared backbone network, acquiring general industrial image feature extraction capabilities. The dataset used in this stage contains industrial appearance defect images of multiple categories and scenarios, without distinguishing specific attribute categories, and is pre-trained only on a general object detection task, enabling the shared backbone network to have the general ability to extract basic texture, edge, and contour features of industrial images.

[0034] The second stage freezes all parameters of the shared backbone network and independently trains two detection branches using single-morphological defect samples and single-material defect samples, respectively. In this stage, the feature output of the shared backbone network is fixed, and only the network parameters of the two detection branches themselves are optimized. The morphological detection branch is trained using only samples labeled with morphological attributes, and the material detection branch is trained using only samples labeled with material attributes. The two branches independently complete backpropagation and parameter updates without interference. The purpose of this stage is to enable each branch to independently identify its corresponding attribute, ensuring the accuracy of single-attribute recognition.

[0035] The third stage unfreezes the top-level parameters of the shared backbone network and uses mixed single-attribute defect samples for joint fine-tuning, simultaneously optimizing feature extraction accuracy and attribute recognition accuracy. This stage grants access to update parameters of the high-level feature layers of the shared backbone network, and simultaneously inputs samples mixing various single-attribute defects for joint training, adding attribute orthogonal constraint loss to the total loss function. This stage, on the one hand, fine-tunes the high-level features of the backbone network to better adapt it to the detection scenarios of electronic components, and on the other hand, further constrains the feature independence of the two branches to ensure attribute decoupling.

[0036] Step 3: Spatial Alignment Matching and Validity Verification Spatial alignment matching is performed on the two types of detection results, and the legality of the matching results is verified based on the prior rules of electronic manufacturing process failure.

[0037] First, spatial alignment matching is performed. A spatial matching degree calculation function is used to determine the correspondence between the two types of detection boxes, establishing a spatial correspondence between the shape detection results and the material detection results. The expression for the spatial matching degree calculation function is: In the formula, This represents the spatial matching degree between the shape detection box and the material detection box. The value ranges from 0 to 1. The higher the value, the greater the spatial overlap between the two detection boxes, and the greater the probability that they belong to the same defect area. This represents the area of ​​the intersection of two detection boxes, that is, the pixel area of ​​the overlapping region between the shape detection box and the material detection box in the same image coordinate system. This represents the area of ​​the defect detection box output by the shape detection branch, which is the total area of ​​pixels covered by the shape detection box. This represents the area of ​​the defect detection box output by the material detection branch, which is the total area of ​​pixels covered by the material detection box.

[0038] Specifically, all shape and material detection boxes are traversed, and the spatial matching degree of each pair of detection boxes is calculated one by one. Two detection boxes with a spatial matching degree higher than a preset threshold are determined to be the corresponding attribute outputs of the same defect area, and a one-to-one correspondence is established. For detection results that do not match a corresponding detection box, they are determined to be single-attribute defects and directly enter the subsequent legality verification process.

[0039] After spatial matching is completed, the legality of the matching results is verified based on the prior rules for failure in electronic manufacturing processes. The prior rules for failure in electronic manufacturing processes are constructed based on the physical failure mechanisms of components, summarizing the attribute correspondence rules of various failure modes in the production process of electronic components, and forming a configurable rule base.

[0040] For example, typical failure patterns include the following categories: Scratch-type defects usually correspond to no abnormal material properties or only abnormal surface reflection, and do not involve large-area oxidation. Oxidation-type defects usually correspond to planar morphological properties, and do not involve sharp linear morphological properties. Coating peeling-type defects usually correspond to irregular planar morphological properties, and do not involve point-like discrete morphological properties.

[0041] The verification process compares each attribute combination with the preset rules, eliminating incorrect combinations that do not conform to physical failure laws and retaining valid attribute combinations that conform to the rules to generate the final defect detection results. The rule base supports targeted adjustments based on the process characteristics of different component categories. For electronic components with different packaging forms and different substrates, corresponding exclusive verification rule sets can be configured to adapt to different testing scenarios.

[0042] Step 4: Generate the final defect detection results The final defect detection result is generated based on the combination of attributes that pass the verification.

[0043] Specifically, for attribute combinations that match successfully and pass verification, the corresponding shape attribute and material attribute are combined to form the final defect attribute for that defect area. For single-attribute defects that do not match but pass verification, the corresponding single attribute is directly retained as the final defect attribute.

[0044] The final defect detection result includes the defect's location coordinates, attribute category, and overall confidence score. The overall confidence score is generated using a confidence score fusion calculation function, expressed as: In the formula, The overall confidence score represents the defect and is used to characterize the overall reliability of the final defect detection results. This represents the confidence weight of the morphology detection branch, which can be adjusted according to the accuracy requirements of different detection scenarios. This indicates the confidence weight of the material testing branch, which can be adjusted according to the accuracy requirements of different testing scenarios. This represents the defect confidence score output by the morphology detection branch. It is output by the detection head of the morphology detection branch and characterizes the reliability of morphological attribute recognition in that region. This represents the defect confidence value output by the material detection branch. It is output by the detection head of the material detection branch and characterizes the reliability of the material property identification in this area.

[0045] For single-attribute defects, the confidence level of the corresponding branch output is directly used as the final confidence level. The generated final defect detection results can be output to the production line control system or data management system for subsequent stages such as defect sorting, process optimization, and quality statistics.

[0046] The present invention also discloses a deep learning-based electronic component quality inspection system, which is applicable to a deep learning-based electronic component quality inspection method, including an image acquisition module, a detection calculation module, a result output module, an attribute splitting configuration module, a dual-branch detection module, a spatial matching verification module, and a defect generation module.

[0047] The attribute splitting configuration module is used to store defect attribute splitting rules that are orthogonal to morphological attributes and material attributes. It supports the configuration and updating of the attribute classification system and provides a benchmark for attribute judgment for the entire detection system.

[0048] The dual-branch detection module incorporates a shared backbone network, a morphology detection branch, and a material detection branch to extract corresponding features from the input image and output two types of attribute detection results. Internally, the module includes a shared feature extraction unit, a morphology attribute detection unit, and a material attribute detection unit. The shared feature extraction unit employs a multi-scale convolutional architecture to extract common basic features of the input image and simultaneously feed feature data to both detection units. The morphology attribute detection unit incorporates a morphology feature enhancement structure to independently complete defect identification and localization output in the morphology dimension. The material attribute detection unit incorporates a material feature enhancement structure to independently complete defect identification and localization output in the material dimension. The two detection units do not interfere with each other's detection processes, ensuring decoupling characteristics in the attribute dimension.

[0049] The spatial matching verification module is used to complete the spatial alignment of two types of inspection results and the legality verification of process rules. The module includes a built-in process prior rule library and a spatial matching calculation unit. The spatial matching calculation unit is responsible for calculating the spatial matching degree of the two types of inspection boxes and establishing a one-to-one correspondence between defect areas. The process prior rule library stores verification rules based on the failure mechanism of electronic manufacturing. It performs legality screening on the matched attribute combinations, eliminates incorrect combinations that do not conform to physical laws, and outputs valid attribute combinations to the defect generation module.

[0050] The defect generation module is used to generate the final defect detection result based on the verified attribute combination, merge the confidence scores of the two branches to obtain the comprehensive confidence score, and output complete defect location, attribute category and confidence score information.

[0051] The image acquisition module is used to acquire images of the appearance of electronic components. It can be used with industrial cameras and lighting systems to complete stable image acquisition and provide clear input images for subsequent inspection.

[0052] The detection and computation module is used to run the deep learning detection network, perform image inference operations, and output the initial detection results of two types of attributes.

[0053] The results output module is used to output the final defect detection results and can be connected to external systems to complete the transmission, display and storage of detection data.

[0054] It also includes a training configuration module with built-in phased training logic and confidence fusion calculation functions, supporting iterative updates of network parameters and dynamic adjustments to the rule base to adapt to the detection needs of different types of components. The training configuration module allows configuration of hyperparameters for each training stage, management of training datasets and training progress, and completion of iterative optimization of the network model.

[0055] Through the above implementation method, defect detection can be decomposed into two independent attribute dimensions for identification. Composite defect identification can be achieved without the need to collect a large number of composite defect samples in advance. At the same time, it can identify legitimate composite defect combinations not covered by the training set, reduce the deployment cost and adaptation cycle of the detection system, and improve the accuracy of composite defect identification and detection coverage.

[0056] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A method for quality inspection of electronic components based on deep learning, comprising acquiring appearance inspection images of electronic components, inputting the appearance inspection images into a deep learning detection network to obtain defect detection results, characterized in that, Includes the following steps: Step 1: Construct a defect attribute decomposition system in which morphological attributes and material attributes are orthogonal. The morphological attributes correspond to the geometric shape and spatial distribution characteristics of the defect, and the material attributes correspond to the surface material and optical properties of the defect. Step 2: A network architecture combining a shared backbone network and two independent detection branches is adopted. The multi-scale basic features of the input image are extracted through the shared backbone network and input into the morphology detection branch and the material detection branch respectively. The morphology attribute detection results and material attribute detection results of the defect area are output simultaneously. Step 3: Spatial alignment and matching of the two types of test results, and legality verification of the matching results based on the prior rules of electronic manufacturing process failure; Step 4: Generate the final defect detection result based on the verified attribute combinations.

2. The method for quality inspection of electronic components based on deep learning according to claim 1, characterized in that, During the joint fine-tuning of the network, an attribute orthogonal constraint loss function is used to constrain the feature independence of the two branches. The expression of the attribute orthogonal constraint loss function is as follows: In the formula, This represents the loss value due to orthogonal constraint on the attribute. This represents the total number of input samples in a single training iteration. Indicates the sample number. This represents the single-sample feature vector output by the morphological detection branch. This represents the feature vector of the same sample output by the material detection branch. The numerator is the result of the dot product of the two feature vectors, and the denominator is the product of the L2 norms of the two feature vectors.

3. The method for quality inspection of electronic components based on deep learning according to claim 1, characterized in that, Spatial alignment matching uses a spatial matching degree calculation function to determine the correspondence between two types of detection boxes. The expression of the spatial matching degree calculation function is as follows: In the formula, This represents the spatial matching degree between the shape detection box and the material detection box. This represents the area of ​​the intersection region between the two detection boxes. This represents the area of ​​the defect detection box output by the morphology detection branch. This represents the area of ​​the defect detection box output by the material inspection branch. The corresponding attribute output is for two detection boxes with a spatial matching degree higher than a preset threshold, which are determined to be in the same defect region.

4. The method for quality inspection of electronic components based on deep learning according to claim 1, characterized in that, The morphology detection branch incorporates a morphology feature enhancement unit at its input, which strengthens the edge contour features of defects by embedding edge operator convolution kernels and introduces deformable convolutions to adapt to geometric defects with different orientations. The material detection branch incorporates a material feature enhancement unit at its input, which enhances the color deviation and texture features of defects through color space transformation and local texture extraction operators. Both detection branches undergo independent backpropagation training, with gradients not propagating between them, ensuring the decoupling of the two attribute dimensions.

5. The method for quality inspection of electronic components based on deep learning according to claim 1, characterized in that, The network training employs a three-stage execution strategy. The first stage uses a large-scale industrial defect dataset to pre-train a shared backbone network, acquiring general industrial image feature extraction capabilities. The second stage freezes all parameters of the shared backbone network and independently trains two detection branches using single-morphology defect samples and single-material defect samples, respectively. The third stage unfreezes the top-level parameters of the shared backbone network and performs joint fine-tuning using mixed single-defect samples, simultaneously optimizing feature extraction accuracy and attribute recognition accuracy.

6. The method for quality inspection of electronic components based on deep learning according to claim 1, characterized in that, The prior rules for electronic manufacturing process failures are built upon the physical failure mechanisms of components, and the validity of matched attribute combinations is verified. The verification process compares each attribute combination with the preset rules, eliminating incorrect combinations that do not conform to the physical failure laws, and retaining valid attribute combinations that conform to the rules to generate the final defect detection results. The rule base supports targeted adjustments based on the process characteristics of different component categories.

7. A deep learning-based electronic component quality inspection system, applicable to the deep learning-based electronic component quality inspection method according to any one of claims 1 to 6, characterized in that, It includes an image acquisition module, a detection and calculation module, a result output module, an attribute splitting and configuration module, a dual-branch detection module, a spatial matching and verification module, and a defect generation module. The attribute splitting and configuration module is used to store defect attribute splitting rules that are orthogonal to morphological and material attributes. The dual-branch detection module has a built-in shared backbone network, a morphological detection branch, and a material detection branch, which are used to extract corresponding features of the input image and output two types of attribute detection results. The spatial matching and verification module is used to complete the spatial alignment of the two types of detection results and the legality verification of process rules. The defect generation module is used to generate the final defect detection result based on the attribute combination that has passed the verification.

8. The electronic component quality inspection system based on deep learning according to claim 7, characterized in that, The dual-branch detection module internally comprises a shared feature extraction unit, a morphological attribute detection unit, and a material attribute detection unit. The shared feature extraction unit employs a multi-scale convolutional architecture, responsible for extracting common basic features from the input image and simultaneously feeding feature data to both detection units. Each of the two detection units has a built-in corresponding feature enhancement structure, independently completing defect identification and localization output for its respective dimension without interfering with the detection process, thus ensuring the decoupling characteristics of the attribute dimensions.

9. The electronic component quality inspection system based on deep learning according to claim 7, characterized in that, The spatial matching verification module incorporates a process prior rule base and a spatial matching calculation unit. The spatial matching calculation unit is responsible for calculating the spatial matching degree of the two types of detection boxes and establishing a one-to-one correspondence between defect areas. The process prior rule base stores verification rules constructed based on the failure mechanism of electronic manufacturing. It performs legality screening on the matched attribute combinations, excludes incorrect combinations that do not conform to physical laws, and outputs valid attribute combinations to the defect generation module.

10. The deep learning-based electronic component quality inspection system according to claim 7, characterized in that, A training configuration module is also included, which incorporates phased training logic and a confidence fusion calculation function. This confidence fusion calculation function generates the final comprehensive confidence score for the defect, and its expression is: In the formula, The overall confidence level value representing the defect. This represents the confidence weight of the morphological detection branch. This indicates the confidence weight of the material testing branch. This represents the defect confidence score output by the morphological detection branch. This represents the defect confidence value output by the material inspection branch. The training configuration module supports iterative updates of network parameters and dynamic adjustments of the rule base to adapt to the inspection needs of different types of components.

Images