Precise electronic component detection method, system and equipment and medium thereof

Abstract

The invention provides a precise electronic component detection method, system and equipment and a medium thereof. The method comprises the following steps: acquiring visual data, electrical data and thermal data of an electronic component; and preprocessing the visual data, the electrical data and the thermal data of the electronic component. According to the invention, electronic component information is comprehensively obtained by fusing visual, electrical and thermal data, and fine appearance defects and complex electrical performance index abnormalities are accurately identified by using a deep learning algorithm and a convolutional neural network. Meanwhile, the detection speed is increased by means of automatic and closely-linked processes of data acquisition, preprocessing, fusion, feature extraction, model detection, classification and sorting and the like, and the performance of the electronic component is comprehensively considered by monitoring multi-aspect performance of the electronic component in different working states, so that high-precision detection is realized, the production efficiency is improved, the quality of the component is comprehensively evaluated, and the production cost is reduced. The high-precision production requirement is met, the large-scale production rhythm is adapted, and the missing potential fault hidden danger is avoided.

Description

Technical Field

[0001] The present invention relates to the technical field of electronic component detection, and particularly to a precise electronic component detection method, system, device and medium thereof. Background Art

[0002] In the technical field of electronic component detection, current electronic devices are continuously developing towards miniaturization, integration and functional diversification. Miniaturization has continuously reduced the size of electronic components, but more functions need to be integrated in a smaller space; integration has closely integrated a large number of components on a tiny circuit board, increasing the mutual influence; functional diversification requires electronic devices to meet complex application scenarios. This series of changes has made the performance and quality requirements for electronic components increasingly stringent. However, there are still certain problems in the detection of electronic components:

[0003] First, in terms of detection accuracy, in the past, relying solely on manual visual inspection or simple measurement tools for detection, it was difficult to detect subtle appearance defects of electronic components, such as tiny cracks and extremely fine pin deformations. At the same time, for some complex electrical performance indicators, the measurement error was large, and it was impossible to accurately judge whether the components met the high-precision production requirements;

[0004] Second, in terms of detection efficiency, traditional methods are mostly piece-by-piece and single-item detection, with a cumbersome process and long time consumption. In a large-scale production environment, the detection speed is far behind the production rhythm, seriously affecting production efficiency and increasing production costs;

[0005] Third, in terms of the comprehensiveness of detection, traditional detection means often can only detect one or several characteristics of electronic components, and it is difficult to comprehensively consider their performance under different working conditions, and it is impossible to obtain comprehensive performance data. For example, it is impossible to simultaneously monitor the electrical performance changes of components under different temperature and voltage conditions, and potential fault hazards are easily missed;

[0006] Therefore, a precise electronic component detection method, system, device and medium thereof are proposed. Summary of the Invention

[0007] In view of this, the embodiments of the present invention hope to provide a precise electronic component detection method, system, device and medium thereof to solve or alleviate the technical problems existing in the prior art and at least provide a beneficial choice.

[0008] To solve the above technical problems, a technical solution adopted in the present application is: a precise electronic component detection method, including the following steps:

[0009] Step 1, obtain visual data, electrical data and thermal data of the electronic component;

[0010] Step 2: Preprocess the visual data, electrical data, and thermal data of the electronic components;

[0011] Step 3: Merge the preprocessed visual data, electrical data, and thermal data using the data layer fusion method to form a dataset containing multi-dimensional information;

[0012] Step 4: Extract features from the data in the fused dataset based on the convolutional neural network in the deep learning algorithm to extract key features;

[0013] Step 5: Based on the extracted key features and the convolutional neural network architecture, construct and train a deep learning model;

[0014] Step 6: Input the data of the electronic components to be detected after feature extraction into the trained deep learning model for detection and generate detection results;

[0015] Step 7: Classify and sort the electronic components according to the detection results.

[0016] Preferably, as a further optimization of this technical solution, in Step 5, the method for constructing and training the deep learning model includes the following steps:

[0017] Step 501: Obtain sample data of multiple normal and defective electronic components, classify and label them to obtain sample labeled data;

[0018] Step 502: Based on the convolutional neural network architecture, combine the dimensions and distribution characteristics of the key features to construct a deep learning model and determine the initial parameters;

[0019] Step 503: Divide the sample labeled data into a training set and a validation set according to a ratio of 7:3 or 8:2;

[0020] Step 504: Use the training set to train the constructed deep learning model;

[0021] Step 505: Use the validation set to validate the deep learning model and generate evaluation metrics;

[0022] Step 506: Judge the deep learning model according to the evaluation metrics and make adjustments according to the judgment results;

[0023] Step 507: Repeat Steps 504-506 until the performance of the deep learning model on the validation set meets the preset performance threshold.

[0024] As a further optimization of this technical solution, in step three, the data layer fusion method is to align the preprocessed visual data, electrical data, and thermal data according to timestamps, and then splice the visual data, electrical data, and thermal data at the same moment along the channel dimension to form a dataset containing multi-dimensional information.

[0025] As a further optimization of this technical solution, in step two, the preprocessing includes removing noise data, handling outliers, and normalizing different types of data.

[0026] As a further optimization of this technical solution, in step one, the visual data is obtained by using an industrial camera to take pictures of electronic components from multiple angles; the electrical data is obtained by using an electronic measuring instrument to measure electronic components; the thermal data is obtained by using an infrared thermal imager to monitor the surface temperature distribution of electronic components in real time.

[0027] As a further optimization of this technical solution, the evaluation metrics include accuracy, recall, F1 score, mean squared error, and mean absolute error;

[0028] The accuracy is used to measure the proportion of samples correctly predicted by the deep learning model, and the formula is:

[0029]

[0030] The recall is used to measure the ability of the deep learning model to correctly identify positive examples, and the formula is:

[0031]

[0032] The F1 score is the harmonic mean of accuracy and recall, and the formula is:

[0033]

[0034] Where:

[0035]

[0036] The mean squared error is used to measure the average squared error between the predicted value and the true value, and the formula is:

[0037]

[0038] The mean absolute error is used to measure the average absolute error between the predicted value and the true value, and the formula is:

[0039]

[0040] Among them, TP represents true positive, TN represents true negative, FP represents false positive, FN represents false negative, n is the number of samples in the validation set, and y i is the true value, and

[0041] Further preferably provided in the technical solution of the present invention, the visual data includes the outer contour, pin shape, number of pins, surface texture, color characteristics of the electronic component, and defect information such as whether there are scratches, cracks, and stains; the electrical data includes the actual resistance value of the resistor, the capacitance value of the capacitor, the tangent of the loss angle of the capacitor, the inductance of the inductor, the quality factor of the inductor, the forward conduction voltage of the diode, the reverse leakage current of the diode, the amplification factor of the triode, and the saturation voltage drop of the triode; the thermal data includes the surface temperature distribution, hot spot position, and temperature gradient change data of the electronic component under various working conditions.

[0042] To solve the above technical problems, another technical solution adopted in this application is: a precise electronic component detection system, the system includes: a data acquisition module, a data preprocessing module, a data fusion module, a feature extraction module, a model construction and training module, a detection module, and a sorting execution module;

[0043] The data acquisition module is configured to acquire visual data, electrical data, and thermal data of the electronic component;

[0044] The data preprocessing module is configured to preprocess the visual data, electrical data, and thermal data of the electronic component;

[0045] The data fusion module is configured to merge the preprocessed visual data, electrical data, and thermal data in a data layer fusion manner to form a data set containing multi-dimensional information;

[0046] The feature extraction module is configured to extract key features from the data in the fused data set based on the convolutional neural network in the deep learning algorithm;

[0047] The model construction and training module constructs and trains a deep learning model based on the convolutional neural network architecture according to the extracted key features;

[0048] The detection module is configured to input the data of the electronic component to be detected after feature extraction into the trained deep learning model for detection and generate a detection result;

[0049] The sorting module is configured to classify and sort the electronic components according to the detection result.

[0050] To solve the above technical problems, another technical solution adopted in this application is: an electronic device, which includes a processor and a memory coupled to the processor. Program instructions are stored in the memory. When the program instructions are executed by the processor, the processor executes the steps of a precise electronic component detection method as described above.

[0051] To solve the above technical problems, another technical solution adopted in this application is: a computer-readable storage medium storing program instructions capable of implementing a precise electronic component detection method as described above.

[0052] Due to the adoption of the above technical solutions in the embodiments of the present invention, it has the following advantages:

[0053] 1. By integrating visual, electrical, and thermal data, the present invention comprehensively obtains electronic component information. Combining with the powerful feature extraction capabilities of deep learning algorithms and convolutional neural networks, it can accurately identify subtle appearance defects and abnormal complex electrical performance indicators, making the detection accuracy far exceed that of traditional manual visual inspection and simple measurement tools, meeting the high-precision production requirements.

[0054] 2. Through the close connection of data collection, preprocessing, fusion, feature extraction, model detection, classification and sorting, etc., the detection process of the present invention has a high degree of automation. Compared with the traditional piece-by-piece and single-item detection methods, it can greatly improve the detection speed, adapt to the large-scale production rhythm, effectively improve production efficiency, and reduce production costs.

[0055] 3. By simultaneously monitoring the performance of electronic components in different working states in multiple aspects, such as thermal data can reflect the surface temperature distribution, hot spot positions, and temperature gradient changes under different working states, and combining with the changes of parameters such as resistance, capacitance, and inductance in electrical data under different conditions, it can comprehensively consider the performance of components, avoid missing potential fault hazards, and comprehensively evaluate the quality of components.

[0056] The above summary is only for the purpose of the specification and is not intended to be limiting in any way. In addition to the above-described illustrative aspects, embodiments, and features, further aspects, embodiments, and features of the present invention will become apparent by reference to the drawings and the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] To more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0058] Figure 1 It is a schematic flowchart of a precise electronic component detection method of the present invention;

[0059] Figure 2 It is a schematic flowchart of a method for constructing and training a deep learning model of the present invention;

[0060] Figure 3 It is a schematic diagram of the functional modules of a precise electronic component detection system of the present invention;

[0061] Figure 4 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. Detailed implementation manners

[0062] The following describes the embodiments of the present disclosure in detail with reference to the accompanying drawings.

[0063] It should be clear that the following illustrates the implementation manners of the present disclosure through specific specific examples, and those skilled in the art can easily understand other advantages and effects of the present disclosure from the content disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all the embodiments. The present disclosure can also be implemented or applied through other different specific implementation manners, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present disclosure. It should be noted that, without conflict, the following embodiments and the features in the embodiments can be combined with each other. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present disclosure without creative efforts belong to the scope of protection of the present disclosure.

[0064] It should also be noted that the following describes various aspects of the embodiments within the scope of the appended claims. It should be obvious that the aspects described herein can be embodied in a wide variety of forms, and any specific structure and / or function described herein is illustrative only. Based on the present disclosure, those skilled in the art should understand that one aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number of aspects described herein can be used to implement the device and / or practice the method. Additionally, this device and / or this method can be implemented using other structures and / or functionality in addition to one or more of the aspects described herein.

[0065] It should also be noted that the diagrams provided in the following embodiments only illustrate the basic concept of the present disclosure schematically, and only the components related to the present disclosure are shown in the diagrams, rather than being drawn according to the number, shape, and size of the components in actual implementation. The type, quantity, and ratio of each component in its actual implementation can be an arbitrary change, and the component layout type may also be more complex.

[0066] In addition, in the following description, specific details are provided to facilitate a thorough understanding of the examples. However, those skilled in the art will understand that the described aspects can be practiced without these specific details.

[0067] Figure 1 It is a schematic flowchart of a precise electronic component detection method according to an embodiment of the present invention. It should be noted that if there are substantially the same results, the method of the present application is not limited to Figure 1 the shown process sequence. As Figure 1 - Figure 2 shown: A precise electronic component detection method includes the following steps:

[0068] Step 1: Obtain visual data, electrical data, and thermal data of the electronic component;

[0069] Specifically, first, use an industrial camera to take pictures of the electronic component from multiple angles. According to the shape, size, and type of defects of the component, determine the appropriate shooting angles and quantities to ensure that the appearance features of the component can be comprehensively captured; for example, for a chip with dense pins, it is necessary to take pictures from multiple inclined angles to clearly present information such as the shape, quantity, and spacing of the pins; at the same time, equip with various lighting devices such as a ring light source and a backlight source, and by adjusting parameters such as the brightness, angle, and color of the light source, highlight the surface texture, color features of the component, and whether there are defects such as scratches, cracks, and stains;

[0070] Then, use an electronic measuring instrument to measure the electronic component. For different types of components, select the corresponding measuring instrument and measuring method; for a resistor, use a multimeter to measure its actual resistance value; for a capacitor, use an LCR tester with a capacitance measurement accuracy of up to ±0.05%, and measure the capacitance value and loss tangent; for an inductor, also use an LCR tester to measure the inductance and quality factor; for a diode and a triode, use a professional semiconductor parameter test instrument to measure the forward conduction voltage and reverse leakage current of the diode, and the amplification factor and saturation voltage drop of the triode, etc.; during the measurement process, ensure the accuracy and stability of the measuring instrument, and set appropriate measurement conditions according to the rated parameters of the component, such as the measurement voltage and current range;

[0071] Next, use an infrared thermal imager to monitor the surface temperature distribution of electronic components in real time; place the electronic components in a normal working state, scan the surface of the components through the lens of the infrared thermal imager, and the infrared thermal imager will generate a corresponding temperature distribution image according to the infrared radiation intensity emitted by different parts of the component surface, so as to obtain the surface temperature distribution data; at the same time, by analyzing the temperature distribution image, the hot spot position, that is, the area where the temperature abnormally rises, and the temperature gradient change data can be determined to understand the temperature change trend on the surface of the component; during the measurement process, pay attention to the influence of environmental temperature and humidity to avoid interference from external factors on the measurement results.

[0072] Step 2: Preprocess the visual data, electrical data, and thermal data of the electronic components;

[0073] Among them, the preprocessing of the visual data, electrical data, and thermal data of the electronic components includes three aspects: one is to remove noise. For visual data, median or Gaussian filtering is used, for electrical data, moving average filtering is adopted, and for thermal data, wavelet denoising is utilized; the second is to process outliers, which can be identified and processed by calculating the mean and standard deviation based on statistical methods to set thresholds, or the Isolation Forest algorithm can be used to judge with the help of a tree model, and then repair, adjust, or correct different types of data respectively; the third is normalization. For visual data, the pixel value is divided by 255, for electrical data, it is processed with the maximum-minimum normalization formula, and for thermal data, normalization is calculated according to its own temperature range to make the data more conducive to subsequent analysis.

[0074] Step 3: Merge the preprocessed visual data, electrical data, and thermal data in a data-level fusion manner to form a dataset containing multi-dimensional information;

[0075] Specifically, first, when collecting visual, electrical, and thermal data, synchronously record the time points of each data collection to add accurate timestamps to the data; for example, while the industrial camera takes pictures, the electronic measuring instrument reads the data, and the infrared thermal imager scans, the high-precision clock module records the time, accurate to the millisecond level; after preprocessing, sort the data according to the timestamps so that different types of data obtained at the same moment correspond and match; if the data collection frequencies are different, the low-frequency data can be supplemented with data points by interpolation, and the high-frequency data is sampled at the time interval of the low-frequency data to ensure that the data time points are strictly aligned; for example, if the electrical data collection frequency is 100Hz and the thermal data collection frequency is 50Hz, linearly interpolate the thermal data or sample the electrical data every other point to ensure that the two time points are the same;

[0076] Then, align the timestamped data and concatenate it along the channel dimension; if the visual data is an RGB three-channel image, its size after preprocessing is H×W×3 (H is the height and W is the width); the electrical data is represented as a one-dimensional array of length N after processing; the thermal data is processed into a two-dimensional temperature distribution matrix of H×W. The electrical data can be extended to a two-dimensional matrix with the same height and width as the visual data (such as by repeated padding or mapping according to certain rules), and extended to three channels. The thermal data is also extended to three channels; finally, the three channels of the visual data, the three channels after the extension of the electrical data, and the three channels after the extension of the thermal data are concatenated in sequence to form a multi-dimensional dataset of H×W×9, which integrates multi-type data information.

[0077] Step 4: Based on the convolutional neural network in the deep learning algorithm, extract features from the data in the fused dataset and extract key features;

[0078] Specifically, first, construct a convolutional neural network architecture including an input layer, convolutional layers, pooling layers, and fully connected layers, as follows;

[0079] The design of the input layer should match the dimension of the fused dataset. If the fused dataset is a multi-channel image data, such as the dataset with the dimension [H, W, 9] (height H, width W, number of channels 9) mentioned above, the number and arrangement of neurons in the input layer should be determined according to this dimension to ensure that the input data can be completely received;

[0080] The convolutional layer is the core part of feature extraction. In this layer, multiple different convolutional kernels (also called filters) are used to perform convolutional operations on the input data. Each convolutional kernel has specific weight parameters. By sliding on the input data and multiplying and summing element by element with the local area of the data, different features are extracted; for example, some convolutional kernels are good at extracting edge features, while others are more suitable for extracting texture features; parameters such as the size, number, and stride of the convolutional kernel need to be adjusted according to specific problems. Usually, the size of the convolutional kernel can be selected as 3x3, 5x5, etc.; the number of convolutional kernels increases as the number of network layers increases. For example, 16 convolutional kernels are used in the shallower convolutional layers and increased to 64 or more in the deeper layers; the stride determines the interval at which the convolutional kernel slides on the input data, and the commonly used stride is 1 or 2;

[0081] After the output of the convolutional layer, an activation function is applied to introduce non-linearity. The commonly used activation function is ReLU (Rectified Linear Unit), and its formula is:

[0082] f(x) = max(0, x);

[0083] The ReLU function can effectively alleviate the vanishing gradient problem, accelerate the training speed of the network, and enable the network to learn more complex features;

[0084] The main role of the pooling layer is to downsample the output of the convolutional layer, reduce the data dimension, reduce the computational amount, and enhance the robustness of features at the same time. Common pooling operations include max pooling and average pooling. Max pooling will select the maximum value in each pooling window as the output, while average pooling calculates the average value of all values in the window. The size and stride of the pooling window are also parameters that need to be adjusted. Generally, the pooling window size is 2x2 and the stride is 2;

[0085] After being processed by multiple convolutional layers and pooling layers, the data will be flattened and input into the fully connected layer. Each neuron in the fully connected layer is connected to all neurons in the previous layer, and is used to combine and classify the features extracted previously. In the fully connected layer, the number of neurons will be gradually reduced, and finally a vector related to the classification task will be output. For example, if it is to judge whether an electronic component is normal and the fault type, a vector of length n will be output, where n is the number of fault types plus 1 (including the normal situation);

[0086] Then, the fused dataset is divided into a training set, a validation set, and a test set; the training set is used for parameter learning of the model, the validation set is used to evaluate the performance of the model during training, adjust hyperparameters, and prevent overfitting, and the test set is used to finally evaluate the generalization ability of the model; usually, the ratio of the training set, the validation set, and the test set can be set to 7:2:1 or 8:1:1;

[0087] Next, select an appropriate loss function according to the specific task objective; if it is a classification task, the commonly used loss function is the cross-entropy loss function, which can measure the difference between the model prediction result and the true label; for multi-classification problems, the cross-entropy loss function can effectively guide the model to learn in the direction of correct classification;

[0088] Subsequently, select an optimizer. The optimizer is used to update the parameters of the model to minimize the loss function; common optimizers include Stochastic Gradient Descent (SGD), Adam, etc.; The Adam optimizer combines the advantages of AdaGrad and RMSProp, can adaptively adjust the learning rate of each parameter, and can achieve good training results in many cases;

[0089] During the training process, the model will continuously iterate on the training set. Each time it iterates, a batch of data is input into the model, the prediction result is calculated through forward propagation, then the loss value is calculated according to the loss function, and then the gradient is calculated through the backpropagation algorithm. Finally, the optimizer is used to update the parameters of the model. This process will be repeated multiple times until the performance of the model on the validation set reaches a satisfactory effect;

[0090] In a trained convolutional neural network, different layers extract features at different levels. Shallow layers usually extract some simple features such as edges and colors; deeper layers can extract more complex and abstract features, which are closely related to the fault types and states of components. An appropriate intermediate layer can be selected, and its output can be used as the key feature. For example, in some experiments, it is found that the output features of the second-to-last layer or the third-to-last layer perform well in classification tasks.

[0091] After extracting the features of the intermediate layer, there may be some redundant or unimportant features. Feature selection methods such as principal component analysis (PCA) and analysis of variance can be used to screen the extracted features, remove noise and redundant information, and retain the most representative and discriminative key features. This can reduce the dimensionality of the features and improve the training efficiency and performance of the subsequent model.

[0092] Step 5: Based on the extracted key features, construct a deep learning model based on the convolutional neural network architecture and train it.

[0093] First, based on the convolutional neural network architecture already used for feature extraction, more convolutional layers can be added to further mine features. For example, convolutional layers with different convolutional kernel sizes can be added, such as 1x1, 3x3, and 5x5 convolutional kernels, to extract features from different scales. The 1x1 convolutional kernel can be used to adjust the number of channels and reduce the computational amount; the 3x3 and 5x5 convolutional kernels are used to capture local features of different sizes. The ReLU activation function is still used after each convolution to introduce non-linearity.

[0094] Then, add a batch normalization (BN) layer after the convolutional layer. The BN layer can accelerate the convergence of the model, reduce the problem of internal covariate shift, and make the model training more stable. It normalizes each mini-batch of data, adjusting the mean of the data to 0 and the variance to 1.

[0095] To prevent overfitting, add a Dropout layer between the fully connected layers. The Dropout layer randomly "drops out" a part of the neurons, so that the model will not overly rely on certain specific neurons during training, enhancing the generalization ability of the model. For example, setting the Dropout rate to 0.5 means that 50% of the neurons will be randomly ignored each time during training.

[0096] Design the output layer according to specific task objectives. If classifying electronic components, such as determining whether a component is normal and what type of fault it belongs to, the number of neurons in the output layer should be equal to the number of classes, and the Softmax activation function is used to convert the output into a probability distribution. If it is a regression task, such as predicting a certain performance index of a component, the linear activation function is used in the output layer;

[0097] To increase the diversity of training data and improve the generalization ability of the model, augmentation operations can be performed on the extracted key feature data. For the visual feature part, operations such as rotation, flipping, scaling, and brightness adjustment can be carried out. For electrical and thermal features, random noise can be added within a certain range;

[0098] Divide the extracted key feature dataset into a training set, a validation set, and a test set, generally in a ratio of 7:2:1 or 8:1:1. The training set is used for parameter learning of the model, the validation set is used to evaluate the performance of the model during training, adjust hyperparameters, and prevent overfitting, and the test set is used to finally evaluate the generalization ability of the model;

[0099] Select an appropriate loss function according to the task type. For classification tasks, the cross-entropy loss function is usually used, which can measure the difference between the model's prediction result and the true label. For regression tasks, the mean squared error (MSE) loss function is commonly used to calculate the average of the squared errors between the predicted value and the true value;

[0100] Select an appropriate optimizer to update the model's parameters to minimize the loss function. Common optimizers include Stochastic Gradient Descent (SGD), Adam, RMSProp, etc. The Adam optimizer combines the advantages of AdaGrad and RMSProp, can adaptively adjust the learning rate of each parameter, and can achieve good training results in many cases. Appropriate learning rates, such as 0.001, and other optimizer-related parameters can be set;

[0101] Determine the number of training epochs, that is, the number of times the model iterates over the entire training dataset; set the batch size, that is, the number of data samples input to the model each time training. A larger batch size can speed up training, but may cause memory shortages. A smaller batch size enables the model to update parameters more frequently during training, but the training speed is slower;

[0102] During the training process, the model continuously iterates over the training set. In each iteration, a batch of data is input into the model, and the prediction results are calculated through forward propagation. Then, the loss value is calculated according to the loss function, and the gradients are calculated through the backpropagation algorithm. Finally, the optimizer is used to update the model's parameters. At the same time, after each epoch, the performance of the model is evaluated using the validation set, and metrics such as the loss and accuracy of the validation set are recorded. If the performance of the validation set no longer improves, overfitting occurs, and the training can be stopped early;

[0103] The trained model is finally evaluated using the test set, and metrics such as accuracy, recall, F1 value, etc. (for classification tasks), or mean squared error, mean absolute error, etc. (for regression tasks) are calculated to evaluate the generalization ability of the model;

[0104] The model is tuned according to the evaluation results. The architecture of the model can be adjusted, such as increasing or decreasing the number of convolutional layers and fully connected layers; hyperparameters can be adjusted, such as the learning rate, batch size, Dropout rate, etc.; different optimizers and loss functions can also be tried until the model reaches satisfactory performance.

[0105] Step 6: Input the data of the electronic component to be detected after feature extraction into the trained deep learning model for detection and generate detection results;

[0106] Specifically, first, according to the method in Step 1, use an industrial camera to capture the appearance of the electronic component to be detected from multiple angles, and at the same time adjust the parameters of the appropriate lighting equipment, such as brightness, angle, and color, to clearly capture the appearance features of the component, including pin shape, surface defects, etc.; use electronic measuring instruments, and adopt corresponding measurement methods for different types of components, such as using a multimeter to measure the resistance value, an LCR tester to measure the capacitance and inductance parameters, a semiconductor parameter tester to measure the parameters of diodes and transistors, etc., ensure the accuracy and stability of the measuring instrument, and set appropriate measurement conditions according to the rated parameters of the component; use an infrared thermal imager to monitor the surface temperature distribution of the electronic component to be detected in the normal working state in real time, generate a temperature distribution image, obtain the hot spot position and temperature gradient change data, and at the same time pay attention to the influence of environmental temperature and humidity on the measurement results;

[0107] Then, preprocess the visual, electrical, and thermal data of the electronic component to be detected according to Step 2, including removing noise (median or Gaussian filtering for visual data, moving average filtering for electrical data, wavelet denoising for thermal data), handling outliers (based on statistical methods or the Isolation Forest algorithm), and normalization (dividing the pixel values of visual data by 255, using the maximum-minimum normalization formula for electrical data, and calculating normalization according to the self-temperature range for thermal data);

[0108] Next, perform data layer fusion on the preprocessed data according to Step 3. Through timestamp alignment and channel dimension concatenation, form a dataset containing multi-dimensional information; and according to Step 4, use the trained convolutional neural network to extract features from the fused data to obtain the key features of the electronic component to be detected;

[0109] Subsequently, input the key feature data of the electronic component to be detected into the deep learning model trained in Step 5, ensuring that the dimension and format of the input data are consistent with the input requirements during model training. For example, if the input during model training is multi-channel image data with a dimension of [H, W, 9], then the data to be detected also needs to be converted to the same dimension and format; after receiving the input data, the model performs forward propagation calculations according to the trained parameters and architecture. Starting from the input layer, the data passes through the convolutional layer, pooling layer, fully connected layer, etc. in sequence, and finally obtains the prediction result at the output layer;

[0110] If it is a classification task, such as determining whether the electronic component is normal and which fault type it belongs to, the output layer of the model usually uses the Softmax activation function to convert the output into a probability distribution. Each category corresponds to a probability value, and the category with the largest probability value is the predicted category; for example, if the output result is [0.1, 0.2, 0.7], corresponding to the probabilities of normal, fault type A, and fault type B respectively, then it is predicted that the component is of fault type B; according to specific requirements, a confidence threshold can be set for each category; if the largest probability value is lower than this threshold, it is considered that the model lacks confidence in this prediction result and needs further inspection or re-evaluation;

[0111] For a regression task, such as predicting a certain performance index of a component, the output layer of the model uses a linear activation function, and the output result is the predicted performance index value; for example, when predicting the resistance value of a resistor, the value output by the model is the predicted resistance value; the deviation between the predicted value and the standard value in the technical specifications of this component can be calculated to evaluate whether the performance of the component meets the requirements. If the deviation exceeds a certain range, it is determined that there is a problem with this component;

[0112] Finally, present the detection results in an intuitive way. For example, use charts to display the predicted categories or performance metric values. For classification tasks, a bar chart can be drawn to show the predicted probabilities for each category. For regression tasks, a line chart can be drawn to compare the predicted values with the standard values. When presenting the detection results, the original visual, electrical, and thermal data of the component to be detected can also be combined, such as showing the appearance image of the component, the temperature distribution image, etc., to help the operator more comprehensively understand the condition of the component. At the same time, record the detection results, including information such as the predicted category, performance metric values, confidence level, etc., and associate the identification information of the component to be detected, such as model number, batch number, etc. These records can be used for subsequent quality traceability, data analysis, and statistics, etc., in order to promptly discover problems in the production process and take corresponding measures.

[0113] Step 7: Classify and sort the electronic components according to the detection results.

[0114] Specifically, first, based on the detection results of the electronic components, combined with production requirements and quality specifications, formulate clear classification criteria. For the electronic components judged to be normal, classify them into the qualified product category. For components with different fault types, such as open circuit, short circuit, parameter abnormality, etc., establish corresponding fault categories respectively.

[0115] Then, use automated sorting equipment to achieve efficient sorting. Common ones include robotic arm sorting systems, conveyor belt sorting systems, etc. These devices obtain the detection result information of the electronic components through data interaction with the detection system and perform precise sorting according to the preset classification criteria. Assist sorting with image recognition technology. By identifying the appearance of the components again, confirm their categories and improve the accuracy of sorting. For some electronic components with doubtful detection results or in a critical state, arrange experienced technicians for manual re-inspection. The technicians, based on professional knowledge and practical experience, combined with the original detection data, re-evaluate and judge the components to ensure the accuracy of classification. During the manual re-inspection process, if it is found that the detection system has misjudged or other problems, record and feedback them in a timely manner for optimizing and improving the detection system.

[0116] Next, according to the classification criteria, store the sorted electronic components in different areas or containers to ensure that components of different categories are isolated from each other to avoid confusion. Set clear labels on the storage areas and containers, indicating information such as the category, quantity, and batch number of the components to facilitate subsequent management and traceability. For qualified products, label them as "qualified products" and mark the production batch and production date; for faulty components, detail the fault types, such as "short - circuit fault", "parameter exceeding standard", etc. At the same time, during the classification and sorting process, record in detail data such as the test results, classification information, and sorting time of each electronic component to form a complete quality record file. Regularly conduct statistical analysis on the classification and sorting data to understand the occurrence frequency, distribution, and change trend of different fault types, providing strong data support for production process improvement and quality control. If it is found through data analysis that a certain fault frequently appears in a specific batch or production line, promptly conduct inspections and adjustments on that batch or production line to reduce the defective rate and improve product quality.

[0117] In one embodiment, specifically, in step five, the method for constructing and training a deep - learning model includes the following steps:

[0118] Step 501: Obtain sample data of multiple normal and defective electronic components, and perform classification and annotation to obtain sample - annotated data;

[0119] Specifically, collect a large number of electronic components from different production batches and different sources, covering normal and various defective components, such as pin deformation, short - circuit, overheating, etc. After using the methods of the previous steps to collect, process data, and extract key features, by professional personnel or according to the technical standards of the components, label the corresponding category for each sample key feature, such as normal, specific fault type; if it is a regression task, label the precise performance index values, such as resistance value, capacitance value.

[0120] Step 502: Based on the convolutional neural network architecture, combine the dimension and distribution characteristics of the key features to construct a deep - learning model and determine the initial parameters;

[0121] Specifically, determine the number and arrangement of neurons in the input layer according to the dimension of the key features. If the key feature is a 100-dimensional vector, set 100 neurons in the input layer. Combine the distribution of key features and select appropriate convolutional kernels. If there are slight local differences in the features, use a 3x3 convolutional kernel to capture details. If you need to focus on features in a larger area, select a 5x5 convolutional kernel. The number of convolutional kernels increases from the shallow layer to the deep layer, for example, 16 in the shallow layer and 64 in the deep layer. The pooling layer selects max or average pooling, sets the window to 2x2, and the stride to 2 to reduce the data dimension. The number of neurons in the fully connected layer depends on the task. For classification tasks, set it according to the number of categories, and for regression tasks, set it according to the number of prediction indicators. Initialize the weights using the Xavier or He initialization method to make the weight values conducive to model convergence. Set the initial learning rate to 0.001 and adjust it according to the training situation later.

[0122] Step 503: Divide the sample labeled data into a training set and a validation set according to a ratio of 7:3 or 8:2.

[0123] Specifically, randomly shuffle the sample labeled data and divide it according to a ratio of 7:3 or 8:2. You can use methods such as drawing lots or a random number table to ensure the randomness of the division. After division, the training set is used for learning model parameters, and the validation set is used to evaluate model performance and adjust hyperparameters.

[0124] Step 504: Use the training set to train the constructed deep learning model.

[0125] Specifically, select the cross-entropy loss function for classification tasks and the mean squared error loss function for regression tasks. Select Adam as the optimizer and utilize its characteristic of adaptively adjusting the learning rate to accelerate model convergence.

[0126] During training, input the training set data into the model in batches. After forward propagation of a batch of data to obtain the prediction results, calculate the loss value, then calculate the gradient through backpropagation, and update the parameters using the optimizer. Repeat continuously according to the set number of training epochs. The number of training epochs is determined according to the complexity of the model and the amount of data, generally dozens to hundreds of epochs.

[0127] Step 505: Use the validation set to validate the deep learning model and generate evaluation metrics.

[0128] Specifically, input the validation set data into the training model to obtain the prediction results. For classification tasks, calculate metrics such as accuracy, recall rate, and F1 value to intuitively reflect the accuracy of the model's classification and its recognition ability for different categories. For regression tasks, calculate the mean squared error and mean absolute error to measure the deviation between the predicted value and the true value.

[0129] Step 506: Judge the deep learning model according to the evaluation metrics and make adjustments according to the judgment results.

[0130] Specifically, if the training set metrics continue to improve while the validation set metrics deteriorate, it indicates that the model is overfitting. In this case, L1 or L2 regularization can be adopted, adding a regularization term to the loss function to constrain the parameter size; or Dropout can be used to randomly discard some neurons in the fully connected layer or convolutional layer, reducing the dependence between neurons and enhancing the generalization ability of the model.

[0131] If both the training set and validation set metrics are low and improve slowly, it means that the model is underfitting. The model complexity can be increased, such as adding convolutional layers or fully connected layers and increasing the number of convolutional kernels; hyperparameters can be adjusted, such as increasing the learning rate and the number of training epochs; more data can also be collected to enrich the diversity of training data and enable the model to learn more features.

[0132] Step 507: Repeat steps 504 - 506 until the performance of the deep learning model on the validation set meets the preset performance threshold.

[0133] Specifically, continuously repeat the process of training, validation, and adjustment. At each iteration, improve the model based on the evaluation metrics and set the performance threshold. For example, in a classification task, the accuracy reaches 95%, and in a regression task, the mean squared error is less than 0.5, etc. When the performance of the model on the validation set reaches the threshold, stop the training to obtain the trained deep learning model for subsequent electronic component detection.

[0134] In one embodiment, specifically, in step three, the data layer fusion method is to align the pre - processed visual data, electrical data, and thermal data according to the timestamp, and then concatenate the visual data, electrical data, and thermal data at the same moment along the channel dimension to form a dataset containing multi - dimensional information.

[0135] Specifically, in the data acquisition stage, to ensure the precise correlation of visual data, electrical data, and thermal data, a high - precision time synchronization device is needed to mark each type of data with a timestamp accurate to the microsecond level. For example, an industrial camera with an accurate clock module is used to collect visual data, and at the same time, the electronic measuring instrument and the infrared thermal imager also record the corresponding time when they are working. After the pre - processing is completed, these data are sorted according to the timestamp. Since there are differences in the acquisition frequencies of different types of data, for example, the acquisition frequency of electrical data is higher, 100 times per second, while the thermal data is acquired 50 times per second; for the thermal data with a lower acquisition frequency, the linear interpolation method is used to estimate the missing data points in the middle based on the data at adjacent time points to make its time interval consistent with that of the electrical data; for the data with a high acquisition frequency, such as electrical data, it can be sampled according to the acquisition frequency of the thermal data, and the data at the corresponding time points are selected to achieve the precise alignment of all data in time.

[0136] After completing the timestamp alignment, splice different types of data at the same moment according to the channel dimension; assume that the visual data is an RGB-format image after preprocessing, with dimensions [height H, width W, number of channels 3], representing the length and width of the image and the three color channels of red, green, and blue; the electrical data exists in the form of a one-dimensional array after preprocessing, such as [resistance value, capacitance value, inductance value...]. At this time, the electrical data needs to be dimensionally expanded to match the visual data in the spatial dimension; the one-dimensional electrical data can be expanded into a two-dimensional matrix according to certain rules, such as repeating and filling each electrical parameter value to make its size [H, W], and then expanding it to three dimensions, that is, [H, W, 3], and the values of each channel are the same, all being the corresponding electrical parameter values; if the thermal data is a two-dimensional temperature distribution matrix [H, W], representing the temperature information at different positions, it is also expanded to three dimensions [H, W, 3], and all three channels are temperature data; finally, the three channels of the visual data, the three channels of the expanded electrical data, and the three channels of the thermal data at the same moment are spliced together in sequence to form a dataset containing multi-dimensional information with dimensions [H, W, 9]. In this way, the dataset integrates visual, electrical, and thermal information in the spatial dimension, providing a rich data basis for subsequent feature extraction and model training.

[0137] In one embodiment, specifically, in step two, the preprocessing includes removing noise data, outlier processing, and normalizing different types of data;

[0138] Among them, the specific steps for removing noise data from visual data, electrical data, and thermal data are as follows:

[0139] For visual data: Visual data mainly comes from images captured by industrial cameras. During the capture process, noise is generated due to unstable light, thermal noise of the camera sensor, etc.; for salt-and-pepper noise, median filtering is an effective processing method. It replaces the value of each pixel point in the image with the median value of its neighboring pixel values to eliminate isolated noise points; for example, when there are some random black and white noise points in the image, after median filtering, these noise points will be replaced by the median value of the surrounding normal pixels, making the image clearer and not affecting the judgment of features such as the outer contour and pin shape of electronic components; for Gaussian noise, Gaussian filtering can be used. Its principle is to perform weighted averaging on neighboring pixels according to the Gaussian function, and the points closer to the central pixel have higher weights. This can smooth the image to a certain extent, suppress noise, and at the same time retain the edge information of the image, avoiding misjudgment of information such as the surface texture and defects of components due to noise interference.

[0140] For electrical data: When measuring the electrical parameters of electronic components, due to factors such as the precision limitation of measuring instruments and external electromagnetic interference, noise is mixed into the electrical data. Moving average filtering is a commonly used denoising method. By averaging a series of consecutive measurement values, it can effectively reduce the influence of random noise. For example, when measuring the actual resistance value, the results of multiple measurements will have small fluctuations. Through moving average filtering, by averaging multiple measurement values within a certain time window, the obtained result can more accurately reflect the true resistance value, making the electrical parameter curve smoother and more conducive to subsequent analysis.

[0141] For thermal data: When an infrared thermal imager acquires thermal data, thermal radiation in the environment and the thermal noise of the instrument itself will interfere with the data. Wavelet denoising technology is suitable for processing such data. It first decomposes the thermal data into different frequency sub-bands, where the noise is mainly concentrated in the high-frequency sub-bands. By setting an appropriate threshold to process the high-frequency sub-bands and removing the noise part, and then performing wavelet reconstruction, the denoised thermal data can be obtained. This can more accurately present the temperature distribution on the surface of electronic components, clearly display the hot spot positions and temperature gradient changes, and provide a reliable basis for judging the thermal performance of the components.

[0142] Among them, statistical methods or machine learning algorithms can be used to process outliers in visual data, electrical data, and thermal data, as follows:

[0143] Based on statistical methods: By calculating the mean and standard deviation of the data, a reasonable threshold range is set to identify outliers. For visual data, electrical data, and thermal data, generally, data points exceeding the mean plus or minus 3 times the standard deviation can be regarded as outliers. Taking visual data as an example, if the gray value of a certain pixel point differs greatly from the gray values of surrounding pixel points, far exceeding the normal fluctuation range, and it is found through calculation that it exceeds the set threshold, then this pixel point is an outlier. For the processing of outliers, if it is an abnormal pixel point in visual data, interpolation method can be used to estimate and replace the outlier according to the gray values of surrounding normal pixels, making the image look more natural and not affecting the analysis of the appearance of the components. For abnormal measurement values in electrical data, such as the measured capacitance value of a certain capacitor being very different from that of other capacitors in the same batch, the abnormal value can be corrected or directly removed by combining multiple measurement data or referring to the standard values of similar components. In terms of thermal data, if the temperature value of a certain area significantly deviates from the overall temperature distribution, the area can be re-measured, or the abnormal temperature value can be reasonably corrected according to the heat conduction principle and the temperature distribution of the surrounding area.

[0144] Based on machine learning algorithms: The Isolation Forest algorithm is an effective outlier detection algorithm. It constructs a tree-like model and determines whether a data point is an outlier based on the path length from the data point to the root node. For visual data, electrical data, and thermal data, taking these data as input, the algorithm will analyze the data. For example, when processing electrical data, if among a series of measurement values of a certain resistor, there is a value whose path length in the Isolation Forest model is significantly different from that of other normal data points, then that value is determined to be an outlier. After determining it as an outlier, for visual data, an image inpainting algorithm can be used for repair to restore the image to normal; for electrical data, historical data or data of other components in the same batch can be referred to for adjustment; for thermal data, it is corrected by combining the heat conduction principle and the surrounding temperature information to ensure the accuracy of the data.

[0145] Among them, the specific steps for separately performing data normalization operations on visual data, electrical data, and thermal data are as follows:

[0146] For visual data: The pixel value range of images captured by industrial cameras is usually 0 - 255 (8-bit images). To make the data have a unified scale in subsequent processing and facilitate the training of deep learning models, it is necessary to normalize the pixel values to the [0, 1] interval. For RGB images, the pixel values of each channel are processed separately, that is, each pixel value is divided by 255. After such processing, all pixel values are within a unified range, avoiding some features being overemphasized or ignored during model training due to excessive differences in pixel value ranges, thereby improving the training effect and stability of the model.

[0147] For electrical data: For different types of electrical parameters, their numerical value ranges vary greatly. For example, the resistance value ranges from a few ohms to megohms, and the capacitance value ranges from picofarads to farads. The maximum - minimum normalization method is adopted to map the value of each electrical parameter to the [0, 1] interval, and the calculation formula is:

[0148]

[0149] Among them, X is the original data, X min and X max are respectively the minimum and maximum values of this parameter in the dataset. In this way, different electrical parameters can have the same weight and influence in model training, improving the model's learning ability for various electrical parameter characteristics.

[0150] For thermal data: The temperature range of thermal data varies depending on the operating conditions and characteristics of electronic components. For example, the operating temperature range of some electronic components is 0 - 80°C. To facilitate the fusion and unified analysis with other types of data, it is necessary to normalize the thermal data. According to its actual temperature range, the formula (T - Tmin )(T max -T min ) is calculated, where T is the actual temperature value, and T min and T max are the set minimum and maximum temperature values respectively. In this way, the thermal data can be normalized to the interval [0, 1], making it on the same scale as the visual data and electrical data, facilitating subsequent data fusion and model analysis, and more accurately mining the information contained in the data.

[0151] In one embodiment, specifically, in step one, the visual data is obtained by using an industrial camera to take pictures of the electronic components from multiple angles; the electrical data is obtained by using an electronic measuring instrument to measure the electronic components; the thermal data is obtained by using an infrared thermal imager to monitor the surface temperature distribution of the electronic components in real time.

[0152] In one embodiment, specifically, the evaluation metrics include accuracy, recall, F1 value, mean squared error, and mean absolute error;

[0153] Accuracy is used to measure the proportion of samples correctly predicted by the deep learning model, and the formula is:

[0154]

[0155] Recall is used to measure the ability of the deep learning model to correctly identify positive examples, and the formula is:

[0156]

[0157] The F1 value is the harmonic mean of accuracy and recall, and the formula is:

[0158]

[0159] Among them:

[0160]

[0161] Mean squared error is used to measure the average squared error between the predicted value and the true value, and the formula is:

[0162]

[0163] Mean absolute error is used to measure the average absolute error between the predicted value and the true value, and the formula is:

[0164]

[0165] Among them, TP represents true positive examples, TN represents true negative examples, FP represents false positive examples, FN represents false negative examples, n is the number of samples in the validation set, and y i is the true value. is the predicted value.

[0166] In one embodiment, specifically, the visual data includes the outer contour, pin shape, number of pins, surface texture, color characteristics of the electronic component, as well as defect information such as the presence of scratches, cracks, and stains; the electrical data includes the actual resistance value of the resistor, the capacitance value of the capacitor, the tangent of the loss angle of the capacitor, the inductance of the inductor, the quality factor of the inductor, the forward conduction voltage of the diode, the reverse leakage current of the diode, the amplification factor of the triode, and the saturation voltage drop of the triode; the thermal data includes the surface temperature distribution, hot spot position, and temperature gradient change data of the electronic component under various working conditions;

[0167] Among them, the visual data can intuitively reflect the appearance condition of the component; the outer contour, pin shape, and number can determine whether it meets the design specifications. For example, deformed pins or missing pins will affect electrical connection and function; the surface texture and color characteristics can reflect the manufacturing process, and defects such as scratches, cracks, and stains will reduce performance or even cause failures. By analyzing these data, the quality of the component can be initially evaluated;

[0168] The electrical data can accurately reflect the electrical performance; the parameters of resistors, capacitors, and inductors determine their functions in the circuit. Excessive deviation will change the circuit characteristics; the parameters of diodes and triodes directly affect their working states in the circuit. Abnormal forward conduction voltage, reverse leakage current, amplification factor, etc. will cause abnormal signal transmission and amplification functions, which are the core basis for judging the electrical performance of components;

[0169] The thermal data can effectively reflect the thermal performance and potential failures; the surface temperature distribution, hot spot position, and temperature gradient change can reveal the heat dissipation and internal loss conditions; the hot spot position implies excessive local power or poor heat dissipation, with potential short - circuit hazards; abnormal temperature gradient changes indicate poor heat conduction, which will affect long - term stability and reliability and are key indicators for evaluating thermal performance.

[0170] Figure 3 is a schematic diagram of the functional modules of a precise electronic component detection system according to an embodiment of the present application. As Figure 3 shown, a precise electronic component detection system includes: a data acquisition module, a data pre - processing module, a data fusion module, a feature extraction module, a model construction and training module, a detection module, and a sorting execution module;

[0171] The data acquisition module is configured to obtain the visual data, electrical data, and thermal data of the electronic component;

[0172] The data pre - processing module is configured to pre - process the visual data, electrical data, and thermal data of the electronic component;

[0173] A data fusion module, configured to merge the preprocessed visual data, electrical data, and thermal data in a data layer fusion manner to form a dataset containing multi-dimensional information;

[0174] A feature extraction module, configured to extract key features from the data in the fused dataset based on a convolutional neural network in a deep learning algorithm;

[0175] A model construction and training module, which constructs and trains a deep learning model based on the extracted key features and a convolutional neural network architecture;

[0176] A detection module, configured to input the data of the electronic component to be detected after feature extraction into the trained deep learning model for detection and generate a detection result;

[0177] A sorting module, configured to classify and sort the electronic components according to the detection result.

[0178] For other details of the technical solutions implemented by each module in the above-mentioned precise electronic component detection system according to the above embodiments, reference can be made to the description in a precise electronic component detection method in the above embodiments, which will not be elaborated here.

[0179] It should be noted that the embodiments in this specification are all described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The same or similar parts among the embodiments can be referred to each other. For system embodiments, since they are basically similar to method embodiments, the description is relatively simple. For related parts, reference can be made to the partial description of the method embodiments.

[0180] An electronic device according to an embodiment of the present disclosure includes a memory and a processor. The memory is used to store non-transitory computer-readable instructions. Specifically, the memory may include one or more computer program products, and the computer program products may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory, etc. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc.

[0181] The processor may be a central processing unit (CPU) or other forms of processing units with data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device to perform desired functions. In an embodiment of the present disclosure, the processor is used to run the computer-readable instructions stored in the memory, so that the electronic device executes all or part of the steps of the precise electronic component detection method of the above embodiments of the present disclosure.

[0182] Those skilled in the art should be able to understand that, in order to solve the technical problem of how to obtain a good user experience effect, this embodiment may also include well-known structures such as communication buses and interfaces, and these well-known structures should also be included in the protection scope of this disclosure.

[0183] Such as Figure 4 FIG. is a schematic structural diagram of an electronic device provided by an embodiment of the present disclosure. It shows a schematic structural diagram of an electronic device suitable for implementing the electronic device in the embodiment of the present disclosure. Figure 4 The shown electronic device is merely an example and should not impose any limitation on the functions and usage scope of the embodiments of the present disclosure.

[0184] Such as Figure 4 As shown, the electronic device may include a processor (such as a central processing unit, a graphics processing unit, etc.), which may perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) or a program loaded from a storage device into a random access memory (RAM). In the RAM, various programs and data required for the operation of the electronic device are also stored. The processor, ROM, and RAM are connected to each other through a bus. The input / output (I / O) interface is also connected to the bus.

[0185] Generally, the following devices may be connected to the I / O interface: input devices including, for example, sensors or visual information acquisition devices; output devices including, for example, a display screen; storage devices including, for example, magnetic tapes, hard disks, etc.; and communication devices. The communication device may allow the electronic device to communicate with other devices (such as edge computing devices) wirelessly or wiredly to exchange data. Although Figure 4 the shown electronic device has various devices, it should be understood that it is not required to implement or have all the shown devices. More or fewer devices may be alternatively implemented or had.

[0186] In particular, according to the embodiments of the present disclosure, the process described above with reference to the flowchart may be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program includes program codes for executing the method shown in the flowchart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication device, or installed from a storage device, or installed from a ROM. When the computer program is executed by the processor, all or part of the steps of a precise electronic component detection method according to the embodiments of the present disclosure are executed.

[0187] For the detailed description of this embodiment, reference may be made to the corresponding descriptions in the foregoing embodiments, and details will not be repeated here.

[0188] A computer-readable storage medium according to an embodiment of the present disclosure stores non-transitory computer-readable instructions thereon. When the non-transitory computer-readable instructions are run by a processor, all or part of the steps of an accurate electronic component detection method according to the various embodiments of the present disclosure described above are executed.

[0189] The above computer-readable storage medium includes but is not limited to: optical storage media (such as CD-ROMs and DVDs), magneto-optical storage media (such as MOs), magnetic storage media (such as magnetic tapes or external hard drives), media with built-in rewritable non-volatile memories (such as memory cards), and media with built-in ROMs (such as ROM cartridges).

[0190] For a detailed description of this embodiment, reference may be made to the corresponding descriptions in the foregoing embodiments, and details are not repeated herein.

[0191] The basic principles of the present disclosure have been described above in connection with specific embodiments. However, it should be noted that the advantages, benefits, effects, etc. mentioned in the present disclosure are only examples and not limitations. It cannot be considered that these advantages, benefits, effects, etc. are essential for each embodiment of the present disclosure. In addition, the above-described specific details are only for illustrative purposes and for ease of understanding, and are not limitations. The above details do not limit the present disclosure to necessarily implement using the above specific details.

[0192] In the present disclosure, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. The block diagrams of devices, apparatuses, equipment, and systems involved in the present disclosure are only illustrative examples and do not intend to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including", "comprising", "having", etc. are open-ended words, meaning "including but not limited to", and can be used interchangeably with each other. The words "or" and "and" used herein refer to the word "and / or", and can be used interchangeably with each other, unless the context clearly indicates otherwise. The word "such as" used herein refers to the phrase "such as but not limited to", and can be used interchangeably with each other.

[0193] In addition, as used herein, the "or" used in the listing of items starting with "at least one" indicates a separate listing, so that for example, the listing of "at least one of A, B, or C" means A or B or C, or AB or AC or BC, or ABC (i.e., A and B and C). In addition, the term "exemplary" does not mean that the described examples are preferred or better than other examples.

[0194] It should also be noted that in the systems and methods of the present disclosure, each component or each step can be decomposed and / or recombined. These decompositions and / or recombinations should be regarded as equivalent solutions of the present disclosure.

[0195] Various changes, substitutions, and alterations to the techniques described herein can be made without departing from the teachings of the technology defined by the appended claims. Additionally, the scope of the claims of the present disclosure is not limited to the specific aspects of the processes, machines, manufactures, compositions of events, means, methods, and acts described above. Processes, machines, manufactures, compositions of events, means, methods, or acts that presently exist or will later be developed that perform substantially the same function or achieve substantially the same result as the corresponding aspects described herein can be utilized. Accordingly, the appended claims include such processes, machines, manufactures, compositions of events, means, methods, or acts within their scope.

[0196] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the aspects shown herein, but rather to the broadest scope consistent with the principles and novel features disclosed herein. The above description has been given for purposes of illustration and description. Additionally, this description is not intended to limit the embodiments of the present disclosure to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and subcombinations thereof.

Claims

1. A precise electronic component detection method, characterized in that: The following steps are involved: Obtain visual, electrical and thermal data of electronic components; Preprocessing of visual, electrical and thermal data of electronic components; The preprocessed visual data, electrical data and thermal data are merged using data layer fusion to form a data set containing multi-dimensional information; Based on the convolutional neural network in the deep learning algorithm, feature extraction is performed on the data in the fused data set to extract key features; Based on the extracted key features, a deep learning model is constructed and trained based on the convolutional neural network architecture; Input the data of the electronic components to be tested after feature extraction into the trained deep learning model for testing and generate the test results; According to the test results, the electronic components are classified and sorted.

2. A precise electronic component detection method according to claim 1, characterized in that: The method for constructing a deep learning model and performing training comprises the following steps: Step 501: Acquire multiple normal and defective electronic component sample data, and classify and label them to obtain sample label data; Step 502: Based on the convolutional neural network architecture, combined with the dimension and distribution characteristics of the key features, a deep learning model is constructed, and initial parameters are determined; Step 503: Divide the sample labeled data into a training set and a validation set in a ratio of 7:3 or 8:2; Step 504: Use the training set to train the constructed deep learning model; Step 505: Use the validation set to validate the deep learning model and generate evaluation indicators; Step 506: judging the deep learning model according to the evaluation index, and adjusting it according to the judgment result; Step 507: Repeat steps 504-506 until the performance of the deep learning model on the validation set meets a preset performance threshold.

3. The method for accurately detecting electronic components according to claim 1, characterized in that: The data layer fusion method is to align the preprocessed visual data, electrical data and thermal data according to timestamps, and then splice the visual data, electrical data and thermal data at the same moment according to channel dimensions to form a data set containing multi-dimensional information.

4. The method for accurately detecting electronic components according to claim 1, characterized in that: The preprocessing includes removing noise data, processing outliers, and normalizing different types of data.

5. The method for accurately detecting electronic components according to claim 1, characterized in that: The visual data is obtained by photographing the electronic components from multiple angles using an industrial camera; the electrical data is obtained by measuring the electronic components using an electronic measuring instrument; and the thermal data is obtained by monitoring the surface temperature distribution of the electronic components in real time using an infrared thermal imager.

6. The method for accurately detecting electronic components according to claim 2, characterized in that: The evaluation indicators include accuracy, recall, F1 value, mean square error and mean absolute error.

7. The method for accurately detecting electronic components according to claim 5, characterized in that: The visual data includes the outline of the electronic components, the shape of the pins, the number of pins, the surface texture, the color characteristics, and the defect information of scratches, cracks, and stains; the electrical data includes the actual resistance of the resistor, the capacitance of the capacitor, the loss tangent of the capacitor, the inductance of the inductor, the quality factor of the inductor, the forward conduction voltage of the diode, the reverse leakage current of the diode, the amplification factor of the transistor, and the saturation voltage drop of the transistor; the thermal data includes the surface temperature distribution, hot spot location and temperature gradient change data of the electronic components under various working conditions.

8. A precise electronic component detection system, applied to a precise electronic component detection method according to any one of claims 1 to 7, characterized in that: The system includes: a data acquisition module, a data preprocessing module, a data fusion module, a feature extraction module, a model building and training module, a detection module and a sorting execution module; The data acquisition module is configured to acquire visual data, electrical data and thermal data of electronic components; The data preprocessing module is configured to preprocess the visual data, electrical data and thermal data of the electronic components; The data fusion module is configured to merge the preprocessed visual data, electrical data and thermal data using a data layer fusion method to form a data set containing multi-dimensional information; The feature extraction module is configured to perform feature extraction on the data in the fused data set based on the convolutional neural network in the deep learning algorithm to extract key features; The model building and training module builds and trains a deep learning model based on the extracted key features and a convolutional neural network architecture; The detection module is configured to input the data of the electronic components to be detected after feature extraction into the trained deep learning model for detection and generate a detection result; The sorting module is configured to classify and sort the electronic components according to the detection results.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the precise electronic component detection method described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute a precise electronic component detection method as described in any one of claims 1-7.

Images