Electronic component defect recognition method and device based on multi-task multi-size network

Abstract

The application discloses a multi-task multi-size network-based electronic component defect identification method and device, and relates to the field of electronic component defect identification.The method comprises the following steps: collecting high-definition industrial images of multiple electronic components including defects as a defect image dataset; determining and labeling the defect positions and defect types of the electronic components in the defect image dataset; dividing the labeled image set into a training set and a test set; training a multi-task multi-size network model using the training set; obtaining a multi-task multi-size network model with minimum loss; testing the multi-task multi-size network model with minimum loss using the test set; obtaining an optimal multi-task multi-size network model with maximum Dice coefficient and accuracy; and inputting the image of an electronic component to be identified into the optimal multi-task multi-size network model to obtain the defect position and defect type of the electronic component to be identified.The application can identify the defect position and defect type of an electronic component, and improve the identification efficiency and accuracy.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image recognition, and particularly relates to an electronic component defect recognition method and device based on a multi-task multi-size network, and especially relates to defect positioning and recognition in different life cycles of electronic component manufacturing, packaging, finished product testing, etc. BACKGROUND

[0002] In the era of high-speed development of the electronic industry, the demand for electronic components has been growing year by year, and at the same time, the requirements for electronic components in different industrial scenarios are also increasing. Therefore, how to improve the production and use efficiency of electronic components and ensure product quality has become a problem to be solved in the electronic industry.

[0003] Electronic component application scenarios cover at least 5 kinds of microwave vacuum devices, microwave solid-state devices, semiconductor integrated circuits, film integrated circuits, and hybrid integrated circuits, including at least 8 types of traveling wave tubes, magnetrons, switch matrices, frequency conversion components, integrated circuit chips, packaging substrates, and ceramic housings. The defect types of electronic components cover at least 15 types of cracks, delamination, scratches, missing corners, oil stains, scratches, string holes, damage, spots, holes, particle contamination, and open / short circuits. Therefore, the defects of electronic components have the characteristics of diverse fault scenarios and complex types. Traditional defect detection methods, such as relying on electrical performance parameter detection and manual visual detection, cannot fully utilize image semantic information, and have obvious defects such as high consumption of manpower and financial resources, low efficiency, and complicated process.

[0004] With the development of deep learning, the method of electronic component defect detection has gradually shifted to automation, high efficiency, and high precision. In the fields of image segmentation, target detection, and classification tasks, a series of defect detection methods based on deep learning have emerged. For example, TDD-Net is proposed based on Faster R-CNN and feature pyramid, which achieves 98% accuracy in electronic component defect detection, but does not have defect type recognition capability. A novel cascaded autoencoder structure is designed for automatic defect detection and recognition based on convolutional neural networks, which is used for segmentation and positioning of defects, but does not realize an integrated detection and recognition system. Due to insufficient information provided by the electronic component defect image dataset, the model does not fully utilize visual and semantic information, resulting in poor dataset effectiveness, low model detection accuracy, low robustness and generalization, and lack of defect type recognition capability in industrial-grade defect detection systems. SUMMARY

[0005] To solve the above problems, the present application provides an electronic component defect recognition method and device based on a multi-task multi-size network, which can accurately and quickly identify the defect position and defect type of an electronic component.

[0006] In one aspect, a method for identifying defects of electronic components based on a multi-task multi-size network comprises:

[0007] S101, collecting high-definition industrial images of a plurality of electronic components including defects as defect image data sets;

[0008] S102, determining and labeling the defect positions and defect types of each electronic component in the defect image data set according to the defined standards for normal operation and defect generation of electronic components, and dividing the labeled image set into a training set and a test set;

[0009] S103, training the multi-task multi-size network model using the training set to obtain a multi-task multi-size network model with the smallest loss, and testing the multi-task multi-size network model with the smallest loss using the test set to obtain an optimal multi-task multi-size network model with the largest Dice coefficient and accuracy;

[0010] S104, inputting the image of the electronic component to be identified into the optimal multi-task multi-size network model to obtain the defect position and defect type of the electronic component to be identified.

[0011] Preferably, S101 specifically comprises:

[0012] Collecting high-definition industrial raw RGB image data of external defects of electronic components including optical, ultrasonic and infrared detection;

[0013] Based on the defined standards for normal operation and defect generation of electronic components, the defect types included in the electronic components are obtained;

[0014] Based on the defined standards for normal operation and defect generation of electronic components, the defect positions of the electronic components are located and labeled, and the defect position mask is obtained, which is stored as a grayscale image;

[0015] According to the defect position, an edge detection algorithm is used to perform edge extraction operation on the obtained defect position mask grayscale image to obtain a defect edge mask, which is stored as a grayscale image;

[0016] The adaptive contour wrapping method is used to segment the defects from the original RGB image data, save the new minimum positive rectangle as a defect local image, and assign a corresponding defect type label;

[0017] The obtained level original RGB image, defect position mask image, defect edge mask image and defect local image are expanded in quantity and form through data enhancement operation on the data set.

[0018] Preferably, the multi-task multi-size network model comprises a defect positioning module and a defect type identification module.

[0019] The defect positioning module comprises a defect positioning branch, a defect edge positioning branch and an information fusion unit; the defect positioning branch takes RGB feature information of an original RGB image as input, obtains multi-resolution information through a multi-scale branch network; the defect edge positioning branch takes LAB feature information of an original RGB image converted into an LAB image as input, learns edge information through a symmetrical network structure and a feature connection operation, and obtains a defect edge mask; the edge information and the multi-resolution information flow into the information fusion unit to obtain a predicted defect position mask.

[0020] The defect type identification module sequentially inputs defect information obtained by segmenting the defect through an adaptive contour outermost rectangle extraction method into a backbone network with residual connection, an attention unit and a classifier, and outputs a predicted defect type.

[0021] Preferably, after feature extraction through the backbone network, the extracted feature information is input into the attention unit, as follows:

[0022]

[0023] wherein, d k is a scaling factor, K is a matrix obtained through a U-Net, V is an output of the backbone network, and Q is a matrix obtained through position encoding, wherein the position encoding is represented as:

[0024]

[0025]

[0026] wherein, PE (pos,2i) represents a result of position encoding for a 2i position; PE (pos,2i+1) represents a result of position encoding for a 2i+1 position; PE is a two-dimensional matrix and has the same dimension as an input vector, and is represented as a matrix after position encoding; pos is a position of the defect, i represents a dimension number, and C is a set channel number.

[0027] Preferably, the loss calculation method of the multi-task multi-size network model comprises:

[0028] calculating a defect positioning loss L label_mask The predicted defect position mask generated through the defect positioning module and the actual defect position mask are subjected to loss calculation of pixel points through a focal loss function.

[0029] calculating an edge positioning loss L label_edgeThe defect edge mask obtained through the defect edge positioning branch of the defect positioning module is compared with the actual defect edge mask to calculate the edge information loss using a cross-entropy loss function.

[0030] The defect type recognition loss L is calculated by comparing the predicted defect type obtained through the defect type recognition module with the actual defect type label using a multi-class cross-entropy loss function. label_recog The defect type recognition loss L is calculated by comparing the predicted defect type obtained through the defect type recognition module with the actual defect type label using a multi-class cross-entropy loss function.

[0031] The total loss function L is obtained as follows:

[0032] L total =μ0L label_recog +μ1L label_edge +μ2L label_mask

[0033] wherein μ0, μ1 and μ2 are the adjustment coefficients of the defect type recognition loss, the edge positioning loss and the defect positioning loss, respectively, μ0 is set to 0.5, and μ1 and μ2 are set to 0.25, respectively.

[0034] Preferably, in S103, the Dice coefficient is obtained by calculating the similarity between the predicted value and the true value of the defect positioning, as follows:

[0035]

[0036] wherein X is the predicted defect position of the model, and Y is the true defect position.

[0037] The accuracy Accuracy is the proportion of the number of correctly classified samples to the total number of samples, i.e.:

[0038]

[0039] wherein TP i represents the correctly predicted samples with "true label and true classification", FP i represents the incorrectly predicted samples with "false label but true classification"; and n represents the total number of samples.

[0040] On the other hand, an electronic component defect recognition device based on a multi-task multi-size network comprises:

[0041] A defect image dataset acquisition module is configured to acquire high-definition industrial images of a plurality of electronic components including defects as a defect image dataset.

[0042] An image labeling module is configured to determine and label the defect positions and defect types of each electronic component in the defect image dataset according to the defined standards for normal operation and defect generation of the electronic components, and divide the labeled image set into a training set and a test set.

[0043] an optimal model obtaining module, configured to train a multi-task multi-size network model using the training set, to obtain a multi-task multi-size network model with minimum loss, and to test the multi-task multi-size network model with minimum loss using the test set, to obtain an optimal multi-task multi-size network model with maximum Dice coefficient and accuracy;

[0044] a defect positioning and identification module, configured to input an image of an electronic component to be identified into the optimal multi-task multi-size network model, and to obtain a defect position and a defect type of the electronic component to be identified.

[0045] Compared with the prior art, the present application has the following beneficial effects:

[0046] The present application uses multi-scale information of a defect image of an electronic component, combines a defect position mask, a defect edge mask and a defect type label, trains and tests a multi-task multi-size network model, obtains an optimal multi-task multi-size network model, and completes defect positioning and defect type identification of an electronic component based on the optimal multi-task multi-size network model, thereby improving the efficiency and accuracy of defect identification of an electronic component. BRIEF DESCRIPTION OF DRAWINGS

[0047] Figure 1 A flowchart of a multi-task multi-size network-based electronic component defect identification method according to an embodiment of the present application is shown in the figure.

[0048] Figure 2 A flowchart of a defect image data set obtaining method according to an embodiment of the present application is shown in the figure.

[0049] Figure 3 A defect positioning network framework of an electronic component according to an embodiment of the present application is shown in the figure.

[0050] Figure 4 A defect type identification network framework of an electronic component according to an embodiment of the present application is shown in the figure.

[0051] Figure 5 A training flowchart of an optimal multi-task multi-size network model according to an embodiment of the present application is shown in the figure.

[0052] Figure 6 A structure block diagram of a multi-task multi-size network-based electronic component defect identification device according to an embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0053] The present application will be further described below in combination with the accompanying drawings and embodiments.

[0054] Referring to Figure 1 The present application is a multi-task multi-size network-based electronic component defect identification method, which comprises:

[0055] S101, collect high-definition industrial images of a plurality of electronic components including defects as defect image data sets;

[0056] S102, determine and label the defect positions and defect types of each electronic component in the defect image data set according to the defined standards for normal operation and defects of the electronic components, and divide the labeled image set into a training set and a test set;

[0057] S103, train the multi-task multi-size network model using the training set to obtain a multi-task multi-size network model with the smallest loss, and test the multi-task multi-size network model with the smallest loss using the test set to obtain an optimal multi-task multi-size network model with the largest Dice coefficient and accuracy;

[0058] S104, input the image of the electronic component to be identified into the optimal multi-task multi-size network model to obtain the defect position and defect type of the electronic component to be identified.

[0059] In this embodiment, referring to Figure 2 S101, specifically includes:

[0060] S1011, collect high-definition industrial original RGB image data of external defects of electronic components including optical, ultrasonic and infrared detection.

[0061] Specifically, a camera can be installed on the inspection equipment to collect high-definition images of a plurality of electronic components including defects, and the images can be stored in the form of RGB three channels.

[0062] S1012, based on the defined standards for normal operation and defects of the electronic components, obtain the defect types included in the electronic components.

[0063] Specifically, according to the industrial standards of the application scene of the electronic components and the current data collected by the industry, the defined standards for normal operation and defects of the key electronic components are determined, and based on the defined standards for normal operation and defects of the electronic components, the defect types included in the electronic components are obtained. According to the industrial standards of the application scene of the electronic components and the current data collected by the industry, the defined standards for normal operation and defects of the key electronic components are determined, and based on the defined standards for normal operation and defects of the electronic components, the defect types included in the electronic components are obtained. The existing technology can be used, and the specific method is not limited in this embodiment.

[0064] S1013, based on the defined standards for normal operation and defects of the electronic components, locate and label the defect positions of the electronic components to obtain a defect position mask, and store the defect position mask in a grayscale image.

[0065] Specifically, according to the defined standard of normal operation and defect of the key electronic components, the complete position of the electronic component defect is labeled from the image, the defect position mask is obtained, and the obtained defect position mask picture is stored in the form of a gray scale picture, wherein 255 is displayed as black to represent the normal operation area, and 0 is displayed as white to represent the defect area.

[0066] S1014, according to the defect position, an edge detection algorithm is used to perform edge extraction operation on the obtained defect position mask gray scale picture to obtain a defect edge mask, and the defect edge mask is stored in the form of a gray scale picture.

[0067] Specifically, according to the edge detection operation in the field of computer vision, a Canny edge detection algorithm is used to perform edge extraction operation on the obtained gray scale mask, the edge display width is set to 1 pixel point, and the extracted edge picture is stored in the form of a gray scale picture, wherein 0 is displayed as white to represent the edge position, the width is 1 pixel point, and the rest is black with a pixel value of 255. The main principle of the Canny algorithm is as follows:

[0068]

[0069]

[0070] wherein, [G x ,G y ] is the first derivative of the horizontal direction and the numerical direction calculated by using the Sobel operator on the smoothed image, and the boundary gradient and direction are found according to the value.

[0071] S1015, the defect is segmented from the original RGB image data by using an adaptive contour wrapping method, a new minimum positive rectangle is saved as a defect local image, and a corresponding defect type label is given.

[0072] Specifically, the electronic component defect area is further segmented from the original image and saved as a new rectangle area with an adaptive size, and the corresponding defect type is saved in the form of a.txt text, wherein the label 0 represents a crack, 1 represents delamination, 3 represents a scratch, 4 represents an angle defect, 5 represents oil stain, and 6 represents a string hole.

[0073] S1016, the obtained level original RGB image, defect position mask picture, defect edge mask picture and defect local image are expanded in quantity and form by data enhancement operation.

[0074] Specifically, in order to further improve the generalization ability of the data set, the obtained data set is uniformly subjected to data enhancement operation, which includes rotation, cropping, original image color conversion, scaling and noise adding operation, and the data set is further expanded in quantity and form.

[0075] Further, after preparing the complete data set by the above method, the corresponding label and image are input into the designed network, and the present application is a multi-task multi-size network model.

[0076] Referring to Figure 3 and Figure 4 As shown in the drawings, the multi-task multi-size network model comprises a defect positioning module and a defect type identification module.

[0077] The defect positioning module comprises a defect positioning branch, a defect edge positioning branch and an information fusion unit; the defect positioning branch takes the RGB feature information of the original RGB image as input, obtains multi-resolution information through a multi-scale branch network; the defect edge positioning branch takes the LAB feature information of the original RGB image converted into an LAB image as input, adopts a symmetrical network structure and a feature connection operation to learn edge information and obtain a defect edge mask; the edge information and the multi-resolution information flow into the information fusion unit to obtain a predicted defect position mask.

[0078] The defect type identification module sequentially inputs the defect information obtained by segmenting the defects by using the adaptive contour outermost rectangle extraction method into a backbone network with residual connection, an attention unit and a classifier, and outputs a predicted defect type.

[0079] The process of defect prediction by the multi-task multi-size network model is described in detail as follows.

[0080] (1) The original image in the RGB space is converted into an image in the LAB space, and the image size is unified to (3, 128, 192), both of which are input into the multi-task multi-size network model.

[0081] (2) The RGB original image and the LAB original image are respectively input into the electronic component defect positioning module, which comprises two branches, a defect positioning branch and a defect edge positioning branch, which respectively learn defect position information and edge information. Finally, the two branch information is fused to output a defect position mask.

[0082] (2.1), the defect positioning branch, taking the RGB mode image as input, extracts defect features through a multi-scale network to obtain multi-resolution information. The multi-scale network has multiple branches of different resolutions, which constantly interact information between different branches to strengthen semantic information and precise location information. The branch is divided into 4 stages, the first stage has one branch (resolution flow), the nth stage has n branches, and the branches in the same stage are fused by different form convolutions. This design adopts 4 stages, and finally outputs multi-scale information of (18, 128, 192), (36, 64, 92), (72, 32, 48) and (144, 16, 24) through feature fusion between different stages. Among them, (C, H, W) represents (channel number, height, width) respectively.

[0083] (2.2), the defect edge positioning branch, taking the LAB mode image as input, learns edge information through a symmetric convolution network. The symmetric convolution network fully retains low-level features and high-level semantic features through repeated upsampling and downsampling, and segments the edge information. The network can be regarded as an Encoder-Decoder structure, wherein the Encoder consists of 2 3x3 convolution layers and a 2x2 max pooling layer to form a down-sampling module; the Decoder repeatedly consists of 2 deconvolution layers, a feature splicing Concat operation and two 3x3 convolution layers. Through the feature splicing operation, the fusion of shallow texture information and deep semantic information is realized, and the loss in the information transmission process is reduced. The input (1, 128, 192) of the present application outputs small resolution (1, 32, 48) edge mask information through the symmetric convolution network, which assists in generating the complete defect mask position.

[0084] (3), by fusing the feature information of the defect positioning branch and the defect edge positioning branch, the multi-scale information is adjusted to a feature map with a size of (128, 192) through a convolution layer, and then input to a classifier to obtain a defect probability. Taking 0.5 as the defect probability threshold, the probability value greater than 0.5 is output as 1, and the probability value less than 0.5 is output as 0, so as to obtain the final predicted defect position mask.

[0085] (4) The obtained defect position is used to extract the smallest positive rectangle covering the contour, and the defect position on the original image corresponding to the part is cut off and input to a defect type recognition module.

[0086] (5) The defect type recognition module mainly consists of a defect recognition backbone network (which can be a convolutional neural network), an attention network and a classifier, and its process is as shown in the accompanying Figure 4

[0087] ​The defect type identification module takes the defect location mask obtained by the defect localization module as input and uses an adaptive contour encapsulation extractor to obtain the minimum regular rectangle of the electronic component defect, which is then input into the defect type identification network. Since electronic component defects vary in shape and size, the adaptive rectangle extraction operation can preserve the feature information of the defect location to the greatest extent. Meanwhile, the backbone network alleviates the gradient vanishing or exploding problems in deep networks through residual connections and combines with the original information to preserve the texture features of the underlying layers, thus achieving feature extraction of the electronic component defect type. After feature extraction by the backbone network, the extracted feature information is input into the attention unit, i.e.:

[0088]

[0089] Where, d k This represents the scaling factor, K is the matrix obtained through a small U-Net, V is the output of the backbone network, and Q is the matrix obtained after positional encoding, where the positional encoding is expressed as:

[0090]

[0091]

[0092] Among them, PE (pos,2i) This represents the result of position encoding at position 2i; PE (pos,2i+1) This represents the result of position encoding at position 2i+1; PE is a two-dimensional matrix with the same dimension as the input vector, representing the matrix after position encoding; pos is the location of the defect, i represents the number of dimensions, and C is the number of channels set.

[0093] See Figure 5 As shown, after the multi-task, multi-size network model is built, this invention uses a multi-supervised end-to-end training method to train the model and update the model parameters to minimize the loss.

[0094] For defect location masks, since the defect area of ​​electronic components usually occupies a small proportion of the entire image, it leads to an imbalance between positive and negative regions, which in turn reduces training efficiency and model performance. Therefore, in the defect localization branch, Focal Loss (FL) is used to reweight the positive and negative samples. For the defect localization mask, FL can be expressed as...

[0095]

[0096] Where, Y = [Y i,k ],P=[P i,j ];Y i,j and P i,jand defect probability at coordinate (i, j) respectively. a is a parameter to balance positive and negative samples, and g is a parameter to balance easy and hard distinguishable samples, in the experiment, a = 0.25 and g = 2 are set empirically.

[0097] For defect edge mask, binary cross-entropy loss function is adopted, that is:

[0098]

[0099] wherein, E = [E i,j ], Q = [Q i,j ]; E i,j and Q i,j represent the predicted defect edge label and defect edge probability at coordinate (i, j) respectively, and the detection edge mask is used to assist the defect positioning task.

[0100] For defect type recognition label, since the electronic component defect recognition task is a multi-classification problem, multi-classification cross-entropy loss function is adopted, that is:

[0101]

[0102] wherein, there are L label values and N samples, and the probability of the i-th sample being predicted as the k-th label value is p i,k , and the probability of the i-th sample being the k-th label value is x i,k .

[0103] The total loss function can be obtained by combining the losses obtained by each module, that is:

[0104] L total = u0L label_recog + u1L label_edge + u2L label_mask

[0105] wherein, u0, u1 and u2 are respectively the loss adjustment coefficients for defect recognition type, defect position mask and defect edge mask, since defect type recognition and defect position positioning are two main tasks, u0 is set to 0.5, u1 + u2 = 0.5, wherein u1 and u2 are respectively set to 0.25. The goal of obtaining the optimal model is to minimize the total loss L total .

[0106] Finally, the training set and the test set are divided according to the ratio of 8:2, L total is calculated on the training set, and the network parameters are updated by minimizing the loss value. After one iteration is completed, the network performance is tested on the test set, and the optimal multi-task multi-size network model is selected according to the evaluation index, and the main evaluation criteria are as follows:

[0107] The Dice coefficient is obtained by calculating the similarity of the predicted value and the true value of the defect location, as follows:

[0108]

[0109] wherein X is the model-predicted defect location, and Y is the true defect location; the network parameters of the defect location module are retained when the Dice coefficient is maximized, achieving the best defect location effect.

[0110] The accuracy Accuracy refers to the proportion of correctly classified samples in the total number of samples, that is:

[0111]

[0112] wherein TP i represents the correctly predicted samples with "true label and true classification", FP i represents the incorrectly predicted samples with "false label but true classification"; n represents the total number of samples. The network parameters of the defect type identification module are retained when the Accuracy is maximized, achieving the best defect type identification effect.

[0113] Referring to Figure 6 as an implementation of the method shown in each of the above figures, an embodiment of an electronic component defect identification device based on a multi-task multi-size network is provided. The device embodiment corresponds to the method embodiment shown in Figure 1 and can be applied to various electronic devices.

[0114] Specifically, an electronic component defect identification device based on a multi-task multi-size network comprises:

[0115] A defect image dataset acquisition module 601 is configured to acquire high-definition industrial images of a plurality of electronic components including defects as a defect image dataset.

[0116] An image labeling module 602 is configured to determine and label the defect locations and defect types of each electronic component in the defect image dataset according to the defined standards for normal operation and defect generation of the electronic components, and divide the labeled image set into a training set and a test set.

[0117] An optimal model acquisition module 603 is configured to train a multi-task multi-size network model using the training set, obtain a multi-task multi-size network model with the smallest loss, and test the multi-task multi-size network model with the smallest loss using the test set, to obtain an optimal multi-task multi-size network model with the largest Dice coefficient and accuracy Accuracy.

[0118] The defect positioning and identifying module 604 is configured to input the image of the electronic component to be identified into the optimal multi-task multi-size network model, and obtain the defect position and defect type of the electronic component to be identified.

[0119] A specific implementation of an electronic component defect identification system based on a multi-task multi-size network is the same as the electronic component defect identification method based on the multi-task multi-size network, and the embodiment will not be repeated.

[0120] The above embodiments are only for illustrating the technical concept and characteristics of the present application, and the purpose is to enable those skilled in the art to understand the content of the present application and implement it, and cannot limit the protection scope of the present application. Any equivalent changes or modifications made in accordance with the spirit and essence of the present application shall be covered within the protection scope of the present application.

Claims

1. A method for electronic component defect recognition based on a multi-task multi-size network, characterized by, The method comprises the steps of: S101, collecting high-definition industrial images of a plurality of electronic components including defects as defect image data sets; S102, determining the defect position and defect type of each electronic component in the defect image data set according to the defined standard of normal operation and defect of the electronic component, and labeling, dividing the labeled image set into a training set and a test set; S103, training the multi-task multi-size network model using the training set to obtain a multi-task multi-size network model with the smallest loss; test the multi-task multi-size network model with the smallest loss using the test set to obtain an optimal multi-task multi-size network model with the largest Dice coefficient and accuracy; S104, inputting the image of the electronic component to be identified into the optimal multi-task multi-size network model to obtain the defect position and defect type of the electronic component to be identified; The S101 specifically comprises: Collecting high-definition industrial original RGB image data of external defects of electronic components including optical, ultrasonic and infrared detection; Based on the defined standard of normal operation and defect of the electronic component, the defect type included in the electronic component is obtained; Based on the defined standard of normal operation and defect of the electronic component, the defect position of the electronic component is located and labeled, and a defect position mask is obtained, which is stored in a grayscale image; According to the defect position, the obtained defect position mask grayscale image is subjected to edge extraction operation to obtain a defect edge mask, which is stored in a grayscale image; Adaptive contour wrapping method is used to segment the defects from the original RGB image data, save the new minimum positive rectangle as a defect local image, and assign a corresponding defect type label; The obtained level original RGB image, defect position mask image, defect edge mask image and defect local image are expanded in quantity and form through data enhancement operation on the data set; The multi-task multi-size network model comprises a defect positioning module and a defect type identification module; The defect positioning module comprises a defect positioning branch, a defect edge positioning branch and an information fusion unit; the defect positioning branch takes the RGB feature information of the original RGB image as input, and obtains multi-resolution information through a multi-scale branch network; the defect edge positioning branch takes the LAB feature information of the original RGB image converted into an LAB image as input, uses a symmetric network structure and a feature connection operation to learn edge information, and obtains a defect edge mask; the edge information and the multi-resolution information flow into the information fusion unit to obtain a predicted defect position mask; The defect type identification module inputs the defect information obtained by using the adaptive contour wrapping minimum rectangle extraction method to segment the defects into a backbone network with residual connection, an attention unit and a classifier in turn, and outputs a predicted defect type. 2.The electronic component defect recognition method based on multi-task multi-size network of claim 1, wherein, After feature extraction by the backbone network, the extracted feature information is input into the attention unit as follows: where d k is a scaling factor, K is a matrix obtained by a U-Net, V is an output of a backbone network, and Q is a matrix obtained by position encoding, where the position encoding is represented as: where PE (pos,2i) represents the result of position encoding for the 2i position; PE (pos,2i+1) represents the result of position encoding for the 2i+1 position; PE is a two-dimensional matrix and has the same dimension as the input vector, represented as a matrix after position encoding; pos is the position where the defect is located, i represents the number of dimensions, and C is the number of channels set. 3.The electronic component defect recognition method based on multi-task multi-size network according to claim 2, characterized in that, The loss calculation method of the multi-task multi-size network model comprises: The calculation of the defect positioning loss L label_mask The predicted defect position mask generated by the defect positioning module and the actual defect position mask are subjected to pixel loss calculation by using a focal loss function. Compute edge localization loss L label_edge The defect edge mask obtained by the defect edge localization branch of the defect localization module is compared with the actual defect edge mask to calculate edge information loss using a cross-entropy loss function. The defect type recognition loss L is calculated label_recog The defect type recognition loss is calculated by using a multi-classification cross-entropy loss function on the predicted defect type by the defect type recognition module and the actual defect type label. Obtain the total loss function: L total = μ0L label_recog + μ1L label_edge + μ2L label_mask Wherein, μ0, μ1 and μ2 are respectively the adjustment coefficients of the defect type identification loss, the edge positioning loss and the defect positioning loss, μ0 is set to 0.5, and μ1 and μ2 are respectively set to 0.

25. 4.The electronic component defect recognition method based on multi-task multi-size network according to claim 3, characterized in that, In S103, the Dice coefficient is obtained by calculating the similarity of the predicted value and the true value of the defect positioning, as follows: Wherein, X is the defect position predicted by the model, and Y is the true defect position. The accuracy Accuracy refers to the proportion of the number of correctly classified samples to the total number of samples, that is: where TP i represents "the label is true and the classification is true" correctly predicted samples, FP i represents "the label is false but the classification is true" incorrectly predicted samples; n represents the total number of samples.

5. An electronic component defect recognition apparatus based on a multi-task multi-size network, characterized by, Based on the method of any one of claims 1-4, comprising: A defect image data set acquisition module is configured to acquire high-definition industrial images of a plurality of electronic components including defects as a defect image data set; An image labeling module is configured to determine and label the defect positions and defect types of each electronic component in the defect image data set according to the defined standards for normal operation and defect generation of the electronic components, and divide the labeled image set into a training set and a test set; An optimal model acquisition module is configured to train the multi-task multi-size network model using the training set, obtain a multi-task multi-size network model with the smallest loss, test the multi-task multi-size network model with the smallest loss using the test set, and obtain an optimal multi-task multi-size network model with the largest Dice coefficient and accuracy Accuracy; A defect positioning and identification module is configured to input the image of an electronic component to be identified into the optimal multi-task multi-size network model, and obtain the defect position and defect type of the electronic component to be identified.

Images