Chip pin bridging identification method, system and device
By using preset neural networks and classification prediction models in the SMT mounting process, the problems of low manual detection efficiency and high error rate in the prior art are solved, and efficient and accurate identification results are achieved.
Patent Information
- Application Number
- CN202411659538.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-19
- Publication Date
- 2025-03-07
AI Technical Summary
In the SMT mounting process, manual visual inspection is commonly used in the prior art to identify whether the chip pin is bridged, which has low efficiency and high error rate.
By determining the training image set and the test image set, image features are extracted using preset neural networks, and the classification prediction model is trained and tested based on these features, to achieve automatic identification of whether the chip pin is bridged.
Improves the efficiency and accuracy of chip pin bridging identification, reduces manual intervention and reduces error rate.
Smart Images
Figure CN119579980A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of image recognition, and in particular to a chip pin bridge recognition method, system and device. Background Art
[0002] In the SMT (Surface Mounted Technology) chip process, there may be solder bridges between different pins of the chip, which will cause the chip to not work properly. Therefore, it is necessary to identify whether the chip is bridged. The common method in the prior art is to use visual inspection, that is, to identify whether the chip has a bridge problem with the naked eye, but this method is inefficient and has a high error rate. Summary of the invention
[0003] The purpose of the present invention is to provide a chip pin bridging identification method, system and device, which does not require manual identification of whether an image is bridged. By calling a preset neural network to extract image features, a classification prediction model is used to classify the image as to whether it is bridged based on image features. This not only improves the classification efficiency, but also has a higher accuracy.
[0004] In order to solve the above technical problems, the present invention provides a chip pin bridge identification method, comprising:
[0005] Determine a training image set and a test image set, wherein the training image set and the test image set include chip pin bridge images and chip pin non-bridge images;
[0006] Extracting image features of each image in the training image set and the test image set using a preset neural network, and determining image labels corresponding to each of the image features; the image labels are chip pin bridge image labels or chip pin non-bridge image labels;
[0007] The classification prediction model is trained based on the image features and image labels of each image in the training image set, and the trained classification prediction model is tested based on the image features and image labels of each image in the test image set to obtain the classification test accuracy of the classification prediction model;
[0008] If the classification test accuracy is not less than the first preset accuracy, the preset neural network and the tested classification prediction model are used to classify the chip pin image to be classified.
[0009] Preferably, determining a training image set and a test image set, wherein the training image set and the test image set include chip pin bridge images and chip pin non-bridge images, comprises:
[0010] Merging a chip pin bridge image set and a chip pin non-bridge image set to generate a merged image set; the chip pin bridge image set includes each of the chip pin bridge images; the chip pin non-bridge image set includes each of the chip pin non-bridge images;
[0011] The images in the merged image set are divided into the training image set and the test image set according to a preset rule; the ratio of the number of images in the training image set to the number of images in the test image set is a preset ratio.
[0012] Preferably, after determining a training image set and a test image set, wherein the training image set and the test image set include chip pin bridge images and chip pin non-bridge images, the method further includes:
[0013] The training image set and each image in the test image set are enhanced according to the image processing requirements of the preset neural network.
[0014] Preferably, the classification prediction model is trained based on the image features and image labels of each image in the training image set, and the trained classification prediction model is tested based on the image features and image labels of each image in the test image set to obtain the classification test accuracy of the classification prediction model, including:
[0015] Inputting the image features and image labels of each image in the training image set into the classification prediction model, so that the classification prediction model learns the corresponding relationship between the image features and image labels of each image in the training image set, so as to train the classification prediction model;
[0016] The trained classification prediction model is called to classify each image in the test image set based on the image features of each image in the test image set to obtain a classification test label for each image; the classification test label includes a predicted chip pin bridge image label and a predicted chip pin no bridge image label;
[0017] The classification test accuracy of the classification prediction model is determined based on the classification test labels and image labels of each image.
[0018] Preferably, after determining the classification test accuracy of the classification prediction model based on the classification test labels and image labels of each image, the method further includes:
[0019] Determining whether the classification test accuracy is not less than the first preset accuracy;
[0020] If not, returning to the step of calling the trained classification prediction model to classify the images in the test image set based on the image features of each image in the test image set, and determining the classification label of each image;
[0021] If so, the process proceeds to the step of using the preset neural network and the tested classification prediction model to classify the chip pin images to be classified.
[0022] Preferably, the classification prediction model is trained based on the image features and image labels of each image in the training image set, and the trained classification prediction model is tested based on the image features and image labels of each image in the test image set, and after obtaining the classification test accuracy of the classification prediction model, the method further includes:
[0023] If the classification test accuracy of the classification prediction model is not less than the first preset accuracy, then determining whether the classification test accuracy of the classification prediction module is not less than a second preset accuracy, and the second preset accuracy is greater than the first preset accuracy;
[0024] If not, the feature extraction parameters of the preset neural network are adjusted, and the process returns to the step of extracting the image features of each image in the training image set and the test image set using the preset neural network, and determining the image label corresponding to each of the image features;
[0025] If so, the process proceeds to the step of using the preset neural network and the tested classification prediction model to classify the chip pin images to be classified.
[0026] Preferably, before classifying the chip pin image to be classified using the preset neural network and the tested classification prediction model, the method further includes:
[0027] If the classification test accuracy is not less than the second preset accuracy, performing transfer learning on the preset neural network and the classification prediction model;
[0028] If the classification test accuracy of the classification prediction model after transfer learning is not less than the second preset accuracy, the step of using the preset neural network and the tested classification prediction model to classify the chip pin image to be classified is entered.
[0029] Preferably, using the preset neural network and the tested classification prediction model to classify the chip pin images to be classified includes:
[0030] Obtain the chip pin image to be classified;
[0031] Extracting image features of the chip pin image to be classified using a preset neural network;
[0032] The tested classification prediction model is called to classify the chip pin image to be classified based on the image features.
[0033] In order to solve the above technical problems, the present invention also provides a chip pin bridge identification system, comprising:
[0034] An image determination unit, used to determine a training image set and a test image set, wherein the training image set and the test image set include chip pin bridge images and chip pin non-bridge images;
[0035] A feature extraction unit, used to extract image features of each image in the training image set and the test image set using a preset neural network, and determine an image label corresponding to each image feature; the image label is a chip pin bridge image label or a chip pin non-bridge image label;
[0036] A model training unit, used to train a classification prediction model based on the image features and image labels of each image in the training image set, and to test the trained classification prediction model based on the image features and image labels of each image in the test image set to obtain a classification test accuracy of the classification prediction model;
[0037] The model application unit is used to classify the chip pin image to be classified by using the preset neural network and the tested classification prediction model when the classification test accuracy is not less than a first preset accuracy.
[0038] In order to solve the above technical problems, the present invention also provides a chip pin bridge identification device, comprising:
[0039] Memory for storing computer programs;
[0040] The processor is used to implement the steps of the chip pin bridge identification method as described above when executing a computer program.
[0041] The present application provides a chip pin bridge recognition method, system and device. After determining a training image set and a test image set including chip pin bridge images and chip pin non-bridge images, a preset neural network is used to extract the image features of each image in the training image set and the test image set, and the image label corresponding to each image feature is determined. The classification prediction model is trained based on the training image set, and the trained classification prediction model is tested based on the test image set. If the classification test accuracy is not less than the first preset accuracy, the preset neural network and the classification prediction model after the test can be used to classify the chip pin image to be classified. It can be seen that in the present application, there is no need to manually identify whether the image is bridged. By calling the preset neural network to extract the features of the image, the classification prediction model is used to classify the image based on the image features. The classification efficiency is not only improved, but also the accuracy is relatively high. BRIEF DESCRIPTION OF THE DRAWINGS
[0042] In order to more clearly illustrate the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0043] Figure 1 A schematic flow chart of a chip pin bridge identification method provided by the present invention;
[0044] Figure 2 This is the first chip pin bridge schematic diagram;
[0045] Figure 3 This is the second chip pin bridge schematic diagram;
[0046] Figure 4 A schematic diagram of the structure of a chip pin bridge identification system provided by the present invention;
[0047] Figure 5 A schematic structural diagram of a chip pin bridge identification device provided by the present invention. DETAILED DESCRIPTION
[0048] The core of the present invention is to provide a chip pin bridging identification method, system and device, which does not require manual identification of whether the image is bridged. By calling a preset neural network to extract image features, a classification prediction model is used to classify the image as to whether it is bridged based on image features. This not only improves the classification efficiency, but also has a higher accuracy.
[0049] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0050] Please refer to Figure 1 , Figure 1 A schematic flow chart of a chip pin bridge identification method provided by the present invention, the method comprising:
[0051] S11: determining a training image set and a test image set, wherein the training image set and the test image set include chip pin bridge images and chip pin non-bridge images;
[0052] S12: using a preset neural network to extract image features of each image in the training image set and the test image set, and determining an image label corresponding to each image feature; the image label is a chip pin bridge image label or a chip pin non-bridge image label;
[0053] S13: training the classification prediction model based on the image features and image labels of each image in the training image set, and testing the trained classification prediction model based on the image features and image labels of each image in the test image set to obtain the classification test accuracy of the classification prediction model;
[0054] S14: If the classification test accuracy is not less than the first preset accuracy, the chip pin image to be classified is classified using a preset neural network and a tested classification prediction model.
[0055] In the prior art, in the SMT mounting process, chip pin bridging may occur, that is, different pins of the same chip are bridged together, such as Figure 2 As shown, Figure 2 This is a schematic diagram of the chip pin bridging in the prior art. This will cause the chip to be unable to output or receive data normally, and thus the chip cannot work properly. Therefore, it is necessary to detect whether the chip has a pin bridging problem after the mounting process is completed. The prior art usually uses manual inspection, that is, manually checking whether the pins of each chip are bridged with the naked eye, which is not only inefficient but also has a low accuracy rate.
[0056] Please refer to Figure 2 and Figure 3 , Figure 2 This is the first chip pin bridge schematic diagram. Figure 3 This is a schematic diagram of the second type of chip pin bridge.
[0057] In the present application, a preset neural network and a classification prediction model are used to determine whether the chip pins are bridged. Specifically, a training image set and a test image set are first determined, wherein both the training image set and the test image set include chip pin bridging images and chip pin non-bridging images. The chip pin bridging image is an image in which the chip pins are bridged after welding, that is, an image that needs to be identified and processed, while the chip pin non-bridging image is an image in which the chip pins are welded normally and there is no bridging problem. Both the training image set and the test image set include chip pin bridging images and chip pin non-bridging images, and the specific number of chip pin bridging images and chip pin non-bridging images is not limited, so as to facilitate the training of the preset neural network and the classification prediction model.
[0058] After determining the training image set and the test image set, a preset neural network extracts image features of each image in the training image set and the test image set, such as extracting image features based on the pixels of each image, and each image corresponds to an image label, such as the image label of an image with no bridging problem of chip pins is a chip pin no bridging image label, and the image label of an image with bridging problem of chip pins is a chip pin bridging image label, that is, each image has its corresponding label in advance.
[0059] The classification prediction model is trained based on the image features and image labels of each image in the training image set. Specifically, the image features and image labels of each image in the training image set can be input into the classification prediction model, so that the classification prediction model pre-learns the image features corresponding to images whose image labels are chip pin bridge image labels, and the image features corresponding to images whose image labels are chip pin non-bridge image labels, that is, learns the correspondence between image labels and image features. After the classification prediction model is trained, the trained classification prediction model is tested based on the image features and image labels of each image in the test image set. The classification prediction model that has learned the correspondence between image labels and image features classifies each image according to the image features of each image in the test image set, and divides each image into images with bridging problems and images without bridging problems. The classification test accuracy of the classification prediction model is determined based on the classification results of the classification prediction model and the image labels corresponding to each image. If the classification prediction model determines that a certain image has a bridging problem based on the image features of the image, and the image label of the image is a chip pin bridging image label, then the classification prediction model's prediction result for the image is correct. Conversely, if the classification prediction model determines that a certain image has a bridging problem based on the image features of the image, but the image label of the image is a chip pin no-bridging image label, then the classification prediction model's prediction result for the image is wrong. After all images in the test image set are classified, the accuracy of the classification test is performed based on the classification results of all images by the classification prediction model.
[0060] If the classification test accuracy of the classification prediction result reaches the first preset accuracy, the preset neural network and the classification prediction model after the test will be used to classify the chip pin image to be classified, such as taking a photo of the chip pin generated by the SMT patch process, that is, obtaining the chip pin image to be classified. At this time, it is not yet determined whether there is a bridging problem in the chip pin image to be classified. The preset neural network is required to extract features of the chip pin image to be classified, and the classification prediction model after the test is used to classify the chip pin image to be classified based on the image features of the chip pin image to be classified, that is, the chip pin image to be classified is divided into a chip pin bridging image or a chip pin non-bridging image. Only when the chip pin image to be classified is a chip pin non-bridging image, the chip corresponding to the chip pin image to be classified meets the requirements, otherwise the chip needs to be reworked or destroyed, that is, it cannot be put into use. Since the classification test accuracy of the classification prediction model after the test has reached the first preset accuracy, the classification accuracy of the chip pin image to be classified meets the first preset accuracy, and the accuracy is relatively high. Of course, the first preset accuracy can be set according to one's own needs in this application.
[0061] It should be noted that before obtaining the training image set and the test image set, the image label of each image has been determined, that is, the image label of each image is a chip pin bridge image label or a chip pin no bridge image label. Here, a preset neural network is used to extract the image features of each image in the training image set and the test image set, and determine the image label corresponding to each image feature, which means that the image features of each image are matched with the image label of the image, so as to train and test the classification prediction model.
[0062] In summary, in the present application, there is no need to manually identify whether an image is bridged. By calling a preset neural network to extract image features, the classification prediction model classifies the image as bridged or not based on image features, which not only improves the classification efficiency but also has higher accuracy.
[0063] Based on the above embodiments:
[0064] As a preferred embodiment, determining a training image set and a test image set, wherein the training image set and the test image set include chip pin bridge images and chip pin non-bridge images, includes:
[0065] The chip pin bridge image set and the chip pin non-bridge image set are merged to generate a merged image set; the chip pin bridge image set includes each chip pin bridge image; the chip pin non-bridge image set includes each chip pin non-bridge image;
[0066] The images in the merged image set are divided into a training image set and a test image set according to a preset rule; the ratio of the number of images in the training image set to the number of images in the test image set is a preset ratio.
[0067] In this embodiment, each chip pin bridge image and each chip pin non-bridge image are obtained in advance, and each image is merged into an image set, and then each image in the image set is divided into a training image set and a test image set according to a preset rule.
[0068] Among them, each image in the chip pin bridge image set and the chip pin non-bridge image set can be an image obtained from the Internet or photographed based on the produced products, and this application does not limit this.
[0069] The chip pin bridge image set and the chip pin non-bridge image set can be two different folders, the image label of each image in the chip pin bridge image set is the chip pin bridge image label, and the image label of each image in the chip pin non-bridge image set is the chip pin non-bridge image label.
[0070] When dividing the training set images and the test set images, the division can be specifically performed according to a preset ratio. For example, 70% of the images in the merged image set are divided into the training image set, and 30% of the images are divided into the test image set. The ratio of the number of images in the training image set to the number of images in the test image set is 7:3. Of course, the present application does not impose any limitation on this and it can be set according to actual needs.
[0071] Since the number of chip pin bridging images and the number of chip pin non-bridging images in the training image set and the test image set are randomly allocated, that is, the ratio between the chip pin bridging images and the chip pin non-bridging images in the training image set and the test image set is not limited, the accuracy of the classification prediction model in classifying the chip pin images to be classified after testing can be improved.
[0072] As a preferred embodiment, after determining the training image set and the test image set, wherein the training image set and the test image set include chip pin bridge images and chip pin non-bridge images, the method further includes:
[0073] The images in the training image set and the test image set are enhanced according to the image processing requirements of the preset neural network.
[0074] After determining each image in the training image set and the test image set, each image can be enhanced based on the image processing requirements when the preset neural network is used for image processing, such as horizontally flipping, randomly rotating, and randomly translating each image to keep the direction of each image consistent, or the size of each image can be adjusted. If the preset neural network can process images of size 224×224, the size of each image is adjusted to 224×224 to facilitate processing by the preset neural network.
[0075] Of course, after merging the chip pin bridge image set and the chip pin non-bridging image set to generate a merged image set, image enhancement processing may be performed, and then each image in the merged image set may be divided into a training image set and a test image set according to preset rules. This application does not limit this.
[0076] After the image enhancement process is performed, each image can be displayed so that a person can visually check whether the image enhancement effect meets the preset requirements. If not, the image enhancement process can be continued.
[0077] When calling the preset neural network, specifically, when analyzing the network architecture, the first layer, that is, the image input layer, requires an input image of size 224×224×3, where 3 is the number of color channels. Based on this, the image size can be adjusted to 224×224. When performing feature extraction, the height and width of each image in the enhanced training image set and test image set are taken. The global pooling layer at the end of the network collects the input features of all spatial positions and corresponds the image features and image labels of each image.
[0078] It should be noted that before using the preset neural network and the tested classification prediction model to classify the chip pin image to be classified, the chip pin image to be classified may also be subjected to image enhancement processing to improve the classification accuracy.
[0079] As a preferred embodiment, the classification prediction model is trained based on the image features and image labels of each image in the training image set, and the trained classification prediction model is tested based on the image features and image labels of each image in the test image set to obtain the classification test accuracy of the classification prediction model, including:
[0080] Inputting the image features and image labels of each image in the training image set into the classification prediction model, so that the classification prediction model learns the corresponding relationship between the image features and image labels of each image in the training image set, so as to train the classification prediction model;
[0081] The trained classification prediction model is called to classify each image in the test image set based on the image features of each image in the test image set to obtain a classification test label for each image; the classification test label includes a predicted chip pin bridge image label and a predicted chip pin no bridge image label;
[0082] The classification test accuracy of the classification prediction model is determined based on the classification test labels of each image and the image labels.
[0083] In this embodiment, when the classification prediction model is trained, the image features and image labels corresponding to each image in the training image set are specifically input into the classification prediction model, so that the classification prediction model learns the correspondence between the image features and image labels in the training image set, that is, the classification prediction model learns which image features correspond to the chip pin bridging image and which image features correspond to the predicted chip without bridging image. Based on this, the trained classification prediction model can classify the image features of each image in the test set according to the correspondence between the learned image features and image labels, that is, the classification test label of each image in the test image set is divided into a predicted chip pin image label and a predicted chip pin without bridging image label according to each image feature. After the classification prediction model is classified, the classification test label and image label of each image are compared. If the classification test label is consistent with the image label, the classification result of the classification test model is accurate, but if not, it is inaccurate. The classification test accuracy is determined based on the ratio of the number of accurately classified images to the total number of images in the test image set. Specifically, if there are 100 images in the test classification image, in the classification test results, there are 80 images with accurate classification and 20 images with inaccurate classification, then the classification test accuracy is 0.8. Of course, this application does not limit this, and it can be set according to actual needs.
[0084] As a preferred embodiment, after determining the classification test accuracy of the classification prediction model based on the classification test labels and image labels of each image, the method further includes:
[0085] Determining whether the classification test accuracy is not less than a first preset accuracy;
[0086] If not, return to the step of calling the trained classification prediction model to classify the images in the test image set based on the image features of each image in the test image set, and determine the classification label of each image;
[0087] If yes, proceed to the step of deploying the preset neural network and classification prediction model to the actual working conditions.
[0088] In this embodiment, if the classification test accuracy reaches the first preset accuracy, the classification test label and the preset neural network classify the chip pin image to be classified in the actual working conditions; however, if the classification test accuracy does not meet the first preset accuracy, it is necessary to iterate the test of the classification prediction model so that the classification test model continues to learn to improve the classification test accuracy of the classification test model until the classification test accuracy reaches the first preset accuracy.
[0089] If the first preset accuracy is 0.999, when the classification test accuracy is less than 0.999, the classification prediction model is retrained to iteratively learn image classification until the classification test accuracy reaches 0.999. When calculating the classification test accuracy, the ratio of the classification test labels of each image generated by the classification prediction model to the real image labels of each image can be calculated, which is the classification test accuracy.
[0090] As a preferred embodiment, the classification prediction model is trained based on the image features and image labels of each image in the training image set, and the trained classification prediction model is tested based on the image features and image labels of each image in the test image set. After obtaining the classification test accuracy of the classification prediction model, the method further includes:
[0091] If the classification test accuracy of the classification prediction model is not less than the first preset accuracy, then determining whether the classification test accuracy of the classification prediction module is not less than the second preset accuracy, and the second preset accuracy is greater than the first preset accuracy;
[0092] If not, the feature extraction parameters of the preset neural network are adjusted, and the process returns to the step of extracting the image features of each image in the training image set and the test image set using the preset neural network, and determining the image label corresponding to each image feature;
[0093] If so, the process proceeds to the step of classifying the chip pin image to be classified using the preset neural network and the tested classification prediction model.
[0094] In this embodiment, if the classification test accuracy of the classification prediction model is not less than the first preset accuracy, the preset neural network and classification prediction model are not sufficient to classify the chip pin image to be classified. Instead, the preset neural network and classification prediction model need to continue learning until the classification test accuracy meets the higher second preset accuracy before they can be deployed.
[0095] Specifically, if the classification test accuracy meets the first preset accuracy but does not meet the second preset accuracy, the feature extraction parameters of the preset neural network are adjusted, and the classification prediction model performs image classification based on the features extracted by the preset neural network after the parameter adjustment. If the classification prediction accuracy still does not meet the second preset accuracy, the feature extraction parameters of the preset neural network continue to be adjusted until the classification prediction accuracy meets the second preset accuracy. By improving the feature extraction accuracy of the preset neural network, the classification test accuracy is improved.
[0096] That is, after the classification test accuracy of the classification prediction model has reached 0.999, it is not deployed directly, but the feature extraction parameters of the preset neural network are adjusted to allow the preset neural network to continue learning image feature extraction until the classification test accuracy of the classification prediction model after image classification based on the features extracted by the preset neural network reaches a second preset accuracy, such as 0.9999, and then the classification prediction model and the preset neural network can be deployed to improve the classification accuracy of the classification prediction model and the preset neural network after deployment. Of course, the specific settings of the first preset accuracy and the second preset accuracy are not limited in this application, and can be set according to the chip pin bridging probability in actual working conditions.
[0097] As a preferred embodiment, before classifying the chip pin image to be classified using the preset neural network and the tested classification prediction model, the method further includes:
[0098] If the classification test accuracy is not less than the second preset accuracy, transfer learning is performed on the preset neural network and classification prediction model;
[0099] If the classification test accuracy of the classification prediction model after transfer learning is not less than the second preset accuracy, the step of using the preset neural network and the tested classification prediction model to classify the chip pin image to be classified is entered.
[0100] In this embodiment, considering that the images in the training image set and the test image set are all manually preset images, that is, images with relatively obvious features of bridging or no bridging found manually, in order to further improve the classification test accuracy of the classification preset model, it is also necessary to perform transfer learning on the preset neural network and the classification prediction model, so as to further improve the classification test accuracy of the preset neural network and the classification prediction model. Specifically, the welding image of two adjacent resistors can be selected for transfer learning of the preset neural network and the classification prediction model. For example, under the desired situation, the pins of the two adjacent resistors are not connected during welding, but due to welding errors, bridging may also occur between the pins of the two resistors. The welding image of the two adjacent resistors can be feature extracted by the preset neural network, so that the classification prediction model can classify the image features extracted by the preset neural network into bridged or no-bridged images. The feature extraction parameters of the preset neural network can be adjusted, or the classification process of the classification prediction model can be iterated until the classification test accuracy of the classification prediction model during transfer learning is not less than the second preset accuracy, thereby improving the classification accuracy of the preset neural network and the classification prediction model for the pin images of the chip to be classified.
[0101] Specifically, during transfer learning, first load the preset neural network after training, adjust the network structure, obtain the input size of the network, replace the last layer of the preset neural network with a layer suitable for the binary classification task, freeze the convolution layer of the pre-trained network, check whether the current layer is a fully connected layer, and if it is a fully connected layer, exit the loop. Set the weight learning rate factor to 0, create a new network architecture, train and test the preset neural network with the training image set and the test image set, store the training and test results, calculate and verify the classification test accuracy, and when the classification test accuracy is greater than 0.9999, the transfer learning is completed.
[0102] As a preferred embodiment, the chip pin images to be classified are classified using a preset neural network and a tested classification prediction model, including:
[0103] Obtain the chip pin image to be classified;
[0104] The image features of the chip pin images to be classified are extracted using a preset neural network;
[0105] The tested classification prediction model is called to classify the chip pin images to be classified based on the image features.
[0106] When using the preset neural network and classification prediction model to classify the chip pin images to be classified, the trained preset neural network and classification prediction model are deployed in the actual production environment for real-time detection and diagnosis of SMT bridging problems. Specifically, first export the preset neural network and classification prediction model that have been trained and tested, including: preset neural network, feature extractor, classification prediction model and transfer learning network architecture, etc., and save the tested preset neural network and classification prediction model to a file named "trainedResNet_SVM_Model.mat". By calling the save function and specifying the variables and file names to be saved, these models can be saved for future use. This makes it easy to load and reuse these models without retraining. When optimizing the model, call the preset neural network that has completed transfer learning, freeze some network layers, reduce calculations and reduce model complexity, load the pre-trained preset neural network, set the model to evaluation mode, use MATLAB Coder to generate optimized code C++ or Python as needed, and then test and optimize the generated code. Adjust and deploy according to the actual usage scenario, integrate into the SMT production line image inspection equipment and MES system; save and convert to Python code (call "work withPython" and other commands in JMP and MATLAB). After the classification test model in the imaging device identifies the bridge, the quality engineer confirms and saves the analysis conclusion based on the bridge image it identifies. Analyze the process based on the analysis results, formulate improvement measures, and then upload the processed information to the server.
[0107] Specifically, in the image detection equipment of the SMT production line, the image capturing device captures the image of the chip pin to be classified. After the image enhancement processing is performed on the image of the chip pin to be classified, the preset neural network extracts the image features of the image of the chip pin to be classified, so that the classification prediction model classifies the image of the chip pin to be classified based on the image features extracted by the preset neural network, that is, identifies whether there is a chip pin bridging problem and whether it is a bridging picture.
[0108] As a preferred embodiment, the preset neural network is a ResNet-18 network, and the classification prediction model is a SVM model.
[0109] SVM (Support Vector Machine) is a flexible supervised machine learning algorithm that has high accuracy in identifying images and distinguishing qualified and unqualified products. ResNet18 deep learning network has the advantages of high accuracy and low computing power in the fields of graphic classification and detection and recognition.
[0110] The present invention samples an innovative combination of ResNet-18 pre-trained model, SVM classifier and transfer learning, and this combination is more efficient, especially in the case where the electronic manufacturing and assembly industry has only a relatively small data set (picture set), but hopes to obtain a high-performance, high-precision classifier. By using a deep learning model pre-trained with a large amount of open source data / picture set to extract features, combined with the traditional machine learning classifier SVM, the performance and efficiency of classification tasks can be effectively improved. Therefore, this application not only solves the problem that the computer configuration of the automatic production line is low and there is no GPU to accelerate network training, but also improves the recognition rate of the image detection equipment, providing a timely and effective decision-making basis for the production line. By optimizing key classification information (bridging), it can better serve actual production and can be applied to similar detection and production lines in a platform-based manner.
[0111] Please refer to Figure 4 , Figure 4 A schematic diagram of the structure of a chip pin bridge identification system provided by the present invention, the system comprising:
[0112] An image determination unit 41 is used to determine a training image set and a test image set, wherein the training image set and the test image set include chip pin bridge images and chip pin non-bridge images;
[0113] The feature extraction unit 42 is used to extract the image features of each image in the training image set and the test image set by using a preset neural network, and determine the image label corresponding to each image feature; the image label is a chip pin bridge image label or a chip pin non-bridge image label;
[0114] A model training unit 43 is used to train the classification prediction model based on the image features and image labels of each image in the training image set, and to test the trained classification prediction model based on the image features and image labels of each image in the test image set to obtain the classification test accuracy of the classification prediction model;
[0115] The model application unit 44 is used to classify the chip pin image to be classified by using a preset neural network and a tested classification prediction model when the classification test accuracy is not less than a first preset accuracy.
[0116] For an introduction to the chip pin bridge identification system provided by the present invention, please refer to the above method embodiment, and the present invention will not be described in detail here.
[0117] Please refer to Figure 5 , Figure 5 A schematic diagram of the structure of a chip pin bridge identification device provided by the present invention, the device comprises:
[0118] A memory 51, used for storing computer programs;
[0119] The processor 52 is used to implement the steps of the chip pin bridge identification method as described above when executing the computer program.
[0120] For an introduction to the chip pin bridge identification device provided by the present invention, please refer to the above method embodiment, and the present invention will not be described in detail here.
[0121] It should also be noted that, in this specification, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprises", "comprising" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the statement "comprising a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element.
[0122] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A chip pin bridge identification method, characterized in that: include: Determine a training image set and a test image set, wherein the training image set and the test image set include chip pin bridge images and chip pin non-bridge images; Extracting image features of each image in the training image set and the test image set using a preset neural network, and determining image labels corresponding to each of the image features; the image labels are chip pin bridge image labels or chip pin non-bridge image labels; The classification prediction model is trained based on the image features and image labels of each image in the training image set, and the trained classification prediction model is tested based on the image features and image labels of each image in the test image set to obtain the classification test accuracy of the classification prediction model; If the classification test accuracy is not less than the first preset accuracy, the preset neural network and the tested classification prediction model are used to classify the chip pin image to be classified.
2. The chip pin bridge identification method according to claim 1, characterized in that: Determining a training image set and a test image set, wherein the training image set and the test image set include chip pin bridge images and chip pin non-bridge images, including: Merging a chip pin bridge image set and a chip pin non-bridge image set to generate a merged image set; the chip pin bridge image set includes each of the chip pin bridge images; the chip pin non-bridge image set includes each of the chip pin non-bridge images; The images in the merged image set are divided into the training image set and the test image set according to a preset rule; the ratio of the number of images in the training image set to the number of images in the test image set is a preset ratio.
3. The chip pin bridge identification method according to claim 1, characterized in that: After determining a training image set and a test image set, wherein the training image set and the test image set include chip pin bridge images and chip pin non-bridge images, the method further includes: The training image set and each image in the test image set are enhanced according to the image processing requirements of the preset neural network.
4. The chip pin bridge identification method according to claim 1, characterized in that: The classification prediction model is trained based on the image features and image labels of each image in the training image set, and the trained classification prediction model is tested based on the image features and image labels of each image in the test image set to obtain the classification test accuracy of the classification prediction model, including: Inputting the image features and image labels of each image in the training image set into the classification prediction model, so that the classification prediction model learns the corresponding relationship between the image features and image labels of each image in the training image set, so as to train the classification prediction model; The trained classification prediction model is called to classify each image in the test image set based on the image features of each image in the test image set to obtain a classification test label for each image; the classification test label includes a predicted chip pin bridge image label and a predicted chip pin no bridge image label; The classification test accuracy of the classification prediction model is determined based on the classification test labels and image labels of each image.
5. The chip pin bridge identification method according to claim 4, characterized in that: After determining the classification test accuracy of the classification prediction model based on the classification test labels and image labels of each image, the method further includes: Determining whether the classification test accuracy is not less than the first preset accuracy; If not, returning to the step of calling the trained classification prediction model to classify the images in the test image set based on the image features of each image in the test image set, and determining the classification label of each image; If so, the process proceeds to the step of using the preset neural network and the tested classification prediction model to classify the chip pin images to be classified.
6. The chip pin bridge identification method according to claim 1, characterized in that: The classification prediction model is trained based on the image features and image labels of each image in the training image set, and the trained classification prediction model is tested based on the image features and image labels of each image in the test image set. After the classification test accuracy of the classification prediction model is obtained, the method further includes: If the classification test accuracy of the classification prediction model is not less than the first preset accuracy, then determining whether the classification test accuracy of the classification prediction module is not less than a second preset accuracy, and the second preset accuracy is greater than the first preset accuracy; If not, the feature extraction parameters of the preset neural network are adjusted, and the process returns to the step of extracting the image features of each image in the training image set and the test image set using the preset neural network, and determining the image label corresponding to each of the image features; If so, the process proceeds to the step of using the preset neural network and the tested classification prediction model to classify the chip pin images to be classified.
7. The chip pin bridge identification method according to claim 6, characterized in that: Before classifying the chip pin image to be classified using the preset neural network and the tested classification prediction model, the method further includes: If the classification test accuracy is not less than the second preset accuracy, performing transfer learning on the preset neural network and the classification prediction model; If the classification test accuracy of the classification prediction model after transfer learning is not less than the second preset accuracy, the step of using the preset neural network and the tested classification prediction model to classify the chip pin image to be classified is entered.
8. The chip pin bridge identification method according to any one of claims 1 to 7, characterized in that: Using the preset neural network and the tested classification prediction model to classify the chip pin image to be classified includes: Obtain the chip pin image to be classified; Extracting image features of the chip pin image to be classified using a preset neural network; The tested classification prediction model is called to classify the chip pin image to be classified based on the image features.
9. A chip pin bridge identification system, characterized in that: include: An image determination unit, used to determine a training image set and a test image set, wherein the training image set and the test image set include chip pin bridge images and chip pin non-bridge images; A feature extraction unit, used to extract image features of each image in the training image set and the test image set using a preset neural network, and determine an image label corresponding to each image feature; the image label is a chip pin bridge image label or a chip pin non-bridge image label; A model training unit, used to train a classification prediction model based on the image features and image labels of each image in the training image set, and to test the trained classification prediction model based on the image features and image labels of each image in the test image set to obtain a classification test accuracy of the classification prediction model; The model application unit is used to classify the chip pin image to be classified by using the preset neural network and the tested classification prediction model when the classification test accuracy is not less than a first preset accuracy.
10. A chip pin bridge identification device, characterized in that: include: Memory for storing computer programs; A processor is used to implement the steps of the chip pin bridge identification method as described in any one of claims 1 to 8 when executing a computer program.