PCBA online detection platform based on AI vision detection technology

Abstract

The application discloses a kind of PCBA online detection platform based on AI vision detection technology, including PCBA intelligent AI real-time detection system, target detection algorithm model system, PCBA detection data system and detection product image storage system, a kind of PCBA online detection platform based on AI vision detection technology described in the application, based on the detection mode of AI, with the help of PCBA related design or manufacturing information realizes fast locking detection target and component key area and carries out marking, then through modern neural network learning and training obtains the dataset of detection component, finally through learning migration, new PCBA product is detected using existing learning dataset, by using the above scheme, improve detection efficiency and detection precision, enhance the generalization ability of algorithm, shorten production cycle and reduce cost, enhance the competitiveness of enterprise, and use the data of intelligent detection, can be used to construct quality database, realizes tracking feedback system.

Description

TECHNICAL FIELD

[0001] The application relates to the field of PCBA online detection, in particular to a PCBA online detection platform based on AI vision detection technology. BACKGROUND

[0002] The PCBA online detection platform is a supporting system for PCBA component detection, in the industrial production process, visual detection is a key link, and defective parts need to be accurately and quickly identified, for a long time, the step of analyzing and checking products usually needs to be recognized by the human eye, so the inspection personnel need to be systematically trained, qualified visual inspectors can comprehensively use various identification knowledge and skills, with the continuous development of artificial intelligence technology, the ability of artificial intelligence to reliably perform tasks according to programs is continuously developed. In face recognition, artificial intelligence analyzes input images, identifies faces and positions of eyes, noses and ears, and outputs a bounding box to determine the positions and sizes of these parts. However, in the detection of printed circuit board (PCBA) images in the electronic manufacturing industry, the environment is much more complex than face recognition, such as various electronic components of different packaging types, different circuit line forms, different identification marks, silk screen text, component soldering points and different color solder masks, which will affect the detection of PCBA. Fortunately, since PCBA is produced according to the designed assembly document, the above environmental factors can be easily filtered out by the document to lock the detection target. The intelligent detection device replaces manual detection and has high efficiency, high accuracy and stability. The main detection items include part missing detection, surface defect feature detection, pin packaging integrity detection, component damage detection and terminal pin detection. Non-contact measurement will not cause any damage to the observer and the observed object, thereby improving the reliability of the system, long-term stable operation, and human beings are difficult to observe the same object for a long time, but machine vision can perform measurement, analysis and identification tasks for a long time. By using the machine vision solution, a large amount of labor resources can be saved. With the continuous development of science and technology, people's requirements for the PCBA online detection platform are also getting higher and higher.

[0003] The existing PCBA online detection platform has certain disadvantages in use. Firstly, the error rate of manual visual inspection is usually 20%-30%, some of which are caused by operation errors, and some are caused by limited operation space, so it is impossible to avoid errors, which is not conducive to the use of people. Secondly, the current main AOI technology of production line applies traditional target detection algorithm, which has many defects compared with AI. The mainstream method based on sliding window region selection is not targeted and has high time complexity, which leads to low detection accuracy and weak generalization ability of traditional method, seriously affecting the universal applicability of the algorithm, and bringing certain adverse effects to the use process of people. Therefore, we propose a PCBA online detection platform based on AI visual detection technology. SUMMARY

[0004] (I) Technical problems solved

[0005] In view of the defects of the prior art, the present application provides a PCBA online detection platform based on AI visual detection technology. The AI-based detection method is introduced into the common and obvious component fault detection, greatly improving the detection accuracy and enhancing the generalization ability of the algorithm, greatly improving the detection efficiency. Based on the intelligent detection result, a quality tracking feedback system is constructed, which has higher integration, richer information, more accurate and rapid quality problem tracing, and the system integrates the results of intelligent algorithm and PCBA barcode information, which can quickly trace the quality problem, and can effectively solve the problems in the background technology.

[0006] (II) Technical solutions

[0007] To achieve the above purpose, the technical scheme adopted by the present application is: a PCBA online detection platform based on AI visual detection technology, comprising a PCBA intelligent AI real-time detection system, a target detection algorithm model system, a PCBA detection data system and a detection product image storage system, characterized by: based on the detection method of AI, with the help of PCBA related design or manufacturing data to realize rapid locking of detection target and component key area and mark it, then through modern neural network learning and training to obtain the data set of detection components, finally through learning migration to realize the detection of new PCBA products by using the existing learning data set.

[0008] By adopting the above scheme, the detection efficiency and detection accuracy are improved, the generalization ability of the algorithm is enhanced, the production cycle is shortened and the cost is reduced, the competitiveness of the enterprise is enhanced, and the data of intelligent detection can be used to construct a quality database to realize a tracking feedback system, and relevant stations in production form a closed loop continuous improvement, and the quality monitoring and continuous improvement of the production line are more automated and systematic. In addition, since intelligent detection can read the PCBA barcode and take pictures, the detection results and product photos are finally associated through the barcode ID and stored in the database, and quality problems can be quickly traced. Finally, the intelligent detection equipment is integrated with a mechanical arm to realize full detection process automation, reduce manual inspection, and avoid personnel contact with products due to unmanned operation, thereby reducing the risk of chip ESD damage on the product. The target detection algorithm model system is output after being constructed, and PCBA detection platform, according to BOM and CAD data, can automatically mark data and be applied to the following tasks: Efficient Net for jack classification; and twin network for jack and SMT small sample metric learning, wherein all models are written using PyTorch tools, NVIDIA Tesla P100 GPU is trained, and data enhancement schemes include filtering denoising, increasing contrast, grayscale, sharpening, etc.

[0009] The PCBA intelligent AI real-time detection system includes the following operation steps:

[0010] First, the target is locked by using the CAD or ODB++ data related to the design or manufacture of the PCBA, and the components are classified according to the BOM and the type and package of the components, and the area of the AI detection components and the Regional of Interest (ROI) of the components are determined, and then the components are labeled. Second, the AI learns and trains the labeled component products using modern neural networks, and saves the feature parameters of the component detection in the AI detection model. By using the data to lock the target of the PCBA and using the BOM to classify and label the target, and then through AI learning and training, the PCBA dataset is converted into a dataset of PCBA components and solder joints classified according to the type and package of the components. Finally, by using the component and solder joint dataset obtained through related detection, when a new PCBA product is detected, only the learned component detection feature parameters need to be migrated to the new PCBA detection, and the convolution layer parameters remain unchanged during training, and the fully connected layer is fine-tuned (such as the size or direction of the components, which can be achieved through fine-tuning), and through a small amount of sample learning, the precise detection effect can be achieved. This process is transfer learning, and through this process, the learned component feature parameters can be used to detect new PCBA products. From the perspective of AI detection learning, the difference between different PCBA detections is the type, size, direction of the components, and the number of components assembled.

[0011] wherein the component position and bounding box are obtained according to the CAD or ODB++ file, the type and package of the BOM components are used to classify the components, and the components are divided into the Regional of Interest (ROI) within the bounding box, including the following contents:

[0012] A. The center area of the bounding box is used to identify whether the component is missing or not, and the content of the inscription;

[0013] The inspection system needs to confirm the following elements of the component:

[0014] (a) Whether the component position installation is appropriate: no missing components or component offset;

[0015] (b) The component model inscription is correct;

[0016] B. The solder pad or solder area in the bounding box is used to detect the quality of the solder joint;

[0017] C. The adjacent solder pad or plug hole is detected around the bounding box to intercept tin bridges or short circuits;

[0018] D. Inspired by the concept of ROI, a spatial attention mechanism is used in the detection model to give more weight to the ROI;

[0019] Wherein, the element and solder joint detection model transfer learning includes the following contents:

[0020] A. Once the predefined elements are labeled based on location and package type, AI can perform detection training on any PCBA sample labeled with elements, and save the element detection parameters in the AI detection model. Based on the ability to learn about element types, there is no need to learn the entire PCBA product. Only the learned element detection parameters need to be transferred to different PCBA products. Keep the convolution layer parameters unchanged during training, fine-tune the fully connected layer. As AI model training and detection of more PCBA products, the accumulated bad data according to element type can be continuously stored. When the learning data of different element types can be transferred, any new PCBA product can be learned through a small amount of samples to achieve accurate detection results.

[0021] B. Transfer learning can make each new PCBA detected without excessive repeated training and learning. Through transfer learning function, a large amount of labeled and learned learning data can be used to detect different PCBA, which saves a lot of training and learning time. The labeled data can be associated with the element location number.

[0022] C. Effective transfer learning is to use artificial intelligence to detect similarities. From the perspective of PCBA detection standards, inspectors have certain professional skills or use PCBA industry inspection standard IPC-A-610 for detection. AI needs to perform similar detection tasks, that is, reference industry standards to detect and judge the detected object. From the perspective of AI detection learning, the difference between different PCBA detection is the element type, size, direction and the number of assembled elements.

[0023] As a preferred technical solution of the present application, in the Efficient Net for jack classification, the Efficient Net is 3*3 convolution layer + 7 Mobile Net convolution blocks, which contain ordinary 1*1 convolution, depth separable convolution, SE layer giving different weights to different channels + 1*1 convolution + pooling + full connection, and finally output the category. Thanks to BOM and CAD, the position and category of electronic components on PCBA can be easily labeled, and the training and test data set can be automatically generated. Moreover, the labeling effect is better than that of artificial labeling because the obtained image data has less redundant information. When these data are applied to train the Efficient Net, 1-2 Mobile Net convolution blocks can be reduced, and the effect of classifying various electronic components can still be achieved. The trained Efficient Net can be directly used for PCBA detection. According to the BOM, the electronic component pictures on the PCBA are input into the Efficient Net in turn, and then the component category at the position on the BOM is compared. If the category is not the component category that should appear at the position, an error is reported.

[0024] As a preferred technical solution of the present application, in the twin network for small sample metric learning of jacks and SMT components, the basic idea is: input two pictures, extract features using the same convolutional neural network, calculate the distance between the two feature vectors to determine whether the contents of the two pictures belong to the same category; the network structure is: first use a ResBlock structure, i.e. batch normalization + ReLU activation + convolution layer + maximum pooling downsampling + residual connection convolutional neural network to extract features of the two input pictures, then calculate the distance between the two feature vectors, then input the distance vector into another convolutional neural network, i.e. convolution layer + ReLU activation + maximum pooling downsampling to extract higher-dimensional features between the two input pictures, and finally enter the full connection layer to get a scalar, and then use the Sigmoid activation function to output the final result. If they are of the same category, the result is 1, and if they are of different categories, the result is 0. During training, only whether the two input electronic component pictures belong to the same category needs to be input. The twin network can be used for PCBA missing and wrong component recognition. For the detection of a certain PCBA, first, there is a "golden standard" PCBA that has been manually compared without errors. According to the BOM and CAD data, the components on the PCBA are input into the twin network as one of the inputs. Then the components at the same position on the PCBA to be detected are also input into the network. The network determines whether they belong to the same category. Such a network structure can be used for small sample learning. For a trained twin network, even if the input is an electronic component category that has not been seen during training, it can still be used to compare with the same position electronic component on the golden standard PCBA to output whether the two electronic components belong to the same category.

[0025] As a preferred technical solution of the present application, the PCBA intelligent AI real-time detection system adopts AI technology, the AI technology uses component types and packaging to create a detection model and detect components, and the AI technology detects each component, uses existing technical data to quickly obtain relevant information of the detected component, and uses these known data to predefine the position and classification of the assembled component, combines the PCBA assembly document to label the detected component, converts the PCBA data set into a PCBA component data set, and performs identification and comparison. The AI technology focuses on detecting defects that cannot be intercepted by the above test procedures.

[0026] As a preferred technical solution of the present application, the PCBA intelligent AI real-time detection system includes the following operation steps:

[0027] S1: detecting PCBA by using AI technology, the PCBA detection focuses on the bad phenomenon that cannot be covered in circuit test, and the detection is an important link in the whole test system. The value of AI detection lies in that it can detect the bad conditions that cannot be completely covered by the existing detection method;

[0028] S2: classifying the components in the bounding box according to the known component types, obtaining the bounding box by using the component position information, defining the component category by using the component type, greatly reducing the learning process of AI, not only can the size of the bounding box of the component in the PCBA image be predefined by combining the component shape and its pad in the CAD data, but also the type of the bounding box can be determined by the type and packaging of the component;

[0029] S3: predefining the component classification according to the type and packaging of the component;

[0030] S4: dividing the key attention area in the bounding box, after the bounding box and the key attention area are pre-defined according to the component type and its packaging, the component features can be quickly obtained, and unnecessary information in the picture can be removed;

[0031] S5: component and pad detection model transfer learning, through the transfer learning function, a large amount of labeled and learned learning data can be used to detect different PCBA, saving a lot of training learning time. The labeled data can be associated with the component position number, and artificial intelligence is used to detect similar work;

[0032] S6: simulating the online detection of inspectors, the online AI detection not only has reliable bad interception, but also can avoid personnel contact with products by using integrated mechanical arms, thereby reducing the risk of damage to PCBA caused by ESD or improper manual operation.

[0033] (Three) beneficial effects

[0034] Compared with the prior art, the application provides a PCBA online detection platform based on an AI vision detection technology, which has the following beneficial effects: the PCBA online detection platform based on the AI vision detection technology of the application introduces the detection mode based on AI into common and obvious component fault detection, greatly improves detection precision, enhances the generalization ability of an algorithm, greatly improves detection efficiency, constructs a quality tracking feedback system based on intelligent detection results, has higher integration, richer information, more accurate and rapid quality problem tracing, the system integrates the results of intelligent algorithms and information such as PCBA bar codes, can quickly trace quality problems, continuously and accurately measures batch detection targets on a production line in the mode of AI vision detection, greatly reduces error rates, costs and improves efficiency;

[0035] 1. PCBA intelligent detection based on an AI detection mode

[0036] The application comprises a PCBA intelligent AI real-time detection system, a target detection algorithm model system, a PCBA detection data system and a detection product image storage system, and the PCBA online detection platform based on the AI vision detection technology of the application has the detection mode based on AI, can quickly lock detection targets and component key areas and mark them by means of PCBA related design or manufacturing data, then obtains a detection component data set through modern neural network learning and training, finally realizes detection of new PCBA products by using the existing learning data set through learning migration, and through the above scheme, detection efficiency and detection precision are improved, the generalization ability of an algorithm is enhanced, the production cycle is shortened and costs are reduced, and the competitiveness of an enterprise is enhanced. Moreover, intelligent detection data can be used to construct a quality database and realize a tracking feedback system, and the tracking feedback system and relevant stations in production constitute a closed loop for continuous improvement, quality monitoring and continuous improvement of a production line are more automated and systematic, again, since intelligent detection involves reading PCBA bar codes and taking photos, finally, detection results and product photos are associated through bar code IDs and stored in a database, and quality problems can be quickly traced, finally, a mechanical arm is integrated with the intelligent detection equipment to realize full detection process automation, reduce manual inspection, and since unmanned operation is adopted, the opportunity for personnel to contact products is avoided, and the risk of chip ESD damage on products is reduced, the target detection algorithm model system outputs a PCBA detection platform after construction, according to a bill of material (BOM) and CAD data, data can be automatically marked, and is applied to the following tasks: Efficient Net for jack classification; and a twin network for jack and SMT small sample metric learning.

[0037] 2. Constructing a product database to realize a higher quality tracking feedback management system

[0038] Through the circuit board and component recognition based on intelligent technology, storage technology, the product database that can trace production process, quality tracking management is built, its basic content includes storage, find the product, component image and detection information, intelligent identification product bar code, material number information management etc., one of its keys lies in the product bar code processing and its association with product real object, whether the reserved area on the circuit board in this project is missing bar code is also the key detection problem, this project will detect whether the bar code is missing and accurately identify it in the complex environment of production line, associate it with product material number information, detection precision information etc., constitute high quality product detection information database that can be traced back, the intelligent detection system detects fast, saves labor cost, completely solves the fatigue misjudgment of human eye, and can detect 24 hours without stopping, accurately identifies the subtle defects, improves the detection efficiency, solves the problem of subtle defects, easy to miss and misjudge by naked eye, different sorting conditions can be set for different types of defects according to the work order to meet the quality requirements of different batches of products, detection data and picture information are saved in the database server, the defect detection results are summarized and analyzed, the product quality traceability is carried out in real time, the intelligent detection technology is introduced to detect the product appearance defects, which reduces the labor cost and greatly improves the detection accuracy and efficiency, brings better reputation and greater benefit to the enterprise, the artificial intelligence detection equipment has amazing effect on product appearance defect detection, compared with manual operation, it has great advantages, the recognition rate of PCBA component missing piece test reaches more than 90%, each PCBA bar code is accurately read and the related information is stored, the PCBA detection data system runs stably, the whole PCBA online detection platform has simple structure and convenient operation, and the use effect is better than that of traditional mode. BRIEF DESCRIPTION OF DRAWINGS

[0039] Fig. 1 It is a whole structure schematic diagram of the PCBA online detection platform based on AI vision detection technology.

[0040] Fig. 2 It is a structure schematic diagram of the target detection model construction module in the PCBA online detection platform based on AI vision detection technology.

[0041] Fig. 3 It is a CAD schematic diagram of PCBA product in the PCBA online detection platform based on AI vision detection technology, which covers component position information.

[0042] Fig. 4 It is an ODB++ schematic diagram of PCBA product in the PCBA online detection platform based on AI vision detection technology, which covers component position and component contour information.

[0043] Fig. 5A structure diagram of one of the divided key attention areas in a boundary box in an AI vision detection technology-based PCBA online detection platform of the present application.

[0044] Fig. 6 A structure diagram of the second divided key attention area in a boundary box in an AI vision detection technology-based PCBA online detection platform of the present application. DETAILED DESCRIPTION

[0045] The technical solutions of the present application will be described clearly and completely in combination with the drawings and specific embodiments, but those skilled in the art will understand that the following described embodiments are part of the embodiments of the present application, not all the embodiments, and are only used to illustrate the present application, and should not be regarded as limiting the scope of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application. If the specific conditions are not specified in the embodiments, the conventional conditions or the conditions recommended by the manufacturer are used. If the reagents or instruments used are not specified by the manufacturer, they are all conventional products that can be purchased on the market.

[0046] In the description of the present application, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer" and the like indicate the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the devices or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as limiting the present application. In addition, the terms "first", "second", "third" are only for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0047] In the description of the present application, it should be noted that unless otherwise explicitly specified and limited, the terms "mounting", "connection", "connection" should be understood broadly, for example, it can be fixedly connected, or it can be detachably connected, or integrally connected; it can be mechanically connected, or it can be electrically connected; it can be directly connected, or it can be indirectly connected through an intermediate medium; it can be the communication inside two elements. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0048] Example one:

[0049] As Figs. 1-6As shown, an AI vision detection technology-based PCBA online detection platform includes a PCBA intelligent AI real-time detection system, a target detection algorithm model system, a PCBA detection data system, and a detection product image storage system. The system is characterized by an AI-based detection method that uses PCBA-related design or manufacturing data to quickly lock in detection targets and key component areas and label them. Then, through modern neural network learning and training, a dataset of detection components is obtained. Finally, through learning migration, existing learning datasets are used to detect new PCBA products.

[0050] By using the above scheme, the detection efficiency and accuracy are improved, the algorithm generalization ability is enhanced, the production cycle is shortened and the cost is reduced, the competitiveness of enterprises is enhanced, and the data of intelligent detection can be used to build a quality database to realize a tracking feedback system and form a closed loop with relevant stations in production to continuously improve the quality monitoring and continuous improvement of the production line. In addition, since intelligent detection involves reading PCBA barcodes and taking photos, the detection results and product photos are associated through barcode IDs and stored in the database, and quality problems can be quickly traced. Finally, the intelligent detection equipment is integrated with a mechanical arm to realize full detection process automation and reduce manual inspection. At the same time, due to unmanned operation, the opportunity for personnel to come into contact with products is avoided, and the risk of chip ESD damage on products is reduced. The target detection algorithm model system outputs a PCBA detection platform after being built. According to BOM and CAD data, the data can be automatically labeled and applied to the following tasks: Efficient Net for jack classification; and twin network for small sample metric learning of jacks and SMT components. All models are written using the PyTorch tool and trained using the NVIDIA Tesla P100 GPU. Data enhancement schemes include filtering and denoising, increasing contrast, grayscale, and sharpening.

[0051] The PCBA intelligent AI real-time detection system includes the following operation steps:

[0052] First, the target is locked by using the CAD or ODB++ data related to the design or manufacture of the PCBA, and the components are classified according to the BOM and the type and package of the components, and the area of the AI detection components and the Regional of Interest (ROI) of the components are determined, and then the components are labeled. Second, the AI learns and trains the labeled component products using modern neural networks, and saves the feature parameters of the component detection in the AI detection model. By using the data to lock the target of the PCBA and using the BOM to classify and label the target, and then through AI learning and training, the PCBA dataset is converted into a dataset of PCBA components and solder joints classified according to the type and package of the components. Finally, by using the component and solder joint dataset obtained through related detection, when a new PCBA product is detected, only the learned component detection feature parameters need to be migrated to the new PCBA detection, and the convolution layer parameters remain unchanged during training, and the fully connected layer is fine-tuned (such as the size or direction of the components, which can be achieved through fine-tuning), and through a small amount of sample learning, the precise detection effect can be achieved. This process is transfer learning, and through this process, the learned component feature parameters can be used to detect new PCBA products. From the perspective of AI detection learning, the difference between different PCBA detections is the type, size, direction of the components, and the number of components assembled.

[0053] wherein the component position and bounding box are obtained according to the CAD or ODB++ file, the components are classified according to the type and package of the components in the BOM, and the components are divided into the Regional of Interest (ROI) within the bounding box, including the following contents:

[0054] A. The center area of the bounding box is used to identify whether the component is missing or not, and the content of the inscription;

[0055] The inspection system needs to confirm the following elements of the component:

[0056] (a) Whether the component position installation is appropriate: no missing components or component offset;

[0057] (b) The component model inscription is correct;

[0058] B. The solder pad or solder area in the bounding box is used to detect the quality of the solder joint;

[0059] C. The adjacent solder pad or plug hole is detected around the bounding box to intercept tin bridges or short circuits;

[0060] D. Inspired by the concept of ROI, a spatial attention mechanism is used in the detection model to give more weight to the ROI;

[0061] Wherein, the element and solder joint detection model transfer learning includes the following contents:

[0062] A. Once the predefined elements are labeled based on location and package type, AI can perform detection training on any PCBA sample labeled with elements, and save the element detection parameters in the AI detection model. Based on the ability to learn element types, there is no need to learn the entire PCBA product. Only the learned element detection parameters are transferred to different PCBA products. The convolution layer parameters remain unchanged during training, and the fully connected layer is fine-tuned. As the AI model trains and detects more PCBA products, it can continuously store the accumulated bad data according to the element type. When the learning data of different element types can be transferred, any new PCBA product can be learned through a small amount of samples to achieve accurate detection results.

[0063] B. Transfer learning can make each new PCBA detected without excessive repeated training and learning. Through transfer learning, a large amount of labeled and learned learning data can be used to detect different PCBA, which saves a lot of training and learning time. The labeled data can be associated with the element location number.

[0064] C. Effective transfer learning is to use artificial intelligence to detect similarities. From the perspective of PCBA detection standards, inspectors have certain professional skills or use PCBA industry inspection standard IPC-A-610 for detection. AI needs to perform similar detection tasks, that is, reference industry standards to detect and determine the detected object. From the perspective of AI detection learning, the difference between different PCBA detection is the type, size, direction and number of assembled elements.

[0065] Further, the PCBA intelligent AI real-time detection system uses AI technology to create a detection model using element types and packaging and detect elements. AI technology detects each element, quickly obtains relevant information about the detected element using existing technical materials, and uses this known data to predefine the location and classification of assembled elements. Combined with the PCBA assembly document, the detected elements are labeled, the PCBA data set is converted into a PCBA element data set, and recognition and comparison are performed. AI technology focuses on detecting defects that cannot be intercepted by the above test procedures.

[0066] Further, the PCBA intelligent AI real-time detection system includes the following operation steps:

[0067] S1: Use AI technology to detect PCBA, PCBA detection focuses on the bad phenomenon that cannot be covered in circuit test, detection is an important link in the whole test system, the value of AI detection is that it can detect the bad condition that the existing detection method cannot completely cover;

[0068] 1. PCBA lacks bypass capacitor, this defect is usually found by visual inspection with template;

[0069] 2. Lack of components, which can be found during ICT or visual inspection;

[0070] 3. Damaged components that cannot be intercepted by testing can be found by visual inspection;

[0071] 4. ICT / FCT test cannot intercept polarity error capacitor components, which can be found by visual inspection;

[0072] 5. If the passive components with incorrect package size are assembled, they can be found by visual inspection;

[0073] 6. Component assembly offset defects that cannot be intercepted during testing can be found by visual inspection;

[0074] 7. Soldering process, solder residue, solder bridge and other defects can be found by visual inspection;

[0075] 8. To reduce the risk of ESD caused by human contact during detection, integrate mechanical arm to assist detection process and reduce the risk.

[0076] S2: Classify the components in the bounding box according to the known component type, use component position information to obtain the bounding box, use component type to define component category, greatly reduce AI learning process, not only can combine component shape and its pads in CAD data to predefine the size of the component in PCBA image, but also can determine the type of bounding box through the type and package of component;

[0077] S3: Predefine component classification according to component type and package;

[0078] 1. Capacitors are classified according to package as follows

[0079] a. Through hole / jack type

[0080] i. Electrolytic / electrolytic capacitor

[0081] ii. Ceramic / ceramic capacitor

[0082] b. SMT / patch type

[0083] i.Aluminum capacitors

[0084] ii. Tantalum capacitors

[0085] iii.Chip type per package size such as 0201,0402…

[0086] Chip capacitors, if classified by package size, include 0201, 0402…

[0087] 2. Resistors are classified according to their package as follows:

[0088] a. Through hole / socket type

[0089] i. Through hole / socket type

[0090] ii. SIP / Single In-line Package

[0091] b. SMT / Surface Mount Technology

[0092] i. Chip type per package size, such as 0201, 0402… (This refers to chip resistors, categorized by package size as 0201, 0402…)

[0093] 3. Inductors are classified according to their packaging as follows:

[0094] a. Through hole / socket type

[0095] b. SMT / Surface Mount Technology

[0096] i. Chip type per package size, such as 0603, 0402… (e.g., chip inductors, if categorized by package size, there are 0603, 0402…)

[0097] ii. Power / Power Inductor

[0098] 4. Diodes are classified according to their packaging as follows:

[0099] a. Through hole / socket type

[0100] b. SMT / Surface Mount Technology

[0101] i.SOT23 / Small Outline Transistor

[0102] ii. SOD / Small Shape Diode

[0103] 5. Transistors are classified according to their packaging as follows:

[0104] a. Through hole / Jack type

[0105] b. SMT / SMD type

[0106] i. SOT23 / Small Outline Transistor

[0107] ii. SOT4 / Small Outline Transistor

[0108] 6. Connectors are classified as follows according to package

[0109] a. Through hole / Jack type

[0110] b. SMT / SMD type

[0111] 7. Chips are classified as follows according to package

[0112] a. Through hole / Jack type

[0113] i. DIP / Dual In-line Package

[0114] b. SMT / SMD type

[0115] i. SOIC / Small Outline Integrated Circuit

[0116] ii. SOP / Small Outline Package

[0117] iii. QFP / Quad Flat Package

[0118] BGA / Ball Grid Array Package.

[0119] S4: Divide the key attention area in the bounding box, after the bounding box and key attention area are predefined according to the element type and its package, the element features can be quickly obtained, and unnecessary information in the picture can be removed;

[0120] S5: Element and solder joint detection model transfer learning, through the transfer learning function, a large amount of labeled and learned learning data can be used to detect different PCBAs, saving a lot of training and learning time. The labeled data can be associated with the element position number, and artificial intelligence is used to detect the similarity;

[0121] S6: Simulate the online detection of inspectors, online AI detection not only has reliable bad blocking, but also avoids personnel contact with products by using integrated mechanical arms, thereby reducing the risk of damage to PCBA caused by ESD or improper manual operation.

[0122] Example Two:

[0123] On the basis of example one, such as Figs. 1-6As shown, an AI vision detection technology-based PCBA online detection platform includes a PCBA intelligent AI real-time detection system, a target detection algorithm model system, a PCBA detection data system, and a detection product image storage system. The system is characterized by an AI-based detection method that uses PCBA-related design or manufacturing data to quickly lock in detection targets and key component areas and label them. Then, through modern neural network learning and training, a dataset of detection components is obtained. Finally, through learning migration, existing learning datasets are used to detect new PCBA products.

[0124] By using the above scheme, the detection efficiency and accuracy are improved, the algorithm generalization ability is enhanced, the production cycle is shortened and the cost is reduced, the competitiveness of the enterprise is enhanced, and the data of intelligent detection can be used to build a quality database to realize a tracking feedback system and form a closed loop with relevant stations in production for continuous improvement. The quality monitoring and continuous improvement of the production line are more automated and systematic. Again, during intelligent detection, the PCBA barcode is read and photographed, and finally the detection results and product photos are associated through the barcode ID and stored in the database. Quality problems can be quickly traced. Finally, the intelligent detection equipment is integrated with a mechanical arm to realize full detection process automation and reduce manual inspection. At the same time, due to unmanned operation, the opportunity for personnel to come into contact with the product is avoided, and the risk of chip ESD damage on the product is reduced. The target detection algorithm model system outputs the PCBA detection platform after being built. According to the BOM and CAD data, the data can be automatically labeled and applied to the following tasks: Efficient Net for jack classification; and twin network for small sample metric learning of jacks and SMT components. All models are written using the PyTorch tool and trained using the NVIDIA Tesla P100 GPU. Data enhancement schemes include filtering and denoising, increasing contrast, grayscale, and sharpening.

[0125] The PCBA intelligent AI real-time detection system includes the following operation steps:

[0126] First, the target is locked by using the CAD or ODB++ data related to the design or manufacture of the PCBA, and the components are classified according to the BOM and the type and package of the components, and the area of the AI detection components and the Regional of Interest (ROI) of the components are determined, and then the components are labeled. Second, the AI learns and trains the labeled component products using modern neural networks, and saves the feature parameters of the component detection in the AI detection model. By using the data to lock the target of the PCBA and using the BOM to classify and label the target, and then through AI learning and training, the PCBA dataset is converted into a dataset of PCBA components and solder joints classified according to the type and package of the components. Finally, by using the component and solder joint dataset obtained through related detection, when a new PCBA product is detected, only the learned component detection feature parameters need to be migrated to the new PCBA detection, and the convolution layer parameters remain unchanged during training, and the fully connected layer is fine-tuned (such as the size or direction of the components, which can be achieved through fine-tuning), and through a small amount of sample learning, the precise detection effect can be achieved. This process is transfer learning, and through this process, the learned component feature parameters can be used to detect new PCBA products. From the perspective of AI detection learning, the difference between different PCBA detections is the type, size, direction of the components, and the number of components assembled.

[0127] wherein the component position and bounding box are obtained according to the CAD or ODB++ file, the components are classified according to the type and package of the components in the BOM, and the components are divided into the Regional of Interest (ROI) within the bounding box, including the following contents:

[0128] A. The center area of the bounding box is used to identify whether the component is missing or not, and the content of the inscription;

[0129] The inspection system needs to confirm the following elements of the component:

[0130] (a) Whether the component position installation is appropriate: no missing components or component offset;

[0131] (b) The component model inscription is correct;

[0132] B. The solder pad or solder area in the bounding box is used to detect the quality of the solder joint;

[0133] C. The adjacent solder pad or plug hole is detected in the periphery of the bounding box to intercept tin bridges or short circuits;

[0134] D. Inspired by the concept of ROI, a spatial attention mechanism is adopted in the detection model to give more weight to the ROI;

[0135] Wherein, the element and solder joint detection model transfer learning includes the following contents:

[0136] A. Once the predefined elements are labeled based on location and package type, AI can perform detection training on any PCBA sample labeled with elements, and save the element detection parameters in the AI detection model. Based on the ability to learn about element types, there is no need to learn the entire PCBA product. Only the learned element detection parameters need to be transferred to different PCBA products. Keep the convolution layer parameters unchanged during training, fine-tune the fully connected layer. As AI model training and detection of more PCBA products, the accumulated bad data according to element type can be continuously stored. When the learning data of different element types can be transferred, any new PCBA product can be learned through a small amount of samples to achieve accurate detection results.

[0137] B. Transfer learning can make each new PCBA detected without excessive repeated training and learning. Through transfer learning function, a large amount of labeled and learned learning data can be used to detect different PCBA, which saves a lot of training and learning time. The labeled data can be associated with the element location number.

[0138] C. Effective transfer learning is to use artificial intelligence to detect similarities. From the perspective of PCBA detection standards, inspectors have certain professional skills or use PCBA industry inspection standard IPC-A-610 for detection. AI needs to perform similar detection tasks, that is, reference industry standards to detect and judge the detected object. From the perspective of AI detection learning, the difference between different PCBA detection is the element type, size, direction and the number of assembled elements.

[0139] Further, in the Efficient Net for jack classification, the Efficient net is 3*3 convolution layer + 7 Mobile Net convolution blocks, which contain ordinary 1*1 convolution, depth separable convolution, SE layer giving different weights to different channels + 1*1 convolution + pooling + full connection, and finally output the category. Thanks to BOM and CAD, the position and category of electronic components on PCBA can be easily labeled, and the training and test data set can be automatically generated. Moreover, the labeling effect is better than artificial because the obtained image data has less redundant information. Applying these data to train the Efficient Net can reduce 1-2 MobileNet convolution blocks, and still achieve the effect of classifying various electronic components. The trained Efficient Net can be directly used for PCBA detection. According to the BOM, the electronic component pictures on the PCBA are input into the Efficient Net in turn, and then the component categories at the same position on the BOM are compared. If the category is not the component category that should appear at that position, an error is reported.

[0140] Further, in the twin network for small sample metric learning of jacks and SMT parts, the basic idea is: input two pictures, extract features with the same convolutional neural network, calculate the distance between the two feature vectors to determine whether the content of the two pictures belongs to the same category; Network structure: first use a ResBlock structure, that is, batch normalization + ReLU activation + convolution layer + maximum pooling downsampling + residual connection convolutional neural network to extract the features of the two input pictures, then calculate the distance between the two feature vectors, then input the distance vector into another convolutional neural network, that is, convolution layer + ReLU activation + maximum pooling downsampling to extract higher-dimensional features between the two input pictures, and finally enter the full connection layer to get a scalar, then use the Sigmoid activation function to output the final result, if they are of the same category, it is 1, and if they are of different categories, it is 0. During training, only the input of whether the two electronic component pictures belong to the same category is needed. The twin network can be used for PCBA missing and wrong part recognition. For the detection of a certain PCBA, first, there is a "golden standard" PCBA that has been manually compared without errors. According to the BOM and CAD data, the components on the PCBA are input into the twin network as one of the inputs, and then the components at the same position on the PCBA to be detected are also input into the network. The network determines whether they belong to the same category. Such a network structure can be used for small sample learning. For a trained twin network, even if the input is an electronic component category that has not been seen during training, it can still be used to compare with the same position electronic component on the golden standard PCBA to output whether the two electronic components belong to the same category.

[0141] Working principle: including PCBA intelligent AI real-time detection system, target detection algorithm model system, PCBA detection data system and detection product image storage system, the PCBA online detection platform based on AI vision detection technology, the detection mode based on AI, with the help of PCBA related design or manufacturing information realizes fast locking detection target and component key area and carries out marking, then through modern neural network learning and training obtains the data set of detection component, finally through learning migration realizes using the existing learning data set to detect new PCBA product, through adopting the above scheme, improve the detection efficiency and detection precision, enhance the generalization ability of algorithm, shorten the production cycle and reduce the cost, enhance the competitiveness of enterprises. Moreover, the data of intelligent detection can be used to build a quality database to realize a tracking feedback system and form a closed loop with the relevant stations in production to continuously improve the quality monitoring and continuous improvement of the production line are more automated and systematic. Again, since intelligent detection, PCBA bar code is read and photographed, finally the detection results and product photos are associated through the bar code ID and stored in the database, and quality problems can be quickly traced. Finally, the intelligent detection equipment is integrated with a mechanical arm to realize full detection process automation, reduce manual inspection, and since unmanned operation, the opportunity of personnel contacting products is avoided, and the risk of chip ESD damage on products is reduced. The target detection algorithm model system outputs the PCBA detection platform after being built, according to the bill of material (BOM) and CAD data, the data can be automatically labeled and applied to the following tasks: Efficient Net for jack classification; twin network for small sample metric learning of jack and SMT parts; and whether the subsequent expected results can be achieved. In image-based data analysis, the quality of the image directly affects the design and accuracy of the recognition algorithm, so data preprocessing is required before image analysis. The main purpose of image preprocessing is to eliminate irrelevant information in the image, recover useful true information, enhance the detectability of relevant information, maximize data simplification, and thus improve the reliability of feature extraction, image segmentation, matching and recognition. Image filtering is to suppress the noise of the target image while preserving the details of the image. In the image, the high-frequency part represents the edge information of the image, that is, the sharpness; the low-frequency part is just the opposite, representing the gray level change information of the image, that is, the content. Filtering is to filter out part of the noise wave through the filter operator to highlight the detail information. When processing color images, the RGB three channels need to be processed in turn, which is time-consuming. To improve the processing speed of the entire application system, the amount of data to be processed needs to be reduced, and grayscale images can effectively reduce the data operation amount. The obtained grayscale image is binarized. The purpose is to classify the PCBA image by background to prepare for subsequent electronic component recognition.Image sharpening and image smoothing are opposite operations. Sharpening reduces the blur in an image by enhancing high-frequency components, enhancing image detail edges and contours, enhancing gray scale contrast, facilitating later identification and processing of the target. Sharpening processing increases the noise of the image while enhancing the edges of the image.

[0142] Based on the project requirements, a deep learning-based method is developed and used for real-time quality intelligent detection of industrial products on the production line. The dataset used in the experimental project is the PCBA image on the production line. In the actual PCBA production process, a large number of component images are collected, and the images are intelligently detected and classified to establish the dataset. With the dataset as input and the PCBA packaging intelligent detection category as output, a computer vision target detection model is established to realize intelligent detection type detection and recognition and corresponding labeling in the PCBA image. The pre-trained model is saved on the factory production line PC, and the model prediction interface is connected to the image detection button in the interface. The user first clicks the image input button to read the latest PCBA image and display it in the center of the interface. Click the image detection button to perform intelligent detection, cut out the barcode, and identify the corresponding string of the barcode. Network and database methods can be used to quickly process the string. Barcode recognition should consider the characteristics of the barcode. The project targets the relative vertical position of the barcode in the picture without various inclinations. First, perform morphological gradient operation, retain X direction features, and remove Y direction interference. Image blurring is performed to facilitate image connection in the later stage. Image thresholding is performed to accelerate algorithm processing and reasonably use the effect of morphological blurring. Close operation inflation and corrosion are performed to further determine the location of the barcode. The contour of the barcode is found, the maximum area of the contour is calculated, the contour rectangle is fitted, and the final result is obtained. The fitted rectangle is used as the recognition boundary of the barcode for image cutting, and the cut barcode is transmitted to the server or network to read the corresponding string. The string is saved on the PC or server and displayed in the integrated interface. Further, the string information is associated with the detection system related information. Then, the information contained in the PCBA is stored in the PC or remote server. It mainly includes PCBA component information, PCBA barcode storage information, and information such as whether to return to the factory or repair. This database can help the project quickly trace quality problems and provide more accurate and relevant information to improve the PCBA maintenance mechanism. The above detection functions are integrated into the software system. The main functional modules of the designed PCBA image detection process system include image input, image detection, detection result display, and manual confirmation module. The main interface layout from left to right displays the intelligent detection status of ten PCBA boards in a tree menu on the left side; the PCBA is displayed in the center of the interface; the intelligent detection position information, intelligent detection type, and material information are displayed on the right side. The manual confirmation button and the detail display window are integrated in the lower right. The image input button, image detection button, and other menu bars are integrated with the top of the interface. Further, the PCBA intelligent detection, PCBA result display, and PCBA barcode information reading are integrated into a unified integrated system. The main functions of the system include user account login, PCBA intelligent detection, and PCBA detection information statistics.The user login function is set up, and different user permissions are given to different accounts. After the detection process is completed, the detection result storage and statistics are performed. During the implementation of the project, the system platform construction will be further adjusted and improved according to the implementation of specific functions, debugging results and on-site operation. AI focuses on the detection of PCBA, which is aimed at the bad phenomenon that cannot be covered in circuit testing. According to the known component type, the components in the bounding box are classified, and the component classification is predefined according to the type and packaging of the components. After the predefined components are labeled based on location and packaging type, AI can detect and train any labeled PCBA sample, and save the component detection parameters in the AI detection model. Based on the ability to learn component types, there is no need to learn the entire PCBA product, and only the learned component detection parameters need to be transferred to different PCBA products. We only need to keep the convolution layer parameters unchanged during training, and fine-tune the fully connected layer. With more PCBA products trained and detected by AI model, the accumulated bad data according to component type can be continuously stored. When the learning data of different component types can be transferred, any new PCBA product can achieve accurate detection effect through a small amount of sample learning. Transfer learning can make each new PCBA be detected without repeated training and learning. Through transfer learning, a large amount of labeled and learned learning data can be used to detect different PCBA, which can save a lot of training and learning time. The labeled data can be associated with the component position number. Effective transfer learning is to use artificial intelligence to detect similarities. From the perspective of PCBA detection standard, inspectors have certain professional skills or use PCBA industry inspection standard IPC-A-610 for detection, and AI needs to perform similar detection tasks, that is, reference industry standards to detect and judge the detected object. From the perspective of AI detection learning, the difference between different PCBA detection is the size of PCBA and the number of components assembled. Online AI detection not only has reliable bad blocking, but also avoids personnel contact with products by using integrated mechanical arm, thereby reducing the risk of damage to PCBA caused by ESD or improper manual operation. In the manufacturing process of SigmaTron company, artificial intelligence is used for visual detection, which integrates machine vision and AI deep learning functions. In the manufacturing environment, AI can quickly obtain relevant information of detection components based on existing technical materials, such as component position and classification. Once the components of the PCBA board are identified, the feature parameters of the components can be learned and trained according to the component type, and transferred to new PCBA boards, which can reduce the number of learning samples and speed up the learning process. Therefore, through the above way, the existing AI technology can be improved and applied to PCBA online detection. The function of AI detection model is not to detect the entire PCBA, but to detect each component.AI is to create detection model and detect components using component type and package, such as missing parts, displacement, component inscriptions, component solder joints or short circuits and other component defects.

[0143] According to BOM and CAD data, we can automatically annotate the data and apply it to the following tasks: (all models are written using PyTorch tools, NVIDIA Tesla P100 GPU training. Data enhancement schemes include filtering and denoising, increasing contrast, grayscale, sharpening, etc.)

[0144] 1. Efficient Net for jack classification:

[0145] Efficient Net: 3*3 convolution layer + 7 Mobile Net convolution blocks (including ordinary 1*1 convolution, depth separable convolution, SE layer gives different channels different weights) + 1*1 convolution + pooling + full connection, finally output class.

[0146] Thanks to BOM and CAD, we can easily annotate the location and category of electronic components on PCBA, automatically generate training and testing data sets, and the annotation effect is better than artificial because the image data has less redundant information. Apply these data to train Efficient Net, which can reduce 1-2 Mobile Net convolution blocks and still achieve the effect of classifying various electronic components.

[0147] The trained Efficient Net can be directly used for PCBA detection. According to the BOM, input the electronic component pictures on the PCBA into the Efficient Net one by one, and then compare the component category at that position on the BOM. If the category is not the component category that should appear at that position, an error will be reported.

[0148] 2. Twin network for small sample metric learning of jack and SMT:

[0149] Basic idea: input two pictures, use the same convolutional neural network (share weights) to extract features respectively, calculate the distance between the two feature vectors to determine whether the content of the two pictures belongs to the same category.

[0150] Network structure: first use a ResBlock structure (batch normalization + ReLU activation + convolution layer + maximum pooling down sampling + residual connection) of convolutional neural network to extract the features of two input pictures, then calculate the distance between the two feature vectors, and then input the distance vector into another convolutional neural network (convolution layer + ReLU activation + maximum pooling down sampling) to extract higher-dimensional features between the two input pictures, and finally enter the fully connected layer to get a scalar, and then use the Sigmoid activation function to output the final result, if it is the same class, it is 1, and if it is different, it is 0.

[0151] For us, we only need to input two electronic component pictures during training, and input whether the two components belong to the same category.

[0152] In our production environment, the twin network can be used for PCBA missing and wrong part recognition. For the detection of a certain PCBA, first there is a "golden standard" PCBA compared by artificial, according to BOM and CAD data, the components on the PCBA are taken as one of the inputs of the twin network, and then the same position components of the PCBA to be detected are also input into the network, and the network judges whether they belong to the same category.

[0153] Such network structure can be used for small sample learning. For a trained twin network, even if the input is an electronic component category that has not been seen during training, it can still be used to compare with the same position electronic components on the golden standard PCBA to output whether the two electronic components belong to the same category.

[0154] It should be noted that in this article, relational terms such as first and second (one and two) are used only to distinguish one entity or action from another entity or action, without necessarily requiring or implying any such actual relationship or order between such entities or actions. Moreover, the terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or equipment including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or equipment. Without more limitations, the element defined by the statement "including a" does not exclude the presence of other identical elements in the process, method, article or equipment including the element.

[0155] The above shows and describes the basic principles and main features of the present application and the advantages of the present application. Those skilled in the art should understand that the present application is not limited to the above-mentioned embodiments, and the above-mentioned embodiments and descriptions in the specification are only to illustrate the principles of the present application. Without departing from the spirit and scope of the present application, various changes and improvements can be made to the present application, and these changes and improvements all fall within the scope of the claimed present application.

Claims

1. A PCBA online inspection platform based on AI visual inspection technology, comprising a PCBA intelligent AI real-time inspection system, a target detection algorithm model system, a PCBA inspection data system, and an inspection product image storage system, characterized in that: AI-based detection methods use PCBA-related design or manufacturing data to quickly locate and label key areas of the detection targets and components. Then, a dataset of the detected components is obtained through modern neural network learning and training. Finally, through learning transfer, the existing learning dataset is used to detect new PCBA products. After the target detection algorithm model system is built, it outputs a PCBA detection platform. Based on the BOM and CAD data, the data is automatically labeled and applied to the following tasks: Efficient Net for through-part classification; Siamese network for few-shot metric learning; YOLO V5 for weld point defect detection. All models are written using PyTorch and trained on an NVIDIA Tesla P100 GPU. Data augmentation schemes include filtering and denoising, increasing contrast, grayscale conversion, and sharpening. The PCBA intelligent AI real-time detection system includes the following operation steps: First, using PCBA-related design or manufacturing data, and further processing PCBA CAD or ODB++ data, the inspection target is located. Based on the BOM, components are categorized according to component type and package, and the areas for AI inspection of components and the regions of interest (ROIs) are determined. The system first identifies the ROI (Region of Interest) and then labels it. Next, the AI ​​performs modern neural network learning and training on the labeled components, storing the component detection feature parameters within the AI ​​detection model. This process utilizes data to target PCBAs, and uses BOM (Bill of Materials) to classify and label the targets. After further AI learning and training, the PCBA dataset is transformed into a dataset of PCBA components and solder joints categorized by component type and package. Finally, using the component and solder joint dataset obtained through relevant detection, when detecting new PCBA products, the learned component detection feature parameters are simply transferred to the new PCBA detection. During training, the convolutional layer parameters remain unchanged, and the fully connected layers are fine-tuned. This process is called transfer learning. Through this process, the learned component feature parameters are used to detect new PCBA products. From the perspective of AI detection learning, the differences between different PCBA detection methods lie in the component type, size, orientation, and the number of assembled components. This process involves obtaining component locations and bounding boxes from CAD or ODB++ files, classifying components by their BOM type and package, and defining Regions of Interest (ROIs) within the bounding boxes for each component. This includes the following: A. The central area of ​​the bounding box is used to identify whether a component is missing, as well as the content of the inscription; The inspection system needs to verify the following elements of the component: (a) Whether the component positions are properly installed: no missing parts or misaligned components; (b) The component model inscription is correct; B. The pads or soldering areas within the bounding box are used to inspect the quality of the solder joints; C. Detect adjacent pads or through holes around the boundary frame to intercept solder bridges or short circuits; D. Inspired by the concept of ROI, spatial attention mechanism is used in the detection model to give more weight to ROI; The transfer learning of component and solder joint detection models includes the following: A. Once predefined components are labeled based on location and package type, the AI ​​trains to detect any PCBA sample with labeled components and stores the detection parameters of these components in the AI ​​detection model. Based on the ability to learn component types, it is not necessary to learn the entire PCBA product. It is only necessary to transfer the learned component detection parameters to different PCBA products. During training, the convolutional layer parameters are kept unchanged, and the fully connected layers are fine-tuned. As the AI ​​model trains and detects more PCBA products, it continuously stores the accumulated defect data according to component type. When the learning data of different component types is transferred, any new PCBA product can learn through a small number of samples. B. Transfer learning eliminates the need for excessive repetitive training when each new PCBA is inspected. Through the transfer learning function, a large amount of labeled and learned data is used to inspect different PCBAs. The labeled data is associated with the component location number. C. Effective transfer learning involves using artificial intelligence to detect similar tasks. In terms of PCBA testing standards, inspectors have certain professional skills or use the PCBA industry testing standard IPC-A-610 for testing. AI only needs to perform similar testing tasks, that is, refer to industry standards to test and judge the object being tested. From the perspective of AI testing learning, the difference between different PCBA tests is the type, size, orientation and number of assembled components.

2. The PCBA online inspection platform based on AI visual inspection technology according to claim 1, characterized in that: The EfficientNet used for component classification consists of a 3x3 convolutional layer and 7 MobileNet convolutional blocks. These include regular 1x1 convolutions, depthwise separable convolutions, SE layers with different weights for different channels, 1x1 convolutions, pooling, and fully connected layers. The final output is the category. Thanks to BOM and CAD, the location and category of electronic components on the PCBA can be easily labeled, automatically generating training and testing datasets. The labeling effect is better than manual labeling because the obtained image data has less redundant information. Applying this data to train the EfficientNet, reducing the number of MobileNet convolutional blocks by 1-2, still achieves the effect of classifying various electronic components. The trained EfficientNet can be directly used for PCBA detection. Based on the BOM, images of electronic components on the PCBA are sequentially input into the EfficientNet, and then compared with the component category at that location on the BOM. If the category is not the component category that should appear at that location, an error is reported.

3. The PCBA online inspection platform based on AI visual inspection technology according to claim 2, characterized in that: The basic idea of ​​Siamese networks for few-shot metric learning is as follows: Two images are input, and features are extracted separately using the same convolutional neural network (CNN). The distance between the two feature vectors is then calculated to determine whether the two images belong to the same category. The network structure is as follows: First, a ResBlock structure (batch normalization + ReLU activation + convolutional layer + max-pooling downsampling + residual connections) is used to extract features from the two input images. Then, the distance between the two feature vectors is calculated. Next, this distance vector is input into another CNN (convolutional layer + ReLU activation + max-pooling downsampling) to extract higher-dimensional features between the two input images. Finally, a fully connected layer is used to obtain a scalar, and the Sigmoid activation function is used to output the final result: 1 for images of the same category and 0 for images of different categories. During training, only two images of electronic components need to be input, along with whether the two components belong to the same category. The twin network can be used to identify missing or incorrect PCBA components. For the detection of a certain type of PCBA, there is first a "gold standard" PCBA that has been manually compared without errors. According to the BOM and CAD data, the components on the PCBA are used as one of the inputs to the twin network. Then, the components in the same position of the PCBA to be detected are also input into the network. The network determines whether they belong to the same category. This network structure is used for few-shot learning. For a well-trained twin network, even if the input is an electronic component category that has not been seen during training, it is still used to compare with the electronic components in the same position on the gold standard PCBA to output whether the two electronic components belong to the same category.

4. The PCBA online inspection platform based on AI visual inspection technology according to claim 3, characterized in that: The PCBA intelligent AI real-time inspection system employs AI technology. This AI technology uses component type and packaging to create inspection models and inspect components. For each component, the AI ​​technology quickly obtains relevant information using existing technical data and uses this known data to predefine the location and classification of assembled components. Combined with PCBA assembly documents, the inspected components are labeled, transforming the PCBA dataset into a dataset of PCBA components for identification and comparison. The AI ​​technology focuses on detecting defects that cannot be intercepted by the above testing processes.

5. The PCBA online inspection platform based on AI visual inspection technology according to claim 4, characterized in that: The PCBA intelligent AI real-time detection system includes the following operation steps: S1: Utilize AI technology to inspect PCBAs. PCBA inspection focuses on defects that are not covered by circuit testing. Inspection is an important part of the entire testing system. The value of AI inspection lies in detecting defects that cannot be fully covered by existing inspection methods. S2: Classify the components within the bounding box according to the known component types, obtain the bounding box using component location information, and define the component category using component type. This not only combines the component shape and its pads in the CAD data to predefine the size of the bounding box of the component in the PCBA image, but also determines the type of the bounding box by the component type and package. S3: Predefine component classification based on component type and package; S4: Define the key areas of interest within the bounding box. After predefining the bounding box and key areas of interest based on the component type and its package, remove unnecessary information from the image. S5: Component and solder joint detection model transfer learning. Through the transfer learning function, a large amount of labeled and learned data is used to detect different PCBAs. The labeled data is associated with the component location number, and artificial intelligence is used to detect similar work. S6: Simulate an inspector conducting online testing.

Images