A circuit board quality inspection method and system for AOI machine vision

Abstract

The present invention provides a circuit board quality inspection method and system for AOI machine vision, belonging to the technical field of machine vision. The method includes: training an AI quality inspection model, which integrates a feature extraction module and a feature comparison module; obtaining the machine vision inspection result of the AOI device for the circuit board to be inspected; inputting the machine vision inspection result into the AI quality inspection model to obtain the quality inspection result; and determining a supplementary strategy for the component reference diagram for reference feature extraction according to the access characteristics of the target component in the circuit board. The circuit board quality inspection method and system for AOI machine vision of the present invention train the AI quality inspection model with diverse training samples, automatically output the quality inspection result according to the machine vision inspection result by using the AI quality inspection model, improve the first-pass rate of the quality inspection system, greatly reduce the labor cost of re-inspection, and make the subsequent reference feature extraction more comprehensive.

Description

Technical Field

[0001] The present invention relates to the technical field of machine vision, and particularly to a circuit board quality inspection method and system for AOI machine vision. Background Art

[0002] At present, the equipment detection first-pass rate of the AOI equipment circuit board detection process is 11%, that is, 89% of the circuit boards need to be manually reinspected, while the truly defective circuit boards only account for 4.9% of the total number of circuit boards, and the detection accuracy is only 5.5%. That is, among the circuit boards that are manually reinspected, 94.5% are normal and problem-free. The accuracy rate of the AOI equipment circuit board detection process is low, a large amount of manpower is required for subsequent reinspection, the advantages of automated quality inspection are not utilized, and the detection process is delayed.

[0003] In view of this, there is an urgent need for a circuit board quality inspection method and system for AOI machine vision to at least solve the above deficiencies. Summary of the Invention

[0004] One of the purposes of the present invention is to provide a circuit board quality inspection method and system for AOI machine vision. The AI quality inspection model is trained with diverse training samples. The AI quality inspection model integrates two AI models, namely a feature extraction module and a feature comparison module. The AI quality inspection model automatically outputs the quality inspection result according to the machine vision detection result, improving the first-pass rate of the quality inspection system and greatly reducing the labor cost of reinspection.

[0005] A circuit board quality inspection method for AOI machine vision provided by an embodiment of the present invention includes:

[0006] Step 1: Train the AI quality inspection model, and the AI quality inspection model integrates a feature extraction module and a feature comparison module;

[0007] Step 2: Obtain the machine vision detection result of the AOI equipment for the circuit board to be detected;

[0008] Step 3: Input the machine vision detection result into the AI quality inspection model, use the feature extraction module to extract the comparison features in the machine vision detection result and the reference features in the component reference diagram, and use the feature comparison module to compare the reference features and the comparison features to obtain the quality inspection result.

[0009] Preferably, the result range of the quality inspection result includes:

[0010] OK, missing parts, reverse, wrong parts, and false soldering.

[0011] A circuit board quality inspection method for AOI machine vision provided by an embodiment of the present invention further includes:

[0012] When using the feature extraction module to extract the reference features in the component reference diagram, perform all-round feature extraction.

[0013] Preferably, when extracting reference features from the component reference diagrams using the feature extraction module, all-round feature extraction is performed, including:

[0014] Determine the group of component reference diagrams belonging to the same target component;

[0015] According to the group of component reference diagrams, determine the shooting environment condition - camera pose pairing group when shooting the target component;

[0016] Obtain the access features of the target component, where the access features include: the three-dimensional data of the target component, the pins accessing the circuit board, and the positional relationship of adjacent components within a preset range;

[0017] According to the access features, plan the ideal camera pose sequence of the target component;

[0018] Traverse the ideal camera poses in the ideal camera pose sequence in turn, configure the ideal shooting environment conditions according to the simulated shooting images of the ideal camera poses being traversed, and pair the ideal shooting environment conditions with the ideal camera poses being traversed;

[0019] After all the ideal camera poses in the ideal camera pose sequence have been traversed, obtain the pairing item group;

[0020] Based on the support analysis rules, determine the support analysis results of each pairing item according to the shooting environment condition - camera pose pairing group and the pairing item group;

[0021] According to the support analysis results, supplement the group of component reference diagrams.

[0022] Preferably, step 1: Train the AI quality inspection model, including:

[0023] Obtain training samples, where the training samples include NG samples and OK samples, and the information included in each training sample includes: the position coordinates of each component on the circuit board and the quality inspection result label of the component at the corresponding position;

[0024] Train the AI quality inspection model according to the training samples.

[0025] Preferably, obtaining the training samples includes:

[0026] Obtain the current training sample set;

[0027] Perform a diversity test on the current training sample set, and supplement the current training sample set according to the diversity test results.

[0028] Preferably, performing a diversity test on the current training sample set and supplementing the current training sample set according to the diversity test results includes:

[0029] Divide the current training sample set into multiple sample subsets according to different circuit board types;

[0030] Sort the sample subsets in descending order according to the proportion of the quality inspection circuit boards of different circuit board types in the historical quality inspection records to obtain a sample subset sequence;

[0031] Traverse the sample subset sequence from beginning to end, and use the circuit board type of the currently traversed sample subset as the target circuit board type;

[0032] Extract quality inspection features from the quality inspection circuit boards of the target circuit board type in the historical quality inspection records to obtain a first quality inspection feature set;

[0033] Extract quality inspection features from the sample subset of the target circuit board type to obtain a second quality inspection feature set;

[0034] Calculate the feature similarity between the first quality inspection feature set and the second quality inspection feature set;

[0035] If the feature similarity is greater than or equal to the preset feature similarity threshold corresponding to the target circuit board type, continue to traverse the next sample subset;

[0036] If the feature similarity is less than the feature similarity threshold corresponding to the target circuit board type, obtain the sum of the proportions of the circuit board types of the traversed sample subsets;

[0037] Determine the sample set supplement condition according to the sum of the proportions, the sum of the proportions threshold, and the preset sample set supplement condition library;

[0038] Supplement the target sample subset specifically according to the sample set supplement condition; the target sample subset is the sample subset of the target circuit board type.

[0039] Preferably, determine the sample set supplement condition according to the sum of the proportions, the sum of the proportions threshold, and the preset sample set supplement condition library, including:

[0040] Judge whether the sum of the proportions is greater than or equal to the sum of the proportions threshold;

[0041] If so, obtain the first sample set supplement condition according to the target difference between the sum of the proportions and the sum of the proportions threshold; the target difference is the difference between the sum of the proportions and the sum of the proportions threshold;

[0042] If not, obtain the preset second sample set supplement condition.

[0043] Preferably, supplement the target sample subset specifically according to the sample set supplement condition, including:

[0044] Determine the feature difference items and feature similarity items between the first quality inspection feature set and the second quality inspection feature set;

[0045] Obtain the associated value between the targeted circuit board type and the circuit board types of the traversed sample subset;

[0046] If the associated value is greater than or equal to the preset associated value threshold, index the target quality inspection features in the sample subset of the circuit board types of the traversed sample subset according to the quality inspection feature type of the feature difference item in the order from largest to smallest; The target quality inspection feature is: the quality inspection feature that is consistent with the quality inspection feature type of the feature difference item extracted from the sample subset corresponding to the circuit board type of the traversed sample subset where the associated value is greater than or equal to the preset associated value threshold;

[0047] If the indexing is successful, obtain the first source sample of the corresponding target quality inspection feature;

[0048] Use the local samples corresponding to the feature similarity items in the targeted sample set as the second source samples;

[0049] Input the first source sample and the second source sample into the circuit analysis model to analyze the coexistence conflict of component states;

[0050] If there is no coexistence conflict of component states, based on the preset sample simulation model, according to the first source sample and the second source sample, simulate and supplement the samples, and supplement the supplemented samples to the targeted sample subset;

[0051] If the indexing fails, obtain the supplementary samples based on big data according to the targeted circuit board type and the feature difference item.

[0052] A circuit board quality inspection system for AOI machine vision provided by an embodiment of the present invention includes:

[0053] A training subsystem for training an AI quality inspection model, and the AI quality inspection model integrates a feature extraction module and a feature comparison module;

[0054] A visual detection result acquisition subsystem for obtaining the machine vision detection result of the AOI device for the detected circuit board;

[0055] A quality inspection result output subsystem for inputting the machine vision detection result into the AI quality inspection model, using the feature extraction module to extract the comparison features in the machine vision detection result and the reference features in the component reference diagram, using the feature comparison module to compare the reference features and the comparison features, and obtaining the quality inspection result.

[0056] The beneficial effects of the present invention are:

[0057] The present invention trains an AI quality inspection model with diverse training samples. The AI quality inspection model integrates two AI models, namely a feature extraction module and a feature comparison module, and automatically outputs a quality inspection result according to the machine vision inspection result by using the AI quality inspection model. A supplementary strategy for the component reference diagram for reference feature extraction is determined according to the access features of the target component in the circuit board, which improves the first-pass rate of the quality inspection system, greatly reduces the labor cost of re-inspection, and makes the subsequent reference feature extraction more comprehensive.

[0058] Other features and advantages of the present invention will be described in the following specification, and, in part, will be obvious from the specification, or will be understood by implementing the present invention. The objectives and other advantages of the present invention can be achieved and obtained by the structures specifically pointed out in this application document.

[0059] The technical solution of the present invention will be further described in detail below through the drawings and embodiments. Description of the Drawings

[0060] The drawings are used to provide a further understanding of the present invention, and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention, but do not constitute a limitation to the present invention. In the drawings:

[0061] Figure 1 It is a schematic diagram of a circuit board quality inspection method for AOI machine vision in an embodiment of the present invention;

[0062] Figure 2 It is a schematic diagram of a circuit board quality inspection system for AOI machine vision in an embodiment of the present invention. Detailed Embodiments

[0063] The following describes the preferred embodiments of the present invention with reference to the drawings. It should be understood that the preferred embodiments described herein are only for the purpose of illustrating and explaining the present invention, and are not used to limit the present invention.

[0064] The embodiment of the present invention provides a circuit board quality inspection method for AOI machine vision, as Figure 1 shown, including:

[0065] Step 1: Train an AI quality inspection model, and the AI quality inspection model integrates a feature extraction module and a feature comparison module;

[0066] The said Step 1: Training the AI quality inspection model includes:

[0067] Obtain training samples, where the training samples include NG samples and OK samples. The information included in each training sample is: the position coordinates of each component on the circuit board and the quality inspection result label of the component at the corresponding position; the OK sample is a qualified sample that passes all quality inspection items and meets the design specifications; the NG sample is an unqualified sample that fails the quality inspection.

[0068] Train the AI quality inspection model based on training samples;

[0069] Step 2: Obtain the machine vision inspection results of the AOI device for the detected circuit board;

[0070] Step 3: Input the machine vision inspection results into the AI quality inspection model, use the feature extraction module to extract the comparison features in the machine vision inspection results and the reference features in the component reference diagram, use the feature comparison module to compare the reference features and the comparison features, and obtain the quality inspection results;

[0071] The result range of the quality inspection results at least includes: OK, missing parts, reverse, wrong parts, and false soldering.

[0072] The working principle and beneficial effects of the above technical solution are as follows:

[0073] The feature extraction module and the feature comparison module are two AI models, which are trained and obtained by using training samples that have marked the position coordinates of each component on the circuit board and the labels (OK or NG and defect categories, such as wrong parts, skew, virtual soldering, etc.) of the components at the corresponding positions. The training samples need to include as many circuit boards as possible. For example: 1000 samples for each type of circuit board, a total of 200 different circuit boards, that is, a total of 200,000 samples. The more diverse the training samples are, the better the model can be trained; the AI quality inspection model integrates the above two AI models and can realize the function of automatically outputting the quality inspection results corresponding to the input AOI device inspection results (machine vision inspection results). The core process of the quality inspection system is: input the components to be detected (obtained by parsing the machine vision inspection results), extract its feature FA (comparison feature), and compare the feature FA with the reference feature FR (considering the influence of shooting conditions, multiple component reference diagrams can be set for the same component to improve accuracy), and obtain results such as OK, missing parts, reverse, wrong parts, false soldering, etc. (quality inspection results).

[0074] The present invention trains the AI quality inspection model through diverse training samples. The AI quality inspection model integrates two AI models, namely the feature extraction module and the feature comparison module. The AI quality inspection model automatically outputs the quality inspection results according to the machine vision inspection results, improving the first-pass rate of the quality inspection system and greatly reducing the labor cost of re-inspection.

[0075] The embodiment of the present invention provides a method for inspecting the quality of a circuit board by AOI machine vision, which further includes:

[0076] When using the feature extraction module to extract the reference features in the component reference diagram, perform omnidirectional feature extraction;

[0077] Among them, when using the feature extraction module to extract the reference features in the component reference diagram, performing omnidirectional feature extraction includes:

[0078] Determine the component reference diagram group belonging to the same target component;

[0079] According to the component reference diagram group, determine the shooting environment condition - camera pose pairing group when shooting the target component;

[0080] Obtain the access characteristics of the target component, where the access characteristics include: three - dimensional data of the target component, access circuit board pins, and the positional relationship of adjacent components within a preset range;

[0081] According to the access characteristics, plan the ideal camera pose sequence of the target component;

[0082] Traverse the ideal camera poses in the ideal camera pose sequence in turn, configure the ideal shooting environment conditions according to the simulated shooting images of the currently traversed ideal camera pose, and pair the ideal shooting environment conditions with the currently traversed ideal camera pose;

[0083] After all the ideal camera poses in the ideal camera pose sequence have been traversed, obtain the pairing item group;

[0084] Based on the support analysis rule, according to the shooting environment condition - camera pose pairing group and the pairing item group, determine the support analysis result of each pairing item;

[0085] According to the support analysis result, supplement the component reference diagram group.

[0086] The working principle and beneficial effects of the above - mentioned technical solution are as follows:

[0087] The component reference diagram group belonging to the same target component means that all the pictures in the group are of the target component to which they belong; by performing image feature extraction on the component reference diagrams, the shooting environment conditions and camera poses can be read. Performing such processing on all the images in the component reference diagram group, each component reference Figure 1 corresponding shooting environment condition and camera pose can be obtained, and summarizing them can obtain the shooting environment condition - camera pose pairing group.

[0088] To ensure the comprehensiveness of the reference features of the subsequent target component, the access features of the target component in the corresponding accessed circuit board are extracted. The access features at least include: the three-dimensional data of the target component, the pins of the accessed circuit board, and the positional relationship of adjacent components within a preset range (for example, within 0.5 cm from the target component). The ideal camera pose sequence of the target component planned according to the access features is: the aggregated sequence of camera poses that can maximize the capture of the entire target component. Moreover, the ideal shooting environment conditions for each ideal camera pose are also different (for example, the brightness requirements for the shooting points corresponding to different ideal camera poses). Therefore, traverse the ideal camera poses in the ideal camera pose sequence in sequence, configure the ideal shooting environment conditions based on the simulated shooting images of the currently traversed ideal camera pose (for example, how much the brightness of which position of the target component needs to reach). The simulated shooting images are jointly determined based on the currently traversed ideal camera pose and the access features, and the ideal shooting environment conditions and the currently traversed ideal camera pose are paired. The support analysis rule includes two analysis levels. First, analyze whether there is a corresponding matching camera pose for the ideal camera pose in the paired item. If so, conduct the second-level analysis: analyze whether the ideal environment conditions and the shooting environment conditions match. If both analyses pass, the final analysis result of this paired item is support. According to the different support analysis results of each paired item, determine the supplement strategy for the supplementary component reference diagram. For example, if there is no corresponding matching ideal camera pose, additional shots need to be taken for this ideal camera pose and the ideal shooting environment conditions of this ideal camera pose; if there is a corresponding matching ideal camera pose and the shooting environment conditions do not match, the image processing of the corresponding component reference diagram can be adaptively performed according to the environment matching result. By introducing the access features of the target component to plan the ideal camera pose sequence and determining the paired item group according to the ideal camera pose sequence, and combining the paired item group and the shooting environment condition-camera pose paired group to conduct the support analysis for each paired item and supplement the component reference diagram group, the comprehensiveness of the subsequent extraction of reference features is improved.

[0089] In one embodiment, obtaining training samples includes:

[0090] Obtain the current training sample set;

[0091] Divide the current training sample set into multiple sample subsets according to different circuit board types;

[0092] Sort the sample subsets according to the proportion of the quality inspection circuit boards of different circuit board types in the historical quality inspection records from large to small to obtain a sample subset sequence;

[0093] Traverse the sample subset sequence from beginning to end, and use the circuit board type of the currently traversed sample subset as the targeted circuit board type;

[0094] Extract quality inspection features from the quality inspection circuit boards of the targeted circuit board type in the historical quality inspection records to obtain the first quality inspection feature set;

[0095] Extract quality inspection features from the sample subset of the targeted circuit board type to obtain the second quality inspection feature set;

[0096] Calculate the feature similarity between the first quality inspection feature set and the second quality inspection feature set;

[0097] If the feature similarity is greater than or equal to the preset feature similarity threshold corresponding to the targeted circuit board type, continue to traverse the next sample subset;

[0098] If the feature similarity is less than the feature similarity threshold corresponding to the targeted circuit board type, obtain the sum of the proportions of the circuit board types in the traversed sample subset; the sum of the proportions is: the sum of the sizes of the proportions of the circuit board types in the traversed sample subset in the quality inspection circuit boards of the corresponding circuit board types in the historical quality inspection records; determine the sample set supplementation condition according to the sum of the proportions, the proportion sum threshold, and the preset sample set supplementation condition library;

[0099] Supplement the targeted sample subset specifically according to the sample set supplementation condition; the targeted sample subset is the sample subset of the targeted circuit board type.

[0100] The working principle and beneficial effects of the above technical solution are:

[0101] The current training sample set is the current training data of the AI quality inspection model; the diversity inspection refers to inspecting the diversity of the circuit board types, the positions of faulty components, and the types of faults in the current training sample set. During the inspection, the current training sample set is divided into multiple sample subsets according to the circuit board type, where the circuit board type refers to the type of circuit board, such as: all SMT boards, SMT+DIP hybrid boards, and pure DIP boards, etc.; the proportion of the quality inspection circuit boards of different circuit board types in the historical quality inspection records represents the probability of subsequent quality inspection of the circuit boards of the corresponding circuit board type. The sample subsets are sorted in descending order according to the proportion. The greater the impact of the diversity of the sample subsets earlier in the sequence on the subsequent model quality inspection effect. Extract quality inspection features from the quality inspection circuit boards of the targeted circuit board type. The quality inspection features at least include: the position coordinates of each component and the label of the component at the corresponding position. The first quality inspection feature set represents the description of the quality inspection situation of the quality inspection circuit boards of the targeted circuit board type in the local history. The second quality inspection feature set represents the description of the quality inspection situation of the quality inspection circuit boards of the targeted circuit board type in the current training sample set. Calculate the feature similarity between the first quality inspection feature set and the second quality inspection feature set. The higher the feature similarity, the more comprehensive the quality inspection situation description. The feature similarity is a quantitative value of the feature matching similarity degree between the first quality inspection feature set and the second quality inspection feature set. The higher the position of the sample subset of the targeted circuit board type in the sample subset sequence, the higher the preset feature similarity threshold corresponding to the targeted circuit board type. For example: at the first sequence position in the sample subset sequence, the feature similarity threshold is: 90; at the second sequence position in the sample subset sequence, the feature similarity threshold is: 85. If the feature similarity is greater than the corresponding feature similarity threshold, it means that the quality inspection situation of the targeted sample subset is comprehensive, and continue to traverse the next sample subset. If the feature similarity is less than the corresponding feature similarity threshold, it means that the quality inspection situation of the targeted sample subset is not comprehensive and needs to be analyzed specifically.

[0102] In the prior art, when performing diversity analysis on model training samples, generally, the current training sample set is evaluated macroscopically and then adaptively supplemented. The sample supplementation lacks pertinence. Therefore, it is urgent to solve this problem.

[0103] The present invention proposes a method for targeted supplementation by determining the sample set supplementation conditions based on the proportion of the circuit board types of the traversed sample subsets and the proportion sum threshold when the feature similarity is less than the corresponding feature similarity threshold of the targeted circuit board type. The greater the difference between the proportion sum and the proportion sum threshold, the stricter the sample set supplementation conditions. Specifically supplement the samples of the incomplete targeted circuit board type, which improves the suitability of sample supplementation and also improves the acquisition efficiency of model training data.

[0104] In one embodiment, determining the sample set supplementation conditions according to the proportion sum, the proportion sum threshold, and the preset sample set supplementation condition library includes:

[0105] Judge the proportion sum and whether it is greater than or equal to the proportion sum threshold;

[0106] If so, obtain the first sample set supplementary condition according to the target difference between the proportion sum and the proportion sum threshold; the target difference is the difference between the proportion sum and the proportion sum threshold;

[0107] If not, obtain the preset second sample set supplementary condition.

[0108] The working principle and beneficial effects of the above technical solution are as follows:

[0109] The proportion sum threshold is set manually in advance, for example: 80%; if the proportion sum is greater than or equal to the proportion sum threshold, it means that the sample subsets of the circuit board types with high quality inspection probability have all passed the sample diversity inspection. For the current targeted sample subset, the greater the target difference between the proportion sum and the proportion sum threshold, the smaller its corresponding quality inspection probability, and the looser the obtained first sample set supplementary condition; for the case where the proportion sum is less than the proportion sum threshold, it means that the sample subsets of the circuit board types with high quality inspection probability have not all passed the sample diversity inspection. Obtain the preset second sample set supplementary condition. The second sample set supplementary condition is: comprehensively supplement the targeted sample subset strictly according to the feature differences between the first quality inspection feature set and the second quality inspection feature set until the feature similarity is less than the feature similarity threshold, reducing the sample supplementary resources and improving the utilization rate of the supplementary resources.

[0110] In one embodiment, the targeted supplementary of the targeted sample subset according to the sample set supplementary condition includes:

[0111] Determine the feature difference items and feature similarity items between the first quality inspection feature set and the second quality inspection feature set;

[0112] Obtain the correlation value between the targeted circuit board type and the circuit board types of the traversed sample subsets;

[0113] If the correlation value is greater than or equal to the preset correlation value threshold, index the target quality inspection features in the sample subsets of the circuit board types of the traversed sample subsets according to the quality inspection feature types of the feature difference items in the order from large to small; the target quality inspection features are: the quality inspection features extracted from the sample subsets corresponding to the circuit board types of the traversed sample subsets with the correlation value greater than or equal to the preset correlation value threshold and consistent with the quality inspection feature types of the feature difference items; if the indexing is successful, obtain the first source samples of the corresponding target quality inspection features;

[0114] Take the local samples corresponding to the feature similarity items in the targeted sample set as the second source samples;

[0115] Input the first source samples and the second source samples into the circuit analysis model to analyze the coexistence conflicts of component states;

[0116] If there is no conflict in the coexistence of component states, based on a preset sample simulation model, supplement samples according to the first source sample and the second source sample, and supplement the supplemented samples to the target sample subset;

[0117] If the indexing fails, obtain supplementary samples based on big data according to the target circuit board type and the feature difference items.

[0118] The working principle and beneficial effects of the above technical solution are as follows:

[0119] The feature difference items are a pair of features where the first quality inspection feature and the second quality inspection feature do not match. For example, the first quality inspection feature is: the surface of the electrolytic capacitor bulges at point A, and the second quality inspection feature is the normal surface feature of the electrolytic capacitor; the feature similarity items are a pair of features where the first quality inspection feature and the second quality inspection feature match. For example, the first quality inspection feature is: the surface of the electrolytic capacitor bulges at point A, and the second quality inspection feature is the surface of the electrolytic capacitor bulges at point B;

[0120] Because the traversed sample subset has undergone feature extraction processing, and the sample subset is sorted in descending order of proportion, and the circuit boards of the circuit type in the traversed sample subset have a high proportion of quality inspections in history, while the untraversed sample subset only conducts feature classification and has a small proportion of quality inspections in history. Therefore, index the target quality inspection feature from the sample subset of the circuit board type in the traversed sample subset to improve the extraction efficiency of the target quality inspection feature;

[0121] The correlation value is a quantitative value of the correlation degree between the target circuit board type (such as: all SMT boards) and the circuit board types of the traversed sample subsets. For example, the correlation value between all SMT boards and SMT+DIP hybrid boards is: 80, and the correlation value between all SMT boards and pure DIP boards is: 20. The larger the correlation value, the more likely it is to extract the sample supplementary information of the target sample subset from the sample subsets of the circuit board types of the corresponding traversed sample subsets. Extract the quality inspection feature types (such as: the working state of electrolytic capacitors) of the target quality inspection features (such as: bulging at point C on the surface of the electrolytic capacitor) in the sample subsets of the circuit board types (such as: SMT+DIP hybrid boards) of the traversed sample subsets whose correlation values are greater than or equal to the preset correlation value threshold in descending order of the correlation values. The preset correlation value threshold is set manually in advance, such as: 60. The first source sample is the AOI machine vision image for target quality inspection feature extraction. The local sample corresponding to the feature similarity item is: a partial image extracted from the target sample set corresponding to the second quality inspection feature of the feature similarity item. For example: the equivalent series inductance image of a certain electrolytic capacitor. The circuit analysis model is an AI model for analyzing the connection status of electronic components in the first source sample and the second source sample, and analyzing whether they can be normally connected and present the corresponding status, which is obtained by training according to manual circuit analysis records. Component state coexistence conflict refers to the contradiction of component states in circuit logic or physical rules after different source samples are combined. For example: the combination of high impedance and low capacitance leads to abnormal resonance frequency. If there is no component state coexistence conflict, based on the preset sample simulation model, according to the first source sample and the second source sample, supplement the sample. The preset sample simulation model is a circuit board simulation model that simulates the coexistence of the target quality inspection feature and the second quality inspection feature corresponding to the feature similarity item on the circuit board of the same target circuit board type. It automatically generates supplementary samples according to the input first source sample and second source sample, and supplements the obtained supplementary samples to the target sample subset. If the target quality inspection feature cannot be indexed in the sample subsets of the circuit board types of the relevant traversed sample subsets, supplementary samples are obtained from big data.

[0122] The present invention adaptively supplements when relevant supplementary samples can be generated locally, and docks with big data to obtain supplementary samples when adaptive supplementation is not possible, without frequently docking with big data nodes, improving the acquisition efficiency of supplementary samples.

[0123] An embodiment of the present invention provides a circuit board quality inspection system for AOI machine vision, as Figure 2 shown, including:

[0124] A training subsystem 1 for training an AI quality inspection model, and the AI quality inspection model integrates a feature extraction module and a feature comparison module;

[0125] A visual detection result acquisition subsystem 2 for acquiring the machine vision detection results of the AOI device for the detected circuit board;

[0126] The quality inspection result output subsystem 3 is used to input the machine vision inspection results into the AI quality inspection model, extract the comparison features in the machine vision inspection results and the reference features in the component reference diagram by using the feature extraction module, and compare the reference features and the comparison features by using the feature comparison module to obtain the quality inspection results;

[0127] The circuit board quality inspection system of AOI machine vision also performs the following operations:

[0128] When extracting the reference features in the component reference diagram by using the feature extraction module, perform omnidirectional feature extraction, including:

[0129] Determine the group of component reference diagrams belonging to the same target component;

[0130] According to the group of component reference diagrams, determine the shooting environment condition - camera pose pairing group when shooting the target component;

[0131] Obtain the access features of the target component, where the access features include: the three-dimensional data of the target component, the pins accessing the circuit board, and the positional relationship of adjacent components within a preset range;

[0132] According to the access features, plan the ideal camera pose sequence of the target component;

[0133] Traverse the ideal camera poses in the ideal camera pose sequence in turn, configure the ideal shooting environment conditions according to the simulated shooting images of the ideal camera pose being traversed, and pair the ideal shooting environment conditions with the ideal camera pose being traversed;

[0134] After all the ideal camera poses in the ideal camera pose sequence have been traversed, obtain the pairing item group;

[0135] Based on the support analysis rules, determine the support analysis results of each pairing item according to the shooting environment condition - camera pose pairing group and the pairing item group;

[0136] According to the support analysis results, supplement the group of component reference diagrams.

[0137] The circuit board quality inspection system of AOI machine vision can be used to implement the above-mentioned circuit board quality inspection method of AOI machine vision.

[0138] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalent technologies, the present invention also intends to include these changes and modifications.

Claims

1. A circuit board quality inspection method for AOI machine vision, characterized in that, Including: Step 1: Train an AI quality inspection model, which integrates a feature extraction module and a feature comparison module; Step 2: Obtain the machine vision inspection results of the AOI device for the detected circuit board; Step 3: Input the machine vision inspection results into the AI quality inspection model. Use the feature extraction module to extract the comparison features in the machine vision inspection results and the reference features in the component reference diagram, and use the feature comparison module to compare the reference features and the comparison features to obtain the quality inspection results; When using the feature extraction module to extract the reference features in the component reference diagram, perform omnidirectional feature extraction, including: Determine the group of component reference diagrams belonging to the same target component; According to the group of component reference diagrams, determine the shooting environment condition - camera pose pairing group when shooting the target component; Obtain the access features of the target component, where the access features include: the three-dimensional data of the target component, the pins accessing the circuit board, and the positional relationship of adjacent components within a preset range; According to the access features, plan the ideal camera pose sequence of the target component; Traverse the ideal camera poses in the ideal camera pose sequence in turn. Configure the ideal shooting environment conditions according to the simulated shooting images of the ideal camera pose being traversed, and pair the ideal shooting environment conditions with the ideal camera pose being traversed; After all the ideal camera poses in the ideal camera pose sequence have been traversed, obtain the pairing item group; Based on the support analysis rules, determine the support analysis results of each pairing item according to the shooting environment condition - camera pose pairing group and the pairing item group; Supplement the group of component reference diagrams according to the support analysis results; Among them, Step 1: Train the AI quality inspection model, including: Obtain training samples, where the training samples include NG samples and OK samples. The information included in each training sample includes: the position coordinates of each component on the circuit board and the quality inspection result label of the component at the corresponding position; Train the AI quality inspection model according to the training samples; Among them, obtaining training samples includes: Obtain the current training sample set; Conduct a diversity test on the current training sample set. According to the diversity test results, supplement the current training sample set, including: Divide the current training sample set into multiple sample subsets according to different circuit board types; Sort the sample subsets according to the proportion of the quality inspection circuit boards of different circuit board types in the historical quality inspection records from large to small to obtain the sample subset sequence; Traverse the sample subset sequence from beginning to end, and use the circuit board type of the sample subset being traversed as the targeted circuit board type; Perform quality inspection feature extraction on the quality inspection circuit boards of the targeted circuit board type in the historical quality inspection records to obtain the first quality inspection feature set; Perform quality inspection feature extraction on the sample subset of the targeted circuit board type to obtain the second quality inspection feature set; Calculate the feature similarity between the first quality inspection feature set and the second quality inspection feature set; If the feature similarity is greater than or equal to the preset feature similarity threshold corresponding to the targeted circuit board type, continue to traverse the next sample subset; If the feature similarity is less than the feature similarity threshold corresponding to the targeted circuit board type, obtain the sum of the proportions of the circuit board types of the traversed sample subsets; Determine the sample set supplement condition according to the sum of the proportions, the sum of the proportions threshold, and the preset sample set supplement condition library; Supplement the targeted sample subset according to the sample set supplement conditions; the targeted sample subset is the sample subset of the targeted circuit board type.

2. The circuit board quality inspection method for AOI machine vision according to claim 1, characterized in that, The result range of the quality inspection results includes: OK, missing parts, reverse, wrong parts, and false soldering.

3. The circuit board quality inspection method for AOI machine vision according to claim 1, characterized in that, According to the sum of proportions, the sum of proportions threshold, and the preset sample set supplement condition library, determine the sample set supplement conditions, including: Judge whether the sum of proportions is greater than or equal to the sum of proportions threshold; If so, obtain the first sample set supplement condition according to the target difference between the sum of proportions and the sum of proportions threshold; the target difference is the difference between the sum of proportions and the sum of proportions threshold; If not, obtain the preset second sample set supplement condition.

4. The circuit board quality inspection method for AOI machine vision according to claim 1, characterized in that, Supplement the targeted sample subset according to the sample set supplement conditions, including: Determine the feature difference items and feature similarity items between the first quality inspection feature set and the second quality inspection feature set; Obtain the correlation value between the targeted circuit board type and the circuit board types of the traversed sample subsets; If the correlation value is greater than or equal to the preset correlation value threshold, index the target quality inspection features in the sample subsets of the circuit board types of the traversed sample subsets according to the quality inspection feature types of the feature difference items in descending order of the correlation value; the target quality inspection features are: the quality inspection features extracted from the sample subsets corresponding to the circuit board types of the traversed sample subsets with the correlation value greater than or equal to the preset correlation value threshold and consistent with the quality inspection feature types of the feature difference items; If the indexing is successful, obtain the first source sample of the corresponding target quality inspection feature; Use the local samples corresponding to the feature similarity items in the targeted sample set as the second source samples; Input the first source sample and the second source sample into the circuit analysis model to analyze the coexistence conflict of component states; If there is no coexistence conflict of component states, based on the preset sample simulation model, according to the first source sample and the second source sample, simulate and supplement the samples, and supplement the supplementary samples into the targeted sample subset; If the indexing fails, obtain supplementary samples based on big data according to the targeted circuit board type and the feature difference items.

5. A circuit board quality inspection system for AOI machine vision, characterized in that, Including: A training subsystem for training an AI quality inspection model, and the AI quality inspection model integrates a feature extraction module and a feature comparison module; A visual inspection result acquisition subsystem for acquiring the machine vision inspection results of the AOI device on the inspection circuit board; A quality inspection result output subsystem for inputting the machine vision inspection results into the AI quality inspection model, using the feature extraction module to extract the comparison features in the machine vision inspection results and the reference features in the component reference diagram, using the feature comparison module to compare the reference features and the comparison features, and obtaining the quality inspection results; The circuit board quality inspection system of the AOI machine vision also performs the following operations: When using the feature extraction module to extract the reference features in the component reference diagram, perform comprehensive feature extraction, including: Determine the group of component reference diagrams belonging to the same target component; According to the group of component reference diagrams, determine the shooting environment condition - camera pose pairing group when shooting the target component; Obtain the access features of the target component, and the access features include: the three-dimensional data of the target component, the access circuit board pins, and the positional relationship of adjacent components within a preset range; According to the access features, plan the ideal camera pose sequence of the target component. Traverse the ideal camera poses in the ideal camera pose sequence in turn, configure the ideal shooting environment conditions according to the simulated shooting images of the currently traversed ideal camera pose, and pair the ideal shooting environment conditions with the currently traversed ideal camera pose; After all the ideal camera poses in the ideal camera pose sequence have been traversed, obtain a set of paired items; Based on the support analysis rules, determine the support analysis results of each paired item according to the shooting environment condition-camera pose pairing group and the set of paired items; Supplement the component reference drawing set according to the support analysis results; Among them, the training subsystem performs the following operations: Obtain training samples, where the training samples include NG samples and OK samples, and the information included in each training sample includes: the position coordinates of each component on the circuit board and the quality inspection result label of the component at the corresponding position; Train the AI quality inspection model according to the training samples; Among them, obtaining training samples includes: Obtain the current training sample set; Conduct a diversity test on the current training sample set, and supplement the current training sample set according to the diversity test results, including: Divide the current training sample set into multiple sample subsets according to different circuit board types; Sort the sample subsets according to the proportion of the quality inspection circuit boards of different circuit board types in the historical quality inspection records from largest to smallest to obtain a sample subset sequence; Traverse the sample subset sequence from beginning to end, and use the circuit board type of the currently traversed sample subset as the targeted circuit board type; Extract the quality inspection features of the quality inspection circuit boards of the targeted circuit board type from the historical quality inspection records to obtain a first quality inspection feature set; Extract the quality inspection features of the sample subset of the targeted circuit board type to obtain a second quality inspection feature set; Calculate the feature similarity between the first quality inspection feature set and the second quality inspection feature set; If the feature similarity is greater than or equal to the preset feature similarity threshold corresponding to the targeted circuit board type, continue to traverse the next sample subset; If the feature similarity is less than the feature similarity threshold corresponding to the targeted circuit board type, obtain the sum of the proportions of the circuit board types of the traversed sample subsets; Determine the sample set supplement condition according to the sum of proportions, the sum of proportions threshold, and the preset sample set supplement condition library; Perform targeted supplementation on the targeted sample subset according to the sample set supplement condition; the targeted sample subset is the sample subset of the targeted circuit board type.

Images