Defect detection system, PCB line defect detection method and readable storage medium

Abstract

The invention belongs to the technical field of digital information transmission, and particularly relates to a defect detection system, a PCB line defect detection method and a readable storage medium, the defect detection system comprises an optical detection device, a cloud server and an all-in-one machine; wherein each optical detection device uploads original data to the cloud server so as to form label data corresponding to the original data in the cloud server, and a defect detection model is constructed in the cloud server and is trained and optimized; after the optical detection equipment sends the acquired data to the all-in-one machine, the all-in-one machine identifies the data through the defect detection model and outputs a judgment result to the optical detection equipment; according to the method, the isolation of production, data and operation can be realized, the safety and stability of the data are effectively ensured, the computing power can be effectively distributed, the data processing amount is improved to meet the production line detection requirement, and meanwhile, the index database and the defect detection model can be timely optimized and updated to improve the detection precision and the detection efficiency.

Description

Technical Field

[0001] The present invention belongs to the technical field of digital information transmission, and specifically relates to a device suitable for a networking environment, and more particularly to a defect detection system, a PCB circuit defect detection method and a readable storage medium. Background Art

[0002] In the PCB production process, after the etching and detinning of the inner and outer layers of the board are completed, the circuit inspection needs to be carried out by automatic optical inspection equipment to ensure that the quality of the PCB board is up to standard and does not affect the subsequent delivery and use. The production of PCB boards requires multiple process flows, and automatic optical inspection is reflected in the circuit production stage. If the quality inspection at the automatic optical inspection stage fails, it will cause huge waste of resources in the subsequent processes.

[0003] After completing the image acquisition of the PCB board, the current automatic optical inspection equipment will output pictures that may contain defective points based on the traditional image algorithm. During the production process, only one production line can output millions of defective pictures per day. Among these pictures, there are a large number of false points, reaching about 85%. In order to correctly screen out the true points and filter out the false points that do not need to be repaired later, it is necessary to operate the automatic optical inspection equipment for visual inspection. Manual visual inspection will incur additional labor costs, and because the speed of workers' visual inspection has physiological limits, it will also affect production efficiency. In addition, the standards for manual visual inspection are not uniform and unstable, which will lead to the loss of true points, with a loss rate of about 0.5%.

[0004] Therefore, it is urgent to develop a new defect detection system, PCB line defect detection method and readable storage medium to solve the technical problem of how to overcome the inability of traditional automatic optical inspection equipment to effectively allocate computing power.

[0005] It should be noted that the above information disclosed in this background technology section is only used to understand the background technology of the present application concept, and therefore, the above description is not considered to constitute information of the prior art. Summary of the invention

[0006] The embodiments of the present disclosure at least provide a defect detection system, a PCB circuit defect detection method and a readable storage medium.

[0007] In a first aspect, an embodiment of the present disclosure provides a defect detection system, which includes: at least one optical detection device, a cloud server and an all-in-one machine; wherein each of the optical detection devices uploads raw data to the cloud server respectively to form label data corresponding to the raw data in the cloud server, and constructs a defect detection model in the cloud server and trains and optimizes it; the cloud server sends the optimized defect detection model to the all-in-one machine; and when the optical detection device sends the acquired data to the all-in-one machine, the all-in-one machine identifies the data through the defect detection model, and performs deep convolution on the data through the defect detection model to output the deep convolution result as follows: ;and ;in, X is a three-dimensional real number matrix representing the image features, H For the input image width, W To input the image height, C in is the number of channels of the input image, K is the convolution kernel size, ( i , j ) is the coordinate of the output feature map, c is the channel index; the data is convolved point by point through the defect detection model to output the point by point convolution result: ;and ;in, C out is the number of channels of the output result, ( i , j,c ) is the coordinate of the depth convolution result, c’ is the output channel index; the final feature map is output by the defect detection model: Y(i,j,c ’ ) = Swish (Y point (i, j, c ’ ）） ;and Swish(x)=x×σ(βx) ,in σ represents the sigmoid function, β is a learnable parameter; the defect detection model generates a corresponding judgment result through the final feature map to output it to the optical detection device.

[0008] In an optional embodiment, the cloud server includes: a cloud platform, a cloud storage unit and a cloud computing unit; the cloud platform obtains the original data sent by each optical detection device, and annotates each original data to form corresponding label data, and the cloud platform constructs a corresponding index database through the original data and label data; the cloud platform splits the original data and saves it in the cloud storage unit and the cloud computing unit, and the cloud platform splits the label data and saves it in the cloud storage unit and the cloud computing unit.

[0009] In an optional implementation, the cloud platform saves the document information in the original data to a cloud storage unit, and the cloud platform saves the images and binary files in the original data to a cloud computing unit; the cloud platform saves the document information in the labeled data to a cloud storage unit, and the cloud platform saves the binary files in the labeled data to a cloud computing unit.

[0010] In an optional implementation, the cloud platform constructs a defect detection model based on the EfficientNet_b0 architecture and the FPN structure, and adjusts the loss function weight of the defect detection model by weighting the amount of training set data; the cloud platform generates corresponding training data based on various types of data in the index database; the defect detection model learns and trains the training data in the cloud computing unit until the cloud platform sends the defect detection model to the all-in-one machine.

[0011] In an optional embodiment, the all-in-one machine includes: an industrial computer, a storage medium and a GPU computing core; when the industrial computer obtains the data sent by the optical inspection equipment, the GPU computing core identifies the data through a defect detection model to send the judgment result to the industrial computer; the industrial computer saves the data and the corresponding judgment result to the storage medium, and the industrial computer returns the judgment result to the optical inspection equipment.

[0012] In the second aspect, the embodiments of the present disclosure also provide a PCB circuit defect detection method, which includes: uploading the original data to the cloud server through each optical inspection device respectively to form label data corresponding to the original data in the cloud server, and constructing a defect detection model in the cloud server and training and optimizing it; sending the optimized defect detection model to the all-in-one machine through the cloud server; when the optical inspection device sends the acquired data to the all-in-one machine, the all-in-one machine identifies the data through the defect detection model and outputs the judgment result to the optical inspection device.

[0013] In an optional embodiment, the cloud platform in the cloud server obtains the original data sent by each optical detection device, and annotates each original data to form corresponding label data, and the cloud platform builds a corresponding index database through the original data and the label data; the original data is split and saved in the cloud storage unit and the cloud computing unit in the cloud server through the cloud platform, and the label data is split and saved in the cloud storage unit and the cloud computing unit through the cloud platform.

[0014] In an optional embodiment, a defect detection model is constructed with the EfficientNet_b0 architecture and the FPN structure in the cloud platform of the cloud server, and the loss function weight of the defect detection model is adjusted by the amount of training set data; the cloud platform generates corresponding training data according to various types of data in the index database; the defect detection model learns and trains the training data in the computing power unit of the cloud server; when the defect detection model performs initial parameter training, the corresponding training data is extracted from the index database to construct a training set to train the defect detection model; when the defect detection model performs full parameter training, the corresponding training data is extracted from the index database to construct a training set and a test set to train the defect detection model.

[0015] In an optional implementation, the category of training data input to the defect detection model is adjusted so as to train the defect detection model to output detection results of corresponding categories.

[0016] In a third aspect, an embodiment of the present disclosure further provides a computer-readable storage medium having a computer program / instruction stored thereon, which implements the steps of the above-mentioned PCB circuit defect detection method when the computer program / instruction is executed by a processor.

[0017] The beneficial effect of the present invention is that the present invention can achieve isolation of production, data, and calculations by setting up a cloud storage unit to obtain data unidirectionally, and at the same time, the cloud computing unit realizes the operation of the defect detection model. It can also effectively allocate computing power and increase data processing capacity to meet production line detection needs. At the same time, it can also timely optimize and update the index database and defect detection model to improve detection accuracy and efficiency.

[0018] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention are realized and obtained by the structures particularly pointed out in the description, claims and drawings.

[0019] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0021] Figure 1 A principle block diagram of a defect detection system provided by an embodiment of the present disclosure; Figure 2 A schematic diagram of a defect detection model provided by an embodiment of the present disclosure; Figure 3 The embodiments of the present disclosure provide Figure 2 Schematic diagram of module 1; Figure 4 The embodiments of the present disclosure provide Figure 2 Schematic diagram of module 2; Figure 5 A flowchart of a training defect detection model provided in an embodiment of the present disclosure; Figure 6 A flow chart of a defect detection model measured in an embodiment of the present disclosure; Figure 7 A flow chart of actual measurement of a defect detection system provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0023] The terms used herein are only used to describe specific exemplary configurations and are not intended to be limited. As used herein, the singular articles "one", "an" and "the" may also be intended to include plural forms, unless it is clearly indicated above that this is not the case. The terms "comprise", "include" and "have" are inclusive, and therefore specify the presence of features, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, elements, components and / or combinations thereof. The method steps, processes and operations described herein should not be interpreted as necessarily requiring them to be performed in the specific order discussed or shown, unless specifically identified as an execution order. Additional or alternative steps may be adopted.

[0024] As used herein, the phrases "in one embodiment," "according to one embodiment," "in some embodiments," and the like generally refer to the fact that the particular feature, structure, or characteristic following the phrase may be included in at least one embodiment of the present disclosure. Therefore, a particular feature, structure, or characteristic may be included in more than one embodiment of the present disclosure, so that these phrases do not necessarily refer to the same embodiment. As used herein, the terms "example," "exemplary," and the like are used to "serve as an example, instance, or illustration." Any implementation, aspect, or design described herein as "example" or "exemplary" is not necessarily to be construed as preferred or superior to other implementations, aspects, or designs. On the contrary, the use of the terms "example," "exemplary," and the like is intended to present concepts in a concrete manner.

[0025] False points refer to defect points that are identified by traditional optical inspection equipment but do not actually affect the performance of the PCB and its subsequent use.

[0026] True points refer to defect points that are identified by traditional optical inspection equipment and will indeed affect the performance and subsequent use of the PCB.

[0027] The miss rate refers to the proportion of real points that are misjudged as false points to all defective points.

[0028] The shielding rate refers to the proportion of defect points that are accurately identified as false points to all false points.

[0029] Gerber (GB) diagram refers to the PCB schematic diagram rendered from the engineering file of PCB production, which guides the standard appearance of the PCB product after specific steps.

[0030] Copper surface, substrate, and hole refer to the components of the inspection target. When the PCB is produced and inspected by optical inspection equipment, the product only includes the plate (substrate) used for insulation and support, the metal circuit (copper surface) on the surface, and the drill holes (holes) at the corresponding positions.

[0031] MongoDB is a document-type database, in which each data is stored in the format of a document, denoted as a document.

[0032] Convolutional Neural Network (CNN) refers to Convolution Neural Network, a deep neural network for image data processing.

[0033] EfficientNet-FPN is a CNN deep neural network structure.

[0034] The spatial\channel attention mechanism refers to a weight calculation method in deep neural networks, which is the application of the self-attention mechanism in convolutional neural networks.

[0035] FC refers to the fully connected layer, which is a computing unit in a deep neural network that maps calculation results to multiple categories.

[0036] The loss function refers to the function used to calculate the difference between the model results and the actual results. The process of model training is the process of reducing the value of the loss function.

[0037] The optimizer is a container for model training. The model parameters are input into the optimizer, and the optimizer will optimize the model parameters in the direction of reducing the loss function value based on the loss function value calculated by the model each time.

[0038] Batch refers to the batch of model training at the same time, and the number of data contained in a batch is recorded as BatchSize.

[0039] The training set or test set is a data set used to train a model or test a model.

[0040] Autoencoder (AE) refers to Auto Encoder, a generative deep neural network used to generate more complex image data.

[0041] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0042] Some embodiments of the present invention are described in detail below in conjunction with the accompanying drawings. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0043] like Figures 1 to 7 As shown, at least one embodiment provides a defect detection system, which includes: at least one optical detection device, a cloud server and an all-in-one machine; wherein each of the optical detection devices uploads raw data to the cloud server respectively, so as to form label data corresponding to the raw data in the cloud server, and constructs a defect detection model in the cloud server and trains and optimizes it; the cloud server sends the optimized defect detection model to the all-in-one machine; and when the optical detection device sends the acquired data to the all-in-one machine, the all-in-one machine identifies the data through the defect detection model, and performs deep convolution on the data through the defect detection model to output the deep convolution result as follows: ;and ; among them, among them, X is a three-dimensional real number matrix representing the image features, H For the input image width, W To input the image height, C in is the number of channels of the input image, K is the convolution kernel size, ( i ,j ) is the coordinate of the output feature map, c is the channel index; the data is convolved point by point through the defect detection model to output the point by point convolution result: ;and ;in, C out is the number of channels of the output result, ( i , j,c ) is the coordinate of the depth convolution result, c’ is the output channel index; the final feature map is output by the defect detection model: Y(i,j,c ’ ) = Swish (Y point (i, j, c ’ ）） ;and Swish(x)=x×σ(βx) ,in σ represents the sigmoid function, β is a learnable parameter; the defect detection model generates a corresponding judgment result through the final feature map to output it to the optical detection device.

[0044] Specifically, each optical detection device is wirelessly connected to the cloud server and the all-in-one machine through a network terminal. At the same time, the all-in-one machine can also be connected to the cloud server to realize edge computing and storage of data.

[0045] In at least one embodiment, by setting up a cloud storage unit to obtain data unidirectionally, and at the same time using a cloud computing unit to perform operations on the defect detection model, it is possible to achieve isolation of production, data, and operations, effectively ensuring the security and stability of the data, and effectively allocating computing power, thereby increasing data processing capacity to meet production line inspection needs. At the same time, it is also possible to timely optimize and update the index database and defect detection model to improve detection accuracy and efficiency.

[0046] In at least one embodiment, the cloud server includes: a cloud platform, a cloud storage unit and a cloud computing unit; the cloud platform obtains the original data sent by each optical detection device, and annotates each original data to form corresponding label data, and the cloud platform constructs a corresponding index database through the original data and label data; the cloud platform splits the original data and saves it in the cloud storage unit and the cloud computing unit, and the cloud platform splits the label data and saves it in the cloud storage unit and the cloud computing unit.

[0047] Specifically, the cloud storage unit may be a storage server for implementing data storage.

[0048] Specifically, the cloud computing unit can provide computing power processing data for the GPU server.

[0049] Specifically, the index database adopts a mode that separates original image data from label data, wherein the original image data is stored in a triple mode of the database MongoDB framework, local image files, and local binary files, while the label data is stored in a dual mode of the MongoDB framework and local binary files.

[0050] In at least one embodiment, the cloud platform saves the document information in the original data to a cloud storage unit, and the cloud platform saves the images and binary files in the original data to a cloud computing unit; the cloud platform saves the document information in the tagged data to a cloud storage unit, and the cloud platform saves the binary files in the tagged data to a cloud computing unit.

[0051] Specifically, in the MongoDB framework, each piece of original image data corresponds to a document (document information) in the MongoDB framework. The document will record all the information of the image, including production date, material number, board number, front and back sides, index, etc., and match the corresponding Gerber image slice; the MongoDB framework will generate a unique _id for each document, which can be used to completely locate the image corresponding to the index, and the document is saved in the cloud storage unit.

[0052] Specifically, the images in the original image data are all saved in a fixed file path, and every 1,000 images are assigned to a subfolder path. The images correspond one-to-one to the document information in the MongoDB database, and the images are saved in the cloud computing unit.

[0053] Specifically, the binary files in the original image data are packaged files of the data in the subfolders corresponding to every 1,000 images under the image path. The binary files correspond one-to-one to the subfolders under the image path, and all binary files are stored in the cloud computing unit.

[0054] Specifically, in the MongoDB framework, the tag corresponding to each original image data is recorded as a document in MongoDB (document information in the tag data). Each document contains an index id and tag information. The index id is used to find the corresponding original image data in MongoDB. The index id corresponds to the _id of the original image data in MongoDB one by one and is stored in the cloud storage unit.

[0055] Specifically, multiple copies of binary files in the tag data are retained, and the files are named by date to record the information of each tag modification. For example, the "20240801tags.pkl" file is modified relative to the "20240731tags.pkl". Each binary file contains the index id and tag tuple of the image and is stored in the cloud computing unit.

[0056] Specifically, the separation of original image data and label data can effectively prevent accidental data loss, while increasing the speed of repeated label modification, making it convenient to quickly generate label copies, label migration, and data set reading and construction.

[0057] Specifically, optical inspection equipment is used in the PCB manufacturing process after the inner and outer circuits are etched. Its inspection target is the copper line defects of the PCB, mainly including open circuits, short circuits, incomplete etching, poor exposure, and other series of defects that have a serious impact on the functionality of the circuit. According to the PCB production process, combined with the defect manifestations and actual needs encountered in actual production, 135 categories in 3 categories and 5 levels were formulated.

[0058] Specifically, the classification definition information is recorded in the configuration file of the cloud platform and indexed by a 5-tuple, for example, (1, 0, 1, 17, -1) corresponds to the 1st level label (copper surface anomaly), the 0th level label (three-dimensional structure), the 1st level label (irregular structure), and the 17th level label (plugged hole).

[0059] Specifically, the cloud platform will add a special label of "all copper" to the data covered by normal copper surfaces in the scanned images in the index database to generate a copper surface texture database.

[0060] Specifically, the cloud platform will add a special label of "full substrate" to the data in the index database that all scan images are normal substrate surfaces, and generate a substrate texture database.

[0061] Specifically, the cloud platform will add a special label of "large hole" to the data in the index database where more than 90% of the scanned images are covered by holes, and generate a hole texture database.

[0062] Specifically, the cloud platform labels the data in the index database, and can realize functions such as data initial labeling, labeling review, labeling training, job distribution, statistics, etc. The previous day's job demerits are collected for review and evaluation, and the labeled data is finally uploaded to MongoDB, and the binary file of the data label for the day is updated at the same time.

[0063] In at least one embodiment, see Figures 2 to 4The cloud platform constructs a defect detection model with the EfficientNet_b0 architecture and the FPN structure, and adjusts the weight of the loss function of the defect detection model by weighting the amount of training set data; the cloud platform generates corresponding training data according to various types of data in the index database; the defect detection model learns and trains the training data in the cloud computing unit until the cloud platform sends the defect detection model to the all-in-one machine.

[0064] Specifically, EfficientNet-FPN structure + self-attention: The EfficientNet deep neural network has been proven to be able to effectively extract graphic features of image data. At the same time, the FPN structure can improve the model's recognition rate for small target detection in images. The spatial\channel self-attention mechanism increases the model's weight distribution when identifying spatial table\color channels, further increasing the detection accuracy.

[0065] Specifically, a defect detection model is constructed with the EfficientNet_b0 architecture and FPN structure to improve the recognition accuracy of small target detection. At the same time, the spatial + channel attention mechanism is added to improve the model's ability to extract spatial\channel information weights.

[0066] See also Figure 2 The EfficientNet_b0 architecture has a total of 7 modules. The FPN structure is added after the 2nd, 3rd, 4th, 5th, and 6th layers. Each layer of the structure outputs a 256-dimensional vector, and a total of 1024-dimensional vectors enter the FC layer.

[0067] Specifically, see Figure 2 , Figure 2 The starting layer of the network is responsible for converting the original input image into a higher-level feature representation; the executable matrix includes several executable modules; the modules are used to implement a processing sequence of a certain mathematical calculation process; the terminal layer of the network is a fully connected layer, which is responsible for mapping the network results to the target structure.

[0068] Specifically, the image is preprocessed first, the original image is scaled and adjusted to the resolution most suitable for the model. The original resolution 108x108 is scaled to 128x128, and the scaling uses a bilinear interpolation algorithm.

[0069] Specifically, see 3, Figure 3 In the two-dimensional convolution of the middle channel, the convolution kernel calculates each channel, but the result is not channel fused; batch normalization, regularization is performed on the calculated data of each batch.

[0070] Specifically, see Figure 4 , Figure 4Padding the edges of the image with 0, that is, expanding the edges of the image and filling them with 0.

[0071] Specifically, the feature information of the image is extracted through depth convolution, pointwise convolution, activation function, and residual connection.

[0072] Specifically, depth convolution means applying a convolutional kernel to each input channel separately to extract spatial features, reducing the number of parameters and computational complexity.

[0073] Specifically, during depth convolution, the input feature map is applied to the depth convolutional kernel to output the depth convolution result.

[0074] Specifically, the input feature map is ; where X is a three-dimensional real number matrix representing the image features, H is the width of the input image, W is the height of the input image, C in is the number of channels of the input image.

[0075] Specifically, the depth convolutional kernel is , where each channel has an independent convolutional kernel; K is the convolutional kernel size.

[0076] Furthermore, the depth convolution result is , that is ; where ([[]] i , j ) are the coordinates of the output feature map, c is the channel index.

[0077] Specifically, pointwise convolution means using a 1x1 convolutional kernel for cross-channel combination and non-linear transformation to adjust the feature representation and further optimize the network performance.

[0078] Specifically, during pointwise convolution, the input depth convolution result is applied to the pointwise convolutional kernel to output the pointwise convolution result.

[0079] Specifically, the depth convolution result is .

[0080] The pointwise convolutional kernel is ; where C out is the number of channels of the output result.

[0081] Furthermore, the pointwise convolution result is , that is ; where ([[]] i , j, c ) are the coordinates of the depth convolution result, c’ is the output channel index.

[0082] Specifically, in the activation function process, the Swish function is used to introduce nonlinearity and enhance the performance of the model.

[0083] Specifically, first input the point-by-point convolution result as , and then output the final feature map as ,Right now Y(i,j,c ’ ) = Swish (Y point (i, j, c ’ ）） , and the Swish function is defined as Swish(x)=x×σ(βx) ,in σ represents the sigmoid function, β are learnable parameters.

[0084] Specifically, when the input and output have the same dimension, a residual connection is added to help gradient propagation and improve training results.

[0085] Specifically, if the input feature map X and the output feature map Y have the same dimension (i.e. C in =C out ), then a residual connection can be added as Y final =Y+X .

[0086] Specifically, through depth convolution, point-by-point convolution, activation function and residual connection, the resolution of the image will be gradually reduced, while the number of channels will be increased. After multiple steps, the original data is changed from a 128x128x3 image to a 4x4x1280 high-level feature map.

[0087] Specifically, the average value of all current results is calculated through global average pooling; renormalization is to readjust the dimension and size of the data.

[0088] Specifically, global average pooling can apply the global average pooling operation to the extracted high-level feature map, compress the information of the spatial dimension into a feature vector, and finally obtain a 1280-dimensional vector.

[0089] Specifically, the feature vector is mapped to each node of the category number through the fully connected layer. Here, the fully connected layer is a 1280x(1000+135)-dimensional matrix, where 1000 corresponds to the common classification of ImageNet and 135 corresponds to various classifications of PCB-AOI. Finally, the feature vector is transformed from a 1-dimensional vector of length 1280 to a 1-dimensional classification vector of length 1135.

[0090] Specifically, the Softmax function is applied to transform the result into a probability distribution, which represents the predicted probability of each category.

[0091] Specifically, the calculation process of softmax is ;in, y i and Z j They correspond to the values ​​of each element of the 1-dimensional classification vector before and after softmax, n=1135, e is the natural logarithm.

[0092] Specifically, the classification model uses the cross entropy loss function to measure the difference between the prediction and the true label, which is expressed as ;in, L represents the cross entropy loss, n is the sample size, y i is the probability distribution of the true performance, i is one-hot encoding, is the probability distribution predicted by the model.

[0093] Specifically, during the defect detection model training process, different loss weights are used for different defect classifications to ensure the recognition rate of "non-missing" defects.

[0094] Specifically, non-missing refers to a defect type that cannot be misjudged. This type of defect will have a great impact on the functionality of the PCB. To address this type of problem, data that meets the "non-missing" standard is individually marked as "non-missing", recorded as intolerable, and the "non-missing" data is given more weight during model training.

[0095] Specifically, secondary review refers to the data that is reviewed again after model training and prediction, for data whose prediction results are different from the original annotation results. This type of data is denoted as solid. This type of data has absolute accuracy, so its weight is increased during model training.

[0096] Specifically, the model training weights can usually be adjusted by adjusting the loss function weights of the "optimizer". However, Zhiyuan found that when the ratio of data volume between different categories is >20 and the number of categories is >100, the optimization effect of model training by only adjusting the parameters of the "optimizer" is very low. This is because in a single Batch training, the maximum value of the Batch Size is usually limited by the video memory and data size. Under a limited BatchSize, it is very easy for the weight value of a single category to be equivalent to the BatchSize in a single Batch training. In this case, the training results under the Batch will inevitably be determined by the high-weight classification, which is very unfavorable for the model to learn the graphic features of different categories.

[0097] Specifically, the optimizer is used to update parameters, minimize losses and improve model performance. Its core steps include: first-order moment estimation, second-order moment estimation, bias correction and parameter update.

[0098] Specifically, the first-order moment estimate (momentum) is m t =β 1 m t-1 +(1-β 1 )g t ; in, m t is the first-order moment estimate at time step t, β 1 Set to 0.9, g t is the current gradient.

[0099] Specifically, the second moment estimate (variance) is V t =β 2 v t-1 +（1-β 2 )g2 t ; in, V t is the second-order moment estimate of the time step, β 2 Set to 0.999.

[0100] Specifically, the bias correction includes: .

[0101] Specifically, parameter updating includes: ;in, θ are model parameters, η is the initial learning rate, ∈ is a very small number (set to 1 here e -8) to prevent division by zero errors.

[0102] Specifically, weighting the amount of training set data means that when constructing the training set, all the data to be trained are read into RAM and converted to a new format. At the same time, an index variable index is added, which is a mapping set of M->N (M>N), where N is an integer from 0 to N-1, corresponding to the sequence number of N training data in the training set; M is an integer from 0 to M-1, corresponding to the design that the nth picture data needs to be indexed at the mth position. In this way, weighting of different categories can also be achieved, while avoiding local gradient iteration anomalies caused by severe imbalance in data proportions.

[0103] Specifically, OK data generation means that the original image data collected from the production line are all production data after being sliced ​​by the optical inspection equipment, and are judged as NG by the optical inspection equipment. Therefore, the original NG data in the index database usually has slight defects (does not constitute an NG judgment). At the same time, the original OK data and NG data in the index database do not have a direct correlation, which will lead to the defect detection model being difficult to learn the graphic difference characteristics of the OK and NG data when training the defect detection model. Instead, it is more inclined to learn the characteristics of the image itself, which is not conducive to the defect detection model to identify OK and NG. Based on this, the method of generating the corresponding perfect OK image according to the Gerber image of the defect image is used to enrich the index database, and the parts corresponding to the "copper surface", "substrate" and "hole" in the Gerber image are randomly filled with textures in the texture database. At the same time, for the "copper surface" part, random color temperature adjustment is added on the basis of the original color temperature to enrich a variety of lighting conditions, and light and shadow textures are added at the junction of different parts to simulate real shooting scenes.

[0104] Specifically, NG data generation refers to the discovery, when analyzing the NG raw data, that the graphs of some NG defect classification data are very simple. For example, the image data of blind holes and skew holes has a very high degree of repetition in the index database, resulting in less than 10% of the actual effective training data. For example, for defects such as triangular scratches, each scanned image is different. Defect classification caused by process abnormalities will cause a large number of repetitive defects, while defects caused by humans are not repetitive. Therefore, a separate NG data generation is added for defects caused by process abnormalities. In addition to using the same rendering method of "copper surface", "substrate" and "hole" as OK data generation, the rendering of the defective part is added. For simple defect generation, the defect range is marked on the Gerber image and filled with the adjusted "copper surface" or "hole" texture; for complex defect generation, a separate autoencoder (AE) generation model is trained for each defect type to generate defects.

[0105] Specifically, the generation of OK data or NG data, generating target data based on actual data, can not only increase the richness of data graphic features, but also increase the randomness of the background to improve the generalization ability of the deep neural network model. At the same time, the generated data can effectively make up for the amount of data that is more difficult to obtain directly, so as to balance the training set of the model.

[0106] Specifically, the defect detection system can adapt to various optical inspection equipment, and with the help of powerful algorithm capabilities, it can quickly and intelligently complete the identification and classification of defect points with extremely low leakage rate and high shielding rate. The defect detection system provides powerful computing power support and adapts and optimizes the defect detection model, so that the defect detection model can run with higher efficiency. In addition, the defect detection system is also the data center of the production line. All defect point images will be retained and classified in the defect detection system. Users can intuitively understand the operation of the production line through the data center, and can optimize and adjust the production line in a targeted manner according to the type of situation. It can even track problems that are not discovered until after leaving the factory, locate specific products, and recall problematic products. The defect detection system also reserves upgrade space to cope with the subsequent increase in computing power demand for intelligent manufacturing, supports remote control upgrades, and can quickly and conveniently complete algorithm model updates to obtain better detection accuracy and faster detection efficiency.

[0107] In at least one embodiment, the all-in-one machine includes: an industrial computer, a storage medium and a GPU computing core; when the industrial computer obtains the data sent by the optical inspection device, the GPU computing core identifies the data through a defect detection model to send the judgment result to the industrial computer; the industrial computer saves the data and the corresponding judgment result to the storage medium, and the industrial computer returns the judgment result to the optical inspection device.

[0108] Specifically, the storage medium may be a hard disk or a virtual storage medium, and the GPU computing core may be a GPU processor.

[0109] Specifically, cloud storage is used to manage production data. The result data of the optical inspection equipment is transmitted through the local area network and saved in the storage medium. After the industrial computer detects the new data, it will read the data and send it to the GPU computing core. After the GPU computing core completes the calculation, it will read the result and return the result to the storage medium. In this process, data only flows from the optical inspection equipment to the storage medium in one direction. Users cannot read data directly from the storage medium and need to obtain permissions through the industrial computer. When the plate repair machine and the manual re-inspection station read the data, an application must be submitted to the industrial computer, and the data can only be read after the permission is confirmed. It can be deployed in a location far away from the production line to achieve isolation between people and data and ensure data security. At the same time, the industrial computer will regularly detect the status of the storage medium, clean up invalid data, and maintain the health of the storage medium.

[0110] Specifically, the whole machine solution can provide sufficient AI computing power, customize the whole machine configuration according to the user's production capacity and data volume, and fully call on the capabilities of the GPU computing core during calculations to maximize computing efficiency.

[0111] Specifically, the whole machine provides a complete upgrade solution. You can submit requirements and provide upgrade solutions for the scenarios you encounter by logging into the cloud service platform. The data\computing power separation mode is adopted to facilitate equipment upgrades according to specific needs. At the same time, it also provides defect detection model upgrade services. According to the submitted demand sheet, the defect detection model is remotely upgraded online, or the defect detection model is downloaded through the service platform to complete the offline upgrade.

[0112] Specifically, the AI ​​calculation results will retain sufficient analysis result information. At the same time, the industrial computer provides complete data analysis and data visualization functions. The data of a certain day in history can be retrieved at any time, and the AI ​​can be used to complete the screening and analysis to optimize the production process.

[0113] Specifically, a PCB production line is equipped with 4 AOI inspection lines according to its production capacity. Each inspection line generates about 1 million inspection images every day and needs to meet the following requirements: the defect detection model is updated with new data in the early morning every day; each production line needs to process 500 inspection requests per minute in real time during the day (peak value 1000 images per minute), and the cloud server is equipped with at least 4 NVIDIA RTX GEFORCE 4090 GPUs, and each all-in-one machine is equipped with 1 NVIDIA RTX GEFORCE 3060 GPU, with a total of 4 all-in-one machines deployed.

[0114] In the training phase, computing power allocation is used to build a defect detection model with EfficientNet_b0 architecture and FPN structure, which can achieve an image input size of 108×108 and a batch size (BatchSize) = 4096. The amount of data for a single training is about 500,000 images based on the initial data volume and daily new data, that is, 112 steps / epoch (1,000,000 / 4096≈1562), and the training round is 300epochs. Single card computing power: RTX4090FP32 computing power ≈ 82.6TFLOPS; model FLOPs: EfficientNet_b0 single-image reasoning FLOPs ≈ 0.39GFLOPs, the amount of back propagation calculation during training is about 2-3 times that of the forward one, taking 1.17GFLOPs / image; total computing power: 1,000,000 images × 300epochs × 1.17GFLOPs = 335,100,000GFLOPs ≈ 335.1 PFLOPs; Single card training time (FP32): (335.1 PFLOPs / 82.6 TFLOPS) seconds ≈ 4057 seconds ≈ 1.13 hours; Multi-card parallelism: 4-card parallelism (data parallelism) efficiency ≈ 90%, then the total time: (1.13 / 4) × (1 / 0.9) ≈ 0.31 hours (adding the delay time of data IO, it still meets the 4-hour requirement and resources are sufficient); Cloud computing unit: 0:00-4:00 at night, 4 4090 GPUs run training tasks at full load.

[0115] In the inference phase, computing power allocation requires peak throughput of 1,000 images / minute ≈ 17 images / second; single-image inference FLOPs: 0.39 GFLOPs; 3060 GPU performance: 12.7 TFLOPS (FP32) → theoretical maximum throughput: {(12.7×10 12 FLOPs / second) / (0.39×10 9 FLOPs / picture)}≈32,564 pictures / second; actual throughput (considering I / O and scheduling): about 5,000 pictures / second / card (conservative estimate); single all-in-one machine demand: peak 17 pictures / second → single 3060 can easily handle (5,000 pictures / second ≫ 17 pictures / second); redundant design: each all-in-one machine is responsible for one detection line, with a throughput of 5,000 pictures / second, far exceeding the peak demand (17 pictures / second); at the same time, the industrial computer assigns tasks to idle GPUs according to the real-time request volume, and detection tasks marked as "cannot be missed" are given priority.

[0116] Specifically, when a conflict in computing power allocation occurs, that is, an urgent model update is required during the day, which may occupy cloud server resources, the all-in-one GPU needs to process reasoning and data storage at the same time, optimize the computing power allocation, and reuse the cloud computing power in time: training is only performed at night, and the cloud server GPU can assist in edge computing when it is idle during the day (network bandwidth support is required), and elastic cloud computing (such as AWS / GCP temporary instances) is enabled for emergency training tasks; use TensorRT to deploy the inference engine, optimize GPU utilization, and complete preprocessing (such as image normalization) in parallel on the CPU to reduce GPU idle time.

[0117] Based on the same technical concept, at least one embodiment also provides a PCB circuit defect detection method, which includes: uploading the original data to the cloud server through each optical inspection device respectively to form label data corresponding to the original data in the cloud server, and building a defect detection model in the cloud server and training and optimizing it; sending the optimized defect detection model to the all-in-one machine through the cloud server; when the optical inspection device sends the acquired data to the all-in-one machine, the all-in-one machine identifies the data through the defect detection model and outputs the judgment result to the optical inspection device.

[0118] In at least one embodiment, the original data sent by each optical detection device is acquired through a cloud platform in a cloud server, and each original data is annotated to form corresponding label data, and the cloud platform builds a corresponding index database through the original data and the label data; the original data is split and saved in a cloud storage unit and a cloud computing unit in the cloud server through the cloud platform, and the label data is split and saved in a cloud storage unit and a cloud computing unit through the cloud platform.

[0119] In at least one embodiment, a defect detection model is constructed in a cloud platform of a cloud server using an EfficientNet_b0 architecture and an FPN structure, and the weight of the loss function of the defect detection model is adjusted by weighting the amount of training set data; the cloud platform generates corresponding training data based on various types of data in an index database; the defect detection model learns and trains the training data in a computing power unit of the cloud server; when the defect detection model performs initial parameter training, the corresponding training data is extracted from the index database to construct a training set to train the defect detection model; when the defect detection model performs full parameter training, the corresponding training data is extracted from the index database to construct a training set and a test set to train the defect detection model.

[0120] In at least one embodiment, see Figure 5When the defect detection model performs initial parameter training, the corresponding training data is extracted from the index database to construct a training set to train the defect detection model; when the defect detection model performs full parameter training, the corresponding training data is extracted from the index database to construct a training set and a test set to train the defect detection model.

[0121] Specifically, when all parameters are trained, the training set refers to all "cannot be missed" and "secondary review" data, and ordinary data is added at the same time. Ordinary data is selected from the training set and the test set based on similarity. The training set data balances the amount of data for each category, with the largest category and the smallest category, and the data volume ratio is <5. During the training process, the results of the defect detection model will be checked to find out the data that the defect detection model does not predict accurately. If the data label is wrong, correct the label; if the data label is correct, increase the repetition of the data; if the above operation is not effective, reinitialize the defect detection model parameter training, and add the checked labels to the "secondary review" data set. After the above steps, the parameters of the defect detection model are trained from the initial random parameters to fully and accurately identify the "cannot be missed data". At the same time, the weights of "cannot be missed" and "secondary review" in the training set are increased, "cannot be missed" is repeated 5 times, and "secondary review" is repeated 3 times; the test set refers to the test set selected based on the similarity of ordinary data. It is found that the data in the original image database has a high repetitiveness. If the training set\test set is divided only by random screening, a large amount of repeated data will enter the training set. This not only ineffectively occupies computing resources, but also causes a particular image to have too high a weight. Therefore, the training set and test set are divided by image similarity screening. The eigenvalues ​​of each image data are calculated using the pre-trained model on the non-PCB database, and the Euclidean distance or Minder distance between the eigenvalues ​​is calculated. The similarity of the images is confirmed based on the size of the distance, and a threshold is set to screen similar data. During the training process, attention is paid to the accuracy of the training set, the accuracy of the test set, the accuracy of "cannot be missed", and the prediction probability. When the minimum prediction probability of "cannot be missed" is <50%, training should be stopped in time to check the label accuracy of the data. If the label is wrong, correct the label; if the label is correct, increase the repetition of the data. The checked labels are added to the "secondary review" data set.

[0122] In at least one embodiment, the category of training data input into the defect detection model is adjusted so as to train the defect detection model to output detection results of corresponding categories.

[0123] Specifically, parameter fine-tuning can effectively solve the OK\NG output problem in special scenarios. This method has low requirements on the cost of model training and it is very easy to adjust the overall performance of the model based on only a small sample in a short period of time.

[0124] Specifically, multi-classification output adjustment: In some application scenarios, there will be more detailed requirements for specific defect classifications, so it is necessary to adjust the already designed classification result output. Based on this, when training the defect detection model, all parameters before the FC layer will be frozen, and the parameters of the FC layer will be randomly initialized and trained. When the defect detection model results converge, all parameters are added to the training until the defect detection model results are stable.

[0125] Specifically, OK\NG results are directly output: Most application scenarios only require the defect detection model to provide OK\NG result outputs. Therefore, the FC layer is replaced with a binary classification output instead of a multi-classification output, and all parameters before the FC layer are frozen. The parameters of the FC layer are randomly initialized and trained until the defect detection model converges.

[0126] Specifically, the composite classification weights are adjusted: Under two extreme conditions, the defect detection model gives correct predictions, but incorrect OK\NG results. When two or more graphic features appear in different positions in the image, the defect detection model will be more inclined to the result with larger and more obvious features; when two or more graphic features appear in the same position in the image, and these graphic features merge to form another graphic feature, the defect detection model will not be able to recognize the newly generated graphic feature; in the above two cases, the graphic recognition ability of the defect detection model is accurate, so a new FC layer is added after the multi-classification output results, and a special data set is built for the above two special scenarios to train the newly added FC parameters.

[0127] Specifically, during the iteration process, the defect detection model will continuously review the accuracy of the existing database, correct the results at any time, and add a "second review" data set. The "second review" data will participate in the training process with a high weight. If there is data with the same label as the "second review" data, due to the different training weights, such data will be misidentified during the training process regardless of whether it is in the training set or the test set. The misidentified data will be modified after the "second review" by the personnel, and will participate in the next training with a high weight again. In this way, the index database can be optimized during the continuous training process of the defect detection model.

[0128] Specifically, see Figure 6 to Figure 7 , automatically obtain the data of the optical inspection equipment, and obtain the prediction results after defect detection model inference. The prediction results will be compared with the results of the first manual marking to obtain the data of the missing part (result comparison). The missing data will be reviewed again, and the real missing part will be selected (loaded missing) for subsequent operations. The actual effect of the defect detection model on the production line can be obtained, so as to make improvements.

[0129] Specifically, the use of semi-supervision and label review methods can improve the efficiency of data labeling and simultaneously optimize data and model parameters during the continuous iteration of the defect detection model.

[0130] Based on the same technical concept, at least one embodiment further provides a computer-readable storage medium on which a computer program / instruction is stored. When the computer program / instruction is executed by a processor, the steps of the above-mentioned PCB line defect detection method are implemented.

[0131] To summarize, the present invention sets a cloud storage unit to acquire data unidirectionally, and at the same time the cloud computing unit realizes the operation of the defect detection model, which can not only realize the isolation of production, data, and operation, effectively ensure the security and stability of the data, but also effectively allocate computing power and increase the data processing volume to meet the production line detection needs, and at the same time, it can timely optimize and update the index database and defect detection model to improve the detection accuracy and efficiency; the index database optimization process is fast; the defect detection model can give corresponding weighted outputs for problems of different degrees; the whole machine is easy to deploy, highly adaptable, data secure, and easy to manage; in terms of the most critical leakage index and shielding rate index, based on the full data of a single production line in a factory, the true point leakage rate of less than 1% and the shielding rate of more than 50% can be achieved. Through this indicator, 50% of the manual inspection workload can be saved, and the quality of the production line can be greatly improved.

[0132] The disclosure and other solutions, examples, embodiments, modules and functional operations described in this document can be implemented in digital electronic circuits, or computer software, firmware or hardware, including the structures disclosed in this document and their structural equivalents, or a combination of one or more thereof. The disclosure and other embodiments can be implemented as one or more computer program products, i.e., one or more modules of computer program instructions encoded on a tangible and non-volatile computer-readable medium for execution by a data processing device or to control the operation of the data processing device. The computer-readable medium can be a machine-readable storage device, a machine-readable storage substrate, a storage device, a material composition that affects a machine-readable propagation signal, or a combination of one or more thereof. The term "data processing unit" or "data processing device" includes all devices, equipment and machines for processing data, including, for example, a programmable processor, a computer or a multiprocessor or a computer group. In addition to hardware, the device may also include code that creates an execution environment for a computer program, for example, code constituting processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more thereof. A propagated signal is an artificially generated signal, for example, a machine-generated electrical, optical or electromagnetic signal, which is generated to encode information for transmission to a suitable receiver device.

[0133] A computer program (also referred to as a program, software, software application, script, or code) may be written in any form of programming language (including compiled or interpreted languages) and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program may be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program, or in multiple coordinated files (e.g., files that store one or more modules, subroutines, or portions of code). A computer program may be deployed for execution on one or more computers that are located at one site or distributed across multiple sites and interconnected by a communications network.

[0134] The processes and logic flows described in this document may be performed by one or more programmable processors executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows may also be performed by, and the apparatus may also be implemented as, special purpose logic circuits, such as an FPGA (field programmable gate array) or an ASIC (application specific integrated circuit).

[0135] For example, processors suitable for executing computer programs include general and special purpose microprocessors, and any one or more of any type of digital computer. Typically, the processor will receive instructions and data from a read-only memory or a random access memory or both. The basic components of a computer are a processor that executes instructions and one or more storage devices that store instructions and data. Typically, a computer will also include one or more mass storage devices for storing data, such as magnetic disks, magneto-optical disks, or optical disks, or operably coupled to receive data from a mass storage device or transfer data to a mass storage device, or both. However, a computer does not necessarily have such a device. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, for example, semiconductor memory devices, such as erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), and flash memory devices; magnetic disks, such as internal hard disks or removable disks; magneto-optical disks; and compact disk read-only memory (CD ROM) and digital versatile disk read-only memory (DVD-ROM) disks. The processor and memory can be supplemented by dedicated logic circuits, or incorporated in dedicated logic circuits.

[0136] Although this patent document contains many details, it should not be construed as limiting any invention or the scope of any claim, but rather as a description of the features of particular embodiments of a particular invention. Certain features that are described in the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, the various functions that are described in the context of a single embodiment may also be implemented separately in multiple embodiments, or in any suitable sub-combination. Additionally, although the above features may be described as acting in certain combinations, and even initially claimed as such, in some cases, one or more features from a claimed combination may be removed from the combination, and the claimed combination may be directed to a sub-combination or a variant of a sub-combination.

[0137] Likewise, although the operations are depicted in the drawings in a particular order, this should not be construed to mean that such operations must be performed in the particular order shown, or in sequential order, to achieve the desired results, or that all illustrated operations must be performed. Additionally, the separation of various system components in the embodiments of this patent document should not be construed to mean that such separation is required in all embodiments.

[0138] Only some implementations and examples are described, and other implementations, enhancements, and variations can be made based on what is described and illustrated in this patent document.

[0139] A first component is directly coupled to a second component when there is no intermediate component other than a line, trace, or another medium between the first and second components. A first component is indirectly coupled to a second component when there is an intermediate component other than a line, trace, or another medium between the first and second components. The term "coupled" and its variants include both direct coupling and indirect coupling. Unless otherwise specified, the use of the term "about" means including a range of plus or minus 10% of the numerical value.

[0140] Although several embodiments are provided in this disclosure, it should be understood that the disclosed systems and methods may be embodied in many other specific forms without departing from the spirit or scope of the disclosure. The current examples are considered illustrative and not restrictive, and are not limited to the details given. For example, various elements or components may be combined or integrated in another system, or some features may be omitted or not implemented.

[0141] In several embodiments provided herein, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, program segment or a part of the code contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0142] In addition, without departing from the scope of the present disclosure, the discrete or separate techniques, systems, subsystems, and methods described and illustrated in the various embodiments may be combined or integrated with other systems, modules, techniques, or methods. Other items shown or discussed as coupled may be directly connected, or may be indirectly coupled or communicated electrically, mechanically, or otherwise through some interface, device, or intermediate component. Other examples of changes, substitutions, and alterations may be determined by those skilled in the art without departing from the spirit and scope disclosed herein.

Claims

1. A defect detection system, characterized in that: include: At least one optical inspection device, cloud server and all-in-one machine; in Each of the optical inspection devices uploads the raw data to the cloud server respectively, so as to form label data corresponding to the raw data in the cloud server, and to construct, train and optimize a defect detection model in the cloud server; The cloud server sends the optimized defect detection model to the integrated machine; as well as When the optical inspection device sends the acquired data to the integrated machine, the integrated machine identifies the data through the defect detection model; The data is deeply convolved through the defect detection model to output the deep convolution result as: ； and, ; in, X is a three-dimensional real number matrix representing the image features, H For the input image width, W To input the image height, C in is the number of channels of the input image, K is the convolution kernel size, ( i , j ) is the coordinate of the output feature map, c is the channel index; The data is convolved point by point through the defect detection model to output the point by point convolution result as: ； and ; in, C out is the number of channels of the output result, ( i , j, c ) is the coordinate of the depth convolution result, c’ is the output channel index; Output the final feature map through the defect detection model: Y(i,j,c ’ ) = Swish (Y point (i, j, c ’ ）） ； and Swish(x)=x×σ(βx) ,in σ represents the sigmoid function, β are learnable parameters; The defect detection model generates a corresponding judgment result through the final feature map to output to the optical detection device.

2. The defect detection system according to claim 1, characterized in that: The cloud server includes: a cloud platform, a cloud storage unit and a cloud computing unit; The cloud platform acquires the raw data sent by each optical detection device, and annotates each raw data to form corresponding label data, and the cloud platform constructs a corresponding index database through the raw data and the label data; The cloud platform splits the original data and saves them in a cloud storage unit and a cloud computing unit, and the cloud platform splits the label data and saves them in a cloud storage unit and a cloud computing unit.

3. The defect detection system according to claim 2, characterized in that: The cloud platform saves the document information in the original data to the cloud storage unit, and the cloud platform saves the pictures and binary files in the original data to the cloud computing unit; The cloud platform saves the document information in the tag data to the cloud storage unit, and the cloud platform saves the binary file in the tag data to the cloud computing unit.

4. The defect detection system according to claim 2, characterized in that: The cloud platform constructs a defect detection model using the EfficientNet_b0 architecture and the FPN structure, and adjusts the loss function weight of the defect detection model by weighting the amount of training set data; The cloud platform generates corresponding training data according to various types of data in the index database; The defect detection model learns and trains the training data in the cloud computing unit until the cloud platform sends the defect detection model to the all-in-one machine.

5. The defect detection system according to claim 1, characterized in that: The all-in-one machine includes: an industrial computer, a storage medium and a GPU computing core; When the industrial computer acquires the data sent by the optical inspection device, the GPU computing core identifies the data through the defect detection model to send the judgment result to the industrial computer; The industrial computer saves the data and the corresponding judgment result to a storage medium, and the industrial computer returns the judgment result to the optical detection device.

6. A PCB circuit defect detection method, characterized in that: include: The original data is uploaded to the cloud server through each optical inspection device to form label data corresponding to the original data in the cloud server, and a defect detection model is constructed, trained and optimized in the cloud server; Send the optimized defect detection model to the integrated machine through the cloud server; When the optical inspection device sends the acquired data to the integrated machine, the integrated machine identifies the data through the defect detection model and outputs the judgment result to the optical inspection device.

7. The PCB circuit defect detection method according to claim 6, characterized in that: The cloud platform in the cloud server obtains the original data sent by each optical detection device, and annotates each original data to form corresponding label data, and the cloud platform constructs a corresponding index database through the original data and the label data; The original data is split and saved in the cloud storage unit and cloud computing unit in the cloud server through the cloud platform, and the label data is split and saved in the cloud storage unit and cloud computing unit through the cloud platform.

8. The PCB circuit defect detection method according to claim 6, characterized in that: In the cloud platform of the cloud server, a defect detection model is built with the EfficientNet_b0 architecture and FPN structure, and the loss function weight of the defect detection model is adjusted by the amount of training set data; The cloud platform generates corresponding training data based on various types of data in the index database; The defect detection model learns and trains the training data in the computing power unit of the cloud server; When the defect detection model is initially trained on parameters, corresponding training data is extracted from the index database to construct a training set to train the defect detection model; When the defect detection model is fully trained on parameters, the corresponding training data is extracted from the index database to construct a training set and a test set to train the defect detection model.

9. The PCB circuit defect detection method according to claim 8, characterized in that: By adjusting the training data category of the input defect detection model, the defect detection model is trained to output the detection results of the corresponding category.

10. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: When the computer program / instruction is executed by a processor, the steps of the PCB circuit defect detection method described in claims 6-9 are implemented.

Images