Circuit board image defect segmentation and model training method, storage medium and computer equipment

Abstract

The invention discloses a circuit board image defect segmentation and model training method, a storage medium and computer equipment. The method comprises the following steps: acquiring a defect image pair; the defect image pair is utilized to perform iterative training on the to-be-trained segmentation model, and a training mode of each round in the iterative training comprises the following steps: fusing the features of the PCBA defect image and the features of the template image through a gating fusion network in the to-be-trained segmentation model to obtain fused features; the current round of training is carried out on the to-be-trained segmentation model based on the fusion features, so that the detection capability of tiny defects in the PCBA image can be improved, the comparison detection capability of logic defects such as wrong parts of elements can be improved, and the detection efficiency can be improved. The trained model can utilize the gating fusion network to automatically learn the weight so as to automatically adjust the fusion proportion of the original image defect features and the template prior features, so that the combined use of the two types of images is further enhanced.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a circuit board image defect segmentation method and model training method thereof, a storage medium, and a computer device. Background Art

[0002] Printed Circuit Board Assembly (PCBA) is a core component of electronic devices. During its manufacturing process, defects such as cold solder joints, scratched gold fingers, broken circuits, component offset, and component misplacement are prone to occur.

[0003] Related technologies typically use single-branch deep learning segmentation networks (such as those represented by U-Net++ and DeepLabV3+) to directly perform end-to-end defect segmentation on physical images. However, this technology has poor detection capabilities for small defects (such as circuit cracks with a width of less than 0.1mm) and lacks the ability to detect logical defects such as component misalignment. Summary of the Invention

[0004] In order to solve the above technical problems, the embodiments of the present application propose a circuit board image defect segmentation and its model training method, storage medium and computer equipment, which can improve the detection ability of tiny defects in PCBA images and improve the comparative detection ability of logical defects such as component mismatch.

[0005] In a first aspect, an embodiment of the present application provides a method for training a circuit board image defect segmentation model, comprising:

[0006] Acquire a defect image pair, wherein the defect image pair includes a PCBA defect image and its corresponding template image;

[0007] The defect image pairs are used to iteratively train the segmentation model to be trained, wherein the training method of each round of the iterative training includes:

[0008] By using a gated fusion network in the segmentation model to be trained, the features of the PCBA defect image and the features of the template image are fused to obtain fused features; and

[0009] Based on the fusion features, the segmentation model to be trained is trained in a current round.

[0010] Optionally, acquiring the defect image pair includes:

[0011] Get the original PCBA image;

[0012] Based on the original PCBA image, the template image is obtained by using template matching technology;

[0013] The PCBA defect image corresponding to the original PCBA image is obtained, wherein the PCBA defect image is obtained by marking defects in the original PCBA image.

[0014] Optionally, the segmentation model to be trained further includes a feature extractor, the feature extractor is obtained through pre-training, and its parameters remain unchanged after the pre-training is completed;

[0015] Before fusing the features of the PCBA defect image and the features of the template image, the training method further includes:

[0016] Based on the PCBA defect image and the template image, feature extraction is performed by the feature extractor to obtain features of the PCBA defect image and features of the template image.

[0017] Optionally, the feature extractor is constructed based on the MIT-B1 network.

[0018] Optionally, the extracting features based on the PCBA defect image and the template image by the feature extractor to obtain features of the PCBA defect image and features of the template image includes:

[0019] Performing pixel-by-pixel coding processing based on defect types on the PCBA defect image and the template image respectively;

[0020] scaling the encoded PCBA defect image and the encoded template image according to a target resolution, wherein the target resolution is suitable for indicating a resolution of the scaled image;

[0021] Normalizing the scaled PCBA defect image and the scaled template image respectively;

[0022] The feature extractor performs feature extraction on the normalized PCBA defect image and the normalized template image, respectively, to obtain features of the PCBA defect image and features of the template image.

[0023] Optionally, the segmentation model to be trained further includes an MLP layer and a decoder of a segmentation network, the segmentation network includes a SegFormer, and the current round of training of the segmentation model to be trained based on the fusion features includes:

[0024] Upsampling the fused features through a decoder of the segmentation network, and concatenating the upsampled fused features to obtain concatenated features;

[0025] Obtaining a segmentation prediction result based on the splicing features through the MLP layer;

[0026] Based on the difference between the annotation information corresponding to the PCBA defect image and the segmentation prediction result, the segmentation model to be trained is trained in the current round using a preset loss function.

[0027] Optionally, the loss function is suitable for adjusting the weights corresponding to different defect types through joint optimization of Dice Loss and Focal Loss.

[0028] In a second aspect, an embodiment of the present application provides a circuit board image defect segmentation method, comprising:

[0029] Acquire an image pair to be identified, wherein the image pair to be identified includes a PCBA image to be identified and its corresponding target template image;

[0030] Call a pre-trained circuit board image defect segmentation model to perform defect segmentation based on the image pair to be identified, wherein the pre-trained circuit board image defect segmentation model is trained using any one of the methods described in the first aspect above.

[0031] In a third aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of any of the above-described methods when the computer program is executed by a processor.

[0032] In a fourth aspect, an embodiment of the present application provides a computer device comprising a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor implements the steps of any of the above methods when executing the computer program.

[0033] In summary, the embodiments of the present application have at least the following beneficial effects:

[0034] According to an embodiment of the present application, a defect image pair is obtained, wherein the defect image pair includes a PCBA defect image and its corresponding template image; the defect image pair is used to iteratively train the segmentation model to be trained, wherein the training method of each round in the iterative training includes: fusing the features of the PCBA defect image and the features of the template image through a gated fusion network in the segmentation model to be trained to obtain a fusion feature; and, based on the fusion feature, training the segmentation model to be trained in the current round, thereby combining the features of the template image to improve the detection capability of tiny defects in the PCBA image and improve the comparative detection capability of logical defects such as component mismatches. In addition, the trained model can use the gated fusion network to automatically learn weights to automatically adjust the fusion ratio of the original image defect features (i.e., the features of the PCBA defect image) and the template prior features (features of the template image) to further enhance the combined use of the two types of images. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 This is a flow chart of a circuit board image defect segmentation model training method provided in an embodiment of the present application;

[0036] Figure 2 This is a flow chart of the training method for each round of iterative training provided in an embodiment of the present application;

[0037] Figure 3 Schematic diagram of a circuit board image defect segmentation model provided in an embodiment of the present application;

[0038] Figure 4 This is a flow chart of a circuit board image defect segmentation method provided by an embodiment of the present application;

[0039] Figure 5 It is a structural diagram of the computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0040] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0041] In the description of this application, the terms "first", "second", "third", etc. are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, features defined as "first", "second", "third", etc. may explicitly or implicitly include one or more of such features. In the description of this application, unless otherwise specified, "multiple" means two or more. In the description of this application, the term "including" and its variations are open inclusions, i.e., "including but not limited to". The term "based on" means "at least partially based on". The term "according to" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one other embodiment"; the term "some embodiments" means "at least some embodiments".

[0042] In the description of this application, it should be noted that, unless otherwise expressly specified or limited, the terms "mounted," "connected," and "connected" should be understood in a broad sense. For example, they can refer to fixed connections, detachable connections, or integral connections; mechanical connections or electrical connections; direct connections or indirect connections through an intermediate medium; and internal connections between two components. Those skilled in the art will understand the specific meanings of the above terms in this application based on the specific circumstances.

[0043] In the description of this application, it should be noted that, unless otherwise defined, all technical and scientific terms used in this application have the same meanings as those commonly understood by those skilled in the art. The terms used in this specification are only for the purpose of describing specific embodiments and are not intended to limit this application. Those of ordinary skill in the art will understand the specific meanings of the above terms in this application in specific circumstances.

[0044] The following are some explanations of the terms and concepts involved in the embodiments of this application:

[0045] An MLP (Multilayer Perceptron), also known as a feedforward neural network, is a neural network-based machine learning model that uses multiple layers of nonlinear transformations to perform high-level abstraction and classification of input data. Compared to a single-layer perceptron, an MLP has multiple hidden layers, each composed of multiple neurons. Each neuron performs a weighted and processed input from the previous layer, and then performs a nonlinear mapping using an activation function.

[0046] First, see Figure 1 , shows a flow chart of a circuit board image defect segmentation model training method provided by an embodiment of the present application, the method comprising steps S101-S102, specifically as follows:

[0047] S101, obtaining a defect image pair, wherein the defect image pair includes a PCBA defect image and its corresponding template image;

[0048] It should be noted that the above-mentioned template image may refer to an ideal or standard PCBA image, representing an image of a PCBA without defects, which is used as a reference for comparison with defective images of PCBAs corresponding to actual production, so as to identify deviations or defects through comparison.

[0049] In one example, both the PCBA defect image and the template image may be images captured with a resolution of 256*256.

[0050] S102, using the defect image pair to iteratively train the segmentation model to be trained, wherein each round of the iterative training has a training method.

[0051] In one example, the defect image pairs may be multiple pairs, for example, at least 20 pairs. In each round of iterative training, different defect image pairs may be used to complete the training of that round. In addition, after obtaining the model that has completed the iterative training, the model may be verified using a validation set, wherein the validation set may include at least 10 pairs of "verification defect image pairs," wherein the verification defect image pairs may include PCBA defect images and template images, and the PCBA defect images and template images included in the verification defect image pairs may be different from the PCBA defect images and template images included in the defect image pairs used for iterative training. When verifying the model, the verification defect image pairs may be input into the model that has completed the iterative training to obtain the defect segmentation result (e.g., the output defect mask) output by the model, and the accuracy of the model may be verified by comparing the labeled defects (e.g., the labeled defect masks) corresponding to the verification defect image pairs with the defect segmentation result.

[0052] See also Figure 2 , shows a flow chart of a training method for each round of iterative training provided in an embodiment of the present application, the training method includes steps S201-S202, which are specifically as follows:

[0053] S201, fusing the features of the PCBA defect image and the features of the template image through the gated fusion network in the segmentation model to be trained to obtain fused features; and

[0054] In one example, see Figure 3 , gated feature fusion can be achieved through a gated fusion network. A channel attention gating mechanism can be designed in the above gated fusion network, which can automatically adjust the fusion ratio of the original image defect features and the template prior features through learnable weights.

[0055] In one example, the above-mentioned gated fusion network can be implemented by the following formula:

[0056] F used =σ(W g *[F img ,F temp ])⊙F img +(1-σ(W g *[F img ,F temp ]))⊙F temp

[0057] Among them, W g is a learnable gating weight matrix, σ(·) represents the Sigmoid activation function, ⊙ represents element-wise multiplication, and F used is the output fusion feature, F img is the feature of PCBA defect image, F temp is the feature of the template image.

[0058] S202: Based on the fusion features, perform a current round of training on the segmentation model to be trained.

[0059] In one example, it is not difficult to understand that the above-mentioned fusion features have already fused the feature information of the PCBA defect image and the template image. At this time, the segmentation model to be trained can be configured to perform defect recognition or segmentation based on the above-mentioned fusion features to generate a predicted defect segmentation result. Then, based on the difference between the defect segmentation result and the annotation information corresponding to the PCBA defect image, a general model training method can be used to complete the current round of training. The model training method is not specifically limited here.

[0060] In an optional implementation, obtaining a defect image pair includes:

[0061] Get the original PCBA image;

[0062] Based on the original PCBA image, the template image is obtained by using template matching technology. In one example, the template matching technology can be implemented using SIFT (Scale-invariant feature transform) feature matching and RANSAC (Random Sample Consensus) algorithm, thereby achieving alignment between the original PCBA image and the template image.

[0063] The PCBA defect image corresponding to the original PCBA image is obtained, wherein the PCBA defect image is obtained by marking defects in the original PCBA image.

[0064] In one example, the PCBA defect image may include an image with a defect mask obtained by annotating defects in the original PCBA image using a polygon annotation method.

[0065] In an optional embodiment, the segmentation model to be trained further includes a feature extractor, the feature extractor is obtained through pre-training, and its parameters remain unchanged after the pre-training is completed;

[0066] Before fusing the features of the PCBA defect image and the features of the template image, the training method further includes:

[0067] Based on the PCBA defect image and the template image, feature extraction is performed by the feature extractor to obtain features of the PCBA defect image and features of the template image.

[0068] In some existing technologies, the features of the PCBA defect image and the template image may be directly spliced together through an architecture represented by a twin network, such as a dual-branch network, and then defect identification / segmentation is performed based on the features obtained by direct splicing. However, this will cause non-defect differences in the template image (such as silk screen character position offset, solder mask color difference) to be forcibly introduced into the fusion features, interfering with the model's attention distribution to real defects, thereby causing a problem of noise coupling; and because the dual-branch encoder needs to be trained, the number of model parameters increases by 1.8-2.5 times, which will increase the risk of overfitting. Accordingly, in this embodiment, the above-mentioned feature extractor can freeze its parameters after completing pre-training, so that the features of the PCBA defect image and the template image are extracted separately using a feature extractor with frozen parameters, so as to retain the generalization ability of the pre-trained model.

[0069] In some existing technologies, histogram matching, gradient-based edge detection combined with morphological operations may be used to compare the physical image of the PCBA with its corresponding template image pixel by pixel, so as to locate defects based on the differences found in the comparison. However, this technology is prone to mismatching when there are factors such as uneven lighting, PCB (Printed Circuit Board) surface reflection, camera distortion, etc., and requires frequent manual calibration of threshold parameters; and because it only focuses on low-order features (such as grayscale values and gradients), it is unable to distinguish between process errors allowed by the design (such as solder mask color deviation) and real defects. Accordingly, the feature extractor in this embodiment can extract high-order image features so that high-order image features can be effectively combined to perform defect segmentation, without the need for frequent manual calibration of threshold parameters.

[0070] In one example, the parameters of the feature extractor may include ImageNet pre-trained weights, and the feature extractor may be used to output feature maps of three scales {1 / 4, 1 / 8, 1 / 16}.

[0071] In an alternative embodiment, see Figure 3 , the feature extractor is built based on the MIT-B1 network.

[0072] It should be noted that MIT stands for Mix Transformer Encoder. By varying hyperparameters, models B0 to B5 can be obtained. MIT-B0 is a lightweight prediction model, while MIT-B5 is the highest-performing and largest model. MIT originates from the Encoder in the Segformer framework. Unlike the Vision Transformer (VIT), which only generates a single feature map, this model's goal is to take an image as input and generate a multi-layered feature map similar to a CNN. These multi-layered feature maps typically provide high-resolution coarse features and low-resolution fine features, which can improve semantic segmentation performance.

[0073] In an optional embodiment, the extracting features based on the PCBA defect image and the template image by the feature extractor to obtain features of the PCBA defect image and features of the template image includes:

[0074] The PCBA defect image and the template image are each subjected to pixel-by-pixel encoding based on the defect type. In one example, assuming there are N defect types, each of which corresponds to N values, for example, from 1 to N. In this case, the pixel-by-pixel encoding process may include uniformly encoding the background of the image as non-defective as 0, and encoding the pixels in the image as defective as the corresponding value according to the defect type to which the defect belongs. It is not difficult to understand that at this point, there are N+1 categories of encoded values.

[0075] The encoded PCBA defect image and the encoded template image are scaled according to a target resolution, wherein the target resolution is suitable for indicating a resolution of the scaled image. In one example, the target resolution may be 512*512, i.e., the target resolution may indicate that the image is scaled to 512*512.

[0076] Normalizing the scaled PCBA defect image and the scaled template image respectively;

[0077] The feature extractor performs feature extraction on the normalized PCBA defect image and the normalized template image, respectively, to obtain features of the PCBA defect image and features of the template image.

[0078] In an optional embodiment, the segmentation model to be trained further includes an MLP layer and a decoder of a segmentation network, the segmentation network includes a SegFormer, and the current round of training of the segmentation model to be trained based on the fusion features includes:

[0079] The fused features are upsampled by the decoder of the segmentation network, and the upsampled fused features are spliced to obtain spliced features. In one example, the decoder of the segmentation network can upsample the fused features to the same size and splice them. At this time, the SegFormer-based decoder (such as the Mix Transformer decoder) can be used as a lightweight segmentation head, which can achieve 30ms real-time inference at a resolution of 512×512, so as to speed up the inference speed.

[0080] Obtaining a segmentation prediction result based on the splicing features through the MLP layer;

[0081] Based on the difference between the annotation information corresponding to the PCBA defect image and the segmentation prediction result, the segmentation model to be trained is trained in the current round using a preset loss function.

[0082] In an optional embodiment, the loss function is suitable for adjusting the weights corresponding to different defect types through joint optimization of Dice Loss and Focal Loss.

[0083] In this embodiment, the category weights corresponding to the defect types can be adjusted to suppress interference from irrelevant information in the template, so that the model can focus on the judgment area, thereby achieving more accurate background shielding.

[0084] It should be noted that Dice Loss and Focal Loss are two loss functions used to process unbalanced data sets. The combination of the two can further improve model performance. Dice Loss is mainly used to evaluate the similarity between two samples. For example, it is particularly suitable for situations where the foreground and background pixels in an image are extremely unbalanced. Focal Loss is a loss function proposed to solve the problem of category imbalance, especially the foreground-background imbalance problem in target detection tasks. It is improved on the basis of cross entropy loss. By reducing the weight of easily classified samples, the model training pays more attention to difficult-to-classify samples. In this embodiment, Dice Loss and Focal Loss are used in combination to combine the advantages of both. Among them, Dice Loss helps to improve the ability to capture foreground objects in segmentation tasks, especially for small targets or sparsely distributed targets; Focal Loss can help the model better deal with the problem of category imbalance and enhance the learning ability of minority categories.

[0085] In one example, joint optimization can be achieved by linearly combining the functions of Dice Loss and Focal Loss.

[0086] Second, see Figure 4 , shows a schematic flow chart of a circuit board image defect segmentation method provided by an embodiment of the present application, the method including steps S401-S402, specifically as follows:

[0087] S401, obtaining an image pair to be identified, wherein the image pair to be identified includes a PCBA image to be identified and its corresponding target template image;

[0088] S402 , calling a pre-trained circuit board image defect segmentation model to perform defect segmentation based on the image pair to be identified, wherein the pre-trained circuit board image defect segmentation model is trained using any one of the methods described in the first aspect above.

[0089] A specific example is provided below to further illustrate the effects of the embodiments of the present application.

[0090] The hardware resources used in this embodiment are NVIDIA RTX3060 graphics cards, the selected comparison example 1 is the Unet++ algorithm, the comparison example 2 is the DeepLabV3+ algorithm, and the comparison example 3 is the Yolov8-s algorithm. The indicator used to evaluate the segmentation effect is mIoU (Mean Intersection over Union).

[0091] The embodiments are:

[0092] A circuit board image defect segmentation method, comprising:

[0093] Acquire an image pair to be identified, wherein the image pair to be identified includes a PCBA image to be identified and its corresponding target template image;

[0094] Call a pre-trained circuit board image defect segmentation model to perform defect segmentation based on the image pair to be identified, wherein the pre-trained circuit board image defect segmentation model is trained using any one of the methods described in the first aspect above.

[0095] The data obtained from the above test are shown in Table 1 below.

[0096] method mIoU (%) Inference time Remark Unet++ 60.5 50ms Single image input DeepLabV3+ 58.7 60ms Single image input Yolov8-s 55.6 15ms Single image input This embodiment 97.6 30ms Dual image input

[0097] Table 1

[0098] As can be seen from the above, the accuracy of this embodiment is higher than that of the current mainstream segmentation detection networks Unet++, DeepLabV3+, and Yolov8-s.

[0099] In a third aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of any of the above-described methods when the computer program is executed by a processor.

[0100] In a fourth aspect, an embodiment of the present application provides a computer device comprising a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor implements the steps of any of the above methods when executing the computer program.

[0101] See also Figure 5 The computer device of this embodiment includes: a processor 501, a memory 502, and a computer program stored in the memory 502 and executable on the processor 501.

[0102] For example, the computer program may be divided into one or more modules / units, which are stored in the memory 502 and executed by the processor 501 to implement the present application. The one or more modules / units may be a series of computer program instruction segments capable of implementing specific functions, and the instruction segments are used to describe the execution process of the computer program in the computer device.

[0103] The computer device may be a desktop computer, laptop, PDA, cloud server, or other computing device. The computer device may include, but is not limited to, a processor 501 and a memory 502. Those skilled in the art will appreciate that the schematic diagram is merely an example of a computer device and does not limit the computer device. The computer device may include more or fewer components than shown, or a combination of certain components, or different components. For example, the computer device may also include input and output devices, network access devices, buses, and the like.

[0104] The processor 501 may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor, or the processor 501 may be any conventional processor. The processor 501 is the control center of the computer device, connecting various parts of the entire computer device using various interfaces and lines.

[0105] The memory 502 can be used to store the computer programs and / or modules. The processor 501 implements various functions of the computer device by running or executing the computer programs and / or modules stored in the memory 502 and calling the data stored in the memory 502. The memory 502 can mainly include a program storage area and a data storage area. The program storage area can store an operating system and at least one application required for a function (such as a sound playback function, an image playback function, etc.); the data storage area can store data created based on the use of the mobile phone (such as audio data, a phone book, etc.). In addition, the memory 502 can include a high-speed random access memory and can also include non-volatile memory, such as a hard disk, internal memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, at least one disk storage device, a flash memory device, or other volatile solid-state storage device.

[0106] Wherein, if the module / unit integrated in the computer device is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the process in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and when the computer program is executed by the processor 501, it can implement the steps of the above-mentioned various method embodiments. Wherein, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device that can carry the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electrical carrier signal, telecommunication signal and software distribution medium, etc.

[0107] In summary, the embodiments of the present application have at least the following beneficial effects:

[0108] According to an embodiment of the present application, a defect image pair is obtained, wherein the defect image pair includes a PCBA defect image and its corresponding template image; the defect image pair is used to iteratively train the segmentation model to be trained, wherein the training method of each round in the iterative training includes: fusing the features of the PCBA defect image and the features of the template image through a gated fusion network in the segmentation model to be trained to obtain a fusion feature; and, based on the fusion feature, training the segmentation model to be trained in the current round, thereby combining the features of the template image to improve the detection capability of tiny defects in the PCBA image and improve the comparative detection capability of logical defects such as component mismatches. In addition, the trained model can use the gated fusion network to automatically learn weights to automatically adjust the fusion ratio of the original image defect features (i.e., the features of the PCBA defect image) and the template prior features (features of the template image) to further enhance the combined use of the two types of images.

[0109] Through the description of the above implementation methods, those skilled in the art can clearly understand that the present application can be implemented by means of software plus the necessary hardware platform, and of course it can also be implemented entirely by hardware. Based on this understanding, all or part of the contribution of the technical solution of the present application to the background technology can be embodied in the form of a software product, and the computer software product can be stored in a storage medium, such as ROM (Read-Only Memory) / RAM (Random Access Memory), a disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application or certain parts of the embodiments.

[0110] The above is a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications are also considered to be within the scope of protection of the present application.

Claims

1. A circuit board image defect segmentation model training method, characterized in that: include: Acquire a defect image pair, wherein the defect image pair includes a PCBA defect image and its corresponding template image; The defect image pairs are used to iteratively train the segmentation model to be trained, wherein the training method of each round of the iterative training includes: By using a gated fusion network in the segmentation model to be trained, the features of the PCBA defect image and the features of the template image are fused to obtain fused features; and Based on the fusion features, the segmentation model to be trained is trained in a current round.

2. The method according to claim 1, characterized in that The obtaining of the defect image pair comprises: Get the original PCBA image; Based on the original PCBA image, the template image is obtained by using template matching technology; The PCBA defect image corresponding to the original PCBA image is obtained, wherein the PCBA defect image is obtained by marking defects in the original PCBA image.

3. The method according to claim 1, characterized in that The segmentation model to be trained also includes a feature extractor, which is obtained through pre-training and whose parameters remain unchanged after completing the pre-training; Before fusing the features of the PCBA defect image and the features of the template image, the training method further includes: Based on the PCBA defect image and the template image, feature extraction is performed by the feature extractor to obtain features of the PCBA defect image and features of the template image.

4. The method according to claim 3, characterized in that The feature extractor is built based on the MIT-B1 network.

5. The method according to claim 3, characterized in that The step of extracting features based on the PCBA defect image and the template image by the feature extractor to obtain features of the PCBA defect image and features of the template image includes: Performing pixel-by-pixel coding processing based on defect types on the PCBA defect image and the template image respectively; scaling the encoded PCBA defect image and the encoded template image according to a target resolution, wherein the target resolution is suitable for indicating a resolution of the scaled image; Normalizing the scaled PCBA defect image and the scaled template image respectively; The feature extractor performs feature extraction on the normalized PCBA defect image and the normalized template image, respectively, to obtain features of the PCBA defect image and features of the template image.

6. The method according to claim 1, wherein The segmentation model to be trained further includes an MLP layer and a decoder of a segmentation network, the segmentation network includes a SegFormer, and the current round of training of the segmentation model to be trained based on the fusion features includes: Upsampling the fused features through a decoder of the segmentation network, and concatenating the upsampled fused features to obtain concatenated features; Obtaining a segmentation prediction result based on the splicing features through the MLP layer; Based on the difference between the annotation information corresponding to the PCBA defect image and the segmentation prediction result, the segmentation model to be trained is trained in the current round using a preset loss function.

7. The method according to claim 6, characterized in that The loss function is suitable for adjusting the weights corresponding to different defect types through joint optimization of Dice Loss and Focal Loss.

8. A circuit board image defect segmentation method, characterized in that: include: Acquire an image pair to be identified, wherein the image pair to be identified includes a PCBA image to be identified and its corresponding target template image; Calling a pre-trained circuit board image defect segmentation model to perform defect segmentation based on the image pair to be identified, wherein the pre-trained circuit board image defect segmentation model is trained by the method described in any one of claims 1-7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

10. A computer device, characterized in that: The method comprises a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor implements the method according to any one of claims 1 to 8 when executing the computer program.

Images