A printed circuit board defect detection method and system based on cross-modal prompt learning and visual guidance

Abstract

The present invention discloses a method and system for printed circuit board defect detection based on cross-modal prompt learning and visual guidance, comprising the following steps: obtaining defect-free PCB image samples to obtain a training data set; constructing an initial printed circuit board detection model; using the training data set to train the initial printed circuit board detection model to obtain a printed circuit board detection model; and using the printed circuit board detection model to complete printed circuit board defect detection. The method of the present invention proposes a synthetic anomaly generation strategy based on PCB structure and physical priors, and introduces a dual-channel repository mechanism. Through a difficult example mining strategy, the most valuable synthetic anomaly samples for model training are screened out, thereby providing visual anomaly pattern guidance. The present invention adopts a dual-constraint alignment optimization strategy combining contrast loss and triplet loss to adjust the mapping relationship between visual and text features, thereby achieving cross-modal alignment and clear normal / abnormal boundaries. The method proposed in the present invention can significantly improve the accuracy, robustness and generalization ability of PCB defect detection under data sparse conditions.

Description

Technical Field

[0001] The present invention belongs to the field of industrial automation detection and artificial intelligence technology, and specifically relates to a printed circuit board defect detection method and system based on cross-modal prompt learning and visual guidance. Background Art

[0002] Printed circuit boards (PCBs) are essential building blocks of modern electronic devices, and their quality directly impacts the performance, stability, and lifespan of end products. The complex PCB manufacturing process can easily introduce various defects, such as opens, shorts, mousebites, burrs, pinholes, spurious copper, missing components, solder defects, and scratches. As electronic products advance toward miniaturization, high integration, and high frequencies, the density of components and circuits on PCBs is increasing, and defect sizes are also shrinking. This poses unprecedented challenges to the accuracy and efficiency of defect detection. Traditional methods that rely on manual visual inspection are not only inefficient and costly, but also susceptible to subjective factors, resulting in high rates of missed and false positives, making them unable to meet the high standards required by modern industrial production lines. Although Automated Optical Inspection (AOI) systems have been widely used, they are usually based on rules or template matching, have limited generalization capabilities for complex, unexpected, or morphologically variable defects, and require tedious parameter adjustments for different product lines.

[0003] In recent years, deep learning techniques, particularly convolutional neural networks (CNNs) and object detection algorithms (such as YOLO and Faster R-CNN), have demonstrated tremendous potential in PCB defect detection. These supervised learning methods can achieve high detection accuracy when equipped with large, accurately labeled defect datasets. However, their primary bottleneck lies in their strong reliance on large amounts of labeled data. In real-world industrial scenarios, defects are inherently sporadic and diverse, making it extremely difficult and costly to collect large-scale labeled samples covering all possible defect types, especially for new product lines or rare defect types. Supervised learning methods require an impractical amount of labeled defect data, which is often difficult to obtain in rapidly changing and sample-scarce industrial environments.

[0004] To alleviate the reliance on labeled defect data, unsupervised anomaly detection methods have emerged. These methods use only normal (defect-free) samples for training, learning the distribution pattern of normal samples and classifying samples that deviate significantly from this pattern as anomalies. Common techniques include reconstruction-based methods (such as autoencoders (AEs) and variational autoencoders (VAEs), methods based on generative adversarial networks (GANs), and methods based on self-supervised learning (such as contrastive learning). However, unsupervised methods often struggle to accurately model the "normal" boundaries of complex, high-dimensional data, and are prone to misidentifying normal process variations as defects (high false positive rates). They are also insensitive to subtle defects that are not significantly different from normal samples (high false negative rates). Consequently, they often struggle to distinguish complex or subtle normal variations from true defects.

[0005] The rise of visual language models (VLMs), such as CLIP and BLIP, has opened up new avenues for using natural language descriptions for visual recognition and understanding. These models, pre-trained on massive image-text pairs, have learned powerful cross-modal association capabilities and demonstrated excellent zero-shot and few-shot generalization performance. Researchers have begun exploring the application of VLMs to anomaly detection tasks. For example, WinCLIP leverages CLIP for industrial anomaly detection, transforming anomaly detection into an image-text matching problem through hand-crafted "normal" and "abnormal" text prompts, eliminating the need for training with abnormal samples. PromptAD automatically generates abnormal prompts by concatenating defect keywords after normal prompts and optimizes them through contrastive learning.

[0006] Although VLM-based methods have brought hope for few-shot PCB defect detection, existing technologies still have significant limitations: (1) Rough and lack of specificity in semantic description: Existing methods mostly rely on manually designed, category-level coarse-grained text prompts (such as "a photo of a normal PCB" vs "a photo of a defective PCB"), or simple keyword splicing. This approach cannot capture the rich morphological details and complex semantic connotations of PCB defects (for example, "a tin bridge short circuit between the 3rd and 4th pins of a QFP chip" is much more informative than "short circuit"). This semantic deficiency limits the ability of VLM to perform fine-grained image-text alignment, making it difficult to distinguish subtle defects or specific types of defects. The prompts used by existing VLM methods are too general and cannot distinguish key PCB defects. (2) Lack of guidance from real abnormal visual patterns: Most VLM anomaly detection methods follow a single-category learning paradigm, relying only on normal samples for training, or like PromptAD, only generating "pseudo" anomalies in the text space. The model is not exposed to real or realistic abnormal visual patterns during training, resulting in limited ability to distinguish unseen real defects with different morphologies. How to effectively incorporate visual prior knowledge of anomalies into the training process is a key challenge. They have difficulty identifying new or complex defect patterns that have not been seen during training. (3) Imprecise semantic-visual alignment: Anomaly prompts generated solely through text manipulation may deviate from the visual representation of the actual defect (Semantic-Visual Mismatch). At the same time, existing cross-modal alignment loss functions (such as simple contrastive loss) may not be sufficient to establish clear and robust decision boundaries for normal and various abnormal patterns in the feature space. Existing methods are insufficient in ensuring that text descriptions accurately correspond to actual visual defect features. Therefore, developing a new PCB defect detection method that can generate fine-grained, semantically rich text prompts, combine effective, visual pattern-based anomaly guidance, and achieve more accurate and discriminative cross-modal feature alignment is of great significance for promoting the practical application of VLM in demanding industrial inspection scenarios. Summary of the Invention

[0007] One of the objectives of the present invention is to provide a printed circuit board defect detection method based on cross-modal cue learning and visual guidance, which can achieve high-precision and high-robust PCB defect detection under conditions of few or even zero samples.

[0008] A second object of the present invention is to provide a system for implementing the printed circuit board defect detection method based on cross-modal prompt learning and visual guidance.

[0009] The present invention provides a printed circuit board defect detection method based on cross-modal prompt learning and visual guidance, comprising the following steps:

[0010] S1. Obtain normal, defect-free printed circuit board image samples to obtain a training dataset;

[0011] S2. Constructing an initial PCB inspection model based on cross-modal cue learning and visual guidance; the initial PCB inspection model includes a cue construction module, an anomaly generation and screening module, a feature space module, a feature extraction module, and a defect judgment and location module;

[0012] S3. Combining contrast loss and triplet loss for dual-constraint optimization, the initial printed circuit board detection model described in step S2 is trained using the training data set obtained in step S1 to obtain a printed circuit board detection model;

[0013] S4. Using the printed circuit board inspection model obtained in step S3, inputting the real-time acquired PCB image for inspection, and completing the printed circuit board defect inspection.

[0014] In step S2, the initial printed circuit board inspection model includes a prompt construction module, an anomaly generation and screening module, a feature extraction module, a feature space module, and a defect judgment and positioning module; wherein the anomaly generation and screening module is only used in the model training process;

[0015] The prompt construction module constructs normal sample prompts and abnormal sample prompts for the input training PCB image and inputs them into the feature extraction module;

[0016] The anomaly generation and screening module includes an anomaly sample generation module, a dual-channel repository, and a difficult example screening module. The anomaly sample generation module performs enhancement operations based on the input training PCB image to generate synthetic anomaly samples and stores them in the dual-channel repository. The difficult example screening module dynamically screens out difficult synthetic anomaly samples based on the synthetic anomaly samples in the dual-channel repository. The results obtained by the anomaly generation and screening module are input into the feature extraction module and the prompt construction module.

[0017] The feature extraction module includes a pre-trained visual encoder and a pre-trained text encoder; the pre-trained visual encoder is used to extract the visual features of the input PCB image; the pre-trained text encoder is used to extract the text features of the normal sample prompts or abnormal sample prompts corresponding to the input PCB image; during training, the visual features and text features ultimately extracted by the feature extraction module are uploaded to the feature space module; in actual use, the visual features ultimately extracted by the feature extraction module are input into the defect judgment and positioning module;

[0018] The defect judgment and positioning module calculates the similarity or distance between the image features and the text features in the feature space module based on the input visual features to obtain the anomaly score; the image-level defect judgment is completed by comparing the obtained anomaly score with the preset threshold; the feature vector at each spatial position of the input visual features interacts with the text features in the feature space to obtain a similarity map, attention map or heat map, and the obtained similarity map, attention map or heat map is upsampled, and a binary defect positioning mask is obtained through threshold processing to complete the positioning of the image defect area.

[0019] The prompt construction module includes normal sample prompt construction and abnormal sample prompt construction; the normal sample prompt construction is specifically: for normal PCB image data, construct a normal sample prompt ; The normal sample suggests There are N prefix symbols to be learned and fixed category nouns Spliced ​​together, it is expressed using the following formula: in, is the concatenation operation of vector or symbol sequence; N is the number of preset prefix symbols; the prefix symbol to be learned The embedding vectors are optimized through back propagation during the model training process, so that the model can automatically learn and capture the contextual semantic features that match;

[0020] The abnormal sample prompt is specifically constructed by: constructing an abnormal sample prompt based on a pre-trained large language model; the pre-trained large language model includes a pre-input guidance prompt; the guidance prompt includes background knowledge and task requirements in the PCB field; the background knowledge includes standard PCB defect types, morphological feature descriptions, common causes, and related process terms; the task requirement is to instruct the pre-trained large language model to generate several different text descriptions for specific defect types based on the background knowledge;

[0021] Specifically, in actual applications, the pre-trained large language model generates several different text descriptions for specific defect types: For the defect type "open", after guiding the pre-trained large language model, multiple fine-grained descriptions similar to the following may be generated: "Description 1: The image shows a clear white break in the middle of a black conductive trace, forming a gap of no less than 0.1 mm, resulting in an interruption of the electrical connection.", "Description 2: There is a linear interruption on the copper foil trace of the PCB. The trace is completely disconnected here, and the edge of the break may be irregular.", "Description 3: A long and thin black line is divided into two sections by a blank area, and no path can be formed."

[0022] The abnormal sample prompt construction includes two methods: manual design construction and learning construction;

[0023] The artificial design construction is specifically as follows: for known PCB defects, a pre-trained large language model is used to generate fine-grained defect descriptions , and the prefix symbol to be learned and fixed category nouns Perform splicing to obtain artificially designed abnormal sample prompts , expressed using the following formula: , in, Tips for abnormal samples constructed for artificial design construction methods;

[0024] The construction to be learned is specifically: for unknown PCB defects, since they are not To fully cover the potential defect types, we use the to-be-learned construction method to construct abnormal sample prompts and then introduce M adaptive suffix symbols to be learned. , expressed using the following formula: , in, Abnormal sample tips for the construction method to be learned; M is the number of suffix symbols; adaptive suffix symbols to be learned The embedding vector is optimized through back propagation during the model training process, and the semantic features representing abnormal states are automatically learned from the training data.

[0025] The anomaly generation and screening module includes an anomaly sample generation module, a dual-channel storage library, and a difficult example screening module. The anomaly sample generation module uses a synthesis method based on structural and physical prior guidance to generate synthetic anomaly samples while considering PCB design rules and material properties. The synthesis method based on structural and physical prior guidance specifically includes: when generating synthetic anomaly samples of a specific defect type, the corresponding prior knowledge is used to constrain the generation parameters of the corresponding defect, thereby preventing the generated synthetic anomaly sample defects from not conforming to the actual objective situation.

[0026] Specifically, in practical applications, the synthesis method based on structural and physical prior guidance is effective in the following defect types, including elastic deformation, scratches, and solder joint anomalies:

[0027] Elastic deformation: To simulate the physical bending or warping of a PCB under stress, a two-dimensional displacement field is calculated. The pixels of the original image are then resampled according to the displacement field using bilinear interpolation to produce a deformed image. The magnitude and spatial frequency of the displacement field can be constrained by prior knowledge (such as board thickness, size, and fixing method).

[0028] Scratches: To simulate surface scratches, a random path is first generated as the main scratch, with a length and width within a preset range. Then, some branches are randomly generated along the main scratch to simulate the irregularity of the scratch. The pixel values ​​in the scratch area are modified to a specific color that simulates metallic gloss or substrate exposure, and a small amount of noise is added.

[0029] Solder joint anomalies: To simulate defects such as cold solder joints, poor solder joints, and overheating oxidation, a gradient synthesis method is used to weightedly blend the original image with an image layer that simulates metallic gloss (for example, using an HSV color gradient with radial brightness attenuation) and an image layer that simulates the appearance of an oxidation layer (for example, adding noise of a specific color and texture), and the blending ratio is controlled by the weight.

[0030] By combining and applying the structure-based and physical prior-guided synthesis methods, a number of synthetic abnormal samples are generated for each input image and input into a dual-channel repository;

[0031] The dual-channel storage library maintains a corresponding and independent first-in-first-out queue for each normal image sample, and the capacity of the queue is preset to k. When a synthetic abnormal sample is newly generated by a specific normal image sample, it is added to the head of the corresponding first-in-first-out queue. If the number of synthetic abnormal samples in the queue is greater than k at this time, the sample at the end of the queue is removed.

[0032] The difficult example screening module measures the difficulty of synthesizing abnormal samples for the current model state by calculating the loss perception weight of each synthetic abnormal sample, which is expressed by the following formula: , in, To synthesize abnormal samples The loss-aware weight of Based on normal image samples The jth synthetic abnormal sample; For the model to detect abnormal samples The total loss value obtained after forward propagation;

[0033] The hard example screening module performs dual-channel sampling in the dual-channel repository based on loss-aware weights to obtain hard synthetic anomaly samples. The dual-channel sampling includes internal queue sampling and external queue sampling. During training, when a normal image sample is processed, two hard synthetic anomaly samples for the current training step are screened from the repository. The two hard synthetic anomaly samples are obtained by internal queue sampling and external queue sampling respectively. The internal queue sampling is to select the sample with the maximum loss-aware weight from the first-in-first-out queue corresponding to the normal image sample in the dual-channel repository. The external queue sampling is to select the sample with the maximum loss-aware weight from the first-in-first-out queue corresponding to the normal image sample in the dual-channel repository. The internal queue sampling is to allow the model to focus on subtle abnormal features generated in similar backgrounds, which helps to improve the model's local discrimination ability and prevent overfitting. The external queue sampling is to allow the model to face abnormal instances from different normal sample backgrounds, thereby improving the model's generalization ability. Through the dual-channel sampling strategy, the two hard synthetic anomaly samples are trained together with the original normal sample during training, thereby providing targeted visual anomaly guidance.

[0034] The feature extraction module includes a pre-trained visual encoder and a pre-trained text encoder; the pre-trained visual encoder uses a pre-trained visual model as a backbone to extract high-level visual features from the input image; the pre-trained visual model includes a Vision Transformer (ViT) or ResNet series;

[0035] The pre-trained text encoder uses a pre-trained Transformer model to process text prompts and encode them into corresponding text features.

[0036] The feature space module stores the matched sample visual features and the corresponding text features in the feature space according to the received data.

[0037] The defect judgment and positioning module performs defect judgment and defect positioning based on the input sample visual features and the data in the feature space module; the defect judgment is specifically as follows:

[0038] Get predefined normal text prompt And abnormal sample prompts ; The abnormal sample prompt A text prompt corresponding to a general exception or a collection of text prompts corresponding to several specific defect types;

[0039] Compute the input sample visual features and predefined normal text prompts And abnormal sample prompts The similarity or distance between them; the similarity is cosine similarity, and the distance is Euclidean distance;

[0040] Define anomaly scores and calculate anomaly scores based on the obtained similarity or distance;

[0041] The anomaly score is compared with a preset threshold. When the anomaly score is greater than the threshold, the corresponding image is an abnormal defective PCB image; when the anomaly score is less than the threshold, it is judged to be normal. The preset threshold is determined by the model on a validation set containing normal samples and abnormal samples based on the expected performance indicators.

[0042] The defect location is specifically as follows:

[0043] Based on the input sample space feature map, calculate the local feature vector and abnormal sample prompts at each spatial position of the feature map or text prompts for specific defect types similarity between

[0044] The similarity map formed by the obtained similarities is up-sampled, and a binary defect location mask is obtained through threshold processing to mark the defect area in the image.

[0045] The defect localization can also be achieved by:

[0046] Based on the input sample spatial feature map, calculate the gradient of the anomaly score for each spatial position on the visual feature map;

[0047] Use the obtained gradient to perform weighted summation on the feature map to obtain a heat map;

[0048] The obtained heat map is upsampled and a binary defect location mask is obtained through threshold processing to mark the defect area in the image.

[0049] The anomaly score is specifically defined as:

[0050] The similarity-based anomaly score is defined using the following formula: ;

[0051] in, is the anomaly score; Calculate the cosine similarity; is the visual feature of the input sample to be tested; the higher the anomaly score, the more similar the image is to the abnormal description and the less similar it is to the normal description;

[0052] The distance-based anomaly score is defined using the following formula: ;

[0053] in, is the anomaly score; It is the Euclidean distance calculation; is the visual feature of the input sample to be tested; the higher the anomaly score obtained, the farther the image is from the normal description and the closer it is to the abnormal description.

[0054] In step S3, the training goal of the model is to adjust the parameters to be learned contained in the visual encoder, text encoder, and sample prompts by optimizing a joint loss function L; the joint loss function L combines the contrast loss and triplet loss , expressed using the following formula: ; in, is a hyperparameter and ;

[0055] The joint loss function L is minimized through the gradient descent optimization algorithm, and all learnable parameters in the model are updated end-to-end.

[0056] The purpose of the contrast loss is to shorten the distance between the positive sample pairs in the feature space or improve the similarity between the two, and push away the negative sample pairs; the positive sample pairs are matched image features and text features, and the negative sample pairs are mismatched image features and text features;

[0057] The contrast loss is expressed using the following formula: ;

[0058] in, For expectations; Calculate the cosine similarity; is the temperature hyperparameter; Normal visual characteristics; is a normal text feature; is the jth text feature;

[0059] The contrastive loss is responsible for establishing basic correspondences and global alignment between modalities;

[0060] The purpose of the triplet loss is to enforce the discriminative boundary between categories in the feature space; the triplet loss requires that the distance between the same category sample and its positive sample is at least a preset margin m smaller than the distance between it and the negative sample of the different category; the triplet loss Use the following formula to express it: ;

[0061] in, is the triplet constraint loss for normal visual features; is the triplet constraint loss for abnormal visual features;

[0062] For normal visual features , its corresponding normal text features is a positive sample, abnormal text feature For negative samples, the triplet constraint loss is expressed using the following formula: ;

[0063] in, is the triplet constraint loss for normal visual features; To obtain the maximum value operation; It is the Euclidean distance calculation; Normal visual characteristics; is a normal text feature; is the abnormal text feature; m is a hyperparameter and m is greater than 0;

[0064] For abnormal visual features , its corresponding normal text features is a positive sample, normal text feature For negative samples, the triplet constraint loss is expressed using the following formula: ;

[0065] in, is the triplet constraint loss for abnormal visual features; To obtain the maximum value operation; It is the Euclidean distance calculation; Abnormal visual features; is a normal text feature; is the abnormal text feature; m is a hyperparameter and m is greater than 0;

[0066] The present invention also provides a system for implementing the method, comprising a data acquisition module, a model construction module, a model training module, and a circuit board image detection module;

[0067] The data acquisition module obtains normal and defect-free printed circuit board image samples to obtain a training data set, and uploads the data to the model training module;

[0068] The model building module builds an initial PCB inspection model based on cross-modal cue learning and visual guidance, and uploads the data to the model training module;

[0069] The model training module performs dual-constraint optimization based on the received data, combining contrast loss and triplet loss, uses the training dataset to train the initial printed circuit board inspection model, and uploads the data to the circuit board image inspection module;

[0070] The circuit board image detection module uses the printed circuit board detection model based on the received data and inputs the real-time acquired PCB image for detection to complete the printed circuit board defect detection.

[0071] Beneficial effects of the present invention:

[0072] (1) The prompt building module in the proposed model generates fine-grained prompts based on LLM and domain knowledge, enabling the model to understand and identify specific and subtle PCB defect characteristics;

[0073] (2) Based on the prior synthetic anomaly generation strategy and dual-channel difficult example mining mechanism, the proposed method provides the model with realistic and most challenging negative sample visual signals, thereby enhancing the model's ability to discriminate various abnormal patterns;

[0074] (3) The proposed method ensures that the image and text features are not only semantically aligned but also have clear discrimination boundaries between categories through a dual-constraint (contrast + triplet loss) optimization strategy;

[0075] (4) The proposed method framework can achieve performance far exceeding that of existing methods when dealing with small-sample or even zero-sample PCB defect detection tasks with extremely limited labeled data;

[0076] (5) The framework of the method of the present invention is modular, and components can be easily adjusted and replaced. It can adapt to different scenarios and is suitable for industrial applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] Figure 1 Schematic diagram of the process of the present invention;

[0078] Figure 2 Schematic diagram of the structure of the system of the present invention;

[0079] Figure 3 This is a diagram showing defect judgment and location results in an embodiment of the present invention; DETAILED DESCRIPTION

[0080] The present invention provides a printed circuit board defect detection method based on cross-modal prompt learning and visual guidance, the flow chart of which is as follows: Figure 1 As shown, the following steps are included:

[0081] S1. Obtain normal, defect-free printed circuit board image samples to obtain a training dataset;

[0082] S2. Construct an initial PCB inspection model based on cross-modal cue learning and visual guidance. The initial PCB inspection model includes a cue construction module, an anomaly generation and screening module, a feature space module, a feature extraction module, and a defect judgment and location module. Specifically,

[0083] The initial printed circuit board inspection model includes a prompt construction module, an anomaly generation and screening module, a feature extraction module, a feature space module, and a defect judgment and positioning module; wherein the anomaly generation and screening module is only used in the model training process;

[0084] The prompt construction module constructs normal sample prompts and abnormal sample prompts for the input training PCB image and inputs them into the feature extraction module;

[0085] The anomaly generation and screening module includes an anomaly sample generation module, a dual-channel repository, and a difficult example screening module. The anomaly sample generation module performs enhancement operations based on the input training PCB image to generate synthetic anomaly samples and stores them in the dual-channel repository. The difficult example screening module dynamically screens out difficult synthetic anomaly samples based on the synthetic anomaly samples in the dual-channel repository. The results obtained by the anomaly generation and screening module are input into the feature extraction module and the prompt construction module.

[0086] The feature extraction module includes a pre-trained visual encoder and a pre-trained text encoder; the pre-trained visual encoder is used to extract the visual features of the input PCB image; the pre-trained text encoder is used to extract the text features of the normal sample prompts or abnormal sample prompts corresponding to the input PCB image; during training, the visual features and text features ultimately extracted by the feature extraction module are uploaded to the feature space module; in actual use, the visual features ultimately extracted by the feature extraction module are input into the defect judgment and positioning module;

[0087] The defect judgment and positioning module calculates the similarity or distance between the image features and the text features in the feature space module based on the input visual features to obtain the anomaly score; the image-level defect judgment is completed by comparing the obtained anomaly score with the preset threshold; the feature vector at each spatial position of the input visual features interacts with the text features in the feature space to obtain a similarity map, attention map or heat map, and the obtained similarity map, attention map or heat map is upsampled, and a binary defect positioning mask is obtained through threshold processing to complete the positioning of the image defect area.

[0088] The prompt construction module includes normal sample prompt construction and abnormal sample prompt construction; the normal sample prompt construction is specifically: for normal PCB image data, construct a normal sample prompt ; The normal sample suggests There are N prefix symbols to be learned and fixed category nouns Spliced ​​together, it is expressed using the following formula: , in, is the concatenation operation of vector or symbol sequence; N is the number of preset prefix symbols; the prefix symbol to be learned The embedding vectors are optimized through back propagation during the model training process, so that the model can automatically learn and capture the contextual semantic features that match;

[0089] The abnormal sample prompt is specifically constructed by: constructing an abnormal sample prompt based on a pre-trained large language model; the pre-trained large language model includes a pre-input guidance prompt; the guidance prompt includes background knowledge and task requirements in the PCB field; the background knowledge includes standard PCB defect types, morphological feature descriptions, common causes, and related process terms; the task requirement is to instruct the pre-trained large language model to generate several different text descriptions for specific defect types based on the background knowledge;

[0090] Specifically, in actual applications, the pre-trained large language model generates several different text descriptions for specific defect types: For the defect type "open", after guiding the pre-trained large language model, multiple fine-grained descriptions similar to the following may be generated: "Description 1: The image shows a clear white break in the middle of a black conductive trace, forming a gap of no less than 0.1 mm, resulting in an interruption of the electrical connection.", "Description 2: There is a linear interruption on the copper foil trace of the PCB. The trace is completely disconnected here, and the edge of the break may be irregular.", "Description 3: A long and thin black line is divided into two sections by a blank area, and no path can be formed."

[0091] The abnormal sample prompt construction includes two methods: manual design construction and learning construction;

[0092] The artificial design construction is specifically as follows: for known PCB defects, a pre-trained large language model is used to generate fine-grained defect descriptions , and the prefix symbol to be learned and fixed category nouns Perform splicing to obtain artificially designed abnormal sample prompts , expressed using the following formula: , in, Tips for abnormal samples constructed for artificial design construction methods;

[0093] The construction to be learned is specifically: for unknown PCB defects, since they are not To fully cover the potential defect types, we use the to-be-learned construction method to construct abnormal sample prompts and then introduce M adaptive suffix symbols to be learned. , expressed using the following formula: , in, Abnormal sample tips for the construction method to be learned; M is the number of suffix symbols; adaptive suffix symbols to be learned The embedding vector is optimized through back propagation during the model training process, and the semantic features representing abnormal states are automatically learned from the training data.

[0094] The anomaly generation and screening module includes an anomaly sample generation module, a dual-channel storage library, and a difficult example screening module. The anomaly sample generation module uses a synthesis method based on structural and physical prior guidance to generate synthetic anomaly samples while considering PCB design rules and material properties. The synthesis method based on structural and physical prior guidance specifically includes: when generating synthetic anomaly samples of a specific defect type, the corresponding prior knowledge is used to constrain the generation parameters of the corresponding defect, thereby preventing the generated synthetic anomaly sample defects from not conforming to the actual objective situation.

[0095] Specifically, in practical applications, the synthesis method based on structural and physical prior guidance is effective in the following defect types, including elastic deformation, scratches, and solder joint anomalies:

[0096] Elastic deformation: To simulate the physical bending or warping of a PCB under stress, a two-dimensional displacement field is calculated. The pixels of the original image are then resampled according to the displacement field using bilinear interpolation to produce a deformed image. The magnitude and spatial frequency of the displacement field can be constrained by prior knowledge (such as board thickness, size, and fixing method).

[0097] Scratches: To simulate surface scratches, a random path is first generated as the main scratch, with a length and width within a preset range. Then, some branches are randomly generated along the main scratch to simulate the irregularity of the scratch. The pixel values ​​in the scratch area are modified to a specific color that simulates metallic gloss or substrate exposure, and a small amount of noise is added.

[0098] Solder joint anomalies: To simulate defects such as cold solder joints, poor solder joints, and overheating oxidation, a gradient synthesis method is used to weightedly blend the original image with an image layer that simulates metallic gloss (for example, using an HSV color gradient with radial brightness attenuation) and an image layer that simulates the appearance of an oxidation layer (for example, adding noise of a specific color and texture), and the blending ratio is controlled by the weight.

[0099] By combining and applying the structure-based and physical prior-guided synthesis methods, a number of synthetic abnormal samples are generated for each input image and input into a dual-channel repository;

[0100] The dual-channel storage library maintains a corresponding and independent first-in-first-out queue for each normal image sample, and the capacity of the queue is preset to k. When a synthetic abnormal sample is newly generated by a specific normal image sample, it is added to the head of the corresponding first-in-first-out queue. If the number of synthetic abnormal samples in the queue is greater than k at this time, the sample at the end of the queue is removed.

[0101] The difficult example screening module measures the difficulty of synthesizing abnormal samples for the current model state by calculating the loss perception weight of each synthetic abnormal sample, which is expressed by the following formula: , in, To synthesize abnormal samples The loss-aware weight of Based on normal image samples The jth synthetic abnormal sample; For the model to detect abnormal samples The total loss value obtained after forward propagation;

[0102] The hard example screening module performs dual-channel sampling in the dual-channel repository based on loss-aware weights to obtain hard synthetic anomaly samples. The dual-channel sampling includes internal queue sampling and external queue sampling. During training, when a normal image sample is processed, two hard synthetic anomaly samples for the current training step are screened from the repository. The two hard synthetic anomaly samples are obtained by internal queue sampling and external queue sampling respectively. The internal queue sampling is to select the sample with the maximum loss-aware weight from the first-in-first-out queue corresponding to the normal image sample in the dual-channel repository. The external queue sampling is to select the sample with the maximum loss-aware weight from the first-in-first-out queue corresponding to the normal image sample in the dual-channel repository. The internal queue sampling is to allow the model to focus on subtle abnormal features generated in similar backgrounds, which helps to improve the model's local discrimination ability and prevent overfitting. The external queue sampling is to allow the model to face abnormal instances from different normal sample backgrounds, thereby improving the model's generalization ability. Through the dual-channel sampling strategy, the two hard synthetic anomaly samples are trained together with the original normal sample during training, thereby providing targeted visual anomaly guidance.

[0103] The feature extraction module includes a pre-trained visual encoder and a pre-trained text encoder; the pre-trained visual encoder uses a pre-trained visual model as a backbone to extract high-level visual features from the input image; the pre-trained visual model includes a Vision Transformer (ViT) or ResNet series;

[0104] The pre-trained text encoder uses a pre-trained Transformer model to process text prompts and encode them into corresponding text features.

[0105] The feature space module stores the matched sample visual features and the corresponding text features in the feature space according to the received data.

[0106] The defect judgment and positioning module performs defect judgment and defect positioning based on the input sample visual features and the data in the feature space module; the defect judgment is specifically as follows:

[0107] Get predefined normal text prompt And abnormal sample prompts ; The abnormal sample prompt A text prompt corresponding to a general exception or a collection of text prompts corresponding to several specific defect types;

[0108] Compute the input sample visual features and predefined normal text prompts And abnormal sample prompts The similarity or distance between them; the similarity is cosine similarity, and the distance is Euclidean distance;

[0109] Define anomaly scores and calculate anomaly scores based on the obtained similarity or distance;

[0110] The anomaly score is compared with a preset threshold. When the anomaly score is greater than the threshold, the corresponding image is an abnormal defective PCB image; when the anomaly score is less than the threshold, it is judged to be normal. The preset threshold is determined by the model on a validation set containing normal samples and abnormal samples based on the expected performance indicators.

[0111] The defect location is specifically as follows:

[0112] Based on the input sample space feature map, calculate the local feature vector and abnormal sample prompts at each spatial position of the feature map or text prompts for specific defect types similarity between

[0113] The similarity map formed by the obtained similarities is up-sampled, and a binary defect location mask is obtained through threshold processing to mark the defect area in the image.

[0114] The defect localization can also be achieved by:

[0115] Based on the input sample spatial feature map, calculate the gradient of the anomaly score for each spatial position on the visual feature map;

[0116] Use the obtained gradient to perform weighted summation on the feature map to obtain a heat map;

[0117] The obtained heat map is upsampled and a binary defect location mask is obtained through threshold processing to mark the defect area in the image.

[0118] The anomaly score is specifically defined as:

[0119] The similarity-based anomaly score is defined using the following formula: ;

[0120] in, is the anomaly score; Calculate the cosine similarity; is the visual feature of the input sample to be tested; the higher the anomaly score, the more similar the image is to the abnormal description and the less similar it is to the normal description;

[0121] The distance-based anomaly score is defined using the following formula: ;

[0122] in, is the anomaly score; It is the Euclidean distance calculation; is the visual feature of the input sample to be tested; the higher the anomaly score obtained, the farther the image is from the normal description and the closer it is to the abnormal description.

[0123] S3. Combining contrast loss and triplet loss to perform dual-constraint optimization, the initial printed circuit board detection model described in step S2 is trained using the training dataset obtained in step S1 to obtain a printed circuit board detection model, specifically:

[0124] The training goal of the model is to adjust the parameters to be learned contained in the visual encoder, text encoder, and sample prompts by optimizing a joint loss function L; the joint loss function L combines the contrast loss and triplet loss , expressed using the following formula: ; in, is a hyperparameter and ;

[0125] The joint loss function L is minimized through the gradient descent optimization algorithm, and all learnable parameters in the model are updated end-to-end.

[0126] The purpose of the contrast loss is to shorten the distance between the positive sample pairs in the feature space or improve the similarity between the two, and push away the negative sample pairs; the positive sample pairs are matched image features and text features, and the negative sample pairs are mismatched image features and text features;

[0127] The contrast loss is expressed using the following formula: ;

[0128] in, For expectations; Calculate the cosine similarity; is the temperature hyperparameter; Normal visual characteristics; is a normal text feature; is the jth text feature;

[0129] The contrastive loss is responsible for establishing basic correspondences and global alignment between modalities;

[0130] The purpose of the triplet loss is to enforce the discriminative boundary between categories in the feature space; the triplet loss requires that the distance between the same category sample and its positive sample is at least a preset margin m smaller than the distance between it and the negative sample of the different category; the triplet loss Use the following formula to express it: ;

[0131] in, is the triplet constraint loss for normal visual features; is the triplet constraint loss for abnormal visual features;

[0132] For normal visual features , its corresponding normal text features is a positive sample, abnormal text feature For negative samples, the triplet constraint loss is expressed using the following formula: ;

[0133] in, is the triplet constraint loss for normal visual features; To obtain the maximum value operation; It is the Euclidean distance calculation; Normal visual characteristics; is a normal text feature; is the abnormal text feature; m is a hyperparameter and m is greater than 0;

[0134] For abnormal visual features , its corresponding normal text features is a positive sample, normal text feature For negative samples, the triplet constraint loss is expressed using the following formula: ;

[0135] in, is the triplet constraint loss for abnormal visual features; To obtain the maximum value operation; It is the Euclidean distance calculation; Abnormal visual features; is a normal text feature; is the abnormal text feature; m is a hyperparameter and m is greater than 0.

[0136] S4. Using the printed circuit board inspection model obtained in step S3, inputting the real-time acquired PCB image for inspection, and completing the printed circuit board defect inspection.

[0137] The present invention also provides a system for implementing the method, the structural diagram of which is shown in FIG. Figure 2 As shown, it includes a data acquisition module, a model building module, a model training module and a circuit board image detection module;

[0138] The data acquisition module obtains normal and defect-free printed circuit board image samples to obtain a training data set, and uploads the data to the model training module;

[0139] The model building module builds an initial PCB inspection model based on cross-modal cue learning and visual guidance, and uploads the data to the model training module;

[0140] The model training module performs dual-constraint optimization based on the received data, combining contrast loss and triplet loss, uses the training dataset to train the initial printed circuit board inspection model, and uploads the data to the circuit board image inspection module;

[0141] The circuit board image detection module uses the printed circuit board detection model based on the received data and inputs the real-time acquired PCB image for detection to complete the printed circuit board defect detection.

[0142] In order to demonstrate the effectiveness and superiority of the method of the present invention, comprehensive experimental evaluations were carried out on multiple public PCB defect detection benchmark datasets (such as VisA, DeepPCB, or other similar datasets mentioned in the original article).

[0143] The experiments strictly follow the setting of few-shot learning, that is, only normal (defect-free) samples are used in the training phase, while mixed samples containing normal and known / unknown types of defects are used in the testing phase.

[0144] The performance of the method of the present invention is compared with a number of representative existing methods, which may include:

[0145] 1. Traditional supervised learning methods (assuming a small number of labeled defect samples are available).

[0146] 2. Advanced unsupervised anomaly detection methods (such as PatchCore, WinClip, etc.).

[0147] 3. Other VLM-based few-shot / zero-shot anomaly detection methods (such as WinCLIP, PromptAD, etc.).

[0148] The evaluation indicators adopt the standards recognized by industry and academia, such as:

[0149] 1. Image-level classification: accuracy, area under the receiver operating characteristic curve (AUC-ROC), etc.

[0150] 2. Pixel-level positioning: accuracy, area under the receiver operating characteristic curve (AUC-ROC), etc.

[0151] The structure of the method of the present invention for detecting and locating defects in PCB images is as follows: Figure 3 shown.

[0152] The experimental results of different methods are shown in Table 1:

[0153] Table 1 Experimental results of different methods for classification and positioning of PCB images

[0154]

[0155] Experimental results demonstrate that our CoPt method significantly outperforms existing comparison methods across all evaluation metrics, achieving state-of-the-art (SOTA) performance. Furthermore, attention heatmaps demonstrate our model's excellent defect localization capabilities. This strongly demonstrates that our method, by combining fine-grained semantic enhancement, prior-based visual anomaly guidance, and a dual-constrained cross-modal alignment strategy, effectively addresses the shortcomings of existing technologies and significantly improves the accuracy and robustness of few-shot PCB defect detection.

Claims

1. A printed circuit board defect detection method based on cross-modal cue learning and visual guidance, characterized in that: The following steps are involved: S1. Obtain normal, defect-free printed circuit board image samples to obtain a training dataset; S2. Constructing an initial PCB inspection model based on cross-modal cue learning and visual guidance; the initial PCB inspection model includes a cue construction module, an anomaly generation and screening module, a feature space module, a feature extraction module, and a defect judgment and location module; S3. Combining contrast loss and triplet loss for dual-constraint optimization, the initial printed circuit board detection model described in step S2 is trained using the training data set obtained in step S1 to obtain a printed circuit board detection model; S4. Using the printed circuit board inspection model obtained in step S3, input the real-time acquired PCB image for inspection to complete printed circuit board defect detection; The anomaly generation and screening module includes an anomaly sample generation module, a dual-channel storage library, and a difficult example screening module. The anomaly sample generation module uses a synthesis method based on structural and physical prior guidance to generate synthetic anomaly samples while considering PCB design rules and material properties. Specifically, when generating synthetic anomaly samples of a specific defect type, the corresponding prior knowledge is used to constrain the generation parameters of the corresponding defect. The dual-channel storage library maintains a corresponding and independent first-in-first-out queue for each normal image sample, and the capacity of the queue is preset to k. When a synthetic abnormal sample is newly generated by a specific normal image sample, it is added to the head of the corresponding first-in-first-out queue. If the number of synthetic abnormal samples in the queue is greater than k at this time, the sample at the end of the queue is removed. The difficult example screening module measures the difficulty of synthesizing abnormal samples for the current model state by calculating the loss perception weight of each synthetic abnormal sample, which is expressed by the following formula: ， in, To synthesize abnormal samples The loss-aware weight of Based on normal image samples The jth synthetic abnormal sample; For the model to detect abnormal samples The total loss value obtained after forward propagation; The difficult example screening module performs dual-channel sampling in the dual-channel repository based on the loss-aware weight to obtain difficult synthetic anomaly samples; the dual-channel sampling includes internal queue sampling and external queue sampling; during the training process, when a normal image sample is processed, two difficult synthetic anomaly samples for the current training step are screened out from the repository, and the two difficult synthetic anomaly samples are obtained by internal queue sampling and external queue sampling respectively; the internal queue sampling is to select the sample with the maximum loss-aware weight from the first-in-first-out queue corresponding to the normal image sample in the dual-channel repository; the external queue sampling selects the sample with the maximum loss-aware weight from the first-in-first-out queue corresponding to the normal image sample in the dual-channel repository; through the dual-channel sampling strategy, the two difficult synthetic anomaly samples are trained together with the original normal sample during training, thereby providing targeted visual anomaly guidance.

2. The printed circuit board defect detection method based on cross-modal prompt learning and visual guidance according to claim 1 is characterized in that: In step S2, the initial printed circuit board inspection model includes a prompt construction module, an anomaly generation and screening module, a feature extraction module, a feature space module, and a defect judgment and positioning module; wherein the anomaly generation and screening module is only used in the model training process; The prompt construction module constructs normal sample prompts and abnormal sample prompts for the input training PCB image and inputs them into the feature extraction module; The abnormal sample generation module performs enhancement operations based on the input training PCB image to generate synthetic abnormal samples and store them in the dual-channel storage library. The difficult example screening module dynamically screens out difficult synthetic abnormal samples based on the synthetic abnormal samples in the dual-channel storage library. The final results obtained by the abnormal generation and screening module are input into the feature extraction module and the prompt construction module. The feature extraction module includes a pre-trained visual encoder and a pre-trained text encoder; the pre-trained visual encoder is used to extract the visual features of the input PCB image; the pre-trained text encoder is used to extract the text features of the normal sample prompts or abnormal sample prompts corresponding to the input PCB image; during training, the visual features and text features ultimately extracted by the feature extraction module are uploaded to the feature space module; in actual use, the visual features ultimately extracted by the feature extraction module are input into the defect judgment and positioning module; The defect judgment and positioning module calculates the similarity or distance between the image features and the text features in the feature space module based on the input visual features to obtain the anomaly score; the image-level defect judgment is completed by comparing the obtained anomaly score with the preset threshold; the feature vector at each spatial position of the input visual features interacts with the text features in the feature space to obtain a similarity map, attention map or heat map, and the obtained similarity map, attention map or heat map is upsampled, and a binary defect positioning mask is obtained through threshold processing to complete the positioning of the image defect area.

3. The printed circuit board defect detection method based on cross-modal prompt learning and visual guidance according to claim 2 is characterized in that: The prompt construction module includes normal sample prompt construction and abnormal sample prompt construction; the normal sample prompt construction is specifically: for normal PCB image data, construct a normal sample prompt ; The normal sample suggests There are N prefix symbols to be learned and fixed category nouns Spliced ​​together, it is expressed using the following formula: ， in, is the concatenation operation of vector or symbol sequence; N is the number of preset prefix symbols; the prefix symbol to be learned The embedding vectors are optimized through back propagation during the model training process, so that the model can automatically learn and capture the contextual semantic features that match; The abnormal sample prompt is specifically constructed by: constructing an abnormal sample prompt based on a pre-trained large language model; the pre-trained large language model includes a pre-input guidance prompt; the guidance prompt includes background knowledge and task requirements in the PCB field; the background knowledge includes standard PCB defect types, morphological feature descriptions, common causes, and related process terms; the task requirement is to instruct the pre-trained large language model to generate several different text descriptions for specific defect types based on the background knowledge; The abnormal sample prompt construction includes two methods: manual design construction and learning construction; The artificial design construction is specifically as follows: for known PCB defects, a pre-trained large language model is used to generate fine-grained defect descriptions , and compare it with the prefix symbol to be learned and fixed category nouns Splicing is performed to obtain artificially designed and constructed abnormal sample prompts , expressed using the following formula: ， in, Tips for abnormal samples constructed for artificial design construction methods; The construction to be learned is specifically: for unknown PCB defects, since they are not To fully cover the potential defect types, we use the to-be-learned construction method to construct abnormal sample prompts and then introduce M adaptive suffix symbols to be learned. , expressed using the following formula: ， in, Abnormal sample tips for the construction method to be learned; M is the number of suffix symbols; adaptive suffix symbols to be learned The embedding vector is optimized through back propagation during the model training process, and the semantic features representing abnormal states are automatically learned from the training data.

4. The printed circuit board defect detection method based on cross-modal prompt learning and visual guidance according to claim 3 is characterized in that: The synthesis method based on structural and physical prior guidance is suitable for the following defect types, including elastic deformation, scratches, and solder joint anomalies: Elastic deformation: To simulate the physical bending or warping of a PCB under stress, a two-dimensional displacement field is calculated. The original image pixels are then resampled according to the displacement field using bilinear interpolation to produce a deformed image. The magnitude and spatial frequency of the displacement field can be constrained by prior knowledge of the board thickness, size, and mounting method. Scratches: To simulate surface scratches, a random path is first generated as the main scratch, with a length and width within a preset range. Branches are then randomly generated along the main scratch to simulate the irregularities of scratching. The pixel values ​​in the scratched area are modified to a specific color that simulates metallic sheen or exposed substrates, and a small amount of noise is added. Solder joint anomalies: To simulate defects such as cold solder joints, poor solder joints, and overheating oxidation, a gradient synthesis method is used to weightedly blend the original image with an image layer that simulates metallic luster and an image layer that simulates the appearance of an oxide layer, and the blending ratio is controlled by the weight.

5. The printed circuit board defect detection method based on cross-modal prompt learning and visual guidance according to claim 3 is characterized in that: The feature extraction module includes a pre-trained visual encoder and a pre-trained text encoder; the pre-trained visual encoder uses a pre-trained visual model as a backbone to extract high-level visual features from the input image; the pre-trained visual model includes a Vision Transformer or ResNet series; The pre-trained text encoder processes text prompts using a pre-trained Transformer model and encodes them into corresponding text features; The feature space module stores the matched sample visual features and the corresponding text features in the feature space according to the received data.

6. The printed circuit board defect detection method based on cross-modal prompt learning and visual guidance according to claim 5 is characterized in that: The defect judgment and positioning module performs defect judgment and defect positioning based on the input sample visual features and the data in the feature space module; the defect judgment is specifically as follows: Get predefined normal text prompt And abnormal sample prompts ; The abnormal sample prompts A text prompt corresponding to a general exception or a collection of text prompts corresponding to several specific defect types; Compute the input sample visual features and predefined normal text prompts And abnormal sample prompts The similarity or distance between them; the similarity is cosine similarity, and the distance is Euclidean distance; Define anomaly scores and calculate anomaly scores based on the obtained similarity or distance; Compare the anomaly score with a preset threshold. When the anomaly score is greater than the threshold, the corresponding image is an abnormal defective PCB image; when the anomaly score is less than the threshold, it is judged to be normal. The preset threshold is determined by the model on a validation set containing normal samples and abnormal samples based on the expected performance indicators; The defect location is specifically as follows: Based on the input sample space feature map, calculate the local feature vector and abnormal sample prompts at each spatial position of the feature map or text prompts for specific defect types similarity between The similarity map formed by the obtained similarities is up-sampled, and a binary defect location mask is obtained through threshold processing to mark the defect area in the image; The defect localization can also be achieved by: Based on the input sample spatial feature map, calculate the gradient of the anomaly score for each spatial position on the visual feature map; Use the obtained gradient to perform weighted summation on the feature map to obtain a heat map; The obtained heat map is upsampled and a binary defect location mask is obtained through threshold processing to mark the defect area in the image.

7. The printed circuit board defect detection method based on cross-modal prompt learning and visual guidance according to claim 6 is characterized in that: The anomaly score is specifically defined as: The similarity-based anomaly score is defined using the following formula: ； in, is the anomaly score; Calculate the cosine similarity; is the visual feature of the input sample to be tested; the higher the anomaly score, the more similar the image is to the abnormal description and the less similar it is to the normal description; The distance-based anomaly score is defined using the following formula: ； in, is the anomaly score; It is the Euclidean distance calculation; is the visual feature of the input sample to be tested; the higher the anomaly score obtained, the farther the image is from the normal description and the closer it is to the abnormal description.

8. The printed circuit board defect detection method based on cross-modal prompt learning and visual guidance according to claim 7 is characterized in that: In step S3, the training goal of the model is to adjust the parameters to be learned contained in the visual encoder, text encoder, and sample prompts by optimizing a joint loss function L; the joint loss function L combines the contrast loss and triplet loss , expressed using the following formula: ;in, is a hyperparameter and ; Minimize the joint loss function L through the gradient descent optimization algorithm, and update all learnable parameters in the model end-to-end; The purpose of the contrast loss is to shorten the distance between the positive sample pairs in the feature space or improve the similarity between the two, and push away the negative sample pairs; the positive sample pairs are matched image features and text features, and the negative sample pairs are mismatched image features and text features; The contrast loss is expressed using the following formula: ； in, For expectations; Calculate the cosine similarity; is the temperature hyperparameter; Normal visual characteristics; is a normal text feature; is the jth text feature; The contrastive loss is responsible for establishing basic correspondences and global alignment between modalities; The purpose of the triplet loss is to enforce the discriminative boundary between categories in the feature space; the triplet loss requires that the distance between the same category sample and its positive sample is at least a preset margin m smaller than the distance between it and the negative sample of the different category; the triplet loss Use the following formula to express it: ； in, is the triplet constraint loss for normal visual features; is the triplet constraint loss for abnormal visual features; For normal visual features , its corresponding normal text features is a positive sample, abnormal text feature For negative samples, the triplet constraint loss is expressed using the following formula: ； in, is the triplet constraint loss for normal visual features; To obtain the maximum value operation; It is the Euclidean distance calculation; Normal visual characteristics; is a normal text feature; is the abnormal text feature; m is a hyperparameter and m is greater than 0; For abnormal visual features , its corresponding normal text features is a positive sample, normal text feature For negative samples, the triplet constraint loss is expressed using the following formula: ； in, is the triplet constraint loss for abnormal visual features; To obtain the maximum value operation; It is the Euclidean distance calculation; Abnormal visual features; is a normal text feature; is the abnormal text feature; m is a hyperparameter and m is greater than 0.

9. A system for implementing the printed circuit board defect detection method based on cross-modal prompt learning and visual guidance as described in any one of claims 1 to 8, characterized in that: It includes data acquisition module, model building module, model training module and circuit board image detection module; The data acquisition module obtains normal and defect-free printed circuit board image samples to obtain a training data set, and uploads the data to the model training module; The model building module builds an initial PCB inspection model based on cross-modal cue learning and visual guidance, and uploads the data to the model training module; The model training module performs dual-constraint optimization based on the received data, combining contrast loss and triplet loss, uses the training dataset to train the initial printed circuit board inspection model, and uploads the data to the circuit board image inspection module; The circuit board image detection module uses the printed circuit board detection model based on the received data and inputs the real-time acquired PCB image for detection to complete the printed circuit board defect detection.

Images