Anomaly detection method, device and medium for a printed circuit board assembly

Abstract

The application discloses an abnormality detection method, device and medium of a printed circuit board assembly, and belongs to the technical field of image processing. The method comprises the following steps: acquiring a to-be-detected image of the printed circuit board assembly; inputting the to-be-detected image of the printed circuit board assembly and a preset template image into a pre-trained target recognition model to obtain a distribution position image of a target object; adopting a connectivity detection algorithm to detect whether the distribution position image is connected, and if the detection result is that the target object is connected, it is determined that the printed circuit board assembly has an abnormality. The technical scheme separates the recognition and judgment logic, the recognition model only needs to complete the learning of an inductive bias task, avoids the model from falling into a detection performance bottleneck, the connectivity detection algorithm improves the anti-interference robustness of the abnormality detection scheme to external conditions, and by inputting the to-be-detected image of the printed circuit board assembly and the preset template image into the target recognition model, the connectivity abnormality of the target object can be efficiently and accurately recognized.

Description

Technical Field

[0001] This application belongs to the field of image processing technology, specifically relating to a method, device and medium for anomaly detection of printed circuit board assemblies. Background Technology

[0002] In the modern electronics manufacturing industry, the soldering quality of printed circuit board (PCB) assemblies directly affects the performance and reliability of electronic products. Therefore, defect detection on PCBs has become an indispensable and crucial step in ensuring the quality of electronic products. As electronic products become increasingly miniaturized, denser, and more complex, even more stringent and precise requirements are placed on defect detection technology.

[0003] Currently, automated optical inspection systems are generally used to acquire images of printed circuit board (PCB) assemblies, and image processing algorithms are used to analyze these images. For example, in anomaly detection scenarios, PCB assembly images can be input into a pre-trained binary classification model. While binary classification models are easy to implement, they have weak inductive bias and are easily affected by external conditions, making them unable to quickly adapt to real-world application scenarios. This significantly reduces the efficiency and accuracy of anomaly detection.

[0004] Therefore, how to quickly adapt to various practical application scenarios, effectively reduce the difficulty of model training, and at the same time improve the detection accuracy of target objects is a problem that urgently needs to be solved by people in this field. Summary of the Invention

[0005] The purpose of this application is to provide a method, device and medium for anomaly detection of printed circuit board assemblies, which can efficiently obtain accurate distribution location images of target objects and accurately identify connectivity anomalies of target objects.

[0006] In a first aspect, embodiments of this application provide a method for detecting anomalies in a printed circuit board assembly, the method comprising:

[0007] Acquire the image of the printed circuit board assembly to be inspected;

[0008] The image to be detected of the printed circuit board assembly and a preset template image are input into a pre-trained target recognition model to obtain a distribution image of the target object; wherein, the preset template image is used to indicate the background features of the image to be detected to the target recognition model;

[0009] A connectivity detection algorithm is used to detect whether the distributed location images are connected. If the detection result shows that the target object is connected, it is determined that the printed circuit board assembly is abnormal.

[0010] The anomaly detection scheme provided in this application separates the identification and judgment logic, which are implemented through an identification model and a connectivity detection algorithm, respectively. The identification model only needs to complete the learning of one inductive bias task, avoiding the model getting stuck in the detection performance bottleneck caused by simultaneously performing the learning of multiple bias tasks. Moreover, the identification model focuses on identifying solder areas, greatly improving the model's processing accuracy; the connectivity detection algorithm has strong resistance to interference from external conditions, improving the robustness of the anomaly detection scheme against external interference. By inputting the image to be detected of the printed circuit board assembly and the preset template image into the target identification model, this application can efficiently obtain an accurate image of the distribution location of the target object and accurately identify the connectivity anomalies of the target object.

[0011] Furthermore, the pre-trained target recognition model is obtained based on a small number of samples using a meta-learning training mechanism.

[0012] The beneficial effect of this scheme is that the meta-learning training mechanism based on a small number of samples enables the target recognition model to learn general learning strategies and prior knowledge, thereby effectively reducing the difficulty of model training and helping the target recognition model to quickly adapt to various practical application scenarios.

[0013] Furthermore, the image to be detected of the printed circuit board assembly and the preset template image are input into a pre-trained target recognition model to obtain the distribution location image of the target object, including:

[0014] The image to be detected of the printed circuit board assembly, the preset template image, and the prompt text are input into the pre-trained target recognition model to obtain the distribution location image of the target object; wherein, the prompt text is used to indicate the background features of the image to be detected to the target recognition model.

[0015] The beneficial effect of this solution is that by inputting the image to be detected of the printed circuit board assembly, the preset template image, and the prompt text into the pre-trained target recognition model, the distribution location image of the target object is obtained. Compared with only inputting the image to be detected of the printed circuit board assembly and the preset template image into the pre-trained target recognition model, the interference of background features of the image to be detected on the recognition and analysis of the target object can be better eliminated, and the detection accuracy of the target object can be further improved.

[0016] Furthermore, the image to be detected of the printed circuit board assembly, the preset template image, and the prompt text are input into a pre-trained target recognition model to obtain an image showing the distribution location of the target object, including:

[0017] The target recognition model determines the prompt vector based on a preset template image and / or prompt text;

[0018] Based on the cue vector, features are extracted from the image of the printed circuit board assembly to be detected at at least two scales to obtain the basic feature images at each scale.

[0019] The basic feature images at each scale are merged to obtain the output feature image;

[0020] The output feature image is deconvolved to obtain the distribution location image of the target object.

[0021] The beneficial effect of this scheme is that by extracting features from the image of the printed circuit board assembly to be detected at at least two scales based on the cue vector, the basic feature images at each scale are obtained. The basic feature images at each scale are merged to obtain the output feature image. Finally, the output feature image is deconvolved to obtain the distribution location image of the target object. This can extract and integrate feature information at multiple scales, and achieve an effective conversion from the image to be detected to the accurate localization of the target object.

[0022] Furthermore, based on the cue vector, features are extracted from the image of the printed circuit board assembly to be detected at at least two scales to obtain basic feature images at each scale, including:

[0023] Features are extracted from the image of the printed circuit board assembly to be inspected at at least two scales to obtain the original feature images at each scale;

[0024] The basic feature image is obtained by cross-correlation between the cue vector and the original feature image at each scale.

[0025] The beneficial effect of this scheme is that by performing cross-correlation operations between the cue vector and the original feature images at various scales to obtain the basic feature image, the intrinsic relationship between the key information contained in the cue vector and the image features can be fully explored, thereby highlighting the feature information closely related to the detection task and effectively reducing the interference of irrelevant information.

[0026] Furthermore, the basic feature images at each scale are merged to obtain the output feature image, including:

[0027] Perform the first convolution operation, scale scaling operation, and second convolution operation on the basic feature images at each scale to obtain the target scale feature image;

[0028] The feature images of each target scale are stacked in the depth direction to obtain the output feature image.

[0029] The beneficial effect of this scheme is that by performing the first convolution operation, scale scaling operation, and second convolution operation on the basic feature images at each scale to obtain the target scale feature image, and stacking the target scale feature images in the depth direction to obtain the output feature image, it can effectively integrate the multi-dimensional feature information contained in the images at different scales, realize all-round feature fusion from local details to overall layout, and thus provide a rich, comprehensive and hierarchical feature foundation for the subsequent determination of target objects.

[0030] Furthermore, a deconvolution operation is performed on the output feature image to obtain a distribution location image of the target object, including:

[0031] The output feature image is deconvolved sequentially through a first deconvolution layer, a second deconvolution layer, and a third deconvolution layer to obtain the distribution location image of the target object.

[0032] The beneficial effect of this scheme is that by sequentially performing deconvolution operations on the output feature image through the first deconvolution layer, the second deconvolution layer, and the third deconvolution layer, the distribution location image of the target object can be obtained. This can gradually restore the spatial information and detailed features lost in the previous series of convolution and scaling operations of the image to be detected, and achieve an effective transformation from abstract feature representation to intuitive visualization of the location distribution of the target object.

[0033] Furthermore, the second deconvolutional layer is associated with a low-rank decomposition adapter module;

[0034] Accordingly, deconvolution operations are performed through a second deconvolution layer, including:

[0035] The output of the first deconvolution layer is deconvolved using the second deconvolution layer and the low-rank decomposition adapter module.

[0036] The beneficial effects of this scheme are as follows: by associating the second deconvolution layer with the low-rank decomposition adapter module, and performing deconvolution operations on the output of the first deconvolution layer through the second deconvolution layer and the low-rank decomposition adapter module, the number of parameters of the adapter module itself is effectively reduced through low-rank decomposition technology. This significantly reduces the demand for computing resources during fine-tuning, enabling smooth model fine-tuning in resource-constrained environments. It effectively alleviates the overfitting problem caused by the difference between the distribution of new data and the original training data during model fine-tuning, ensuring the model's generalization ability on new data, and flexibly adapting the target recognition model. It can improve the accuracy of the target recognition model in real-world scenarios with only a small number of samples, thereby further improving the model's adaptability to new scenarios. While meta-learning is used to obtain a large segmentation model, training such a large segmentation model requires a large amount of data and a long training time. By using the low-rank decomposition adapter module, the model can be quickly trained to adapt to the new scenario based on the training data in the new scenario, significantly improving the adaptability to new scenarios and the model's detection accuracy.

[0037] Furthermore, after determining that the printed circuit board assembly has an abnormality, the method further includes:

[0038] If a connected region of the target object passes through the opposite boundary of the distributed location image, then it is determined that the printed circuit board assembly has a penetration anomaly.

[0039] The beneficial effect of this solution is that, by determining the presence of penetration anomalies in printed circuit board assemblies when the connected regions of the target object simultaneously pass through the opposing boundaries of the distributed location images, the penetration anomalies on printed circuit board assemblies can be identified quickly and accurately.

[0040] Furthermore, the process of pre-training the target recognition model includes:

[0041] Based on the category information of the sample images, determine the training image set and the validation image set;

[0042] The target recognition model is trained based on the training image set to optimize the model parameters of the target recognition model;

[0043] The training effect of the target recognition model is verified based on the set of verification images in order to adjust and optimize the hyperparameters;

[0044] Repeat the above steps until the preset number of repetitions is reached.

[0045] The beneficial effect of this scheme is that it determines the training image set and the validation image set based on the category information of the sample images, trains the target recognition model based on the training image set to optimize the model parameters of the target recognition model, and validates the training effect of the target recognition model based on the validation image set to adjust and optimize the hyperparameters. This can build a target recognition model with high accuracy and strong generalization ability.

[0046] Furthermore, after training the target recognition model based on the training image set to optimize the model parameters of the target recognition model, the method further includes:

[0047] Acquire historical anomaly detection data, and fine-tune the model parameters of the deconvolution part of the target recognition model based on the historical anomaly detection data.

[0048] The beneficial effect of this scheme is that by fine-tuning the model parameters of the deconvolution part of the target recognition model based on historical anomaly detection data, the number of parameters that need to be learned can be greatly reduced, the risk of overfitting of the target recognition model can be reduced, and thus the learning of the target recognition model can be promoted.

[0049] In a second aspect, embodiments of this application provide an electronic device including a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the method described in the first aspect.

[0050] Thirdly, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the first aspect.

[0051] In this embodiment, a print circuit board (PCB) assembly image to be detected is obtained; the image to be detected and a preset template image are input into a pre-trained target recognition model to obtain a distribution image of the target object; wherein, the preset template image is used to indicate the background features of the image to be detected to the target recognition model; a connectivity detection algorithm is used to detect whether the distribution image is connected; if the detection result indicates that the target object is connected, it is determined that the PCB assembly has an anomaly. The technical solution provided in this embodiment decouples the model's recognition and judgment logic, implementing them separately through a recognition model and a connectivity detection algorithm. The recognition model only needs to complete the learning of an inductive bias task, avoiding the model getting stuck in the detection performance bottleneck. The recognition model focuses on recognizing solder areas, greatly improving the model's recognition accuracy. Moreover, the connectivity detection algorithm has strong resistance to interference from external conditions. Compared with the method of judging connectivity through a model, this application does not require learning and training for judging connectivity, reducing model training data and training time. Simultaneously, the judgment logic of the recognition model is relatively transparent; by recognizing the solder area and visualizing it on the interface, it can help users understand the model's behavior. Furthermore, the model's judgment logic can be inferred from the model results, making the deterministic judgment logic more reliable.

[0052] This application efficiently obtains accurate distribution images of target objects by inputting the images of the printed circuit board assembly to be detected and preset template images into the target recognition model, thereby improving the accuracy of detecting connectivity anomalies in target objects. Furthermore, the meta-learning training mechanism based on a small number of samples enables the target recognition model to learn general learning strategies and prior knowledge, effectively reducing the difficulty of model training and helping the target recognition model quickly adapt to various practical application scenarios. Attached Figure Description

[0053] Figure 1 This is a flowchart illustrating the anomaly detection method for printed circuit board assemblies provided in the embodiments of this application;

[0054] Figure 2 This is a flowchart illustrating the anomaly detection method for printed circuit board assemblies provided in the embodiments of this application;

[0055] Figure 3 This is a schematic diagram of the target recognition model provided in the embodiments of this application;

[0056] Figure 4 This is a flowchart illustrating the anomaly detection method for printed circuit board assemblies provided in the embodiments of this application;

[0057] Figure 5 This is a flowchart illustrating the process of pre-training the target recognition model provided in the embodiments of this application;

[0058] Figure 6 This is a schematic diagram of the structure of the abnormality detection device for printed circuit board assembly provided in the embodiments of this application;

[0059] Figure 7 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0060] To make the objectives, technical solutions, and advantages of this application clearer, specific embodiments of this application will be described in further detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are merely for explaining this application and not for limiting it. It should also be noted that, for ease of description, only the parts relevant to this application are shown in the drawings, not all of them. Before discussing exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe operations (or steps) as sequential processes, many of these operations can be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the operations can be rearranged. The process can be terminated when its operation is completed, but may also have additional steps not included in the drawings. The process can correspond to a method, function, procedure, subroutine, subprogram, etc.

[0061] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0062] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0063] The following description, in conjunction with the accompanying drawings, details the anomaly detection method, equipment, and medium for printed circuit board assemblies provided in this application through specific embodiments and application scenarios.

[0064] In modern electronics manufacturing, the soldering quality of printed circuit board assemblies (PCBAs) is crucial to the performance and reliability of electronic products. A PCB refers to a printed circuit board that has undergone a series of processing steps, including component mounting and soldering. Anomaly detection of PCBs is a key step in ensuring product quality. Anomaly detection includes soldering defect detection (such as solder bridging), scratch detection, and foreign object detection. This application document uses solder bridging detection as an example.

[0065] Currently, common methods for detecting solder bridging involve using automated optical inspection systems to capture images of printed circuit board (PCB) components and analyzing these images using image processing algorithms. The images are input into a pre-trained binary classification model, which outputs the presence or absence of solder bridging. This model requires two inductive bias tasks: identifying the location of the solder paste and determining whether it is connected within the image's field of view. During training, the lack of supervised information makes it difficult for the binary classification model to simultaneously learn these two inductive bias tasks, easily leading to performance bottlenecks. Furthermore, the binary classification model is highly susceptible to the influence of training samples, such as image lighting conditions. Different lighting conditions result in poor accuracy in determining connectivity. Moreover, this model struggles to adapt quickly to real-world application scenarios, leading to low efficiency and accuracy in solder bridging detection.

[0066] To address the aforementioned shortcomings, the anomaly detection scheme provided in this application decouples the identification and judgment logic, implementing them separately through an identification model and a connectivity detection algorithm. The model used in this anomaly detection scheme only needs to complete the learning of one inductive bias task, avoiding the performance bottleneck caused by simultaneously performing multiple bias tasks. Furthermore, the identification model focuses on identifying solder areas, significantly improving model processing accuracy and training efficiency. Compared to connectivity detection schemes that rely on models, connectivity detection algorithms are more robust to external interference. Moreover, the connectivity judgment scheme provided in this application does not require training to determine connectivity, reducing model training data and time. Simultaneously, the judgment logic of the identification model is relatively transparent; by identifying solder paste areas and visualizing them on the interface, it helps users understand the model's behavior. Furthermore, the model's judgment logic can be inferred from the model results, making the deterministic judgment logic more reliable.

[0067] In addition, the meta-learning training mechanism based on a small number of samples enables the target recognition model to learn general learning strategies and prior knowledge, thereby effectively reducing the difficulty of model training and helping the target recognition model to quickly adapt to various practical application scenarios.

[0068] Figure 1 This is a flowchart illustrating the anomaly detection method for printed circuit board assemblies provided in an embodiment of this application. Figure 1 As shown, the specific steps include the following:

[0069] S101, Obtain the image of the printed circuit board assembly to be inspected;

[0070] First, this solution is applicable to scenarios requiring the detection of soldering defects in printed circuit board (PCB) assemblies, such as detecting solder bridging. Based on these application scenarios, it is understood that the subject of this application can be an electronic device with image and data processing capabilities, such as a mobile phone, tablet computer, desktop computer, or a large conference display panel and other smart terminals.

[0071] The image to be detected can be the image corresponding to the area to be tested in the printed circuit board assembly.

[0072] In one embodiment, acquiring the image of a printed circuit board assembly to be inspected can be achieved by using an image acquisition system to obtain an image of the printed circuit board assembly under AOI (Automated Optical Inspection) lighting, and then selecting regions in the image according to a preset region selection rule to obtain the image to be inspected. The image acquisition system may include an industrial high-precision camera; the AOI light may consist of lighthouse-shaped RGB (Red, Green, Blue) LEDs, which can display different RGB colors at different heights of the printed circuit board assembly; the preset region selection rule may be a rule for selecting regions in the image based on more detailed inspection requirements of the printed circuit board assembly. For example, if the user needs to detect whether there is a penetration anomaly in the printed circuit board assembly, the preset region selection rule may be to select regions in the image where penetration anomalies are likely to occur, and the size of the resulting image to be inspected may be similar to the length of the penetration anomaly.

[0073] S102, the image to be detected of the printed circuit board assembly and the preset template image are input into the pre-trained target recognition model to obtain the distribution location image of the target object; wherein, the pre-trained target recognition model is obtained based on a small number of samples using a meta-learning training mechanism, and the preset template image is used to indicate the background features of the image to be detected to the target recognition model;

[0074] Tin-based solders are commonly used for soldering printed circuit boards (PCBs) to create PCB assemblies because they have a lower soldering temperature, resulting in less thermal shock to electronic components. Additionally, tin itself has relatively mild chemical properties and will not corrode or cause other harmful effects on component materials during soldering. The target object refers to the solder paste remaining on the PCB assembly after soldering with tin-based solder.

[0075] The distribution image of the target object can be a binary mask image, where one value indicates that the pixel belongs to the target object and the other value indicates that the pixel does not belong to the target object.

[0076] The target recognition model can be a model that can autonomously analyze and output an image of the distribution location of target objects based on an input image of a printed circuit board assembly to be detected. The target recognition model can be obtained through a meta-learning training mechanism based on a small number of samples.

[0077] In one feasible embodiment, optionally, the pre-trained target recognition model is obtained based on a small number of samples using a meta-learning training mechanism.

[0078] Here, "small number of samples" can refer to the fact that, compared to the large number of training samples required for training traditional deep learning models, object recognition models only need a small number of training samples to complete model training.

[0079] Meta-learning training mechanism can be understood as "learning how to learn". It is a method that can effectively learn using a small number of samples. The core of this mechanism is to enable the model to quickly adapt to new tasks or new data distributions by learning on multiple related small tasks and extracting general learning strategies or prior knowledge.

[0080] This solution, based on a meta-learning training mechanism using a small number of samples, enables the target recognition model to learn general learning strategies and prior knowledge, thereby effectively reducing the difficulty of model training and helping the target recognition model to quickly adapt to various practical application scenarios.

[0081] In one embodiment, the process of pre-training an object recognition model may include: determining a training image set and a validation image set based on the category information of the sample images; training the object recognition model based on the training image set to optimize the model parameters of the object recognition model; validating the training effect of the object recognition model based on the validation image set to adjust and optimize the hyperparameters; and repeating the above steps until the number of repetitions reaches the preset number of training iterations.

[0082] The preset template image can be an image of a printed circuit board assembly without any abnormalities, which can indicate the background features of the image to be detected (such as silkscreen on the printed circuit board assembly) to the target recognition model, thereby preventing the background features from interfering with the recognition and analysis of the target object.

[0083] In one embodiment, the method of inputting the image to be detected of the printed circuit board assembly and the preset template image into a pre-trained target recognition model to obtain the distribution location image of the target object can be achieved by the target recognition model determining a cue vector based on the preset template image, extracting features from the image to be detected of the printed circuit board assembly at at least two scales based on the cue vector to obtain basic feature images at each scale, merging the basic feature images at each scale to obtain an output feature image, and performing a deconvolution operation on the output feature image to obtain the distribution location image of the target object.

[0084] S103, a connectivity detection algorithm is used to detect whether the distributed location image is connected. If the detection result is that the target object is connected, it is determined that the printed circuit board assembly is abnormal.

[0085] Among them, connectivity detection algorithms are algorithms used to determine whether there are connections between elements in a graph. They can include flood fill algorithm, DFS (Depth First Search) or BFS (Breadth First Search), etc.

[0086] Among them, detecting whether the distribution location image is connected means detecting whether the pixels representing the target object in the distribution location image are connected.

[0087] The target object can be solder paste. Correspondingly, if the target object is connected, it means that the solder paste is connected, which means that there is solder bridging in the printed circuit board assembly, that is, there is an abnormality in the printed circuit board assembly.

[0088] The technical solution provided in this application embodiment obtains a print circuit board assembly image to be detected; inputs the print circuit board assembly image to be detected and a preset template image into a pre-trained target recognition model to obtain a distribution location image of the target object; wherein, the preset template image is used to indicate the background features of the image to be detected to the target recognition model; a connectivity detection algorithm is used to detect whether the distribution location image is connected, and if the detection result is that the target object is connected, it is determined that the print circuit board assembly has an anomaly. The above-mentioned anomaly detection method for print circuit board assemblies decouples the model's recognition and judgment logic, which are implemented by the recognition model and the connectivity detection algorithm respectively. The recognition model only needs to complete the learning of an inductive bias task, avoiding the model from getting stuck in the detection performance bottleneck. The recognition model focuses on recognizing solder areas, which greatly improves the model's recognition accuracy. Moreover, the connectivity detection algorithm has strong resistance to interference from external conditions. Compared with the method of judging connectivity through the model, this application does not need to learn and train the judgment connectivity, reducing model training data and training time. Meanwhile, the judgment logic of the recognition model is relatively transparent. By identifying solder areas and visualizing them on the interface, users can understand the model's behavior. Furthermore, the model's judgment logic can be inferred from the model results, making the deterministic judgment logic more reliable. By inputting the image to be detected of the printed circuit board assembly and the preset template image into the target recognition model, the distribution location image of the target object can be obtained efficiently, improving the accuracy of detecting connectivity anomalies in the target object. In addition, the meta-learning training mechanism based on a small number of samples enables the target recognition model to learn general learning strategies and prior knowledge, thereby effectively reducing the difficulty of model training and helping the target recognition model quickly adapt to various practical application scenarios.

[0089] Figure 2 This is a flowchart illustrating the anomaly detection method for printed circuit board assemblies provided in an embodiment of this application. Figure 2 As shown, the specific steps include the following:

[0090] S201, Obtain the image of the printed circuit board assembly to be inspected;

[0091] S202, the image to be detected of the printed circuit board assembly, the preset template image, and the prompt text are input into the pre-trained target recognition model to obtain the distribution location image of the target object; wherein, the prompt text is used to indicate the background features of the image to be detected to the target recognition model;

[0092] The prompt text can be user-input descriptive text describing the background features of the image to be detected, such as "This is a."

[0093] In one embodiment, the method of inputting the image to be detected of the printed circuit board assembly, the preset template image, and the prompt text into a pre-trained target recognition model to obtain the distribution location image of the target object can be achieved by the target recognition model determining the prompt vector based on the preset template image and / or the prompt text, extracting features from the image to be detected of the printed circuit board assembly at at least two scales based on the prompt vector to obtain the basic feature images at each scale, merging the basic feature images at each scale to obtain the output feature image, and performing a deconvolution operation on the output feature image to obtain the distribution location image of the target object.

[0094] In one feasible embodiment, optionally, the image to be detected of the printed circuit board assembly, a preset template image, and prompt text are input into a pre-trained target recognition model to obtain a distribution location image of the target object, including:

[0095] The target recognition model determines the prompt vector based on a preset template image and / or prompt text;

[0096] Based on the cue vector, features are extracted from the image of the printed circuit board assembly to be detected at at least two scales to obtain the basic feature images at each scale.

[0097] The basic feature images at each scale are merged to obtain the output feature image;

[0098] The output feature image is deconvolved to obtain the distribution location image of the target object.

[0099] Among them, a vector is a data form that the target recognition model can read, and the prompt vector is the vector extracted based on the preset template image and / or prompt text.

[0100] In one embodiment, the target recognition model determines the prompt vector based on a preset template image and / or prompt text by using a prompt word encoder. The prompt word encoder is an important component in the field of natural language processing, and its main function is to convert prompt words in natural language form into a vector representation that can be processed by a computer.

[0101] Scale can be a measure of the level of detail in an observed image. Understandably, feature images at different scales represent features of different dimensions, and their dimensions also differ in their representation.

[0102] Features can refer to representative information in an image that can be used to distinguish different objects or states, such as edge information and texture information.

[0103] In one embodiment, based on the cue vector, features are extracted from the image of the printed circuit board assembly to be detected at at least two scales to obtain basic feature images at each scale. This can be achieved by extracting features from the image of the printed circuit board assembly to be detected at at least two scales to obtain original feature images at each scale, and then performing cross-correlation operations between the cue vector and the original feature images at each scale to obtain basic feature images.

[0104] In one feasible embodiment, optionally, based on the cue vector, features are extracted from the image to be detected of the printed circuit board assembly at at least two scales to obtain basic feature images at each scale, including:

[0105] Features are extracted from the image of the printed circuit board assembly to be inspected at at least two scales to obtain the original feature images at each scale;

[0106] The basic feature image is obtained by cross-correlation between the cue vector and the original feature image at each scale.

[0107] Figure 3 This is a schematic diagram of the target recognition model provided in an embodiment of this application. For example... Figure 3 As shown, the target recognition model mainly consists of three parts: feature extraction, scale merging, and deconvolution.

[0108] The feature extraction part of the target recognition model can be used to extract features from the image of the printed circuit board assembly to be detected at at least two scales. The feature extraction part can be a pre-trained backbone network, which is a basic, core supporting deep neural network architecture mainly used to extract image features.

[0109] The original feature image can refer to the feature image that has not yet been referenced and prompted. The original feature image is the output result of the backbone.

[0110] Cross-correlation is a fundamental operation in signal and image processing that can be used to filter images.

[0111] This scheme obtains a basic feature image by cross-correlation between the cue vector and the original feature image at various scales. This fully explores the key information contained in the cue vector and the intrinsic relationship between the image features, thereby highlighting the feature information closely related to the detection task and effectively reducing the interference of irrelevant information.

[0112] The output feature image can be a feature image that integrates feature information from various scales.

[0113] In one embodiment, the method of merging the basic feature images at each scale to obtain the output feature image can be to perform a first convolution operation, a scale scaling operation, and a second convolution operation on the basic feature images at each scale to obtain the target scale feature image, and then stack the target scale feature images in the depth direction to obtain the output feature image.

[0114] In one feasible embodiment, optionally, the base feature images at each scale are merged to obtain an output feature image, including:

[0115] Perform the first convolution operation, scale scaling operation, and second convolution operation on the basic feature images at each scale to obtain the target scale feature image;

[0116] The feature images of each target scale are stacked in the depth direction to obtain the output feature image.

[0117] The first and second convolution operations are both convolution operations. Convolution is a mathematical operation that is widely used in many fields such as signal processing, image processing, and deep learning. The core idea of ​​convolution is to slide a small window called the convolution kernel over the input data and perform weighted summation and other processing on the data within the coverage area, thereby extracting or transforming the corresponding features.

[0118] Among them, scaling is an operation in the field of image processing used to change the size of an image. Through specific algorithms and rules, the image is adjusted in the horizontal and vertical directions according to a certain proportion or specific size requirements, so that the image appears to be enlarged or reduced.

[0119] The target scale feature image is the base feature image scaled to the same size and then convolved. The target scale is generally consistent with the scale of the printed circuit board assembly image to be inspected.

[0120] The depth direction can refer to the dimension perpendicular to the width and height dimensions of the image.

[0121] In one embodiment, stacking the feature images of each target scale in the depth direction can be achieved by arranging the feature images of each target scale sequentially along the depth direction. From a data structure perspective, this is similar to performing a splicing operation in a channel dimension to obtain the output feature image.

[0122] This scheme obtains target-scale feature images by performing a first convolution operation, a scale scaling operation, and a second convolution operation on the basic feature images at each scale, and then stacks the target-scale feature images in the depth direction to obtain the output feature image. This can effectively integrate the multi-dimensional feature information contained in images at different scales, and achieve all-round feature fusion from local details to overall layout, thereby providing a rich, comprehensive and hierarchical feature foundation for the subsequent determination of target objects.

[0123] Among them, deconvolution, also known as transpose convolution, is an operation widely used in deep learning, especially in convolutional neural networks. It aims to reverse the transformation of the feature image obtained after convolution, and to a certain extent recover the image data that corresponds to the original input image in terms of size, feature distribution, etc.

[0124] In one embodiment, the method of performing deconvolution on the output feature image to obtain the distribution location image of the target object can be to sequentially perform deconvolution on the output feature image through a first deconvolution layer, a second deconvolution layer, and a third deconvolution layer to obtain the distribution location image of the target object.

[0125] In one feasible embodiment, optionally, the output feature image is deconvolved to obtain a distribution location image of the target object, including:

[0126] The output feature image is deconvolved sequentially through a first deconvolution layer, a second deconvolution layer, and a third deconvolution layer to obtain the distribution location image of the target object.

[0127] like Figure 3 As shown, the deconvolution part in the target prediction model can be used to perform deconvolution operations on the output feature image to obtain the distribution location image of the target object. The deconvolution part can include a first deconvolution layer, a second deconvolution layer, and a third deconvolution layer.

[0128] The first, second, and third deconvolution layers are all deconvolution layers, and each deconvolution layer can be used to perform a deconvolution operation on the output of the previous layer.

[0129] In one feasible embodiment, optionally, the second deconvolutional layer is associated with a low-rank decomposition adapter module;

[0130] Accordingly, deconvolution operations are performed through a second deconvolution layer, including:

[0131] The output of the first deconvolution layer is deconvolved using the second deconvolution layer and the low-rank decomposition adapter module.

[0132] The Low-Rank Adaptation module is a technical component used for efficient fine-tuning in deep learning models. Its main purpose is to learn new knowledge through low-rank decomposition to adapt to specific downstream tasks without significantly changing the structure of the original pre-trained model.

[0133] In one embodiment, the method of deconvolving the output of the first deconvolution layer through the second deconvolution layer and the low-rank decomposition adapter module can be implemented by freezing the model parameters of the target recognition model when the performance of the target recognition model is poor, training only the low-rank decomposition adapter module, and in the actual deconvolution operation, first deconvolving the output of the first deconvolution layer through the second deconvolution layer, and then optimizing the output of the second deconvolution layer through the low-rank decomposition adapter module, or first deconvolving the output of the first deconvolution layer through the second deconvolution layer and the low-rank decomposition adapter module respectively, and then combining the two outputs.

[0134] This solution associates a second deconvolutional layer with a low-rank decomposition adapter module. The output of the first deconvolutional layer is deconvolved through both the second deconvolutional layer and the low-rank decomposition adapter module. The low-rank decomposition technique effectively reduces the number of parameters in the adapter module, significantly lowering the computational resource requirements during fine-tuning and enabling smooth model fine-tuning even in resource-constrained environments. It effectively alleviates overfitting caused by differences between the distribution of new and original training data during fine-tuning, ensuring the model's generalization ability on new data. It allows for flexible adaptive adjustments to the target recognition model, improving accuracy in real-world scenarios with only a small number of samples, further enhancing the model's adaptability to new scenarios. While meta-learning yields large segmentation models, training such models requires a large amount of data and a long training time. The low-rank decomposition adapter module allows for rapid training of models adapted to new scenarios based on training data from those scenarios, significantly improving adaptability and detection accuracy.

[0135] This scheme sequentially performs deconvolution operations on the output feature image through the first, second, and third deconvolution layers to obtain the distribution location image of the target object. It can gradually restore the spatial information and detailed features lost in the previous series of convolution and scaling operations of the image to be detected, and realize the effective transformation from abstract feature representation to intuitive visualization of the location distribution of the target object.

[0136] This scheme extracts features from the image of the printed circuit board assembly to be detected at at least two scales based on cue vectors to obtain basic feature images at each scale. The basic feature images at each scale are then merged to obtain an output feature image. Finally, the output feature image is deconvolved to obtain the distribution location image of the target object. This method can extract and integrate feature information at multiple scales, achieving an effective conversion from the image to be detected to the precise localization of the target object.

[0137] This solution obtains the distribution location image of the target object by inputting the image of the printed circuit board assembly to be detected, the preset template image, and the prompt text into a pre-trained target recognition model. Compared with only inputting the image of the printed circuit board assembly to be detected and the preset template image into the pre-trained target recognition model, this solution can better eliminate the interference of background features of the image to be detected on the recognition and analysis of the target object, and further improve the detection accuracy of the target object.

[0138] S203, a connectivity detection algorithm is used to detect whether the distributed location image is connected. If the detection result is that the target object is connected, it is determined that the printed circuit board assembly is abnormal.

[0139] This solution obtains the distribution location image of the target object by inputting the image to be detected of the printed circuit board assembly and the preset template image into a pre-trained target recognition model. This can eliminate the interference of background features of the image to be detected on the recognition and analysis of the target object, thereby improving the accuracy of the determined distribution location of the target object.

[0140] Figure 4 This is a flowchart illustrating the anomaly detection method for printed circuit board assemblies provided in an embodiment of this application. Figure 4 As shown, the specific steps include the following:

[0141] S401, Obtain the image of the printed circuit board assembly to be inspected;

[0142] S402, the image to be detected of the printed circuit board assembly and the preset template image are input into the pre-trained target recognition model to obtain the distribution location image of the target object; wherein, the preset template image is used to indicate the background features of the image to be detected to the target recognition model;

[0143] In one feasible embodiment, optionally, after obtaining the distribution location image of the target object, the method further includes:

[0144] An erosion and dilation operation is applied to the distribution location image to remove scattered distribution noise of the target object in the distribution location image.

[0145] Erosion and dilation are commonly used morphological image processing operations in digital image processing and computer vision. Specifically, erosion involves using a structuring element of a specific shape (usually a rectangle, circle, or cross) to slide across the image, thus "shrinking" the target object (such as the object's outline or connected regions). Dilation, on the other hand, is the opposite of erosion, also using a structuring element of a specific shape to slide across the image, but instead of erosion, it "expands" the target object.

[0146] Scattered noise can refer to pixels or small regions in the image that exist in a scattered and isolated state and do not conform to the normal distribution pattern and characteristics of the target object.

[0147] This scheme uses erosion and dilation operations on the distribution location image to remove scattered noise from the target object in the image, which can effectively improve the clarity and purity of the distribution location image, making the distribution shape and outline of the target object more accurately presented. This is beneficial for subsequent detection of whether the target object is connected and reduces errors caused by noise interference.

[0148] S403, a connectivity detection algorithm is used to detect whether the distributed location image is connected. If the detection result is that the target object is connected, it is determined that the printed circuit board assembly is abnormal.

[0149] S404, if a connected region of the target object passes through the opposite boundary of the distributed location image, then it is determined that the printed circuit board assembly has a penetration anomaly.

[0150] Among them, a through-through anomaly can refer to a situation where solder paste is connected and has a certain length.

[0151] Here, the boundary can refer to the edge portion of the distribution location image, which can be understood as the pixel region of the outermost layer of the distribution location image. It can be understood that opposing boundaries can include a set of top and bottom boundaries and / or a set of left and right boundaries.

[0152] Understandably, if a connected region of the target object passes through the opposite boundary of the distributed location image, it means that the length of the target object has exceeded the size of the image to be detected. This connection situation, which exceeds the normal range and crosses the opposite boundary, indicates that there is a through solder bridging anomaly on the printed circuit board assembly.

[0153] In one feasible embodiment, optionally, after acquiring the image to be inspected of the printed circuit board assembly, the method further includes:

[0154] The image to be detected is rotated to the target recognition direction according to a preset rotation rule; wherein, the target recognition direction includes the horizontal direction or the vertical direction;

[0155] Correspondingly, if a connected region of the target object passes through the opposing boundary of the distributed location image, then it is determined that the printed circuit board assembly has a penetration anomaly, including:

[0156] If the target recognition direction is horizontal, and a connected region of the target object passes through both the left and right boundaries of the distributed location image, then it is determined that the printed circuit board assembly has a penetration anomaly.

[0157] or,

[0158] If the target recognition direction is vertical, and a connected region of the target object passes through both the upper and lower boundaries of the distributed location image, then it is determined that the printed circuit board assembly has a penetration anomaly.

[0159] In this process, if the presence of solder bridging in the images to be inspected can be pre-identified, the images containing solder bridging are rotated until the solder bridging directions in each image are relatively consistent. The preset rotated solder bridging direction is the target recognition direction. The target recognition direction can be horizontal or vertical. The horizontal direction can refer to a left-to-right or right-to-left direction, and the vertical direction can refer to a top-to-bottom or bottom-to-top direction. Correspondingly, the preset rotation rule can be a rule used to rotate the images to be inspected to the target recognition direction, and can be implemented by a custom program.

[0160] Understandably, if the target recognition direction is horizontal, the connected regions of the target object can generally only pass through the left and right boundaries of the distribution location image at the same time; if the target recognition direction is vertical, the connected regions of the target object can generally only pass through the upper and lower boundaries of the distribution location image at the same time.

[0161] This solution rotates the image to be detected to the target recognition direction according to preset rotation rules, which can effectively standardize the direction of anomaly detection and avoid misjudgment or omission of detection results due to the randomness of image direction, making the judgment of through anomalies more consistent and accurate.

[0162] This solution can quickly and accurately identify penetration anomalies on printed circuit board assemblies by determining that there are penetration anomalies in the connected regions of the target object and at the opposite boundaries of the distributed location image.

[0163] Figure 5 This is a flowchart illustrating the process of pre-training the target recognition model provided in the embodiments of this application.

[0164] like Figure 5 As shown, the specific steps include the following:

[0165] S501, Based on the category information of the sample images, determine the training image set and the validation image set;

[0166] The sample image can refer to the image of the printed circuit board assembly to be detected, which is used to pre-train the target recognition model. In the sample image, each solder paste position is marked with a polygonal bounding box.

[0167] The category information can refer to the object category included in the sample image, which may include black background, red background, wire-like solder bridging, columnar solder bridging, flux, and silkscreen images, etc.

[0168] The training image set can be a set of sample images that need to be input into the target recognition model to train the target recognition model; the validation image set can be a set of sample images used to validate the training effect of the target recognition model.

[0169] In one embodiment, the method for determining the training image set and the validation image set based on the category information of the sample images can be as follows: randomly selecting a predetermined number of category information from predetermined category information as target category information; randomly selecting a predetermined number of sample images from the sample image sets associated with each target category information as the training image set; and randomly selecting a predetermined number of sample images from the sample image sets associated with each target category information as the validation image set. The training image set and the validation image set are disjoint.

[0170] In one feasible embodiment, optionally, the training image set and the validation image set are determined based on the category information of the sample images, including:

[0171] Randomly select a preset number of category information from the preset category information as the target category information;

[0172] A predetermined number of sample images are randomly selected from the sample image set associated with each target category information to form the training image set;

[0173] A preset number of sample images are randomly selected from the sample image set associated with each target category information to serve as the verification image set; wherein the training image set and the verification image set do not intersect.

[0174] The preset category information can be a set of object categories included in each sample image that has been pre-defined.

[0175] The preset number of categories can be a pre-defined number of categories to be trained in each training task. As an example, the preset number of categories can be 5.

[0176] Random selection is a method for obtaining elements from a given set. Its core characteristics are that each element has an equal probability of being selected, and the selection process is unpredictable and unaffected by any fixed pattern or order set by humans. Accordingly, the category information of the preset number of randomly selected categories is the target category information for this training task.

[0177] Each category information is associated with a set of sample images. A set of sample images associated with a category information can refer to a set of sample images of objects that include that category information. Similarly, the set of sample images associated with target category information can refer to a set of sample images of objects that include target category information.

[0178] The preset sample size can be a pre-defined number of sample images required to train the target category information in each training task. As an example, the preset sample size could be 5. Understandably, with both the preset number of categories and the preset sample size being 5, the number of sample images in both the training and validation image sets would be 25.

[0179] The fact that the training image set and the validation image set are disjoint can mean that the sample images in the training image set and the sample images in the validation image set are not repeated.

[0180] This scheme randomly selects a preset number of category information from preset category information as target category information, and randomly selects a preset number of sample images from the sample image set associated with each target category information as training image set and validation image set, wherein the training image set and validation image set do not overlap. The dataset constructed by this random sampling method can comprehensively and evenly cover the sample features of different categories, allowing the target recognition model to fully access diverse image information during the learning process, thereby improving its ability to recognize target objects in various printed circuit board components and its generalization performance.

[0181] S502, The target recognition model is trained based on the training image set to optimize the model parameters of the target recognition model;

[0182] Here, model parameters can refer to the variables within the target recognition model.

[0183] In one embodiment, training a target recognition model based on a training image set to optimize the model parameters can be achieved by inputting the training image set into the target recognition model, obtaining the output of the target recognition model, and then performing backpropagation based on the output and pre-annotated data of the training image set to optimize the model parameters.

[0184] Among them, such as Figure 3 As shown, after the target prediction model obtains the output feature image, it can first output a judgment result determining whether an anomaly exists based on the output feature image. Based on this judgment result and the pre-annotated results of whether anomalies exist in each sample image of the training image set, the model parameters of the feature extraction part and the scale merging part are optimized. The process of determining whether an anomaly exists based on the output feature image can be implemented by a fully connected layer. In a fully connected layer, each neuron in the previous layer is connected to every neuron in the current layer, meaning that the neurons are in a "fully connected" relationship.

[0185] In one feasible embodiment, optionally, after training the target recognition model based on the training image set to optimize the model parameters of the target recognition model, the method further includes:

[0186] Acquire historical anomaly detection data, and fine-tune the model parameters of the deconvolution part of the target recognition model based on the historical anomaly detection data.

[0187] Historical anomaly detection data refers to all sets of anomaly detection data recorded up to the current time point. Each set of anomaly detection data may include a target image, a preset template image, and the corresponding output image showing the distribution locations of the target objects.

[0188] In one embodiment, after the target recognition model outputs an image showing the distribution location of the target object, the input image to be detected, the preset template image, and the corresponding output image showing the distribution location of the target object are associated and stored to obtain a set of anomaly detection data. Then, the anomaly detection data sets are associated and stored in chronological order to obtain historical anomaly detection data. Accordingly, historical anomaly detection data can be obtained by reading the aforementioned associated and stored data.

[0189] In one embodiment, fine-tuning the model parameters of the deconvolution part of the target recognition model based on historical anomaly detection data can be achieved by determining the loss curve of the target recognition model based on the historical anomaly detection data, determining the contribution of the deconvolution part to the loss based on the loss curve, and determining the direction of fine-tuning based on the contribution of the deconvolution part to the loss. For example, if the loss is mainly caused by the inability of the deconvolutioned feature map to recover the position information of the target object well, the stride of the deconvolution kernel can be appropriately reduced to make the upsampled feature map more refined, thereby enhancing the ability of the deconvolution part to recover position information.

[0190] This scheme, by fine-tuning the model parameters of the deconvolution part of the target recognition model based on historical anomaly detection data, can greatly reduce the number of parameters that need to be learned, reduce the risk of overfitting of the target recognition model, and thus promote the learning of the target recognition model.

[0191] S503, The training effect of the target recognition model is verified based on the verification image set in order to adjust and optimize the hyperparameters;

[0192] Hyperparameters, in this context, refer to parameters that are pre-set before training machine learning and deep learning models. These differ from model parameters, which are learned from data during training. Hyperparameters, on the other hand, control the training process and the model structure itself. Examples of hyperparameters include the learning rate, which specifically determines the step size for updating model parameters in each training iteration.

[0193] In one embodiment, the training effect of the target recognition model is validated based on a set of validation images to adjust and optimize the hyperparameters. This can be achieved by determining the values ​​of each validation metric for the target recognition model based on the set of validation images, and then adjusting and optimizing the hyperparameters accordingly. For example, if the convergence speed of the target recognition model is too slow, the learning rate can be appropriately increased; if the target recognition model oscillates and fails to converge, the learning rate can be decreased.

[0194] S504, Repeat the above steps until the preset number of repetitions is reached.

[0195] The preset number of training sessions can refer to the number of training tasks set under the meta-learning training mechanism.

[0196] This scheme determines the training image set and the validation image set based on the category information of the sample images. The target recognition model is trained based on the training image set to optimize the model parameters. The training effect of the target recognition model is validated based on the validation image set to adjust and optimize the hyperparameters. This can build a target recognition model with high accuracy and strong generalization ability.

[0197] Figure 6 This is a schematic diagram of the structure of the anomaly detection device for a printed circuit board assembly provided in an embodiment of this application. Figure 6 As shown, it specifically includes the following:

[0198] Image acquisition module 610 is used to acquire the image to be inspected of the printed circuit board assembly;

[0199] The target recognition module 620 is used to input the image to be detected of the printed circuit board assembly and the preset template image into the pre-trained target recognition model to obtain the distribution location image of the target object; wherein, the preset template image is used to indicate the background features of the image to be detected to the target recognition model;

[0200] The anomaly identification module 630 is used to detect whether the distributed location image is connected using a connectivity detection algorithm. If the detection result is that the target object is connected, it is determined that the printed circuit board assembly has an anomaly.

[0201] The anomaly detection scheme provided in this application separates the identification and judgment logic, which are implemented through an identification model and a connectivity detection algorithm, respectively. The identification model only needs to complete the learning of one inductive bias task, avoiding the model getting stuck in the detection performance bottleneck caused by simultaneously performing the learning of multiple bias tasks. Moreover, the identification model focuses on identifying solder areas, greatly improving the model's processing accuracy; the connectivity detection algorithm has strong resistance to interference from external conditions, improving the robustness of the anomaly detection scheme against external interference. By inputting the image to be detected of the printed circuit board assembly and the preset template image into the target identification model, this application can efficiently obtain an accurate image of the distribution location of the target object and accurately identify the connectivity anomalies of the target object.

[0202] Furthermore, the pre-trained target recognition model is obtained based on a small number of samples using a meta-learning training mechanism.

[0203] The beneficial effect of this scheme is that the meta-learning training mechanism based on a small number of samples enables the target recognition model to learn general learning strategies and prior knowledge, thereby effectively reducing the difficulty of model training and helping the target recognition model to quickly adapt to various practical application scenarios.

[0204] Furthermore, the target recognition module 620 is specifically used for:

[0205] The image to be detected of the printed circuit board assembly, the preset template image, and the prompt text are input into the pre-trained target recognition model to obtain the distribution location image of the target object; wherein, the prompt text is used to indicate the background features of the image to be detected to the target recognition model.

[0206] The beneficial effect of this solution is that by inputting the image to be detected of the printed circuit board assembly, the preset template image, and the prompt text into the pre-trained target recognition model, the distribution location image of the target object can be obtained. Compared with only inputting the image to be detected of the printed circuit board assembly and the preset template image into the pre-trained target recognition model, the interference of background features of the image to be detected on the recognition and analysis of the target object can be better eliminated, thereby further improving the detection accuracy of the target object.

[0207] Furthermore, the target recognition module 620 is specifically used for:

[0208] The target recognition model determines the prompt vector based on a preset template image and / or prompt text;

[0209] Based on the cue vector, features are extracted from the image of the printed circuit board assembly to be detected at at least two scales to obtain the basic feature images at each scale.

[0210] The basic feature images at each scale are merged to obtain the output feature image;

[0211] The output feature image is deconvolved to obtain the distribution location image of the target object.

[0212] The beneficial effect of this scheme is that by extracting features from the image of the printed circuit board assembly to be detected at at least two scales based on the cue vector, the basic feature images at each scale are obtained. The basic feature images at each scale are merged to obtain the output feature image. Finally, the output feature image is deconvolved to obtain the distribution location image of the target object. This can extract and integrate feature information at multiple scales, and achieve an effective conversion from the image to be detected to the accurate localization of the target object.

[0213] Furthermore, the target recognition module 620 is specifically used for:

[0214] Features are extracted from the image of the printed circuit board assembly to be inspected at at least two scales to obtain the original feature images at each scale;

[0215] The basic feature image is obtained by cross-correlation between the cue vector and the original feature image at each scale.

[0216] The beneficial effect of this scheme is that by performing cross-correlation operations between the cue vector and the original feature images at various scales to obtain the basic feature image, the intrinsic relationship between the key information contained in the cue vector and the image features can be fully explored, thereby highlighting the feature information closely related to the detection task and effectively reducing the interference of irrelevant information.

[0217] Furthermore, the target recognition module 620 is specifically used for:

[0218] Perform the first convolution operation, scale scaling operation, and second convolution operation on the basic feature images at each scale to obtain the target scale feature image;

[0219] The feature images of each target scale are stacked in the depth direction to obtain the output feature image.

[0220] The beneficial effect of this scheme is that by performing the first convolution operation, scale scaling operation, and second convolution operation on the basic feature images at each scale to obtain the target scale feature image, and stacking the target scale feature images in the depth direction to obtain the output feature image, it can effectively integrate the multi-dimensional feature information contained in the images at different scales, realize all-round feature fusion from local details to overall layout, and thus provide a rich, comprehensive and hierarchical feature foundation for the subsequent determination of target objects.

[0221] Furthermore, the target recognition module 620 is specifically used for:

[0222] The output feature image is deconvolved sequentially through a first deconvolution layer, a second deconvolution layer, and a third deconvolution layer to obtain the distribution location image of the target object.

[0223] The beneficial effect of this scheme is that by sequentially performing deconvolution operations on the output feature image through the first deconvolution layer, the second deconvolution layer, and the third deconvolution layer, the distribution location image of the target object can be obtained. This can gradually restore the spatial information and detailed features lost in the previous series of convolution and scaling operations of the image to be detected, and achieve an effective transformation from abstract feature representation to intuitive visualization of the location distribution of the target object.

[0224] Furthermore, the second deconvolutional layer is associated with a low-rank decomposition adapter module;

[0225] Accordingly, the target recognition module 620 is specifically used for:

[0226] The output of the first deconvolution layer is deconvolved using the second deconvolution layer and the low-rank decomposition adapter module.

[0227] The beneficial effects of this scheme are as follows: by associating the second deconvolution layer with the low-rank decomposition adapter module, and performing deconvolution operations on the output of the first deconvolution layer through the second deconvolution layer and the low-rank decomposition adapter module, the number of parameters of the adapter module itself is effectively reduced through low-rank decomposition technology. This significantly reduces the demand for computing resources during fine-tuning, enabling smooth model fine-tuning in resource-constrained environments. It effectively alleviates the overfitting problem caused by the difference between the distribution of new data and the original training data during model fine-tuning, ensuring the model's generalization ability on new data, and flexibly adapting the target recognition model. It can improve the accuracy of the target recognition model in real-world scenarios with only a small number of samples, thereby further improving the model's adaptability to new scenarios. While meta-learning is used to obtain a large segmentation model, training such a large segmentation model requires a large amount of data and a long training time. By using the low-rank decomposition adapter module, the model can be quickly trained to adapt to the new scenario based on the training data in the new scenario, significantly improving the adaptability to new scenarios and the model's detection accuracy.

[0228] Furthermore, the device also includes:

[0229] The penetration anomaly determination module is used to determine that the printed circuit board assembly has a penetration anomaly if a connected region of a target object passes through the opposite boundary of the distributed location image.

[0230] The beneficial effect of this solution is that, by determining the presence of penetration anomalies in printed circuit board assemblies when the connected regions of the target object simultaneously pass through the opposing boundaries of the distributed location images, the penetration anomalies on printed circuit board assemblies can be identified quickly and accurately.

[0231] Furthermore, the device also includes a model training module, specifically used for:

[0232] Based on the category information of the sample images, determine the training image set and the validation image set;

[0233] The target recognition model is trained based on the training image set to optimize the model parameters of the target recognition model;

[0234] The training effect of the target recognition model is verified based on the set of verification images in order to adjust and optimize the hyperparameters;

[0235] Repeat the above steps until the preset number of repetitions is reached.

[0236] The beneficial effect of this scheme is that it determines the training image set and the validation image set based on the category information of the sample images, trains the target recognition model based on the training image set to optimize the model parameters of the target recognition model, and validates the training effect of the target recognition model based on the validation image set to adjust and optimize the hyperparameters. This can build a target recognition model with high accuracy and strong generalization ability.

[0237] Furthermore, the model training module is specifically used for:

[0238] Acquire historical anomaly detection data, and fine-tune the model parameters of the deconvolution part of the target recognition model based on the historical anomaly detection data.

[0239] The beneficial effect of this scheme is that by fine-tuning the model parameters of the deconvolution part of the target recognition model based on historical anomaly detection data, the number of parameters that need to be learned can be greatly reduced, the risk of overfitting of the target recognition model can be reduced, and thus the learning of the target recognition model can be promoted.

[0240] The technical solution provided in this application includes a target recognition module, which inputs a target image of a printed circuit board assembly to be detected and a preset template image into a pre-trained target recognition model to obtain a distribution location image of the target object. The pre-trained target recognition model is obtained based on a small number of samples using a meta-learning training mechanism. The preset template image is used to indicate background features of the target image to the target recognition model. An anomaly detection module is used to detect whether the distribution location image is connected using a connectivity detection algorithm. If the detection result indicates that the target object is connected, then the printed circuit board assembly is determined to have an anomaly. This anomaly detection device for printed circuit board assemblies decouples the model's recognition and judgment logic, implementing them separately through the recognition model and the connectivity detection algorithm. The recognition model only needs to complete the learning of an inductive bias task, avoiding the model getting stuck in a detection performance bottleneck. The recognition model focuses on recognizing solder areas, greatly improving the model's recognition accuracy. Moreover, the connectivity detection algorithm has strong resistance to interference from external conditions. Compared with the method of determining connectivity through a model, this application does not require learning and training for determining connectivity, reducing model training data and training time. Meanwhile, the judgment logic of the recognition model is relatively transparent. By identifying solder areas and visualizing them on the interface, users can understand the model's behavior. Furthermore, the model's judgment logic can be inferred from the model results, making the deterministic judgment logic more reliable. By inputting the image to be detected of the printed circuit board assembly and the preset template image into the target recognition model, the distribution location image of the target object can be obtained efficiently, improving the accuracy of detecting connectivity anomalies in the target object. In addition, the meta-learning training mechanism based on a small number of samples enables the target recognition model to learn general learning strategies and prior knowledge, thereby effectively reducing the difficulty of model training and helping the target recognition model quickly adapt to various practical application scenarios.

[0241] The anomaly detection device for printed circuit board components in this application embodiment can be a device, or it can be a component, integrated circuit, or chip in a terminal. The device can be a mobile electronic device or a non-mobile electronic device. For example, mobile electronic devices can be mobile phones, tablets, laptops, PDAs, in-vehicle electronic devices, wearable devices, ultra-mobile personal computers (UMPCs), netbooks, or personal digital assistants (PDAs), etc., while non-mobile electronic devices can be servers, network attached storage (NAS), personal computers (PCs), televisions (TVs), ATMs, or self-service machines, etc. This application embodiment does not impose specific limitations.

[0242] The anomaly detection device for the printed circuit board assembly in this application embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit it.

[0243] The abnormal detection device for printed circuit board assemblies provided in this application embodiment can realize the various processes implemented in the above method embodiments. To avoid repetition, it will not be described again here.

[0244] Figure 7 This is a schematic diagram of the structure of the electronic device provided in an embodiment of this application. For example... Figure 7 As shown, this application embodiment also provides an electronic device 700, including a processor 701, a memory 702, and a program or instructions stored in the memory 702 and executable on the processor 701. When the program or instructions are executed by the processor 701, they implement the various processes of the above-described printed circuit board assembly anomaly detection device embodiment and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0245] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0246] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described printed circuit board assembly anomaly detection device embodiment and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0247] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0248] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0249] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0250] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

[0251] The above description is merely a preferred embodiment and the technical principles employed in this application. This application is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions that can be made by those skilled in the art will not depart from the scope of protection of this application. Therefore, although this application has been described in detail through the above embodiments, this application is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of this application, the scope of which is determined by the scope of the claims.

Claims

1. A method for detecting anomalies in a printed circuit board assembly, characterized in that, The method includes: Acquire the image of the printed circuit board assembly to be inspected; The image to be detected of the printed circuit board assembly and a preset template image are input into a pre-trained target recognition model to obtain a distribution image of the target object; wherein, the preset template image is used to indicate the background features of the image to be detected to the target recognition model; A connectivity detection algorithm is used to detect whether the distributed location images are connected. If the detection result shows that the target object is connected, it is determined that the printed circuit board assembly is abnormal.

2. The method for detecting anomalies in a printed circuit board assembly according to claim 1, characterized in that, The pre-trained target recognition model is obtained based on a small number of samples using a meta-learning training mechanism.

3. The method for detecting anomalies in a printed circuit board assembly according to claim 1, characterized in that, The image to be detected of the printed circuit board assembly and the preset template image are input into a pre-trained target recognition model to obtain the distribution location image of the target object, including: The image to be detected of the printed circuit board assembly, the preset template image, and the prompt text are input into the pre-trained target recognition model to obtain the distribution location image of the target object; wherein, the prompt text is used to indicate the background features of the image to be detected to the target recognition model.

4. The method for detecting anomalies in a printed circuit board assembly according to claim 3, characterized in that, The image of the printed circuit board assembly to be detected, the preset template image, and the prompt text are input into a pre-trained target recognition model to obtain the distribution location image of the target object, including: The target recognition model determines the prompt vector based on a preset template image and / or prompt text; Based on the cue vector, features are extracted from the image of the printed circuit board assembly to be detected at at least two scales to obtain the basic feature images at each scale. The basic feature images at each scale are merged to obtain the output feature image; The output feature image is deconvolved to obtain the distribution location image of the target object.

5. The method for detecting anomalies in a printed circuit board assembly according to claim 4, characterized in that, Based on the cue vector, features are extracted from the image of the printed circuit board assembly to be detected at at least two scales to obtain basic feature images at each scale, including: Features are extracted from the image of the printed circuit board assembly to be inspected at at least two scales to obtain the original feature images at each scale; The basic feature image is obtained by cross-correlation between the cue vector and the original feature image at each scale.

6. The method for detecting anomalies in a printed circuit board assembly according to claim 4, characterized in that, By merging the basic feature images at various scales, an output feature image is obtained, including: Perform the first convolution operation, scale scaling operation, and second convolution operation on the basic feature images at each scale to obtain the target scale feature image; The feature images of each target scale are stacked in the depth direction to obtain the output feature image.

7. The method for detecting anomalies in a printed circuit board assembly according to claim 4, characterized in that, Performing a deconvolution operation on the output feature image to obtain the distribution location image of the target object includes: The output feature image is deconvolved sequentially through a first deconvolution layer, a second deconvolution layer, and a third deconvolution layer to obtain the distribution location image of the target object.

8. The method for detecting anomalies in a printed circuit board assembly according to claim 7, characterized in that, The second deconvolutional layer is associated with the low-rank decomposition adapter module; Accordingly, deconvolution operations are performed through a second deconvolution layer, including: The output of the first deconvolution layer is deconvolved using the second deconvolution layer and the low-rank decomposition adapter module.

9. The method for detecting anomalies in a printed circuit board assembly according to claim 1, characterized in that, After determining that the printed circuit board assembly is faulty, the method further includes: If a connected region of the target object passes through the opposite boundary of the distributed location image, then it is determined that the printed circuit board assembly has a penetration anomaly.

10. The method for detecting anomalies in a printed circuit board assembly according to claim 1, characterized in that, The process of pre-training an object recognition model includes: Based on the category information of the sample images, determine the training image set and the validation image set; The target recognition model is trained based on the training image set to optimize the model parameters of the target recognition model; The training effect of the target recognition model is verified based on the set of verification images in order to adjust and optimize the hyperparameters; Repeat the above steps until the preset number of repetitions is reached.

11. The method for detecting anomalies in a printed circuit board assembly according to claim 10, characterized in that, After training the target recognition model based on the training image set to optimize the model parameters of the target recognition model, the method further includes: Acquire historical anomaly detection data, and fine-tune the model parameters of the deconvolution part of the target recognition model based on the historical anomaly detection data.

12. An electronic device, characterized in that, The method includes a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the anomaly detection method for a printed circuit board assembly as described in any one of claims 1-11.

13. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of the anomaly detection method for a printed circuit board assembly as described in any one of claims 1-11.

Images