Defect detection model training method and PCB defect detection method

Abstract

The invention relates to a training method of a defect detection model and a PCB defect detection method, and the training method comprises the steps: carrying out the feature extraction based on a first training image through a first network branch of an initial detection model, and obtaining a visual defect feature map; performing feature extraction based on a second training image through a second network branch of the initial detection model to obtain a thermodynamic defect feature map; fusing the visual defect feature map and the thermal defect feature map by using the initial detection model to obtain a composite defect feature map, and performing detection based on the composite defect feature map to obtain a prediction result; and constructing a physical constraint item according to the space gradient information of the thermodynamic defect feature map, constructing a joint loss function according to the defect label, the prediction result and the physical constraint item, and performing iterative training on the initial detection model based on the joint loss function to generate a defect detection model. According to the invention, the problem of low accuracy of PCB defect detection is solved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to training methods for defect detection models and PCB defect detection methods. Background Technology

[0002] With the continuous development of electronic products, printed circuit boards (PCBs), as core components of electronic devices, have become a crucial focus in the manufacturing process due to their quality and reliability. During PCB manufacturing, defects may occur due to various reasons, such as poor printing or soldering issues. In related technologies, automatic defect detection based on deep learning can, to some extent, compensate for the shortcomings of manual quality inspection and improve the yield rate of PCB production. However, existing deep learning-based intelligent defect detection methods suffer from poor accuracy and generalization ability in complex or multi-target inspection environments, thus affecting the accuracy of PCB defect detection.

[0003] Currently, no effective solution has been proposed to address the low accuracy of PCB defect detection in related technologies. Summary of the Invention

[0004] This application provides a method for training a defect detection model and a PCB defect detection method to at least solve the problem of low accuracy in PCB defect detection in related technologies.

[0005] In a first aspect, embodiments of this application provide a method for training a defect detection model, the method comprising:

[0006] Obtain a first training image and a second training image containing the image of the sample PCB, as well as the defect label corresponding to the sample PCB;

[0007] A visual defect feature map is obtained by extracting features from the first training image using the first network branch of the initial detection model; a thermal defect feature map is obtained by extracting features from the second training image using the second network branch of the initial detection model.

[0008] Using the initial detection model, the visual defect feature map and the thermal defect feature map are fused to obtain a composite defect feature map, and detection is performed based on the composite defect feature map to obtain a prediction result;

[0009] A physical constraint term is constructed based on the spatial gradient information of the thermal defect feature map. A joint loss function is constructed based on the defect label, the prediction result, and the physical constraint term. The initial detection model is then iteratively trained based on the joint loss function to generate a defect detection model.

[0010] In some embodiments, the step of obtaining a thermal defect feature map by extracting features from the second training image through the second network branch of the initial detection model includes:

[0011] Initial feature data is obtained by extracting features from the second training image through the second network branch;

[0012] Using a pre-established heat conduction inversion model, heat conduction calculations are performed based on the initial feature data to obtain temperature gradient information and heat flux density information;

[0013] Based on the temperature gradient information and the heat flux density information, local thermal anomaly regions in the second training image are identified, and the thermal defect feature map is generated based on the local thermal anomaly regions.

[0014] In some embodiments, constructing physical constraint terms based on the spatial gradient information of the thermal defect feature map includes:

[0015] Obtain the preset spatial grid parameters;

[0016] Based on the spatial grid parameters, the spatial gradient of the thermal defect feature map is calculated to obtain the spatial gradient information, and the physical constraint terms are constructed based on the spatial gradient information.

[0017] In some embodiments, fusing the visual defect feature map and the thermal defect feature map using the initial detection model to obtain a composite defect feature map includes:

[0018] The visual defect feature map and the thermal defect feature map are assigned corresponding feature weight values, and the visual defect feature map and the thermal defect feature map are weighted and fused based on the assigned feature weight values ​​to obtain the composite defect feature map.

[0019] In some embodiments, the feature weight values ​​include a first learnable weight parameter and a second learnable weight parameter; the weighted fusion of the visual defect feature map and the thermal defect feature map based on the assigned feature weight values ​​to obtain the composite defect feature map includes:

[0020] Obtain the preset bias parameters;

[0021] Based on the first learnable weight parameter, the second learnable weight parameter, and the bias parameter, the visual defect feature map and the thermal defect feature map are linearly transformed to generate an attention weight map;

[0022] Based on the attention weight map, the visual defect feature map and the thermal defect feature map are summed element by element to obtain the composite defect feature map.

[0023] In some embodiments, the detection based on the composite defect feature map to obtain the prediction result includes:

[0024] The classification output head of the initial detection model is used to predict the probability distribution of defects based on the composite defect feature map to obtain the classification prediction result. Then, the thermal risk assessment head of the initial detection model is used to perform linear regression processing on the composite defect feature map to obtain the future temperature rise prediction result.

[0025] The prediction results include the classification prediction results and the future temperature rise prediction results.

[0026] In some embodiments, constructing a joint loss function based on the defect label, the prediction result, and the physical constraint term includes:

[0027] Based on the classification prediction results and the defect labels, a first branch loss function is constructed, and based on the future temperature rise prediction results and the physical constraint terms, a second branch loss function is constructed.

[0028] Assign corresponding loss weight values ​​to the first branch loss function and the second branch loss function, and perform weighted calculation on the first branch loss function and the second branch loss function based on the assigned loss weight values ​​to construct the joint loss function.

[0029] In some embodiments, assigning corresponding loss weight values ​​to the first branch loss function and the second branch loss function includes:

[0030] Monitor the magnitude of the first initial loss gradient corresponding to the first network branch, and the magnitude of the second initial loss gradient corresponding to the second network branch;

[0031] Based on the first initial loss gradient magnitude and the second initial loss gradient magnitude, a balance factor is calculated, and according to the balance factor, the corresponding loss weight values ​​are assigned to the first branch loss function and the second branch loss function.

[0032] Secondly, embodiments of this application provide a PCB defect detection method, the method comprising:

[0033] Acquire a first and a second image of the PCB to be inspected, which contain images of the PCB to be inspected.

[0034] The first image to be inspected and the second image to be inspected are input into the trained defect detection model to obtain the defect detection result of the PCB to be inspected; the defect detection model is trained and generated according to the training method of the defect detection model described in the first aspect above.

[0035] Thirdly, embodiments of this application provide a storage medium storing a computer program thereon, which, when executed by a processor, implements the training method for the defect monitoring model as described in the first aspect above, or the PCB defect monitoring method as described in the second aspect above.

[0036] Compared to related technologies, the training method for the defect detection model and the PCB defect detection method provided in this application embodiment acquire a first training image and a second training image containing the sample PCB image, as well as the defect label corresponding to the sample PCB; through the first network branch of the initial detection model, a visual defect feature map is obtained based on the first training image by feature extraction; through the second network branch of the initial detection model, a thermal defect feature map is obtained based on the second training image by feature extraction; using the initial detection model, the visual defect feature map and the thermal defect feature map are fused to obtain a composite defect feature map, and detection is performed based on the composite defect feature map to obtain a prediction result; a physical constraint term is constructed based on the spatial gradient information of the thermal defect feature map, a joint loss function is constructed based on the defect label, the prediction result, and the physical constraint term, and the initial detection model is iteratively trained based on the joint loss function to generate a defect detection model.

[0037] Based on this, by introducing physical constraint terms, the model can simultaneously optimize the accuracy of thermodynamic parameters and the compliance with physical laws during backpropagation, thus providing a training process for closed-loop design of multimodal perception and physical enhancement. This solves the problem of low accuracy in PCB defect detection caused by low accuracy in latent defect detection and insufficient thermally related defect identification capabilities in related technologies, and ultimately achieves a comprehensive improvement in detection accuracy, real-time performance and environmental robustness.

[0038] Details of one or more embodiments of this application are set forth in the following drawings and description to make other features, objects and advantages of this application more readily apparent. Attached Figure Description

[0039] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0040] Figure 1 This is a hardware structure block diagram of a terminal for a training method of a defect detection model according to an embodiment of this application;

[0041] Figure 2 This is a flowchart of a training method for a defect detection model according to an embodiment of this application;

[0042] Figure 3 This is a flowchart of a heat conduction inversion algorithm according to an embodiment of this application;

[0043] Figure 4 This is a flowchart of a PCB defect detection method according to an embodiment of this application;

[0044] Figure 5 This is a schematic diagram of the training process of a defect detection model according to an embodiment of this application;

[0045] Figure 6 This is a schematic diagram illustrating a statistical comparison of defect detection performance according to an embodiment of this application. Detailed Implementation

[0046] To make the objectives, technical solutions, and advantages of this application clearer, the application is described and illustrated below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the application. All other embodiments obtained by those skilled in the art based on the embodiments provided in this application without inventive effort are within the scope of protection of this application. Furthermore, it is understood that although the efforts made in such a development process may be complex and lengthy, for those skilled in the art related to the content disclosed in this application, modifications to design, manufacturing, or production based on the technical content disclosed in this application are merely conventional technical means and should not be construed as insufficient disclosure of the content of this application.

[0047] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment that is mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0048] Unless otherwise defined, the technical or scientific terms used in this application shall have the ordinary meaning understood by one of ordinary skill in the art to which this application pertains. The terms “a,” “an,” “an,” “the,” and similar words used in this application do not indicate quantity limitation and may indicate singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units not listed, or may include other steps or units inherent to these processes, methods, products, or devices. The terms “connected,” “linked,” “coupled,” and similar words used in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. “Multiple” used in this application means two or more. “And / or” describes the relationship between related objects, indicating that three relationships may exist; for example, “A and / or B” can represent: A alone, A and B simultaneously, and B alone. The terms “first,” “second,” “third,” etc., used in this application are merely to distinguish similar objects and do not represent a specific ordering of the objects.

[0049] The method embodiments provided in this example can be executed on a terminal, computer, or similar computing device. Taking running on a terminal as an example, Figure 1 This is a hardware structure block diagram of a terminal for a defect detection model training method according to an embodiment of this application. For example... Figure 1 As shown, a terminal may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 (which may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.) and a memory 104 for storing data are also shown. Optionally, the terminal may further include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the terminal described above. For example, the terminal may also include components that are more... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0050] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to a defect detection model training method in this embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, thus implementing the aforementioned method. The memory 104 may include high-speed random access memory and non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0051] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the terminal's communication provider. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0052] This embodiment provides a method for training a defect detection model. Figure 2 This is a flowchart of a training method for a defect detection model according to an embodiment of this application, such as... Figure 2 As shown, the process includes the following steps:

[0053] Step S210: Obtain a first training image and a second training image containing the sample PCB image, as well as the defect label corresponding to the sample PCB.

[0054] Specifically, the first training image and the second training image can be captured by two cameras respectively, targeting the same scene where PCB samples are placed, and the image modes of the first training image and the second training image are different. For example, the first training image can be a color image captured by a visible light camera, and the second training image can be an infrared image captured by an infrared camera.

[0055] More specifically, the aforementioned first and second training images are multiple image pairs, such as no fewer than 1500 image pairs, and are stored as a training dataset. In this training dataset, the resolution of each first training image is ≥1024×1024 pixels, and the resolution of each first training image is ≥640×480 pixels. Furthermore, to facilitate subsequent processing, the first and second training images acquired at the same time need to be aligned, with a spatiotemporal alignment error ≤5ms.

[0056] It should also be noted that the training dataset consisting of the first and second training images should cover a greater than or equal to the preset number of defect types. This preset number of types can be set in advance according to actual application conditions, for example, to six types; in other words, the training dataset should cover at least six predefined defect types, including different defect types such as open circuit, short circuit, Annular Ring Break, Missing Pad, Line Scratch, and foreign matter residue, thereby ensuring the comprehensiveness of PCB defect detection. The defect labels corresponding to each defect type can be preset and carried by the first and / or second training images.

[0057] Step S220: Using the first network branch of the initial detection model, a visual defect feature map is obtained by extracting features based on the first training image; using the second network branch of the initial detection model, a thermal defect feature map is obtained by extracting features based on the second training image.

[0058] The network structure of the initial detection model includes network branches corresponding to two types of training images, namely the first network branch and the second network branch; texture features and thermodynamic features are extracted from the first training image and the second training image respectively through these two branches.

[0059] Specifically, this first network branch can effectively capture the texture, geometry, and local anomalies (such as explicit defects like short circuits and open circuits) of the PCB surface through the residual structure, outputting a feature map. It also employs an improved structural similarity algorithm (SSIM+) and introduces an adaptive gamma correction algorithm to highlight low-contrast defects, as shown in the following formula:

[0060] ;

[0061] In the above formula, , These are the average brightness of local regions in the first training image x and the second training image y, respectively. , denoted as the variance of the local regions of the first training image x and the second training image y, respectively, and denoted as the contrast (the larger the variance, the higher the contrast). Let be the local covariance of the first training image x and the second training image y, used to measure structural similarity. C1 and C2 are small constants to prevent the denominator from being zero. , The local gamma values ​​for the first training image x and the second training image y are used to adjust image contrast. Due to subtle optical differences on the PCB surface (such as poor solder joints or copper foil oxidation) and the influence of uneven lighting, traditional methods struggle to distinguish normal textures from defects. Therefore, in this step, the SSIM term preserves the image's structural information (such as edges and textures), while the gamma correction term compensates for uneven lighting. This makes defects (such as pinholes in dark areas or over-etching in bright areas) more easily identifiable in the difference image, thus achieving pixel-level difference enhancement. This image difference enhancement can be achieved by using a difference image created with Python, highlighting differences in red to reduce the model's workload.

[0062] Furthermore, the "50" in the ResNet50 architecture used in the first network branch refers to the network's depth, indicating that it has 50 layers, including convolutional and fully connected layers. The ResNet50 architecture consists of multiple building blocks, each containing several convolutional layers. These blocks are responsible for capturing hierarchical features of the input image. As the network depth increases, these blocks enable the model to learn more abstract and complex representations. The dataset input to this branch contains over 100,000 first training images, distributed across 200 categories, serving as the pre-training set for ResNet50. ResNet (Residual Network) is a neural network that addresses the problem of training deep learning networks by employing skip connections, which "skip" several convolutional layers in each basic module of the network. This makes training more practical and faster because it provides alternative paths for both the original and derived data. These skip connections add the output of the previous module to the next module, as shown in the equation below:

[0063] y = F(x) + x;

[0064] Here, F is defined as the residual function, x is the input, and y is the output.

[0065] On the other hand, the aforementioned second network branch recovers the spatial resolution of the temperature field through an encoder-decoder structure to perform thermodynamic analysis on the second training image. This includes calculating the heat distribution gradient and identifying local temperature anomalies, thereby capturing heat-related defects (such as uneven heat dissipation and short-circuit heating). Furthermore, a thermodynamic prior constraint layer is embedded in the decoding stage. This constraint layer, based on the principle of heat conduction equation inversion, calculates the spatial gradient at the predicted temperature field output and constrains the smoothness of the temperature distribution through an L1 regularization term, ensuring that the model learns heat conduction characteristics that conform to physical laws. This design enables the second network branch not only to characterize thermal anomaly regions but also to implicitly correlate the thermodynamic behavior of defects (such as temperature rise caused by increased local resistance) through physical laws.

[0066] Step S230: Using the initial detection model, the visual defect feature map and the thermal defect feature map are fused to obtain a composite defect feature map, and detection is performed based on the composite defect feature map to obtain the prediction result.

[0067] In this step, an intermediate fusion strategy is first adopted to achieve information complementarity while maintaining modality-specific features. This fusion strategy can employ multimodal fusion techniques such as feature concatenation or weighted superposition. For example, the visual defect feature map (512-dimensional) output from the first network branch is channel compressed (reduced to 256-dimensionality by 1×1 convolution), while the thermal defect feature map (128-dimensionality) generated by the second network branch is channel expanded (increased to 256-dimensionality by 1×1 convolution) and upsampled (bilinear interpolation × 4x) to achieve spatial resolution alignment. Subsequently, a spatial attention mechanism is introduced, generating an attention map (Sigmoid activation) using visible light features and dynamically weighting infrared features. This design allows the model to learn adaptively: enhancing visible light features in areas with clear textures (such as solder pads) and strengthening infrared features in areas with abnormal thermal radiation (such as short-circuit points). Finally, a composite defect feature map is generated through weighted fusion, maintaining 256 channels and spatial resolution consistent with the original input.

[0068] Next, the fused composite defect feature map is input into the detection head module of the initial detection model. A computer vision library is used to analyze the composite feature map, extracting visible light texture features and infrared thermodynamic features to establish feature associations. Defect category probability distribution prediction and thermal risk assessment are then performed separately to obtain the aforementioned prediction results, completing defect localization and classification. For example, the computer vision library "Tetryon-AI" can be used, suitable for various applications such as defect detection, anomaly detection, and quality assurance. It contains a library of pre-trained models with an adaptable architecture, enabling the creation and guidance of unique models for specific applications.

[0069] Step S240: Construct physical constraint terms based on the spatial gradient information of the thermal defect feature map; construct a joint loss function based on the defect label, prediction results, and physical constraint terms; and iteratively train the initial detection model based on the joint loss function to generate a defect detection model.

[0070] Specifically, the model performs spatial gradient analysis on the predicted temperature field distribution through a temperature gradient calculation layer at the end of the decoder in the second network branch, and constructs a physical constraint term based on this. This constraint term forces the temperature field output by the model to satisfy the sparsity characteristic of physical laws—defective regions will exhibit significant abrupt temperature gradient changes due to abnormal heat sources (such as local high temperatures caused by short circuits), while the temperature distribution in non-defective regions should remain smooth. Through this explicit constraint, the model can more accurately focus on thermal anomaly regions during training, while suppressing noise interference from irrelevant regions. Furthermore, to strengthen the coupling between physical laws and deep learning, the inversion mechanism of the heat conduction equation is implicitly incorporated into the loss function design. Thus, during the training phase, the model automatically differentiates and calculates the spatiotemporal derivative of the predicted temperature field, supplementing the loss function as an implicit physical constraint. This design ensures that the temperature field output by the model not only satisfies local gradient sparsity but also conforms to global heat conduction laws, thereby improving the reliability of thermodynamic anomaly predictions.

[0071] More specifically, in the network architecture of the second network branch, the thermodynamic prior constraint layer is implemented as follows: For this network architecture, a temperature gradient calculation layer is added at the feature map output of the decoding stage to calculate the spatial gradient of the predicted temperature field distribution. This gradient information directly affects the loss function through a sparse constraint term, rather than existing as an independent network layer. During training, the loss function of this second network branch contains two core parts: the first part is the mean square error between the predicted heat flux density and the true value, used to supervise the accuracy of thermodynamic parameters; the second part is the temperature gradient sparse constraint, which forces the network to learn the smooth characteristics of the temperature field that conform to physical laws through L1 regularization. These two parts are jointly optimized with the classification loss of the first network branch to ensure the thermodynamic consistency of multimodal features during fusion.

[0072] Through steps S210 to S240, the first and second training images of different modalities are processed by the two network branches in the initial detection model, and physical constraint terms are calculated by embedding a thermal conduction constraint layer in the model structure. The introduction of this constraint term enables the model to simultaneously optimize the accuracy of thermodynamic parameters and the conformity of physical laws during backpropagation, thereby providing a training process for closed-loop design of multimodal perception and physical enhancement. This solves the problem of low accuracy in PCB defect detection caused by low accuracy in latent defect detection and insufficient thermal related defect identification capabilities in related technologies. Ultimately, it achieves a comprehensive improvement in detection accuracy, real-time performance, and environmental robustness, forming a significant technical barrier, especially in the fields of latent defect detection and thermal failure prediction.

[0073] In some embodiments, the above-mentioned method of obtaining a thermal defect feature map by extracting features from a second training image through a second network branch of the initial monitoring model may further include the following steps:

[0074] The second network branch extracts features from the second training image to obtain initial feature data; using a pre-established heat conduction inversion model, heat conduction is calculated based on the initial feature data to obtain temperature gradient information and heat flux density information; based on the temperature gradient information and heat flux density information, local thermal anomaly regions in the second training image are identified, and a thermal defect feature map is generated based on the local thermal anomaly regions.

[0075] This step primarily provides the processing flow for the infrared thermal imaging branch (i.e., the second network branch). Its core lies in achieving thermodynamic characteristic modeling of PCB defects through the synergistic effect of the heat conduction inversion model and multi-scale physical constraints. The specific steps are as follows:

[0076] First, the infrared image is used as the second training image and input into the feature extraction network (U-Net++ architecture) of the infrared branch. This branch extracts initial feature data during the encoding stage through multi-layer convolutional operations, containing low- to high-order semantic information of the temperature field. This feature data not only preserves the spatial resolution of the original infrared image (640×512 pixels) but also progressively abstracts the global features of the heat distribution through downsampling blocks (3 in total). At this point, the feature data implicitly contains the spatial gradient distribution of the PCB surface temperature and the transient thermal conduction characteristics.

[0077] Next, a pre-established heat conduction inversion model is introduced to physically enhance the initial feature data, and the abnormal heat source distribution term is solved by equation deformation, as shown in the following formula:

[0078] ;

[0079] In the above formula, This is used to represent the intensity of an abnormal heat source at the image coordinates (x, y). K represents thermal conductivity. The Laplace quantity used to represent the temperature field. Indicates the density of the material. Indicates specific heat capacity. This represents the rate of temperature change over time. The above formula can be used to identify non-uniform heat source distributions caused by defects.

[0080] Subsequently, anomaly detection is performed on the inversion results based on physical constraints. Specifically, temperature gradient and heat flux density are calculated, and anomaly detection criteria are set, such as a heat flux density mutation threshold of Δq ≥ 20% of the baseline value and a temperature gradient threshold of 0.5℃ / mm. Regions that meet these anomaly detection criteria are then marked as local thermal anomaly regions. Finally, the multi-dimensional physical constraint results are fused to generate a thermal defect feature map.

[0081] Through the above embodiments, by using physical-guided feature learning, the model is forced to focus on thermodynamic anomalies that conform to physical laws, rather than simply relying on data-driven approaches. This can improve the robustness of detecting latent defects (such as cold solder joints and interlayer short circuits). For example, when detecting PCB soldering defects, the temperature field of a normal solder joint exhibits a smooth distribution, while the temperature of a cold solder joint rises sharply (gradient anomaly) and the heat flow direction is disordered (dispersion is not zero) due to increased contact thermal resistance. Through the above steps, the thermal defect feature map can accurately locate such defects and complement the geometric features of the visible light branch (such as solder joint shape distortion), ultimately outputting a joint prediction result that includes the defect category and thermal risk assessment value.

[0082] In some embodiments, the above-described construction of physical constraint terms based on the spatial gradient information of the thermal defect feature map may further include the following steps:

[0083] Obtain the preset spatial grid parameters; perform spatial gradient calculation on the thermal defect feature map based on the spatial grid parameters to obtain spatial gradient information, and construct physical constraint terms based on the spatial gradient information.

[0084] The design of spatial grid parameters is fundamental to numerical computation. First, the actual dimensions (Δx, Δy) of each grid cell need to be determined based on the resolution of the infrared thermal imaging (e.g., 640×480 pixels) and the physical dimensions of the actual detection area (e.g., a 100mm×80mm field of view on a PCB board). For example, if the image resolution is 640×480 pixels and covers a 100mm×80mm area, then Δx = 100 / 640 ≈ 0.156mm / pixel, and Δy = 80 / 480 ≈ 0.167mm / pixel. Simultaneously, the grid topology (e.g., rectangular grid) and boundary treatment (e.g., zero-fill or mirror expansion) need to be defined to ensure the stability of the numerical computation. Furthermore, the time step Δt needs to be set according to the characteristics of the heat conduction problem (e.g., thermal diffusivity) to satisfy the Courant-Friedrichs-Lewym (CFL) condition and avoid numerical oscillations.

[0085] On a discretized grid, the spatial gradient of the temperature field is calculated using the finite difference method. Combining this with the physical laws of the heat conduction equation, the spatial gradient information is transformed into constraint terms, such as heat flux conservation constraints and temperature gradient sparsity constraints. Finally, during the iterative training phase, these constraint terms are added to the sum of the loss functions to complete the iterative training.

[0086] More specifically, please refer to Figure 3 The figure also provides a detailed explanation of the heat conduction inversion algorithm process, which includes the following steps:

[0087] Step S301: Input the original infrared sequence (i.e., the second training image mentioned above).

[0088] Step S302, preprocessing stage, performs spatiotemporal filtering: 3D Gaussian filtering, and emissivity correction: pixel-level ε value mapping based on material library.

[0089] Step S303, variance discretization stage. Set spatial grid: 0.1mm × 0.1mm (corresponding to a 3×3 region of infrared pixels); time step: Δt = 0.1 seconds, satisfying the CFL condition.

[0090] Step S304, Iterative Solution Stage. Initial conditions: T(x,y,0) = ambient temperature (25℃); Boundary conditions: convective heat dissipation (h=25W / m²). 2 K); Convergence criterion: residual < 1e-4℃ or number of iterations ≥ 100.

[0091] Step S305: Output the abnormal heat source distribution map and temperature gradient vector field.

[0092] Through the above embodiments, the gradient information will guide the update of network parameters through the backpropagation algorithm, so that the predicted heat map not only matches the labeled data, but also conforms to the physical laws of heat conduction.

[0093] In some embodiments, the above-mentioned method of fusing visual defect feature maps and thermal defect feature maps using an initial detection model to obtain a composite defect feature map may further include the following steps:

[0094] Assign corresponding feature weight values ​​to the visual defect feature map and the thermal defect feature map, and then perform weighted fusion of the visual defect feature map and the thermal defect feature map based on the assigned feature weight values ​​to obtain a composite defect feature map.

[0095] Furthermore, the aforementioned feature weight values ​​include a first learnable weight parameter and a second learnable weight parameter; the weighted fusion of the visual defect feature map and the thermal defect feature map based on the assigned weight values ​​to obtain a composite defect feature map may further include the following steps:

[0096] Obtain preset bias parameters; based on the first learnable weight parameter, the second learnable weight parameter, and the bias parameter, perform linear transformation on the visual defect feature map and the thermal defect feature map to generate an attention weight map; based on the attention weight map, perform element-wise weighted summation on the visual defect feature map and the thermal defect feature map to obtain a composite defect feature map.

[0097] First, obtain the preset bias parameter b from the model, and the first and second learnable weight parameters (denoted as W) corresponding to the visible light and infrared branches, respectively. v W t Subsequently, the visual defect feature map F v With thermal defect feature diagram F t Each parameter is weighted separately and then a bias term is added to form a linear combination. The result of this linear combination is input into the activation function to generate the attention weight map G, as shown in the following formula:

[0098] G=σ(W v F v +W t F t +b);

[0099] F_{fusion}=G⊙F v +(1-G)⊙F t ;

[0100] In the above formula, The sigmoid activation function is used. The bias parameter b serves as a learnable parameter for gating signal generation, and its role is to adjust the baseline activation threshold for feature fusion. When no features are input, b adjusts the initial activation state of the gating unit to prevent feature weights from completely suppressing a particular modality. During backpropagation, b and the weight matrix are optimized together, enabling the model to dynamically adjust the fusion ratio of the two modalities according to the training objective. Combined with the sigmoid function, b improves the model's ability to capture the complementarity of bimodal features by shifting the non-linear region of the activation curve.

[0101] Finally, the original feature map is summed element-wise based on the attention weight map G, as shown in the following formula:

[0102] F_{fusion}=G⊙F v +(1-G)⊙F t ;

[0103] Where ⊙ denotes element-wise multiplication, and F_{fusion} represents the composite defect feature map. This formula achieves complementary fusion of dual-modal information: the visible light branch provides high-resolution geometric details (such as pad edge fractures), while the infrared branch supplements the physical consistency under the constraints of thermal conduction (such as abnormal temperature gradients in short-circuit regions). Through end-to-end training, the model can automatically balance the reliability of the two modes under complex operating conditions; for example, it enhances the infrared feature weights under uneven illumination and strengthens the dominance of visible light features under thermal noise interference.

[0104] Through the above embodiments, the thermal conduction equations embedded in the infrared branch are used to ensure that the fused features conform to thermodynamic laws, and a cross-modal attention gating mechanism is introduced, which can effectively solve the problem of scale mismatch between visible light and infrared features.

[0105] In some embodiments, the above-mentioned detection based on composite defect feature maps to obtain prediction results may further include the following steps:

[0106] The classification prediction result is obtained by predicting the probability distribution of defects based on the composite defect feature map through the classification output head of the initial detection model, and the future temperature rise prediction result is obtained by performing linear regression processing on the composite defect feature map through the thermal risk assessment head of the initial detection model. The prediction results include the classification prediction result and the future temperature rise prediction result.

[0107] Specifically, the model inputs a composite defect feature map into a classification output head, reduces the spatial dimension of the feature map using fully connected or convolutional layers, and outputs the probability distribution of multiple defect types (such as short circuits, open circuits, welding problems, etc.) through a Softmax activation function, thus obtaining the classification prediction result. Simultaneously, the composite defect feature map is also input into an independent thermal risk assessment head. This branch models the local temperature change rate using a linear regression model (which can incorporate the time derivative term of the heat conduction equation). Based on physical constraints such as the current temperature gradient and sudden changes in heat flux density, it predicts the temperature rise within a certain time window in the future (e.g., triggering an early warning when ΔT ≥ 0.5°C / mm), ultimately forming a comprehensive prediction output that includes the classification result and the temperature rise trend. This design not only achieves explicit identification of defect types but also uncovers latent thermal failure risks through a physically enhanced model, realizing defect detection guided by physical mechanisms.

[0108] In some embodiments, the construction of the joint loss function based on defect labels, prediction results, and physical constraints may further include the following steps:

[0109] Based on the classification prediction results and defect labels, a first branch loss function is constructed, and a second branch loss function is constructed based on the future temperature rise prediction results and physical constraints. Corresponding loss weight values ​​are assigned to the first branch loss function and the second branch loss function. The first branch loss function and the second branch loss function are weighted and calculated based on the assigned loss weight values ​​to construct a joint loss function.

[0110] Specifically, the dynamic weight allocation uses the following formula:

[0111] λ_t = \frac{1}{1+\exp(-γ(L_{visible}-L_{thermal}))};

[0112] In the above formula, γ is the balancing factor, L_{visible} represents the value of the first branch loss function, and L_{thermal} represents the value of the second branch loss function. The final joint loss function is:

[0113] L_{total} = λ_tL_{visible} + (1-λ_t)L_{thermal};

[0114] Through the above embodiments, a multi-task learning framework is used to simultaneously optimize the visible light and infrared detection branches, and model convergence is achieved through dynamic weight allocation, ultimately realizing the technical route of multimodal perception-physical enhancement-dynamic fusion.

[0115] In some embodiments, the above-described assignment of corresponding loss weight values ​​to the first branch loss function and the second branch loss function may further include the following steps:

[0116] Monitor the magnitude of the first initial loss gradient corresponding to the first network branch and the magnitude of the second initial loss gradient corresponding to the second network branch; calculate the balance factor based on the magnitude of the first initial loss gradient and the magnitude of the second initial loss gradient, and assign corresponding loss weight values ​​to the loss function of the first branch and the loss function of the second branch according to the balance factor.

[0117] Specifically, during the initial training phase, the loss gradient magnitudes of the visible light branch (first network branch) and the infrared branch (second network branch) are monitored in real time. and Set the balance factor as follows:

[0118] ;

[0119] In the above formula, To prevent extremely small constants with a denominator of zero, this design automatically reflects the difference in contribution of the two modal features to model optimization, strengthening its weight adjustment capability when the visible light branch gradient is dominant; a time decay coefficient β is introduced to establish an exponential moving average (EMA) mechanism for updating.

[0120] ;

[0121] The above formula dynamically adjusts the balance strength based on the relative difference in loss values. It automatically strengthens the adjustment when the difference between the two branch losses significantly increases, thus avoiding the gradient vanishing problem. γ is parameterized as a differentiable variable, and a regularization term is added to the total loss function.

[0122] ;

[0123] Calculate using the chain rule This method enables end-to-end automatic parameter tuning. It incorporates second-order optimization principles, allowing the γ value to adaptively converge to the optimal solution during backpropagation.

[0124] Compared with the traditional fixed γ value method, the above-mentioned dynamic setting of the balance factor can effectively eliminate the bias of manual parameter tuning, establish a dynamic feedback mechanism through mathematical modeling, and enhance parameter stability by introducing time series analysis. At the same time, it transforms hyperparameters into learnable variables, thereby improving the robustness of the model to multimodal feature imbalance.

[0125] This embodiment also provides a PCB defect detection method. Figure 4 This is a flowchart of a PCB defect detection method according to an embodiment of this application, such as... Figure 4 As shown, the process includes the following steps:

[0126] Step S410: Obtain a first image to be inspected and a second image to be inspected, which contain images of the PCB to be inspected.

[0127] Step S420: Input the first image to be inspected and the second image to be inspected into the trained defect detection model to obtain the defect detection result of the PCB to be inspected; the defect detection model is generated according to the training method of the defect detection model described in any of the above embodiments.

[0128] The aforementioned PCB defect detection method also includes an online detection stage: the visible light and infrared acquisition frame rates are synchronized to 60fps; a lightweight inference engine is deployed to achieve a single-frame processing latency of ≤50ms; and a defect-thermodynamic spectrum database is established for online result verification.

[0129] The present application will now be described in detail with reference to specific embodiments. Figure 5 This is a schematic diagram of the training process of a defect detection model according to an embodiment of this application, as shown below. Figure 5 As shown, the defect detection model before training (i.e., the initial detection model mentioned above) includes an input layer, a feature extraction layer, a fusion module, and an output layer. Wherein:

[0130] The input layers include a visible light branch input layer and an infrared branch input layer. The feature extraction layers include a ResNet visible light branch (outputting a 2048×7×7 tensor) and a U-Net infrared branch (outputting a 512×40×40 tensor). The fusion module performs spatial alignment: unifying feature maps to 256×256 using bilinear interpolation, setting attention gating, and feature concatenation: merging along the channel dimension (outputting 256×256×512). The output layers include a defect classification head and a thermal risk assessment head; the defect classification head passes through a fully connected layer to Sofmax, outputting a 6-class probability distribution; the thermal risk assessment head uses an LSTM algorithm and linear regression to output a predicted future temperature rise.

[0131] To better understand this application, two specific application scenarios that utilize the aforementioned defect detection model are also provided: an online inspection system for SMT production lines and an offline in-depth diagnostic system for rework stations.

[0132] The deployed hardware for the SMT production line online inspection system includes an optical imaging subsystem, an infrared thermal imaging subsystem, and a synchronous control system. The optical imaging subsystem uses a Basler ace2 4096-30gc industrial camera with the following specifications: Sony IMX342 CMOS sensor, 4096×3072 effective pixels (12.5 megapixels), 3.45μm×3.45μm pixel size, 30fps (full resolution) / 60fps (2048×1536 regional readout), and a 10GigE interface. The camera is mounted vertically 350 mm above the conveyor belt, using a pneumatic lifting mechanism allowing for ±50mm height adjustment; it is equipped with an active cooling system (TEC thermoelectric cooler) to maintain an operating temperature of 25±2℃.

[0133] In the application scenario of an online inspection system for SMT production lines, the software implementation process includes: First, preprocessing is performed based on a data preprocessing pipeline. The processing of visible light images to be inspected can be achieved through geometric correction, radial distortion correction, illumination compensation, infrared data processing, emissivity setting, and thermal sequence analysis. For example, geometric correction can be performed using the Zhang Zhengyou calibration method to obtain the camera intrinsic parameter matrix.

[0134] K = \begin{bmatrix}

[0135] 3567.2 & 0 & 2045.3 \\

[0136] 0 & 3565.1 & 1532.8 \\

[0137] 0 & 0 & 1

[0138] \end{bmatrix}

[0139] The radial distortion correction parameters can be set as follows: k1=-0.2153, k2=0.2317. Illumination compensation can employ a block brightness equalization algorithm (Block Size=256×256) and adaptive histogram stretching (CLAHE, Clip Limit=2.0). Infrared data processing can use temperature calibration and non-uniformity correction (NUC) based on Planck's radiation law. Emissivity settings can be 0.92 for the FR-4 substrate, 0.05 for the copper pads, and 0.85 for the component encapsulation. The thermal sequence analysis process can involve first downsampling the time, extracting a 1Hz trend signal from the 60Hz raw data, then spatial filtering, and smoothing based on a 5×5 Gaussian kernel (σ=1.5).

[0140] On the other hand, deep learning models are deployed. The training strategy can employ transfer learning, as shown in Table 1.

[0141] Table 1

[0142]

[0143] Before training, data augmentation processing can be performed on the training images. The processing of visible light training images includes: random rotation (±5°), color jitter (ΔHSV=±0.1), and simulating poor solder joints (randomly removing 5-15% of solder joint pixels). The processing of infrared training images includes: adding thermal noise (σ=0.05℃) and simulating abnormal heat dissipation (local temperature rise of 0.5-2.0℃).

[0144] Inference optimization is then performed, using TensorRT acceleration configuration. Below is a code example generated based on the TensorRT engine:

[0145] trt_logger = trt.Logger(trt.Logger.WARNING)

[0146] explicit_batch = 1 << (int)(trt.NetworkDefinitionCreationFlag.EXPLICIT_BATCH)

[0147] with trt.Builder(trt_logger) as builder, builder.create_network(explicit_batch) as network:

[0148] parser = trt.OnnxParser(network, trt_logger)

[0149] with open("model.onnx", "rb") as model:

[0150] parser.parse(model.read())

[0151] config = builder.create_builder_config()

[0152] config.set_flag(trt.BuilderFlag.FP16)

[0153] config.max_workspace_size = 1 << 30

[0154] engine = builder.build_engine(network, config)

[0155] In addition, memory optimization is required, which is implemented using a double buffering mechanism. The GPU memory allocation strategy is to allocate 70% for inference and 30% for preprocessing.

[0156] Next, the production line integration details were set. First, regarding the mechanical structure design, such as the layout parameters of the inspection station, including conveyor belt speed: 0.8m / s ±0.05m / s, station length: 1200mm, camera field of view coverage: 98.7% (considering mechanical vibration tolerance), maximum allowable board warpage: ≤1.5mm / m, etc., and the design of vibration reduction measures, using an air-floating vibration isolation platform (natural frequency <5Hz, camera bracket damping coefficient set at: 0.65 (silicone vibration damping pad). Second, regarding the human-machine interface, including a real-time monitoring panel with multi-view display: picture-in-picture overlay of visible light and infrared images; abnormal areas marked with pseudo-color (rainbow-colored temperature scale). The alarm handling mechanism adopts a tiered alarm strategy: a level 1 alarm occurs when three consecutive boards have defects in the same location, automatically stopping the machine and triggering an audible and visual alarm; a level 2 alarm occurs when the temperature gradient exceeds the limit for 10 seconds, reducing the conveyor belt speed to 0.3m / s; a level 3 alarm occurs when the number of defects on a single board >5, logging is recorded, and manual sampling is prompted.

[0157] The hardware configuration for the offline deep diagnostic system at the rework station includes a mobile inspection terminal and an end effector. The mobile inspection terminal consists of: a Dell Precision 7760 workstation (CPU: Intel Xeon W-11955M @3.6GHz; GPU: NVIDIA RTX A5000 16GB); a handheld thermal imager (FLIR T1020 with thermal sensitivity of 0.03℃); and a six-DOF robotic arm (UR10e) supporting laser ranging (accuracy ±1mm). The end effector includes a vacuum-adhesive PCB clamp. The repeatability is set to ±0.1mm.

[0158] Next, the depth analysis function was set up. The data acquisition process included a robotic arm driving a thermal imager to scan along the Z-axis (step size 0.5mm, range ±5mm); multi-view image registration included feature point matching: ORB algorithm to extract 500 feature points, and transformation matrix solving: RANSAC+LM optimization. The 3D temperature field generation is shown in the following code:

[0159] def reconstruct_3d_temp(points_cloud):

[0160] voxel_size = 0.1 # mm

[0161] grid = create_voxel_grid(points_cloud, voxel_size)

[0162] apply_laplacian_smoothing(grid, iterations=5)

[0163] return interpolate_missing_data(grid)

[0164] Next, regarding the fault tracing analysis, including thermo-electric coupling simulation, the input parameters are set as follows:

[0165] {

[0166] "material_properties": {

[0167] "FR4": {"k": 0.3, "ρ": 1.85e6},

[0168] "Copper": {"k": 401, "ρ": 1.68e-8}

[0169] },

[0170] "boundary_conditions": {

[0171] "convection_coeff": 25,

[0172] "ambient_temp": 25

[0173] }

[0174] }

[0175] The finite element method solver used was COMSOL Multiphysics® coupled with ANSYS Icepak. The fault location algorithm is shown in the following formula:

[0176] ;

[0177] Set a threshold: a potential fault point is identified when the score is > 15.

[0178] The final diagnostic report is generated. The report template includes: a defect location map (including GPS coordinate mapping, accuracy ±5mm), and thermal history curves (data traced back up to 72 hours). Failure mode prediction is based on MTTF calculations using Weibull analysis.

[0179] MTTF = \eta \cdot \Gamma(1+\frac{1}{\beta}) ;

[0180] The shape parameter β = 1.2 and the characteristic lifetime η = 1500 hours.

[0181] The digital twin interface supports exporting JT format 3D models and integrates with the MES system via the OPC UA protocol.

[0182] The implementation results for the two application scenarios mentioned above are shown in Table 2:

[0183] Table 2

[0184]

[0185] In addition, please see Figure 6 Furthermore, this application provides a multimodal PCB defect detection method, and a histogram comparing the multimodal defect detection performance of the detection method based on a single visible light deep learning model. For example, for the detection of cold solder joint defects and overcurrent defects, the defect detection score of the multimodal solution in this application is much higher than that of the single visible light deep learning model.

[0186] Through the above embodiments, it supports wide temperature range detection from -20℃ to 150℃, adapting to complex working conditions in industrial sites; it provides a full-stack optimization solution from algorithm to hardware, ensuring a smooth transition from laboratory to production line; it helps reduce the labor cost of re-inspection by 70%, shortening the equipment investment payback period to 14 months; and through predictive maintenance, it also effectively reduces after-sales repair costs caused by thermal failure by 30%.

[0187] It should be noted that the steps shown in the above process or in the flowchart of the accompanying figures can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0188] This embodiment also provides a training device for a defect detection model, which is used to implement the above embodiments and preferred embodiments, and will not be repeated as already described. As used below, the terms "module," "unit," "subunit," etc., can refer to a combination of software and / or hardware that performs a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0189] The aforementioned training device includes: an acquisition module, a feature extraction module, a prediction module, and an iterative training module; wherein:

[0190] The system comprises the following modules: an acquisition module for acquiring a first training image and a second training image containing the sample PCB image, and the corresponding defect label of the sample PCB; a feature extraction module for extracting a visual defect feature map based on the first training image using the first network branch of the initial detection model, and extracting a thermal defect feature map based on the second training image using the second network branch of the initial detection model; a prediction module for fusing the visual defect feature map and the thermal defect feature map using the initial detection model to obtain a composite defect feature map, and performing detection based on the composite defect feature map to obtain a prediction result; and an iterative training module for constructing physical constraint terms based on the spatial gradient information of the thermal defect feature map, constructing a joint loss function based on the defect label, prediction result, and physical constraint terms, and iteratively training the initial detection model based on the joint loss function to generate a defect detection model.

[0191] It should be noted that the above modules can be functional modules or program modules, and can be implemented by software or hardware. For modules implemented by hardware, the above modules can reside in the same processor; or the above modules can be located in different processors in any combination. Specific examples in this embodiment can be found in the examples described in the above embodiments and optional implementations, and will not be repeated in this embodiment.

[0192] This embodiment also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0193] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0194] Optionally, in this embodiment, the processor can be configured to perform the following steps via a computer program:

[0195] S1, acquire the first training image and the second training image containing the sample PCB image, as well as the defect label corresponding to the sample PCB.

[0196] S2, through the first network branch of the initial detection model, a visual defect feature map is obtained by extracting features based on the first training image; through the second network branch of the initial detection model, a thermal defect feature map is obtained by extracting features based on the second training image.

[0197] S3 uses the initial detection model to fuse the visual defect feature map and the thermal defect feature map to obtain a composite defect feature map, and performs detection based on the composite defect feature map to obtain the prediction result.

[0198] S4. Construct physical constraint terms based on the spatial gradient information of the thermal defect feature map. Construct a joint loss function based on the defect label, prediction results and physical constraint terms. Iteratively train the initial detection model based on the joint loss function to generate a defect detection model.

[0199] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated here.

[0200] Furthermore, in conjunction with the training method of the defect detection model in the above embodiments, this application embodiment can provide a storage medium for implementation. This storage medium stores a computer program; when executed by a processor, the computer program implements any of the defect detection model training methods in the above embodiments.

[0201] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0202] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0203] Those skilled in the art should understand that the technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments have been described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0204] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A training method for a defect detection model, characterized in that, The method includes: Obtain a first training image and a second training image containing the image of the sample PCB, as well as the defect label corresponding to the sample PCB; A visual defect feature map is obtained by extracting features from the first training image using the first network branch of the initial detection model; a thermal defect feature map is obtained by extracting features from the second training image using the second network branch of the initial detection model. Using the initial detection model, the visual defect feature map and the thermal defect feature map are fused to obtain a composite defect feature map, and detection is performed based on the composite defect feature map to obtain a prediction result; A physical constraint term is constructed based on the spatial gradient information of the thermal defect feature map. A joint loss function is constructed based on the defect label, the prediction result, and the physical constraint term. The initial detection model is then iteratively trained based on the joint loss function to generate a defect detection model.

2. The training method according to claim 1, characterized in that, The step of obtaining a thermal defect feature map by extracting features from the second training image through the second network branch of the initial detection model includes: Initial feature data is obtained by extracting features from the second training image through the second network branch; Using a pre-established heat conduction inversion model, heat conduction calculations are performed based on the initial feature data to obtain temperature gradient information and heat flux density information; Based on the temperature gradient information and the heat flux density information, local thermal anomaly regions in the second training image are identified, and the thermal defect feature map is generated based on the local thermal anomaly regions.

3. The training method according to claim 1, characterized in that, The construction of physical constraint terms based on the spatial gradient information of the thermal defect feature map includes: Obtain the preset spatial grid parameters; Based on the spatial grid parameters, the spatial gradient of the thermal defect feature map is calculated to obtain the spatial gradient information, and the physical constraint terms are constructed based on the spatial gradient information.

4. The training method according to claim 1, characterized in that, The step of fusing the visual defect feature map and the thermal defect feature map using the initial detection model to obtain a composite defect feature map includes: The visual defect feature map and the thermal defect feature map are assigned corresponding feature weight values, and the visual defect feature map and the thermal defect feature map are weighted and fused based on the assigned feature weight values ​​to obtain the composite defect feature map.

5. The training method according to claim 4, characterized in that, The feature weight values ​​include a first learnable weight parameter and a second learnable weight parameter; the weighted fusion of the visual defect feature map and the thermal defect feature map based on the assigned feature weight values ​​to obtain the composite defect feature map includes: Obtain the preset bias parameters; Based on the first learnable weight parameter, the second learnable weight parameter, and the bias parameter, the visual defect feature map and the thermal defect feature map are linearly transformed to generate an attention weight map; Based on the attention weight map, the visual defect feature map and the thermal defect feature map are summed element by element to obtain the composite defect feature map.

6. The training method according to any one of claims 1 to 5, characterized in that, The detection based on the composite defect feature map to obtain the prediction result includes: The classification output head of the initial detection model is used to predict the probability distribution of defects based on the composite defect feature map to obtain the classification prediction result. Then, the thermal risk assessment head of the initial detection model is used to perform linear regression processing on the composite defect feature map to obtain the future temperature rise prediction result. The prediction results include the classification prediction results and the future temperature rise prediction results.

7. The training method according to claim 6, characterized in that, The step of constructing a joint loss function based on the defect label, the prediction result, and the physical constraint term includes: Based on the classification prediction results and the defect labels, a first branch loss function is constructed, and based on the future temperature rise prediction results and the physical constraint terms, a second branch loss function is constructed. Assign corresponding loss weight values ​​to the first branch loss function and the second branch loss function, and perform weighted calculation on the first branch loss function and the second branch loss function based on the assigned loss weight values ​​to construct the joint loss function.

8. The training method according to claim 7, characterized in that, Assigning corresponding loss weight values ​​to the first branch loss function and the second branch loss function includes: Monitor the magnitude of the first initial loss gradient corresponding to the first network branch, and the magnitude of the second initial loss gradient corresponding to the second network branch; Based on the first initial loss gradient magnitude and the second initial loss gradient magnitude, a balance factor is calculated, and according to the balance factor, the corresponding loss weight values ​​are assigned to the first branch loss function and the second branch loss function.

9. A PCB defect detection method, characterized in that, The method includes: Acquire a first and a second image of the PCB to be inspected, which contain images of the PCB to be inspected. The first image to be inspected and the second image to be inspected are input into the trained defect detection model to obtain the defect detection result of the PCB to be inspected; the defect detection model is trained and generated by the training method of the defect detection model according to any one of claims 1 to 8.

10. A storage medium, characterized in that, The storage medium stores a computer program, wherein the computer program is configured to execute, at runtime, the training method of the defect detection model according to any one of claims 1 to 8, or the PCB defect detection method according to claim 9.

Images