A PCB defect detection model automatic optimization method and system based on multi-modal diagnostic representation

Abstract

This invention provides an automatic optimization method and system for a PCB defect detection model based on multimodal diagnostic representations, belonging to the field of automated machine learning technology. The method is automatically scheduled and executed by an intelligent experiment agent, including: acquiring a PCB dataset, an initial model, experimental specifications, and a policy library; parsing labeled files to generate task prior representations and training to obtain a stable baseline; calling a verification analysis module and a feature response extraction module to generate error sample information and feature response information; constructing a multimodal diagnostic representation by combining task priors, error information, feature responses, training process, and historical data; using this representation as a retrieval query to match candidate optimization strategies in the policy library and generate candidate experiments with controlled main variables; executing experiments and updating the historical data through a non-intrusive adapter; updating or restoring the stable baseline based on the experimental results, and determining whether to continue the loop. This invention achieves automated optimization of the PCB defect detection model, significantly reducing manual intervention.

Description

Technical Field

[0001] This invention relates to the field of automated machine learning technology, specifically to an automatic optimization method and system for a PCB defect detection model based on multimodal diagnostic representation. Background Technology

[0002] PCBs are a critical foundational component of electronic products, and their defect detection is crucial for product quality control. In recent years, deep learning-based target detection models (such as YOLO, Faster R-CNN, and DETR) have been widely applied to PCB defect detection tasks and have achieved good results.

[0003] However, the optimization of PCB defect detection models still heavily relies on human experience. Manual parameter tuning requires engineers to repeatedly review logs and error samples, selecting experimental directions based on experience, which is time-consuming, inefficient, and difficult to sustain. General automatic parameter tuning (such as random search and Bayesian optimization) usually only uses a few numerical indicators such as overall mAP as feedback, failing to fully utilize information such as annotation distribution, defect scale, class imbalance, error sample types, and intermediate feature responses of the model. It is difficult to determine whether the performance bottleneck stems from small target missed detections, class confusion, localization bias, or insufficient feature response, leading to blind optimization directions.

[0004] Furthermore, the improvement directions for different problems vary significantly (e.g., small target omissions require adjusting resolution or sample allocation, and positioning deviations require adjusting positioning loss). Existing methods struggle to automatically select suitable candidate modifications based on error patterns and historical experiments, easily leading to invalid attempts or the introduction of multiple variables at once, making it difficult to attribute experimental results. Moreover, general-purpose tools lack unified constraints on experimental phases, modifiable objects, anomaly handling, and model recovery. This is especially problematic when model structure modifications are involved, easily resulting in code clutter, unreproducible results, and difficulty in recovering from invalid modifications. These shortcomings prevent existing technologies from meeting the rapid iteration needs of PCB production line data as product models and imaging conditions change.

[0005] Therefore, there is an urgent need for an automatic optimization method and system for PCB defect detection models that can automatically construct model diagnostic criteria, generate candidate experiments under specification constraints, and execute and iteratively update stable baselines. Summary of the Invention

[0006] The purpose of this invention is to overcome at least one technical problem existing in the prior art and to provide an automatic optimization method and system for PCB defect detection model based on multimodal diagnostic characterization.

[0007] On one hand, this invention provides an automatic optimization method for a PCB defect detection model based on multimodal diagnostic representations. The method is automatically scheduled and executed by an intelligent experimental agent, including: Step S1, the intelligent experimental agent acquires a PCB defect detection dataset, an initial target detection model, a declarative experimental specification, and a retrieval enhancement experimental strategy library; Step S2, the intelligent experimental agent calls a pre-set annotation parsing module to parse the annotation files of the PCB defect detection dataset and generate a PCB task prior representation; Step S3, the intelligent experimental agent, based on the PCB task prior representation and the declarative experimental specification, calls a pre-set training execution module to train the initial target detection model and obtain initial detection results; the initial detection results include model evaluation metrics and training process information; the intelligent experimental agent confirms the model configuration and evaluation metrics of the trained target detection model as the current stable baseline; Step S4, the intelligent experimental agent calls a pre-set verification analysis module and a pre-set feature response extraction module to generate model diagnostic information based on the current stable baseline; the model diagnostic information includes error sample information and feature response information. Step S5: The intelligent experiment agent calls a pre-set multimodal representation construction module, using the PCB task prior representation, error sample information, feature response information, training process information, and historical experiment information stored in the experiment history as input, to construct a multimodal diagnostic representation. Step S6: The intelligent experiment agent uses the multimodal diagnostic representation as retrieval query information, searches the retrieval enhancement experiment strategy library for candidate optimization strategies matching the current optimization stage, and combines it with the declarative experiment specification to generate candidate experiments with controlled main variables for this round. Step S7: The intelligent experiment agent calls a pre-set non-intrusive experiment execution adapter to execute the candidate experiments with controlled main variables and updates the experiment history. Step S8: The intelligent experiment agent compares the evaluation metrics of the candidate experiments in this round with the evaluation metrics of the current stable baseline: if the preset retention conditions are met, the model configuration and evaluation metrics corresponding to the candidate experiments are updated to the new stable baseline; if the preset retention conditions are not met, the system reverts to the previous stable baseline. Step S9: The intelligent experiment agent determines whether to continue executing the next round of loops S4 to S8 based on the exit conditions in the declarative experiment specification.

[0008] Furthermore, the PCB defect detection dataset includes PCB images and their corresponding defect annotation files; the initial target detection model is the PCB defect detection model to be optimized; the declarative experimental specification is used to define the experimental stages, allowed modification objects, evaluation indicators, exit conditions, and anomaly handling rules in the automatic optimization process; the retrieval enhancement experimental strategy library is used to store predefined adjustment methods, candidate modules, experimental templates, applicable conditions, risk constraints, and historical experimental experience related to the optimization of the PCB defect detection model.

[0009] Further, step S2 includes: the intelligent experiment agent extracts defect box coordinates, defect category, and image size information from the PCB defect annotation file by calling a preset annotation parsing module, and statistically analyzes the scale distribution, aspect ratio distribution, small target proportion, category distribution, category imbalance, and defect density information of the defect targets, thereby forming a PCB task prior representation to describe the characteristics of the current PCB defect detection task; the PCB task prior representation is used to constrain the search space and experiment priority of candidate experiments, including: when the proportion of small targets in the PCB task prior representation is higher than a preset small target proportion threshold, the intelligent experiment agent increases the priority of experiments related to input resolution, shallow feature enhancement, small target detection head, or positive sample allocation strategy; when the category imbalance is higher than a preset imbalance threshold, the intelligent experiment agent increases the priority of experiments related to category weight, loss function, or sample allocation strategy.

[0010] Furthermore, step S4 includes: the intelligent experimental agent calls the current stable baseline to make predictions on a fixed validation set, and matches the prediction results with the real annotations through the validation analysis module to obtain information on at least one type of erroneous sample, including missed detection samples, false detection samples, misclassified samples, and positioning deviation samples; the intelligent experimental agent calls the feature response extraction module to extract intermediate features from the intermediate feature layer of the current target detection model; the feature response extraction module performs channel compression, normalization, and spatial alignment processing on the extracted intermediate features to generate a feature response map that can characterize the model's response to PCB defect areas.

[0011] Furthermore, the multimodal diagnostic representation in step S5 includes labeled representation, error representation, feature representation, training representation, and experimental history representation; wherein, the labeled representation is derived from the prior representation of the PCB task, the error representation is derived from the error sample information, the feature representation is derived from the feature response information, the training representation is derived from the evaluation index in the current stable baseline result, and the experimental history representation is derived from the historical experimental information.

[0012] Further, step S6 includes: the intelligent experiment agent generates retrieval query information based on the multimodal diagnostic representation, and obtains candidate optimization strategies matching the current experimental state from the retrieval enhancement experiment strategy library; the retrieval query information includes the current PCB task prior, current error sample type, current feature response performance, current experimental stage, current stable baseline, and historical experimental state; the retrieval enhancement experiment strategy library includes a predefined experiment strategy library, a candidate module library, and a historical experiment experience library; the predefined experiment strategy library is used to store candidate adjustment methods corresponding to different diagnostic conditions, the candidate module library is used to store model structures, loss functions, sample allocation strategies, or training strategy templates that can be called, and the historical experiment experience library is used to store the applicable conditions, experimental results, and retained conclusions of historical experiments; when the retrieval enhancement experiment strategy library returns multiple candidate optimization strategies, the intelligent experiment agent sorts them according to diagnostic matching degree, historical experiment benefits, reduction fit, resource consumption constraints, and abnormal risks, and determines a main optimization strategy from the top-ranked candidate optimization strategies to generate a candidate experiment with controlled main variables in this round; the remaining candidate optimization strategies serve as alternative strategies for subsequent experiments.

[0013] Furthermore, step S7 includes: a non-intrusive experiment execution adapter converts candidate experiments into external configurations or controlled modification instructions executable by the training engine; when the candidate experiment is a parameter-based experiment, the non-intrusive experiment execution adapter is implemented through configuration overriding or command parameter injection; when the candidate experiment is a structure-based or loss function-based experiment, the non-intrusive experiment execution adapter is implemented through module switching, preset candidate module invocation, or controlled code patching; the experiment history includes experiment number, experiment label, candidate modification content, training parameters, model configuration, overall detection index, category-level detection index, error sample change, resource consumption, experiment status, and / or experiment conclusion; the experiment status includes at least one of baseline state, retained state, discarded state, and abnormal state.

[0014] Furthermore, the preset retention conditions in step S8 include that the candidate experiment achieves a preset improvement in the main evaluation index relative to the current stable baseline, or achieves a preset improvement in the category-level index of the key defect category and the overall performance is not lower than the preset tolerance range; if the candidate experiment causes a decrease in the main evaluation index, an increase in the false negative rate, a model complexity exceeding a preset limit, an inference delay exceeding a preset limit, or training anomalies, then the candidate experiment will not be confirmed as a new stable baseline; the key defect category includes PCB defect categories with a sample number less than a preset sample number threshold, a false negative cost higher than a preset false negative cost threshold, or a historical detection effect lower than a preset detection effect threshold.

[0015] Furthermore, the exit conditions in step S9 include at least one of the following: reaching the preset maximum number of experimental rounds; multiple consecutive rounds of candidate experiments not exceeding the current stable baseline; the false negative rate of the key defect category is lower than a preset threshold; the model accuracy and inference speed meet the deployment requirements; the experimental resource consumption exceeds the preset upper limit; the intelligent experimental agent checks the exit conditions at the end of each round of S9, and if any condition is met, the loop terminates and the current stable baseline is output as the final optimization result.

[0016] Secondly, embodiments of the present invention provide an automatic optimization system for a PCB defect detection model based on multimodal diagnostic representations. The system is implemented using the aforementioned automatic optimization method for a PCB defect detection model based on multimodal diagnostic representations, and includes: an intelligent experiment agent, and a label parsing module, a training execution module, a diagnostic information generation module, a multimodal representation construction module, and an automatic experiment execution module, respectively invoked by the intelligent experiment agent; wherein, the intelligent experiment agent is used to: acquire a PCB defect detection dataset, an initial target detection model, a declarative experimental specification, and a retrieval enhancement experiment strategy library; invoke the label parsing module to generate prior representations of the PCB task; and invoke the training execution module to obtain initial detection results. And confirm the current stable baseline; call the diagnostic information generation module to generate error sample information and feature response information; input the PCB task prior representation, error sample information, feature response information, training process information, and historical experiment information into the multimodal representation construction module to generate multimodal diagnostic representation; use the multimodal diagnostic representation as retrieval query information to retrieve candidate optimization strategies in the retrieval enhancement experiment strategy library, and combine it with the declarative experiment reduction to generate candidate experiments with controlled main variables in this round; call the automatic experiment execution module to execute the candidate experiments and update the experiment history; compare the candidate experiment results with the current stable baseline, and update or restore the stable baseline according to the preset retention conditions; according to the declarative experiment... The exit condition in the verification specification determines whether to continue the automatic optimization loop; the annotation parsing module is used to parse the annotation files of the PCB defect detection dataset, extract defect box coordinates, defect categories and image size information, and statistically analyze the scale distribution, aspect ratio distribution, proportion of small targets, category distribution, category imbalance degree and defect density information of defect targets to generate PCB task prior representations; the training execution module is used to train the initial target detection model according to the PCB task prior representations and declarative experimental specifications to obtain initial detection results; the initial detection results include model evaluation indicators and training process information; the diagnostic information generation module includes a verification analysis unit and a feature response extraction unit. The validation analysis unit is used to make predictions on a fixed validation set based on the current stable baseline and match them with the real annotations to obtain information on at least one type of error sample, including missed detection samples, false detection samples, misclassified samples, and localization bias samples. The feature response extraction unit is used to extract intermediate features from the intermediate feature layer of the current target detection model and perform channel compression, normalization, and spatial alignment to generate feature response information. The multimodal representation construction module is used to construct a multimodal diagnostic representation that includes numerical diagnostic information, image-based diagnostic information, textual historical information, and diagnostic labels, using the PCB task prior representation, error sample information, feature response information, training process information, and historical experimental information stored in the experimental history as input.The automated experiment execution module includes a non-intrusive experiment execution adapter, used to convert the candidate experiments into external configurations or controlled modification instructions executable by the training engine, execute the candidate experiments, and feed the experimental results back to the intelligent experiment agent.

[0017] Thirdly, embodiments of the present invention also provide an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the above-described automatic optimization method for PCB defect detection model based on multimodal diagnostic characterization.

[0018] Fourthly, embodiments of the present invention also provide a readable storage medium, which, when the instructions in the storage medium are executed by the processor of an electronic device, enables the electronic device to execute the above-described automatic optimization method for PCB defect detection model based on multimodal diagnostic characterization.

[0019] The beneficial effects of this invention are: 1. The intelligent experiment agent automatically executes the entire closed-loop operation from data parsing, baseline establishment, model diagnosis to candidate experiment generation, execution, evaluation and baseline update without human intervention, which significantly reduces the technical threshold and labor cost of PCB defect detection model optimization.

[0020] 2. By constructing a multimodal diagnostic representation that includes task priors, error samples, feature responses, training history, and experimental history, the performance bottlenecks of the model can be comprehensively identified from multiple perspectives, including numerical, image, and text. The automatic generation of diagnostic labels reduces the subjectivity and omissions of manual analysis and provides a precise basis for subsequent optimization.

[0021] 3. By leveraging a search-enhanced experimental strategy library, the system uses current diagnostic characteristics as query criteria to intelligently match predefined optimization strategies and historical experimental experience. A ranking mechanism prioritizes high-return, low-risk strategies, avoiding blind trial and error and significantly improving optimization efficiency.

[0022] 4. Each candidate experiment modifies only a single main variable and includes necessary adjustments, ensuring the attributability of the optimized variable. Experimental results are compared with the baseline, and clear retention conditions are set. Experiments that do not meet the conditions are automatically rolled back, while a complete history is retained, ensuring the stability and reproducibility of the optimization process.

[0023] 5. Through a non-intrusive experiment execution adapter, the intelligent experiment agent does not need to directly modify the training project code. Instead, it completes experiments through configuration overriding, controlled file editing, and automatic recovery mechanisms. This design reduces the risk of code errors, supports rapid rollback, and improves the system's robustness.

[0024] 6. The declarative experimental specification limits the objects that can be modified in different optimization stages (e.g., only the model size can be switched in the baseline establishment stage, only the learning rate and batch size can be adjusted in the hyperparameter search stage, and the backbone network, neck network, detector head, and loss function can be modified in the structure and loss function exploration stage), ensuring that the optimization process follows the reasonable order of "first select the basic model, then adjust the optimal parameters, and finally improve the structure", avoiding invalid or destructive experiments. Attached Figure Description

[0025] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0026] Figure 1 This is a flowchart of an automatic optimization method for PCB defect detection model based on multimodal diagnostic characterization provided in Embodiment 1 of the present invention.

[0027] Figure 2 This is a schematic diagram of the automatic optimization system structure of a PCB defect detection model based on multimodal diagnostic characterization provided in Embodiment 2 of the present invention.

[0028] Figure 3 This is a partial block diagram of the electronic device provided in Embodiment 3 of the present invention. Detailed Implementation

[0029] Before discussing the exemplary embodiments in more detail, it should be noted that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe the operations as sequential processes, many of these operations can be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the operations can be rearranged. The process can be terminated when its operation is completed, but may also have additional steps not included in the figures. The process can correspond to a method, function, procedure, subroutine, subroutine, etc.

[0030] It should be understood that although the terms "first," "second," etc., may be used herein to describe various units, these units should not be limited by these terms. These terms are used merely to distinguish one unit from another. For example, without departing from the scope of the exemplary embodiments, a first unit may be referred to as a second unit, and similarly, a second unit may be referred to as a first unit. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0031] The present invention will now be described in detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0032] Example 1 The specific implementation method is as follows: like Figure 1 The diagram shown is a flowchart of an automatic optimization method for PCB defect detection model based on multimodal diagnostic characterization provided by the present invention.

[0033] As an example, the method is automatically scheduled and executed by an intelligent experiment agent, including: Step S1, the intelligent experiment agent acquires a PCB defect detection dataset, an initial target detection model, a declarative experimental specification, and a retrieval enhancement experiment strategy library; Step S2, the intelligent experiment agent calls a pre-set annotation parsing module to parse the annotation files of the PCB defect detection dataset and generate a PCB task prior representation; Step S3, the intelligent experiment agent, based on the PCB task prior representation and the declarative experimental specification, calls a pre-set training execution module to train the initial target detection model and obtain initial detection results; the initial detection results include model evaluation metrics and training process information; the intelligent experiment agent confirms the model configuration and evaluation metrics of the trained target detection model as the current stable baseline; Step S4, the intelligent experiment agent calls a pre-set verification analysis module and a pre-set feature response extraction module to generate model diagnostic information based on the current stable baseline; the model diagnostic information includes error sample information and feature response information; Step S5, the intelligent experiment agent calls a pre-set... The multimodal representation construction module takes the PCB task prior representation, error sample information, feature response information, training process information, and historical experimental information stored in the experimental history as input to construct a multimodal diagnostic representation; Step S6: The intelligent experimental agent uses the multimodal diagnostic representation as retrieval query information to retrieve candidate optimization strategies that match the current optimization stage in the retrieval enhancement experimental strategy library, and combines it with the declarative experimental specification to generate candidate experiments with controlled main variables for this round; Step S7: The intelligent experimental agent calls the preset non-intrusive experimental execution adapter to execute the candidate experiments with controlled main variables and update the experimental history; Step S8: The intelligent experimental agent compares the evaluation index of the candidate experiments in this round with the evaluation index of the current stable baseline: if the preset retention conditions are met, the model configuration and evaluation index corresponding to the candidate experiment are updated to the new stable baseline; if the preset retention conditions are not met, the system is restored to the previous stable baseline; Step S9: The intelligent experimental agent determines whether to continue executing the next round of S4 to S8 based on the exit conditions in the declarative experimental specification.

[0034] In some feasible implementations, the Intelligent Experiment Agent is an automated agent program based on large language models and code execution capabilities. It can autonomously complete the entire process of PCB defect detection model optimization according to preset declarative experimental specifications, without manual intervention. In a preferred embodiment of the invention, the Intelligent Experiment Agent is implemented using the Claude Code tool and the Claude series of large language models. Claude Code serves as the agent's execution framework, providing basic capabilities such as file operations, command execution, and log monitoring; the Claude large language model acts as the agent's "brain," responsible for analyzing diagnostic information, retrieving strategies, generating experimental plans, and making comparative decisions. It should be noted that the invention is not limited to the aforementioned tool and model versions. Any automated agent with code understanding and generation capabilities, capable of calling external modules and executing commands (such as GPT-4 with LangChain, AutoGPT, or rule-based reasoning expert systems) can be used as an implementation method for the Intelligent Experiment Agent.

[0035] In some feasible implementations, the PCB defect detection dataset includes PCB images and their corresponding defect annotation files; the initial target detection model is the PCB defect detection model to be optimized; the declarative experimental specification is used to define the experimental stages, allowed modification objects, evaluation indicators, exit conditions, and anomaly handling rules in the automatic optimization process; the retrieval enhancement experimental strategy library is used to store predefined adjustment methods, candidate modules, experimental templates, applicable conditions, risk constraints, and historical experimental experience related to the optimization of the PCB defect detection model.

[0036] Preferably, the intelligent experimental agent acquires the following four types of input data: PCB Defect Detection Dataset: Contains multiple PCB images and corresponding defect annotation files for each image. The annotation files use a common format (such as XML or JSON) and record the bounding box coordinates, defect category label, and image size information for each defect.

[0037] Initial object detection model: A pre-selected object detection model to be optimized, such as a model based on YOLO, Faster R-CNN, or DETR architecture. This model includes a complete network architecture definition, pre-trained weights, and training configuration file.

[0038] Declarative Experiment Specification: A structured configuration file (e.g., in YAML or JSON format) used to define the experimental phases in the automatic optimization process, the objects that can be modified in each phase, evaluation metrics, exit conditions, and anomaly handling rules. Experimental phases include at least a baseline establishment phase, a hyperparameter search phase, and a structure and loss function exploration phase. Evaluation metrics include overall mean precision, recall, precision, and class-level mean precision. Exit conditions include a maximum number of experiment epochs, a threshold for the number of consecutive epochs without improvement, and performance targets for critical defect categories. Anomaly handling rules define the rollback or retry strategies to be taken when training fails, evaluation fails, or resource limits are exceeded.

[0039] Retrieval Enhancement Experiment Strategy Library: A database storing predefined tuning methods, candidate modules, experimental templates, applicable conditions, risk constraints, and historical experimental experience related to PCB defect detection model optimization.

[0040] In some feasible implementations, step S2 includes: the intelligent experiment agent extracts defect bounding box coordinates, defect categories, and image size information from the PCB defect annotation file by calling a pre-set annotation parsing module, and statistically analyzes the scale distribution, aspect ratio distribution, small target proportion, category distribution, category imbalance, and defect density information of the defect targets, thereby forming a PCB task prior representation to describe the characteristics of the current PCB defect detection task; the PCB task prior representation is used to constrain the search space and experiment priority of candidate experiments, including: when the proportion of small targets in the PCB task prior representation is higher than a preset small target proportion threshold, the intelligent experiment agent increases the priority of experiments related to input resolution, shallow feature enhancement, small target detection head, or positive sample allocation strategy; when the category imbalance is higher than a preset imbalance threshold, the intelligent experiment agent increases the priority of experiments related to category weight, loss function, or sample allocation strategy.

[0041] Preferably, the intelligent experiment agent invokes a pre-built annotation parsing module to perform batch parsing of the annotation files in the PCB defect detection dataset. The annotation parsing module extracts the following raw information from each annotation file: The coordinates of the top-left and bottom-right corners of each defect bounding box; the defect category corresponding to each bounding box (e.g., open circuit, short circuit, burr, notch, etc.); and the width and height of each PCB image. Based on the above raw information, the annotation and parsing module further statistically analyzes the following task features: scale distribution of defect targets: dividing the bounding box area into multiple intervals and counting the number of defects in each interval; aspect ratio distribution: calculating the aspect ratio of each bounding box and calculating its distribution histogram; proportion of small targets: based on a preset small target area threshold (e.g., 32×32 pixels), counting the proportion of defects with an area smaller than the threshold to the total number of defects; category distribution: counting the number of samples for each defect category; class imbalance: calculating the ratio of the number of samples in the largest category to the number of samples in the smallest category, or using other imbalance metrics (e.g., Shannon entropy, mean squared error, etc.); defect density information: counting the average number of defects, the standard deviation of the number of defects, and the maximum number of defects in a single image for each image. The above statistical results are integrated into a structured data object, called the PCB task prior representation. This characterization is used to describe the overall characteristics of the current PCB defect detection task and serves as input for subsequent diagnosis and experimental generation.

[0042] In some feasible implementations, step S3 includes: the intelligent experiment agent automatically determines the initial training parameters and search space constraints based on the PCB task prior representation and declarative experiment specification. Specifically, if the task prior representation indicates a high proportion of small targets, the agent preferentially selects a model variant with multi-scale feature fusion capabilities and appropriately increases the input image resolution. If the class imbalance exceeds a preset threshold, the agent enables class weights or sampling strategies in the training configuration. If the defect density is high, the agent selects a detection head structure capable of handling dense targets (e.g., using a smaller positive sample allocation threshold). The agent invokes a pre-set training execution module to start model training on the training and validation sets defined by the declarative experiment specification, according to the initial training parameters determined above. The training execution module is responsible for executing the complete training loop, including forward propagation, loss calculation, backpropagation, parameter updates, and periodic validation set evaluation. After training is complete, the training execution module outputs initial detection results, including: overall detection metrics such as overall mean precision, overall recall, and overall accuracy; category-level detection metrics such as mean precision, recall, and accuracy for each defect category; training process information such as training loss, validation loss, learning rate curve, and gradient norm for each training cycle; and resource consumption information such as total training time, peak memory usage, and CPU utilization. The intelligent experiment agent confirms the currently trained model structure, model weight file, training configuration, and corresponding evaluation metrics together as the current stable baseline.

[0043] In some feasible implementations, step S4 includes: the intelligent experimental agent calling the current stable baseline to make predictions on a fixed validation set, and matching the prediction results with the real annotations through the validation analysis module to obtain information on at least one type of erroneous sample, including missed detection samples, false detection samples, misclassified samples, and positioning deviation samples; the intelligent experimental agent calling the feature response extraction module to extract intermediate features from the intermediate feature layer of the current target detection model; and the feature response extraction module performing channel compression, normalization, and spatial alignment processing on the extracted intermediate features to generate a feature response map that can characterize the model's response to PCB defect areas.

[0044] Preferably, the workflow of the validation analysis module is as follows: The intelligent experiment agent calls the current stable baseline model to make predictions on a fixed validation set, obtaining the predicted bounding box, class confidence, and class label for each validation set image; the validation analysis module uses an intersection-over-union (IoU) based matching algorithm (usually setting the IoU threshold to 0.5) to match the prediction results with the ground truth labels of the validation set image by image; for each ground truth bounding box, if no predicted bounding box matches it (i.e., the maximum IoU is below the threshold), it is marked as a "missed sample," and the class of the sample is recorded. Information such as location and scale is recorded. For each predicted bounding box, if no ground truth bounding box matches it, it is marked as a "false positive" and its category, confidence level, and location are recorded. For a successfully matched predicted bounding box, if the predicted category is inconsistent with the ground truth category, it is marked as a "misclassified sample." For a successfully matched predicted bounding box with the same category, if the IoU between the predicted bounding box and the ground truth bounding box is between a preset low threshold and a high threshold (e.g., 0.5~0.7), it is marked as a "localization deviation sample," and the localization error value (such as center point offset or aspect ratio deviation) is recorded. In addition, "low-confidence samples" can also be recorded, which are samples where the confidence level of the predicted bounding box is lower than the preset threshold but the actual match is successful.

[0045] Preferably, the workflow of the feature response extraction module is as follows: The intelligent experimental agent selects one or more intermediate feature layers for extraction. Optional intermediate feature layers include the output layer of the backbone network, the fusion layer of the neck network, and multiple branch layers of the head. For each selected feature layer, the feature response extraction module extracts the corresponding feature map (a four-dimensional tensor with dimensions of batch, channel, height, and width) from the current model. The extraction module performs channel compression on each feature map: a 3×3 convolutional layer plus a batch normalization layer can be used to compress the multi-channel feature map into a single-channel response map, or channel compression can be achieved by taking the maximum or average absolute value of all channels. The compressed response map is normalized, adjusting its numerical range to between 0 and 1 for easy visualization and comparison. The normalized response map is spatially aligned: bilinear interpolation is used to scale the response map to the same size as the input image, allowing it to be directly superimposed on the original PCB image for display. The feature response map generated by the above processing can intuitively represent the model's response intensity to various defect areas in the PCB image. In the response heatmap, the darker the color of a region, the stronger the model's response to the features in that region, and vice versa.

[0046] In some feasible implementations, the multimodal diagnostic representation in step S5 includes labeled representation, error representation, feature representation, training representation, and experimental history representation; wherein, the labeled representation is derived from the prior representation of the PCB task, the error representation is derived from the error sample information, the feature representation is derived from the feature response information, the training representation is derived from the evaluation index in the current stable baseline result, and the experimental history representation is derived from the historical experimental information.

[0047] Preferably, the intelligent experiment agent invokes a pre-built multimodal representation construction module, taking the following information as input: PCB task prior representation, error sample information, feature response information, training process information, and historical experiment information stored in the experiment history. The multimodal representation construction module generates multimodal diagnostic representations through the following sub-steps: Data Alignment Unit: This unit aligns input data from different sources according to the primary key, based on the experiment number, model configuration, sample number, and defect category. For example, it correlates the statistics on "proportion of small targets" in the task prior with "missed small target samples" in the error samples and "response intensity of small target region" in the feature response map to ensure that all information corresponds to the same experimental state and sample set.

[0048] Numerical Feature Encoding Unit: This unit organizes information such as defect scale distribution, category distribution, proportion of small targets, category AP, false negative rate, localization error, training loss, and resource consumption into numerical diagnostic information.

[0049] Image feature processing unit: This unit processes missed detection samples, false detection samples, positioning deviation samples, and intermediate feature response maps of the model into image-based diagnostic information.

[0050] Textual history organization unit: This unit organizes the modified objects, experimental status (retained, discarded, anomalous) and experimental conclusions (such as "improved small target recall rate", "caused the deterioration of positioning bias", etc.) of historical candidate experiments into textual history information.

[0051] Diagnostic Label Generation Unit: This unit generates at least one diagnostic label for small target misses, class imbalance, class confusion, localization bias, insufficient feature response, training instability, overfitting risk, or excessive resource consumption. Specifically, a small target miss label is generated when the proportion of small targets is higher than a preset threshold (e.g., >50%) and the miss rate of the small target category is higher than a preset threshold (e.g., >30%); a class imbalance label is generated when the ratio of the number of samples in the largest class to the number of samples in the smallest class is higher than a preset threshold (e.g., >5); a class confusion label is generated when the proportion of mutual misclassification between two different classes in the confusion matrix is ​​higher than a preset threshold (e.g., >20%); a positioning deviation label is generated when the proportion of positioning deviation samples to all positive detection samples is higher than a preset threshold (e.g., >20%); a feature response insufficiency label is generated when the average response intensity of the defect region in the feature response map is lower than a preset proportion of the average response intensity of the background region (e.g., <80%); a training instability label is generated when the training loss curve oscillates violently, the gradient norm has an outlier value, or the loss value becomes non-numerical; an overfitting risk label is generated when the validation loss increases for several consecutive cycles while the training loss continues to decrease; and a resource consumption overload label is generated when GPU memory usage exceeds a preset threshold (e.g., >90%) or the training time exceeds a preset upper limit.

[0052] The representation encapsulation unit encapsulates numerical diagnostic information, image-based diagnostic information, text-based history information, and diagnostic tags into a unified multimodal diagnostic representation object. This facilitates subsequent retrieval and analysis by the experimental generation module.

[0053] In some feasible implementations, step S6 includes: The intelligent experiment agent generates retrieval query information based on the multimodal diagnostic representation and obtains candidate optimization strategies matching the current experimental state from the retrieval enhancement experiment strategy library. The retrieval query information includes the current PCB task prior, current error sample type, current feature response performance, current experimental stage, current stable baseline, and historical experimental state. The retrieval enhancement experiment strategy library includes a predefined experiment strategy library, a candidate module library, and a historical experiment experience library. The predefined experiment strategy library stores candidate adjustment methods corresponding to different diagnostic conditions. The candidate module library stores model structures, loss functions, sample allocation strategies, or training strategy templates that can be called. The historical experiment experience library stores the applicable conditions, experimental results, and retained conclusions of historical experiments. When the retrieval enhancement experiment strategy library returns multiple candidate optimization strategies, the intelligent experiment agent sorts them according to diagnostic matching degree, historical experiment benefits, reduction fit, resource consumption constraints, and abnormal risks, and determines a main optimization strategy from the top-ranked candidate optimization strategies to generate a candidate experiment with controlled main variables for this round. The remaining candidate optimization strategies serve as alternative strategies for subsequent experiments.

[0054] Preferably, the internal structure of the retrieval enhancement experiment strategy library includes three sub-libraries: A predefined experimental strategy library stores candidate adjustment methods corresponding to different diagnostic conditions. Each record includes a diagnostic label, the object to be modified (such as learning rate, backbone network, loss function, etc.), the modification method (such as increasing / decreasing, replacing with a certain module), the applicable experimental stage (baseline establishment, hyperparameter search, structure exploration), risk constraints (such as increasing GPU memory, decreasing inference speed), and an execution template (describing how to implement the modification).

[0055] Candidate module library: Stores descriptive information about model component code that can be dynamically invoked, including module name, input / output interface specifications, dependencies, default parameters, and registration method. This library does not store the actual code, but rather the module's metadata and how to obtain it (such as file path and class name).

[0056] Historical Experiment Experience Repository: Stores the applicable conditions, experimental results, and retained conclusions of historical experiments. Applicable conditions may include prior task features (such as the proportion of small objectives, class imbalance), diagnostic label combinations, etc. Experimental results include changes in evaluation metrics, resource consumption, etc. Retained conclusions record whether the experiment successfully updated the stable baseline.

[0057] The detailed steps of the retrieval process are as follows: The intelligent experiment agent extracts key information from the multimodal diagnostic representation as retrieval query conditions, including: a list of diagnostic labels (e.g., "small target missed detection", "insufficient feature response"), the name of the current experimental stage, key numerical features (e.g., small target proportion, class imbalance ratio, current stable baseline index), and historical experimental status (modifications and results of the most recent experiments); the intelligent experiment agent submits the retrieval query to the retrieval enhancement experiment strategy library, which contains three sub-libraries: a predefined experiment strategy library, a candidate module library, and a historical experiment experience library; the retrieval process first recalls all matching strategies in the predefined strategy library based on the diagnostic labels (e.g., "small target missed detection" may match "increase input resolution" and "add P2 detector head"), then filters out strategies that do not conform to the specification based on the current experimental stage (e.g., if modifying the model structure is prohibited in the hyperparameter search stage, then "add P2 detector head" is filtered out), finally obtaining a preliminary set of candidate strategies. If there are more than one candidate strategy, the intelligent experiment agent will perform a weighted ranking based on the following factors and select the optimal one or K strategies (K is usually 1 to ensure that the main variables are controlled): Diagnostic matching degree: the degree to which the diagnostic conditions of the strategy match the current diagnostic label (perfect match scores the highest); Historical experiment benefits: in the historical experiment experience base, find records with similar task characteristics and calculate the average index improvement brought by the strategy (positive values ​​for retained experiments and negative values ​​for discarded experiments); Stage adaptability: whether the applicable stage of the strategy is consistent with the current stage (perfect match scores higher than partial match); Resource consumption constraints: whether the additional resources (GPU memory, training time) required for the strategy implementation are within the range allowed by the declarative experiment specification; if they exceed the range, the priority is reduced or the strategy is directly excluded; Anomaly risk: the failure rate of the strategy in history (the proportion of training anomalies, evaluation crashes, etc.); the higher the failure rate, the lower the score; calculate the comprehensive score based on the above factors, and select the strategy with the highest score as the basis for this round of experiment. Based on the selected strategy and combined with the declarative experiment specification, the intelligent experiment agent generates a structured candidate experiment object. This object contains the following fields: **Main Modification Target:** The core modification point for this round of experiments, limited to one (e.g., loss function, input resolution, backbone network, etc.). **Necessary Supporting Adjustments:** Other parameters that must be modified simultaneously to achieve the main modification (e.g., adjusting anchor box size, batch size, etc. when increasing input resolution). **Trigger Basis:** Records the diagnostic labels or task features on which this experiment was based (e.g., "class imbalance ratio 10:1"). **Expected Goal:** A description of the problem to be improved (e.g., "reducing the false negative rate of 'spurs' and 'gap' classes"). **Execution Method:** Distinguishes between parametric experiments (modifying only hyperparameters) and structure / loss function experiments (modifying model code). **Validation Metrics:** The main metrics used to evaluate the experimental results (e.g., overall mean precision, key class recall).Retention conditions: Numerical thresholds that must be met to update experimental results to a new stable baseline (e.g., "mean precision increases by ≥1 percentage point" or "key category recall increases by ≥5 percentage points and overall mean precision decreases by ≤0.5 percentage points").

[0058] In some feasible implementations, step S7 includes: a non-intrusive experiment execution adapter converts candidate experiments into external configurations or controlled modification instructions executable by the training engine; when the candidate experiment is a parameter-based experiment, the non-intrusive experiment execution adapter is implemented through configuration overriding or command parameter injection; when the candidate experiment is a structure-based or loss function-based experiment, the non-intrusive experiment execution adapter is implemented through module switching, preset candidate module calls, or controlled code patching; the experiment history includes experiment number, experiment label, candidate modification content, training parameters, model configuration, overall detection index, category-level detection index, error sample changes, resource consumption, experiment status, and / or experiment conclusions; the experiment status includes at least one of baseline state, retained state, discarded state, and abnormal state.

[0059] Preferably, the adapter employs two different processing strategies depending on how the candidate experiment is executed: (1) Parameter-based experiments: These are suitable for situations where only hyperparameters or configuration parameters need to be modified, such as adjusting the learning rate, batch size, data augmentation parameters, optimizer type, etc. These experiments do not involve code changes to the model structure.

[0060] The adapter's specific operation flow is as follows: Locate the configuration file used by the training script (common formats include YAML, JSON, and Python dictionary files). Read the original configuration file content and parse it into an in-memory data structure. Modify the corresponding configuration items based on the main changes in the candidate experiments (e.g., changing "learning_rate" from 0.01 to 0.001). If necessary adjustments are required (e.g., adjusting the anchor box size after changing the input resolution), modify the relevant configuration items accordingly. Write the modified configuration to a new temporary file (or pass it directly via command-line arguments). Call the training command, specifying the use of the modified configuration or parameters. After training, decide based on the experimental results: if the experiment is retained, merge the temporary configuration into the official configuration; if it is discarded, delete the temporary file and restore the original configuration.

[0061] (2) Structure or loss function experiments: applicable to experiments that require modification of the model structure (such as replacing the backbone network, adding a detector head), loss function, sample allocation strategy, etc., involving code changes.

[0062] The adapter's specific operation flow is as follows: Backup Phase: Based on the execution template in the candidate experiments, determine which files might be modified (e.g., model definition files, training scripts, registration files). Store complete copies of these files in a temporary directory named after the experiment number. Module Location: Obtain the metadata of the required modules from the candidate module library retrieved from the augmentation experiment strategy library, including the module source file path, class name, registration method, dependencies, etc. Controlled Code Editing: The adapter does not arbitrarily modify files, but performs precise editing according to a preset template. Common editing operations include: adding an import statement in the import area of ​​the model definition file (e.g., "from modules.p2_head import P2DetectHead"). In the model construction function, add or replace code segments according to conditional branches (e.g., replace the original detection head instantiation code with the instantiation code of the new module). If the module needs to be registered (e.g., added to the model's registry), add a registration statement accordingly at the registration location. All editing uses precise replacement based on regular expressions or abstract syntax trees to avoid breaking other code. Execute Training: Invoke the training command, similar to parameter-based experiments. The adapter monitors the training logs and extracts loss values ​​and validation metrics in real time. Rollback or Persistence: If an anomaly occurs during training (such as module import failure, shape mismatch, or memory overflow), the adapter immediately terminates training, copies the backup files back to their original location, and restores the project to its state before modification. If training completes normally, the adapter waits for the evaluation results from the intelligent experiment agent (from S8). If the agent determines that the experiment meets the retention conditions, the adapter retains the modified files (deletes the backup); if it determines that the files should not be retained or that an anomaly has occurred, it performs a rollback operation to restore the original files.

[0063] Both of these methods ensure that the intelligent experiment agent itself does not directly manipulate files or write code; all potentially destructive modifications are encapsulated within the adapter, further improving the system's robustness.

[0064] In some feasible implementations, the preset retention conditions in step S8 include that the candidate experiment achieves a preset improvement in the main evaluation index relative to the current stable baseline, or achieves a preset improvement in the category-level index of the key defect category and the overall performance is not lower than the preset tolerance range; if the candidate experiment causes a decrease in the main evaluation index, an increase in the false negative rate, a model complexity exceeding a preset limit, an inference delay exceeding a preset limit, or training anomalies, then the candidate experiment will not be confirmed as a new stable baseline; the key defect category includes PCB defect categories with a sample number less than a preset sample number threshold, a false negative cost higher than a preset false negative cost threshold, or a historical detection effect lower than a preset detection effect threshold.

[0065] Preferably, if the retention conditions are met, the intelligent experiment agent confirms the model structure, model weight file, training configuration (including hyperparameters, data augmentation parameters, loss function configuration, etc.), and evaluation metrics corresponding to the candidate experiment as a new stable baseline. Simultaneously, the experimental history status of the candidate experiment is marked as "reserved," and the non-intrusive experiment execution adapter is notified to persist and apply temporary modifications made during the experiment (such as configuration file and code changes). The original stable baseline is archived for future reference. If the retention conditions are not met or an anomaly occurs, the intelligent experiment agent maintains the current stable baseline, marks the experimental history status of the candidate experiment as "discarded" (metrics not met) or "abnormal" (execution error), and notifies the non-intrusive experiment execution adapter to perform a rollback operation, i.e., restoring all modified files (such as model definition files and configuration files) to their pre-experiment state, deleting temporarily generated files, and ensuring a complete restoration of the training environment.

[0066] In some feasible implementations, the exit conditions in step S9 include at least one of the following: reaching the preset maximum number of experimental rounds; multiple consecutive rounds of candidate experiments not exceeding the current stable baseline; the false negative rate of the key defect category is lower than a preset threshold; the model accuracy and inference speed meet the deployment requirements; the experimental resource consumption exceeds the preset upper limit; the intelligent experimental agent checks the exit conditions at the end of each round of S9, and if any condition is met, the loop is terminated and the current stable baseline is output as the final optimization result.

[0067] In summary, this invention constructs a complete closed-loop optimization system of "diagnosis-retrieval-experimentation-evaluation-iteration" through the collaborative work of intelligent experimental agents, multimodal diagnostic representations, retrieval-enhanced experimental strategy libraries, and non-intrusive experimental execution adapters. This system enables fully automated, interpretable, and traceable optimization of PCB defect detection models, significantly reducing the cost of manual intervention and improving the efficiency and effectiveness of model optimization. It can be widely applied to the continuous iteration and rapid deployment of models in PCB production lines and various industrial visual inspection scenarios.

[0068] Example 2 Please see Figure 2 This embodiment provides a schematic diagram of an automatic optimization system structure for PCB defect detection model based on multimodal diagnostic characterization.

[0069] As an example, the system employs the automatic optimization method for PCB defect detection model based on multimodal diagnostic characterization described in Example 1, and the system includes: The intelligent experiment agent 1, and the annotation parsing module 2, training execution module 3, diagnostic information generation module 4, multimodal representation construction module 5 and automatic experiment execution module 6 respectively called by the intelligent experiment agent; in, The intelligent experiment agent 1 is used for: acquiring a PCB defect detection dataset, an initial target detection model, a declarative experiment specification, and a retrieval enhancement experiment strategy library; calling the annotation parsing module 2 to generate a PCB task prior representation; calling the training execution module 3 to obtain initial detection results and confirm the current stable baseline; calling the diagnostic information generation module 4 to generate error sample information and feature response information; inputting the PCB task prior representation, error sample information, feature response information, training process information, and historical experiment information into the multimodal representation construction module 5 to generate a multimodal diagnostic representation; using the multimodal diagnostic representation as retrieval query information to retrieve candidate optimization strategies in the retrieval enhancement experiment strategy library, and combining it with the declarative experiment specification to generate candidate experiments with controlled main variables for this round; calling the automatic experiment execution module 6 to execute the candidate experiments and update the experiment history; comparing the candidate experiment results with the current stable baseline, updating or restoring the stable baseline according to preset retention conditions; and determining whether to continue the automatic optimization loop according to the exit conditions in the declarative experiment specification. The annotation parsing module 2 is used to parse the annotation files of the PCB defect detection dataset, extract defect box coordinates, defect categories and image size information, and statistically analyze the scale distribution, aspect ratio distribution, proportion of small targets, category distribution, category imbalance degree and defect density information of defect targets to generate PCB task prior representation. The training execution module 3 is used to train the initial target detection model according to the prior representation of the PCB task and the declarative experimental specification to obtain the initial detection result; the initial detection result includes model evaluation index and training process information; The diagnostic information generation module 4 includes a verification analysis unit and a feature response extraction unit. The verification analysis unit is used to make predictions on a fixed verification set based on the current stable baseline and match them with the real annotations to obtain information on at least one type of erroneous sample, including missed detection samples, false detection samples, misclassified samples, and localization deviation samples. The feature response extraction unit is used to extract intermediate features from the intermediate feature layer of the current target detection model and perform channel compression, normalization, and spatial alignment processing to generate feature response information. The multimodal representation construction module 5 is used to construct a multimodal diagnostic representation that includes numerical diagnostic information, image-based diagnostic information, textual history information, and diagnostic labels, taking the PCB task prior representation, error sample information, feature response information, training process information, and historical experimental information stored in the experimental history as input. The automatic experiment execution module 6 includes a non-intrusive experiment execution adapter, which is used to convert the candidate experiment into an external configuration or controlled modification instruction that can be executed by the training engine, execute the candidate experiment, and feed back the experiment results to the intelligent experiment agent.

[0070] It is not difficult to see that this embodiment is a system implementation corresponding to the first embodiment, and this embodiment can be implemented in conjunction with the first embodiment. The relevant technical details mentioned in the first embodiment are still valid in this embodiment, and will not be repeated here to reduce repetition. Accordingly, the relevant technical details mentioned in this embodiment can also be applied to the first embodiment.

[0071] It is worth mentioning that all modules involved in this embodiment are logical units. In practical applications, a logical unit can be a physical unit, a part of a physical unit, or a combination of multiple physical units. Furthermore, to highlight the innovative aspects of this invention, this embodiment does not introduce units that are not closely related to solving the technical problem proposed by this invention; however, this does not mean that other units are absent from this embodiment.

[0072] Example 3 Please see Figure 3 The present invention also provides an electronic device, including: a memory and a processor; the memory stores at least one program instruction; the processor loads and executes the at least one program instruction to implement the automatic optimization method for PCB defect detection model based on multimodal diagnostic characterization provided in Embodiment 1.

[0073] The memory 702 and processor 701 are connected via a bus, which may include any number of interconnecting buses and bridges, connecting various circuits of one or more processors 701 and memory 702 together. The bus may also connect various other circuits, such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and therefore will not be described further herein. A bus interface provides an interface between the bus and the transceiver. The transceiver may be a single element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. Data processed by processor 701 is transmitted over a wireless medium via an antenna, which further receives data and transmits it to processor 701.

[0074] Processor 701 is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory 702 can be used to store data used by processor 701 during operation.

[0075] Example 4 This invention also proposes a storage medium storing an automatic optimization method for a PCB defect detection model based on multimodal diagnostic characterization. When executed by a processor, the automatic optimization program for the PCB defect detection model based on multimodal diagnostic characterization implements the steps of the automatic optimization method for the PCB defect detection model based on multimodal diagnostic characterization as described above. Since this storage medium employs all the technical solutions of the above embodiments, it possesses at least all the beneficial effects brought about by the technical solutions of the above embodiments, which will not be elaborated upon further here.

[0076] The above descriptions are merely embodiments of the present invention. Commonly known structures and characteristics are not described in detail here. Those skilled in the art are aware of all common technical knowledge in the field prior to the application date or priority date, are aware of all existing technologies in that field, and have the ability to apply conventional experimental methods prior to that date. Those skilled in the art can, based on the guidance provided in this application, improve and implement this solution in combination with their own capabilities. Some typical known structures or methods should not be obstacles for those skilled in the art to implement this application. It should be noted that those skilled in the art can make several modifications and improvements without departing from the structure of the present invention. These should also be considered within the scope of protection of the present invention, and will not affect the effectiveness of the implementation of the present invention or the practicality of the patent. The scope of protection claimed in this application should be determined by the content of its claims, and the specific embodiments described in the specification can be used to interpret the content of the claims.

Claims

1. An automatic optimization method for PCB defect detection models based on multimodal diagnostic characterization, characterized in that, The method is automatically scheduled and executed by the intelligent experiment agent, including: Step S1: The intelligent experiment agent acquires the PCB defect detection dataset, the initial target detection model, the declarative experiment specification, and the retrieval enhancement experiment strategy library; Step S2: The intelligent experimental agent calls the preset annotation parsing module to parse the annotation file of the PCB defect detection dataset and generate the PCB task prior representation. Step S3: The intelligent experiment agent, based on the prior representation of the PCB task and the declarative experimental specification, calls the pre-set training execution module to train the initial target detection model and obtain the initial detection result; the initial detection result includes model evaluation metrics and training process information; the intelligent experiment agent confirms the model configuration and evaluation metrics of the trained target detection model as the current stable baseline; Step S4: The intelligent experimental agent calls the preset verification analysis module and the preset feature response extraction module to generate model diagnostic information based on the current stable baseline; the model diagnostic information includes error sample information and feature response information. Step S5: The intelligent experimental agent calls the pre-set multimodal representation construction module, and constructs a multimodal diagnostic representation by taking the PCB task prior representation, error sample information, feature response information, training process information and historical experimental information stored in the experimental history as input. Step S6: The intelligent experiment agent uses the multimodal diagnostic representation as retrieval query information to retrieve candidate optimization strategies that match the current optimization stage in the retrieval enhancement experiment strategy library, and combines the declarative experiment specification to generate candidate experiments with controlled main variables for this round. Step S7: The intelligent experiment agent calls the preset non-intrusive experiment execution adapter to execute the candidate experiment with controlled main variables and update the experiment history; Step S8: The intelligent experiment agent compares the evaluation metrics of the candidate experiments in this round with the evaluation metrics of the current stable baseline. If the preset retention conditions are met, the model configuration and evaluation metrics corresponding to the candidate experiments are updated to the new stable baseline. If the preset retention conditions are not met, the system is restored to the previous stable baseline. Step S9: The intelligent experiment agent determines whether to continue executing the next round of S4 to S8 based on the exit condition in the declarative experiment specification.

2. The automatic optimization method for PCB defect detection model based on multimodal diagnostic characterization according to claim 1, characterized in that, The PCB defect detection dataset includes PCB images and their corresponding defect annotation files; the initial target detection model is the PCB defect detection model to be optimized; the declarative experimental specification is used to define the experimental stages, allowed modification objects, evaluation indicators, exit conditions, and anomaly handling rules in the automatic optimization process; the retrieval enhancement experimental strategy library is used to store predefined adjustment methods, candidate modules, experimental templates, applicable conditions, risk constraints, and historical experimental experience related to the optimization of the PCB defect detection model.

3. The automatic optimization method for PCB defect detection model based on multimodal diagnostic characterization according to claim 1, characterized in that, Step S2 includes: The intelligent experimental agent extracts the defect box coordinates, defect category and image size information from the PCB defect annotation file by calling the preset annotation parsing module, and statistically analyzes the scale distribution, aspect ratio distribution, small target proportion, category distribution, category imbalance degree and defect density information of the defect target, thereby forming a PCB task prior characterization to describe the characteristics of the current PCB defect detection task. The PCB task prior representation is used to constrain the search space and experiment priority of candidate experiments, including: when the proportion of small targets in the PCB task prior representation is higher than a preset small target proportion threshold, the intelligent experiment agent increases the priority of experiments related to input resolution, shallow feature enhancement, small target detection head, or positive sample allocation strategy; when the class imbalance is higher than a preset imbalance threshold, the intelligent experiment agent increases the priority of experiments related to class weight, loss function, or sample allocation strategy.

4. The automatic optimization method for PCB defect detection model based on multimodal diagnostic characterization according to claim 1, characterized in that, Step S4 includes: The intelligent experimental agent calls the current stable baseline to make predictions on a fixed validation set, and matches the prediction results with the real annotations through the validation analysis module to obtain information on at least one type of erroneous sample, including missed detection samples, false detection samples, misclassified samples, and localization deviation samples. The intelligent experimental agent calls the feature response extraction module to extract intermediate features from the intermediate feature layer of the current target detection model; the feature response extraction module performs channel compression, normalization and spatial alignment processing on the extracted intermediate features to generate a feature response map that can characterize the model's response to PCB defect areas.

5. The automatic optimization method for PCB defect detection model based on multimodal diagnostic characterization according to claim 4, characterized in that, The multimodal diagnostic representation in step S5 includes labeled representation, error representation, feature representation, training representation, and experimental history representation; wherein, the labeled representation is derived from the prior representation of the PCB task, the error representation is derived from the error sample information, the feature representation is derived from the feature response information, the training representation is derived from the evaluation index in the current stable baseline result, and the experimental history representation is derived from the historical experimental information.

6. The automatic optimization method for PCB defect detection model based on multimodal diagnostic characterization according to claim 5, characterized in that, Step S6 includes: The intelligent experimental agent generates retrieval query information based on the multimodal diagnostic representation and obtains candidate optimization strategies matching the current experimental state from the retrieval enhancement experimental strategy library. The retrieval query information includes the current PCB task prior, current error sample type, current feature response performance, current experimental stage, current stable baseline, and historical experimental state. The retrieval enhancement experimental strategy library includes a predefined experimental strategy library, a candidate module library, and a historical experimental experience library. The predefined experimental strategy library stores candidate adjustment methods corresponding to different diagnostic conditions. The candidate module library stores model structures, loss functions, sample allocation strategies, or training strategy templates that can be called. The historical experimental experience library stores the applicable conditions, experimental results, and retained conclusions of historical experiments. When the search enhancement experiment strategy library returns multiple candidate optimization strategies, the intelligent experiment agent sorts them according to diagnostic matching degree, historical experiment benefits, specification fit degree, resource consumption constraints, and abnormal risks, and determines a primary optimization strategy from the top-ranked candidate optimization strategies to generate a candidate experiment with controlled main variables in this round; the remaining candidate optimization strategies serve as alternative strategies for subsequent experiments.

7. The automatic optimization method for PCB defect detection model based on multimodal diagnostic characterization according to claim 1, characterized in that, Step S7 includes: a non-intrusive experiment execution adapter converts candidate experiments into external configurations or controlled modification instructions executable by the training engine; when the candidate experiment is a parameter-based experiment, the non-intrusive experiment execution adapter is implemented through configuration overriding or command parameter injection; when the candidate experiment is a structure-based or loss function-based experiment, the non-intrusive experiment execution adapter is implemented through module switching, preset candidate module invocation, or controlled code patching. The experimental history includes the experimental number, experimental label, candidate modifications, training parameters, model configuration, overall detection metrics, category-level detection metrics, changes in error samples, resource consumption, experimental status, and / or experimental conclusions. The experimental states include at least one of the following: baseline state, retained state, discarded state, and abnormal state.

8. The automatic optimization method for PCB defect detection model based on multimodal diagnostic characterization according to claim 1, characterized in that, The preset retention conditions in step S8 include that the candidate experiment achieves a preset improvement in the main evaluation index relative to the current stable baseline, or achieves a preset improvement in the category-level index of the key defect category and the overall performance is not lower than the preset tolerance range; if the candidate experiment causes the main evaluation index to drop, the false negative rate to increase, the model complexity to exceed the preset limit, the inference delay to exceed the preset limit, or the training to be abnormal, then the candidate experiment will not be confirmed as a new stable baseline. The key defect categories include PCB defect categories where the number of samples is less than a preset sample number threshold, the cost of missed detection is higher than a preset missed detection cost threshold, or the historical detection effect is lower than a preset detection effect threshold.

9. The automatic optimization method for PCB defect detection model based on multimodal diagnostic characterization according to claim 8, characterized in that, The exit condition in step S9 includes at least one of the following: reaching the preset maximum number of experimental rounds; multiple consecutive rounds of candidate experiments have not exceeded the current stable baseline; The false negative rate for key defect categories is lower than the preset threshold; the model accuracy and inference speed meet deployment requirements; and the experimental resource consumption exceeds the preset limit. The intelligent experiment agent checks the exit condition at the end of each round S9. If any condition is met, the loop terminates and the current stable baseline is output as the final optimization result.

10. An automatic optimization system for PCB defect detection models based on multimodal diagnostic characterization, characterized in that, The system is implemented using the automatic optimization method for PCB defect detection model based on multimodal diagnostic characterization as described in any one of claims 1-9, including: The intelligent experiment agent, and the annotation parsing module, training execution module, diagnostic information generation module, multimodal representation construction module and automatic experiment execution module respectively invoked by the intelligent experiment agent; in, The intelligent experiment agent is used for: acquiring a PCB defect detection dataset, an initial target detection model, a declarative experiment specification, and a retrieval enhancement experiment strategy library; calling the annotation parsing module to generate a priori representation of the PCB task; calling the training execution module to obtain initial detection results and confirm the current stable baseline; calling the diagnostic information generation module to generate error sample information and feature response information; inputting the PCB task priori representation, error sample information, feature response information, training process information, and historical experiment information into the multimodal representation construction module to generate a multimodal diagnostic representation; using the multimodal diagnostic representation as retrieval query information to retrieve candidate optimization strategies in the retrieval enhancement experiment strategy library, and combining it with the declarative experiment specification to generate candidate experiments with controlled main variables for this round; calling the automatic experiment execution module to execute the candidate experiments and update the experiment history; comparing the candidate experiment results with the current stable baseline, updating or restoring the stable baseline according to preset retention conditions; and determining whether to continue the automatic optimization loop according to the exit conditions in the declarative experiment specification. The annotation parsing module is used to parse the annotation files of the PCB defect detection dataset, extract defect box coordinates, defect categories and image size information, and statistically analyze the scale distribution, aspect ratio distribution, proportion of small targets, category distribution, category imbalance degree and defect density information of defect targets to generate PCB task prior representation. The training execution module is used to train the initial target detection model based on the prior representation of the PCB task and the declarative experimental specification to obtain the initial detection results; the initial detection results include model evaluation metrics and training process information; The diagnostic information generation module includes a verification analysis unit and a feature response extraction unit. The verification analysis unit is used to make predictions on a fixed validation set based on the current stable baseline and match them with the real annotations to obtain information on at least one type of erroneous sample, including missed detection samples, false detection samples, misclassified samples, and localization bias samples. The feature response extraction unit is used to extract intermediate features from the intermediate feature layer of the current target detection model and perform channel compression, normalization, and spatial alignment to generate feature response information. The multimodal representation construction module is used to construct a multimodal diagnostic representation that includes numerical diagnostic information, image-based diagnostic information, textual history information, and diagnostic labels, taking the PCB task prior representation, error sample information, feature response information, training process information, and historical experimental information stored in the experimental history as input. The automated experiment execution module includes a non-intrusive experiment execution adapter, which is used to convert the candidate experiment into an external configuration or controlled modification instruction that can be executed by the training engine, execute the candidate experiment, and feed back the experiment results to the intelligent experiment agent.

Images