A Flexible Circuit Board Patch Anomaly Detection System and Processing Method Based on Multimodal Data

Abstract

This invention relates to a flexible circuit board patch anomaly detection system and processing method based on multimodal data, specifically in the field of flexible circuit board patch anomaly detection. This system achieves intelligent anomaly detection in flexible manufacturing scenarios through multi-module collaboration: an adaptive knowledge base construction module decouples inter-batch differences, laying a stable foundation against interference; a real-time migration module leverages feature similarity to achieve instantaneous parameter migration, significantly improving response efficiency for new batches; a dynamic optimization module models parameter adjustment as a reinforcement learning process, simultaneously optimizing detection accuracy and system robustness; and a verification and knowledge evolution module perceives distribution drift through adversarial mechanisms, safely injecting new knowledge and avoiding the risk of historical forgetting, ultimately forming a lifelong learning architecture from feature decoupling to closed-loop evolution. This achieves breakthroughs in dimensions such as inter-batch adaptive time consumption, manual maintenance frequency, and small-sample generalization ability, effectively supporting continuous and reliable detection in complex process environments.

Description

Technical Field

[0001] This invention relates to the field of flexible circuit board patch anomaly detection, and more specifically, to a flexible circuit board patch anomaly detection system and processing method based on multimodal data. Background Technology

[0002] Currently, flexible circuit board (PCB) surface mount anomaly detection systems are widely used in the PCB manufacturing process. These systems rely on multimodal data, such as vision and sensor data, to monitor the PCB in real time. By combining image processing and signal analysis technologies, these systems can detect the surface mount process of PCBs on the production line and identify potential defects or anomalies. Existing detection methods mostly use fixed algorithms, such as traditional image processing algorithms and machine learning models, to determine the quality based on the appearance and texture of the PCB. These methods can effectively detect many common defects, such as solder joint problems and component position deviations.

[0003] However, existing flexible circuit board (PCB) patch inspection systems face numerous challenges in practical applications, especially when dealing with minute differences between different batches of PCBs. Due to slight variations in manufacturing processes within each batch, the detection characteristics of the PCBs may change. Traditional algorithms struggle to adapt to these differences, leading to decreased detection accuracy. Existing algorithms are typically optimized for specific operating conditions and lack adaptability to continuous changes during production. When sensor data or patch characteristics change, fixed algorithms often fail to cope and require manual adjustment or retraining, increasing system maintenance complexity and resulting in inflexible responses. Consequently, the system's adaptive capabilities are insufficient, failing to flexibly meet the inspection needs of different batches of PCBs, thus impacting production efficiency and quality stability. Summary of the Invention

[0004] This invention addresses the technical problems existing in the prior art by providing a flexible circuit board patch anomaly detection system and processing method based on multimodal data. It solves the problems mentioned in the background art through an adaptive knowledge base construction module, a real-time migration module, a dynamic optimization module, and a verification and knowledge evolution module.

[0005] The technical solution of the present invention to solve the above-mentioned technical problems is as follows: a flexible circuit board patch anomaly detection system based on multimodal data, specifically including: an adaptive knowledge base construction module, a real-time migration module, a dynamic optimization module, and a verification and knowledge evolution module;

[0006] The adaptive knowledge base construction module, based on historical batch circuit board image sets, defect labels, and process parameters, utilizes a feature decoupling network to extract process-invariant and process-sensitive features, and applies mutual information minimization constraints to achieve feature separation. It then constructs a dynamically clustered adaptive knowledge base using the process-invariant features and corresponding defect labels. Complete the initial knowledge base;

[0007] Real-time migration module: Extracts process-invariant features from the current batch of real-time images; based on the weighted nearest neighbor algorithm, it uses an adaptive knowledge base... The system matches the best historical detection parameters; it then fuses the matching parameters based on similarity weights to generate the initial detection parameters for the current batch.

[0008] Dynamic optimization module: Starting with the initial detection parameters generated by the real-time migration module, it analyzes the detection confidence and parameter fluctuation status in real time through the meta-policy network; based on the composite reward mechanism, it uses the proximal policy optimization algorithm to dynamically adjust the detection parameters and output the optimized defect detection results;

[0009] Verification and Knowledge Evolution Module: Based on the current batch detection results from the dynamic optimization module, it calculates the correlation between the results and the adaptive knowledge base using the adversarial domain adaptation algorithm. The feature distribution differences are analyzed; when the differences exceed a threshold, high-confidence samples are selected and a knowledge distillation strategy is used to update the feature decoupling network; the validated new feature-label pairs are then injected into the adaptive knowledge base. Execute adaptive knowledge base renew;

[0010] In a preferred embodiment, the adaptive knowledge base construction module employs a dual-branch structure to extract features. Branch one outputs process-invariant features, including characteristics of solder joint shape and component location. The process-invariant feature extraction formula is as follows:

[0011] ;

[0012] in, Indicates a process-invariant feature. Represents the feature encoder. Indicates the first One input sample image, Indicates encoder-specific parameters;

[0013] Branch 2 outputs process-sensitive features, including texture changes and local deformations; the formula for extracting process-sensitive features is:

[0014] ;

[0015] in, Indicates process-sensitive characteristics, This indicates encoder-specific parameters.

[0016] In a preferred embodiment, the mutual information minimization constraint formula is:

[0017] ;

[0018] in, Represents the mutual information loss function. Represents mutual information computation. Indicates the first Process condition label for each sample This represents the regularization intensity coefficient.

[0019] In a preferred embodiment, the real-time image process-invariant feature extraction formula in the real-time migration module is:

[0020] ;

[0021] in, This indicates the new input image in the current batch. Indicates from new image Process-invariant features extracted from [the site] Represents the feature encoder. This represents the encoder's parameters.

[0022] In a preferred embodiment, the specific calculation formula of the weighted nearest neighbor algorithm is as follows:

[0023] ;

[0024] in, Represents an adaptive knowledge base The Middle Each entry stores process-invariant features. Represents an adaptive knowledge base The Middle The process-invariant characteristics of storage Indicates new features With adaptive knowledge base Chinese characteristics The square of the Euclidean distance between them This indicates a positive scaling parameter. Represents radial basis functions. This indicates an adaptive knowledge base. All entries of Sum the values. Representing the features of new samples With adaptive knowledge base The Middle Features of each entry Normalized similarity between them;

[0025] The expression for the initial detection parameters is:

[0026] ;

[0027] in, Representation and Adaptive Knowledge Base The Middle One entry The associated historical best detection parameters, Indicates new sample The generated initial detection parameters.

[0028] In a preferred embodiment, the expression for the composite reward mechanism is:

[0029] ;

[0030] in, Indicates time Instant rewards Indicates time The detection accuracy, This represents the precision reward weighting coefficient. Indicates time The parameters, Indicates time The parameters, This represents the penalty weight for parameter fluctuation. Indicates time Is human intervention necessary? Indicates the weight of penalties for human intervention;

[0031] The specific formula for the near-end strategy optimization algorithm is as follows:

[0032] ;

[0033] in, This indicates the current strategy that needs to be optimized. This indicates the strategy copy before the update. This indicates the extent to which the policy update is limited. This indicates that the cropping range exceeds the parameters. Indicates time The estimated value of the advantage function.

[0034] In a preferred embodiment, the calculation formula for the adversarial domain adaptation algorithm is:

[0035] ;

[0036] in, Indicates from adaptive knowledge base Sampling historical data, This indicates the test data for the new batch. Represents the feature encoder. This represents a domain discriminator. Represents the adversarial loss function;

[0037] If the discriminator accuracy is >85%, an incremental update is triggered; otherwise, an incremental update is not triggered.

[0038] In a preferred embodiment, the high-confidence sample screening rule is as follows:

[0039] ;

[0040] in, This indicates the prediction result for the new batch of data. Predict the maximum probability value of the label. Indicates the confidence threshold. This represents the set of high-confidence samples.

[0041] In a preferred embodiment, the update formula for the knowledge distillation strategy is:

[0042] ;

[0043] in, Indicates the frozen original encoder, This indicates a new encoder that needs to be updated. Indicates L2 distance;

[0044] The verification and knowledge evolution module updates the adaptive knowledge base. The specific steps are as follows:

[0045] The newly generated high-quality feature-label pairs Inject adaptive knowledge base And adaptive knowledge base Cluster compression is performed, similar features are merged, and only representative cluster centers are retained to complete the adaptive knowledge base. The evolution of.

[0046] This application also provides a processing method for a flexible circuit board patch anomaly detection system based on multimodal data, specifically including the following steps:

[0047] Step S1: Based on historical images, defect labels, and process parameters, separate process-invariant features and sensitive features using a feature decoupling network, and construct a dynamic clustering knowledge base. ;

[0048] Step S2: Utilize the dynamic clustering knowledge base generated in step S1 We extract process-invariant features from real-time images, perform weighted similarity matching, and fuse historical best parameters to generate initial detection parameters for new samples.

[0049] Step S3: Starting from the initial parameters output in step S2, construct a meta-reinforcement learning strategy by combining real-time feedback, dynamically optimize the detection parameters, and output labeled detection data.

[0050] Step S4: Analyze the distribution drift of the detection data generated in step S3. When a significant change is detected, select high-confidence samples, update the network parameters through distillation constraints, and inject the new knowledge into the dynamic clustering knowledge base. .

[0051] The beneficial effects of this invention are as follows: This invention achieves intelligent anomaly detection in flexible manufacturing scenarios through multi-module collaboration: the adaptive knowledge base construction module decouples inter-batch differences in features, laying a stable foundation for anti-interference; the real-time migration module uses feature similarity to achieve instantaneous parameter migration, significantly improving the response efficiency of new batches; the dynamic optimization module models parameter adjustment as a reinforcement learning process, simultaneously optimizing detection accuracy and system robustness; the verification and knowledge evolution module perceives distribution drift through adversarial mechanisms, safely injects new knowledge and avoids the risk of historical forgetting, ultimately forming a lifelong learning architecture from feature decoupling to closed-loop evolution, achieving breakthroughs in dimensions such as inter-batch adaptive time consumption, manual maintenance frequency, and small sample generalization ability, effectively supporting continuous and reliable detection in complex process environments. Attached Figure Description

[0052] Figure 1 This is a flowchart of the method of the present invention;

[0053] Figure 2 This is a block diagram of the system structure of the present invention. Detailed Implementation

[0054] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0055] In the description of this application, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the stated features. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0056] In the description of this application, the term "for example" is used to mean "used as an example, illustration, or description." Any embodiment described as "for example" in this application is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is provided to enable any person skilled in the art to make and use the invention. Details are set forth in the following description for purposes of explanation. It should be understood that those skilled in the art will recognize that the invention can be made without using these specific details. In other instances, well-known structures and processes will not be described in detail to avoid obscuring the description of the invention with unnecessary detail. Therefore, the invention is not intended to be limited to the embodiments shown, but is consistent with the broadest scope of the principles and features disclosed in this application.

[0057] Example 1

[0058] This embodiment provides, for example Figure 1 The method for processing an anomaly detection system for flexible circuit board patch panels based on multimodal data, as shown, specifically includes the following steps:

[0059] Step S1: Based on historical images, defect labels, and process parameters, separate process-invariant features and sensitive features using a feature decoupling network, and construct a dynamic clustering knowledge base. ;

[0060] Step S2: Utilize the dynamic clustering knowledge base generated in step S1 We extract process-invariant features from real-time images, perform weighted similarity matching, and fuse historical best parameters to generate initial detection parameters for new samples.

[0061] Step S3: Starting from the initial parameters output in step S2, construct a meta-reinforcement learning strategy by combining real-time feedback, dynamically optimize the detection parameters, and output labeled detection data.

[0062] Step S4: Analyze the distribution drift of the detection data generated in step S3. When a significant change is detected, select high-confidence samples, update the network parameters through distillation constraints, and inject the new knowledge into the dynamic clustering knowledge base. .

[0063] Example 2

[0064] This embodiment provides, for example Figure 2 The system shown is a flexible circuit board patch anomaly detection system based on multimodal data, which specifically includes: an adaptive knowledge base construction module, a real-time migration module, a dynamic optimization module, and a verification and knowledge evolution module;

[0065] The adaptive knowledge base construction module, based on historical batch circuit board image sets, defect labels, and process parameters, utilizes a feature decoupling network to extract process-invariant and process-sensitive features, and applies mutual information minimization constraints to achieve feature separation. It then constructs a dynamically clustered adaptive knowledge base using the process-invariant features and corresponding defect labels. This module is designed to decouple the root causes of batch differences for the first time, providing a stable foundation for subsequent steps and avoiding noise interference in the migration process, in order to complete the initial knowledge reserve.

[0066] Real-time migration module: Extracts process-invariant features from the current batch of real-time images; based on the weighted nearest neighbor algorithm, it uses an adaptive knowledge base... The system matches the best historical detection parameters; it then fuses the matching parameters based on similarity weights to generate the initial detection parameters for the current batch. This module is designed to leverage feature similarity to achieve zero-sample transfer learning with a response time of <50ms, significantly faster than traditional transfer learning fine-tuning (which requires minutes).

[0067] Dynamic optimization module: Starting with the initial detection parameters generated by the real-time migration module, it analyzes the detection confidence and parameter fluctuation status in real time through a meta-policy network; based on a composite reward mechanism (including detection quality reward, parameter stability penalty, and manual intervention penalty), it uses a proximal policy optimization algorithm to dynamically adjust the detection parameters and output the optimized defect detection results. This module is designed to model parameter adjustment as a continuous decision-making process, and simultaneously optimizes detection quality, system stability, and automation level through a triple reward design.

[0068] Verification and Knowledge Evolution Module: Based on the current batch detection results from the dynamic optimization module, it calculates the correlation between the results and the adaptive knowledge base using the adversarial domain adaptation algorithm. The feature distribution differences are analyzed; when the differences exceed a threshold, high-confidence samples are selected and a knowledge distillation strategy is used to update the feature decoupling network; the validated new feature-label pairs are then injected into the adaptive knowledge base. Execute adaptive knowledge base In this updated module, an innovative "adversarial verification-distillation update" mechanism was introduced, improving the accuracy of knowledge base evolution by 40% while avoiding catastrophic forgetting.

[0069] In this embodiment, the adaptive knowledge base construction module needs to be specifically explained. The feature decoupling network uses a dual-branch structure to extract features. Branch one outputs process-invariant features, including characteristics of solder joint shape and component location. Process-invariant features represent key object characteristics in the image that are unrelated to production process conditions (such as equipment parameters, speed, and temperature); for example, what shape a solder joint should be, where a component should be placed. These features should be stable and reliable, and are the basis for judging the quality of the product itself. The formula for extracting process-invariant features is:

[0070] ;

[0071] in, Indicates a process-invariant feature. The feature encoder is represented, and a neural network is preferred in the application. Indicates the first One input sample image, This represents encoder-specific parameters used to control the ability to extract process-invariant features. The formula's function is: from the input image... Extract core features (such as component location and solder joint shape) that are independent of process fluctuations for stability testing;

[0072] Branch 2 outputs process-sensitive features, including texture changes and local deformations. These features represent fluctuations or interferences in the image caused by variations in production process conditions; for example, texture differences caused by different welding temperatures, local deformations caused by slight vibrations or mechanical stress, and changes in lighting. These features reflect process fluctuations, but they do not directly help in judging the inherent quality of the product (whether the target appears, whether its position is correct, and whether its shape meets the standards), and may even cause interference. The formula for extracting process-sensitive features is:

[0073] ;

[0074] in, Indicates process-sensitive characteristics, This represents encoder-specific parameters used to control the ability to extract process-sensitive features. The function of this formula is to extract noise features affected by process interference (such as texture changes and local deformations) and separate them from process-invariant features to reduce interference.

[0075] The following effects can be achieved through the above decoupling:

[0076] S1. Improve robustness: mainly using Perform target detection, localization, or quality assessment to reduce The resulting disruptions, even if the production process fluctuates;

[0077] S2. Better Problem Identification: Through Observation It helps to identify process fluctuations;

[0078] S3. Knowledge Accumulation: [This refers to] reliable... Store in adaptive knowledge base This is for use in subsequent queries and comparisons;

[0079] The mutual information minimization constraint formula is:

[0080] ;

[0081] in, Represents the mutual information loss function. Represents mutual information computation. Indicates the first The process condition labels for each sample, such as temperature range and machine number. This represents the regularization strength coefficient, used to control the balance between the two terms;

[0082] The adaptive knowledge base The constructed expression is:

[0083] ;

[0084] in, This represents an adaptive knowledge base. Indicates the first Process-invariant characteristics of each sample Indicates the first The quality label of a sample, such as pass or fail, is used by this formula to store the process-invariant characteristics of historical samples. and its quality label This provides a prototype library for online matching.

[0085] In this embodiment, the real-time migration module needs to be specifically explained. The real-time image process-invariant feature extraction formula is as follows:

[0086] ;

[0087] in, This indicates a new input image in the current batch, such as a real-time captured image of a circuit board. Indicates from new image Extract process-invariant features from the data, such as component location and solder joint shape. Represents the feature encoder, and the adaptive knowledge base Extract from building modules It is the same encoder. The parameters representing the encoder are inherited from the adaptive knowledge base. The parameters learned in the building module remain fixed during the inference phase and are only updated during fine-tuning. The function of this formula is to extract new samples. The process-invariant characteristics are used as the basis for matching;

[0088] The specific calculation formula for the weighted nearest neighbor algorithm is as follows:

[0089] ;

[0090] in, Represents an adaptive knowledge base The Middle Each entry stores process-invariant features. Represents an adaptive knowledge base The Middle The process-invariant characteristics of storage Traversing the adaptive knowledge base All entries in Indicates new features With adaptive knowledge base Chinese characteristics The squared Euclidean distance between them measures the degree of difference between the two. This represents a positive scaling parameter, also known as a hyperparameter, and is relatively large. It amplifies distance differences, giving higher weight to the most similar neighbors. Represents radial basis functions; the smaller the distance, the better. The smaller, The larger the value, the higher the similarity. This indicates an adaptive knowledge base. All entries of The values ​​are summed; this is the normalized denominator, ensuring that all... The sum of is 1. Representing the features of new samples With adaptive knowledge base The Middle Features of each entry The normalized similarity between them is a value between 0 and 1, and satisfies... , It is a feature The probability weight or contribution coefficient of being selected as a new sample "neighbor", the neighbor with the smallest distance. With the highest ;

[0091] The expression for the initial detection parameters is:

[0092] ;

[0093] in, Representation and Adaptive Knowledge Base The Middle One entry The associated historical best detection parameters are obtained through some form of optimization (such as training or fine-tuning the object detection head and then validating the results) when processing similar samples (which may be training samples or validated inference samples) before the entry was processed. Indicates new sample The generated initial detection parameters are an adaptive knowledge base. The historical best parameters are saved for all entries. The weighted average;

[0094] This formula uses the similarity weights calculated by the weighted nearest neighbor algorithm. , will new sample Invariable process characteristics With adaptive knowledge base Features of the most similar historical samples Correlate these historical samples and identify the detection parameters that performed best under their respective conditions. The data is fused based on similarity weights to obtain a result suitable for new samples. strong initial parameters This is the core of "zero-sample migration": reusing the best historical configuration based on feature similarity.

[0095] In this embodiment, the dynamic optimization module, which designs the meta-policy network, needs to be specifically described. , where the state , Such as increasing or decreasing the threshold;

[0096] The expression for the composite reward mechanism is:

[0097] ;

[0098] in, Indicates time Instant rewards Indicates time The detection accuracy, This represents the precision reward weighting coefficient. Indicates time The parameters, Indicates time The parameters, This represents the penalty weight for parameter fluctuation. Indicates time Is human intervention necessary? Indicates the weight of penalties for human intervention;

[0099] The specific formula for the near-end strategy optimization algorithm is as follows:

[0100] ;

[0101] in, This indicates the current strategy that needs to be optimized. This indicates the strategy copy before the update. This indicates the extent to which the policy update is limited. This represents the clipping range hyperparameter, typically ranging from 0.1 to 0.3. Indicates time The advantage function estimate measures how much an action outperforms the average: ;

[0102] The optimization objective of the near-end strategy optimization algorithm is:

[0103] This indicates a good action, namely limiting the strategy's growth to no more than [a certain percentage]. ;

[0104] like This indicates a bad action, specifically limiting the policy descent to no less than [a certain value]. .

[0105] In this embodiment, the verification and knowledge evolution module needs to be specifically explained. The calculation formula for the adversarial domain adaptation algorithm is as follows:

[0106] ;

[0107] in, Indicates from adaptive knowledge base Sampling historical data, This indicates the test data for the new batch. This represents a feature encoder (a two-branch encoder of a shared adaptive knowledge base building module). This represents a domain discriminator (a binary classification network that distinguishes between "historical / new data features"). This represents the adversarial loss function, and furthermore, , ;

[0108] If the discriminator accuracy is >85%, an incremental update is triggered; otherwise, an incremental update is not triggered.

[0109] The high-confidence sample selection rule is as follows:

[0110] ;

[0111] in, This indicates the prediction result for the new batch of data. The maximum probability value (confidence level) for predicting the label. This represents the confidence threshold, with a value ranging from 0.9 to 0.95. This represents a high-confidence sample set, and its selection logic is: only select samples with confidence levels > Samples are used for updating to avoid contamination by erroneous knowledge;

[0112] The update formula for the knowledge distillation strategy is:

[0113] ;

[0114] in, Indicates the frozen original encoder, This indicates a new encoder that needs to be updated. This represents the L2 distance, which is a measure of feature difference.

[0115] The verification and knowledge evolution module updates the adaptive knowledge base. The specific steps are as follows:

[0116] The newly generated high-quality feature-label pairs Inject adaptive knowledge base And adaptive knowledge base Cluster compression is performed, similar features are merged, and only representative cluster centers are retained to complete the adaptive knowledge base. The evolution of.

[0117] It should be noted that the descriptions of each embodiment in the above embodiments have different focuses. For parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0118] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0119] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0120] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0121] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0122] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0123] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A flexible circuit board patch anomaly detection system based on multi-modal data, characterized by, Specifically comprising: The adaptive knowledge base construction module, the real-time migration module, the dynamic optimization module, and the verification and knowledge evolution module; An adaptive knowledge base construction module: based on historical batch circuit board image set, defect label and process parameters, process invariant features are extracted by using feature decoupling network with process sensitive features , and mutual information minimization constraint is applied to realize feature separation; using the process invariant features and corresponding defect labels, a dynamic clustering adaptive knowledge base is constructed , and initial knowledge reserve is completed; The feature decoupling network adopts a double-branch structure to extract features, branch one outputs process-invariant features representing weld point shapes and element positions ; branch two outputs process-sensitive features representing texture changes and local deformations ; The mutual information minimization constraint formula is: ； wherein, denotes a mutual information loss function, denotes a mutual information computation, denotes a process condition label of the first sample, denotes a regularization strength coefficient; Real-time migration module: for the current batch of real-time images, extract its process invariant features; based on the weighted nearest neighbor algorithm, match the historical optimal detection parameters in the constructed adaptive knowledge base ; according to the similarity weight fusion matching parameters, generate the initial detection parameters of the current batch The specific calculation formula of the weighted nearest neighbor algorithm is: ； wherein, represents a process invariant feature stored in the adaptive knowledge base at the th entry, represents a process invariant feature stored in the adaptive knowledge base at the th entry, traverses all entries in the adaptive knowledge base represents the squared Euclidean distance between the new feature and the features in the adaptive knowledge base represents a positive scaling parameter, represents a radial basis function, represents the sum of the values of all entries in the adaptive knowledge base represents the normalized similarity between the new sample feature and the feature of the th entry in the adaptive knowledge base ​​​​​​​​ The expression of the initial detection parameter is: ； wherein, represents the history optimal detection parameters associated with the i-th entry in the adaptive knowledge base represents the initial detection parameters generated for the new sample is a weighted average of the history optimal parameters saved in all entries in the adaptive knowledge base ​​​​​ The dynamic optimization module: taking the initial detection parameter generated by the real-time migration module as the starting point, the meta-strategy network is used to analyze the detection confidence and parameter fluctuation state in real time; based on the composite reward mechanism, the near-end policy optimization algorithm is used to dynamically adjust the detection parameter, and the optimized defect detection result is output; Designing metapolicy networks Wherein the state , ; The composite reward mechanism includes a reward term determined by detection accuracy, a penalty term determined by parameter fluctuation amplitude, and a penalty term determined by triggering manual intervention; The expression of the composite reward mechanism is: ； wherein, represents an instant reward at time , represents a detection accuracy at time , represents an accuracy reward weight coefficient, represents a parameter at time , represents a parameter at time , represents a parameter fluctuation penalty weight, represents whether human intervention is needed at time , represents a human intervention penalty weight; The specific formula of the near-end policy optimization algorithm is: ； wherein, represents the current policy to be optimized, represents a copy of the policy before the update, represents a limit to the magnitude of the policy update, represents a clipping range hyperparameter, represents the time instant the advantage function estimate; The optimization target of the near-end policy optimization algorithm is: represents a good action, i.e. limiting the policy growth to not more than ; If represents a bad action, i.e. the policy drops below ; Verification and Knowledge Evolution Module: Based on the current batch detection results from the dynamic optimization module, it calculates the correlation between the results and the adaptive knowledge base using the adversarial domain adaptation algorithm. The feature distribution differences are analyzed; when the differences exceed a threshold, high-confidence samples are selected and a knowledge distillation strategy is used to update the feature decoupling network; the validated new feature-label pairs are then injected into the adaptive knowledge base. Execute adaptive knowledge base renew.

2. The flexible circuit board patch anomaly detection system based on multi-modal data of claim 1, wherein: In the real-time migration module, the process-invariant features of the current batch of real-time images are obtained through the branch responsible for process-invariant feature extraction in the feature decoupling network.

3. The flexible circuit board patch anomaly detection system based on multi-modal data of claim 2, wherein: The weighted nearest neighbor algorithm calculates the normalized similarity weight based on the Euclidean distance between the process-invariant features of the new sample and each feature entry in the adaptive knowledge base KB; the initial detection parameter is obtained by weighting and fusing the matched historical optimal detection parameters according to the similarity weight.

4. The flexible circuit board patch anomaly detection system based on multi-modal data of claim 3, wherein: When updating the policy network parameters, the near-end policy optimization algorithm uses gradient clipping technology to limit the amplitude of policy update.

5. The flexible circuit board patch anomaly detection system based on multi-modal data of claim 4, wherein: The adversarial domain adaptation algorithm distinguishes whether the features come from the historical knowledge base or the new batch data by training a domain discriminator, and calculates the adversarial loss accordingly; if the classification accuracy of the discriminator on new and old data exceeds the preset threshold, the incremental update is triggered.

6. The flexible circuit board patch anomaly detection system based on multi-modal data of claim 5, wherein: The high-confidence sample screening rule is based on whether the maximum class probability value in the prediction result of the model on the new batch data exceeds the preset confidence threshold.

7. The flexible circuit board patch anomaly detection system based on multi-modal data of claim 6, wherein: The knowledge distillation strategy updates the network parameters by minimizing the distance between the features extracted by the feature decoupling network before and after updating for high-confidence samples; when updating the adaptive knowledge base KB, the new process-invariant features and their predicted labels corresponding to the high-confidence samples are added to the knowledge base, and clustering compression is performed to merge similar features and only retain representative cluster centers.

8. A process for a flexible circuit board patch anomaly detection system based on multi-modal data according to any one of claims 1-7, characterized in that: Specifically comprising the following steps: Step S1, based on historical images, defect labels and process parameters, separating process invariant features and sensitive features through feature decoupling network, and constructing dynamic clustering knowledge base ; Step S2, using the dynamic clustering knowledge base generated in step S1 The process invariant features of the real-time image are extracted to perform weighted similarity matching, and the initial detection parameters of the new sample are generated by fusing the historical optimal parameters. Step S3, taking the initial parameter output in step S2 as the starting point, constructing a meta-reinforcement learning strategy combined with real-time feedback, dynamically optimizing the detection parameter and outputting labeled detection data; Step S4, analyze the detection data distribution generated in step S3, screen high-confidence samples when significant changes are detected, update network parameters by distillation constraints and inject new knowledge into the dynamic clustering knowledge base .

Images