A Closed-Loop Optimization Method and System for AOI Defect Detection in Smart Factories

Abstract

This application relates to the field of industrial computer vision and discloses a closed-loop optimization method and system for AOI defect detection in smart factories. The method includes: dividing a high-resolution industrial image to be detected into multiple image blocks; performing initial defect detection on the images based on a dual-branch architecture in a high-resolution anomaly detection model; outputting preliminary detection results; filtering material images identified as having defects from the preliminary detection results and obtaining their actual defect information; aligning the actual defect information with intermediate detection data generated by the detection model during the detection process to generate a supervised training dataset; adjusting the core parameters of the detection model based on the supervised training dataset; updating and deploying the optimized detection model to the AOI detection equipment; and repeating steps S1 to S4 to form a closed-loop iterative process of detection-feedback-optimization. This solution achieves dynamic self-optimization of the AOI defect detection model and continuous improvement of detection performance.

Description

Technical Field

[0001] This manual relates to the field of industrial computer vision, and in particular to a closed-loop optimization method and system for AOI defect detection in smart factories. Background Technology

[0002] The manufacturing industry is currently accelerating its transformation towards digitalization and intelligence, which has placed core demands on product quality control, namely "efficiency, accuracy, and traceability." Automated Optical Inspection (AOI), as a key link in quality assurance in smart factories, has evolved from the traditional manual sampling model to a core technology that ensures product consistency, reduces defect rates, and improves production efficiency. The performance and iteration efficiency of its inspection models directly affect enterprises' quality decisions.

[0003] However, in specialized industrial fields such as PCB and precision electronics, AOI defect detection faces unique challenges: new types of defects (such as minor scratches and tiny solder joint anomalies) are constantly emerging, and domain knowledge is highly specialized and discrete. Traditional unsupervised / semi-supervised detection models generally suffer from the limitation of "one-time training and static use." Existing technologies mostly adopt initial one-time training or self-supervised fine-tuning strategies, optimizing only during the model building stage using a small number of labeled or synthetic samples. This makes it impossible to dynamically adapt to new defects in real-world scenarios during subsequent production. Although some technologies mention adaptive labeling loops, their core goal is to improve labeling efficiency rather than optimize the performance of the defect detection model itself. This makes it difficult to form a closed loop from defect detection to model feedback optimization, resulting in poor model adaptability to new defects and high false negative and false positive rates. At the same time, existing technologies lack differentiated parameter configurations, and full retraining can easily damage the original model performance and is inefficient. In addition, the training specifications, scoring logic, and threshold rules are vague, leading to high industrial deployment and debugging costs and poor operability, which seriously restricts the large-scale application of AOI technology.

[0004] Therefore, there is an urgent need for a closed-loop optimization method and system for AOI defect detection in smart factories to achieve dynamic iteration of detection-feedback-optimization, adaptation of differentiated parameters, and standardized industrial implementation specifications, thereby addressing the shortcomings of traditional technologies in terms of adaptability to new defects, scenario compatibility, long-term stability, and industrial operability. Summary of the Invention

[0005] In view of this, the present invention aims to propose a closed-loop optimization method and system for AOI defect detection in smart factories, in order to solve the problems that existing AOI defect detection models cannot be dynamically optimized using information from manual review, resulting in poor adaptability of the models to new defects and insufficient long-term detection stability. The invention achieves closed-loop linkage between the detection model and production feedback, thereby continuously improving defect detection performance.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: A closed-loop optimization method for AOI defect detection in smart factories includes: S1. Divide the high-resolution industrial image to be detected into multiple image blocks, perform initial defect detection on the high-resolution industrial image to be detected based on the dual-branch architecture in the high-resolution anomaly detection model, and output preliminary detection results. S2. Filter the high-resolution industrial images that are determined to have defects in the preliminary detection results, and obtain the actual defect information for the high-resolution industrial images; S3. The real defect information is used as a supervision signal and aligned with the intermediate detection data generated by the high-resolution anomaly detection model during the detection process to generate a supervision training dataset. S4. Based on the supervised training dataset, adjust the core parameters of the high-resolution anomaly detection model to optimize the model's detection performance; S5. Update and deploy the optimized high-resolution anomaly detection model to the AOI detection equipment, and repeat S1 to S4 for the next cycle of defect detection and subsequent continuous optimization, forming a closed-loop iterative process of detection-feedback-optimization.

[0007] The beneficial effects of this solution are as follows: In existing technologies, AOI defect detection models are mostly trained in a static, unsupervised mode, which cannot utilize manually reviewed information generated during the production process for dynamic optimization. This results in poor model adaptability to novel defects, decreased detection accuracy with changes in the production environment, and insufficient long-term detection stability. This solution constructs a closed-loop iterative process of detection-feedback-optimization, using manually confirmed real defect information as a supervisory signal to feed back into the model for adjustment. The optimized model is then continuously applied to subsequent detections, achieving dynamic self-optimization of detection performance and effectively improving the model's ability to identify novel defects and its long-term detection stability.

[0008] Furthermore, in S1, during initial defect detection based on the dual-branch architecture, anomaly scores are calculated to characterize the degree to which a pixel belongs to a defect. The calculation logic for the anomaly score is as follows: ; In the formula, For the first Pixels in an image patch Abnormal scores; This is the multi-resolution fusion feature of the pixel; The first normal sample feature library learned during detector training One prototype feature; The number of prototype features; This indicates the calculation of the Euclidean distance between two vectors; Based on the anomaly scores, high-resolution branch anomaly maps and low-resolution branch anomaly maps are generated respectively, and the final anomaly map is obtained by the following fusion logic: ; In the formula, These are the pixel coordinates of the original image; This is a high-resolution branch anomaly graph; This is a low-resolution branch anomaly graph; This is the final anomaly graph.

[0009] Beneficial effects: By defining anomaly score calculation formula based on Euclidean distance of feature vectors, the multi-resolution fusion features of pixels are quantitatively compared with the prototype features in the normal sample feature library, realizing accurate numerical representation of the degree of defects and providing quantifiable objective evidence for the detection results. At the same time, by adopting the fusion rule of taking the maximum value for each pixel, the sensitivity of the high-resolution branch to subtle defects is effectively combined with the ability of the low-resolution branch to capture large-scale defects. This not only retains the detail detection advantage of high-resolution images, but also enhances the ability to identify global defects, avoiding the missed detection problem that may be caused by single-branch detection, and providing an accurate and reliable anomaly map foundation for subsequent closed-loop optimization.

[0010] Furthermore, in S1, the dual-branch architecture includes a backbone network for feature extraction; the high-resolution anomaly detection model also includes a detector pool and a preset detector allocation strategy; the detector pool contains multiple detectors, each detector corresponds to a set of normal sample feature distributions, and each detector includes a bottom layer, an intermediate layer, and an output layer; the detector allocation strategy is used to assign image blocks to the corresponding detectors for processing, and the detector allocation strategy is configured differently according to the type of industrial material.

[0011] Beneficial effects: By setting up multiple detectors and assigning them different normal sample feature distributions, the model can model the multimodal feature distribution of normal samples more precisely. At the same time, by configuring the detector allocation strategy differently according to the type of industrial material, the model can adapt to the image characteristics of different materials, improving the model's generalization ability and detection accuracy in diverse industrial scenarios, and providing a clear structured basis for subsequent adjustments.

[0012] Furthermore, S3 also includes: S3-1. Map the pixel coordinates of the real defect information to the image block division grid of the high-resolution branch to determine the target image block set corresponding to the real defect; S3-2. Extract the multi-resolution fusion features corresponding to the target image patch set, and construct a supervised training dataset with real defect areas as positive samples and non-defect areas as negative samples. S3-3. Calculate the overlap rate between the region corresponding to the real defect information and the defect candidate region in the preliminary detection result. Select samples with an overlap rate lower than a preset threshold as key optimization samples and include the key optimization samples in the supervised training dataset for adjusting the core parameters in S4. The calculation logic for the overlap rate is as follows: ; In the formula, These are manually marked defect areas; Candidate regions for defects detected by a high-resolution anomaly detection model; The pixel area of ​​the region.

[0013] Beneficial effects: Coordinate mapping enables precise alignment of artificial defect regions with image patches, ensuring the accuracy of the supervision signal; by extracting multi-resolution fusion features of target image patches and constructing a supervised training dataset in the form of positive and negative samples, high-quality labeled data is provided for model optimization; by calculating the overlap rate, key optimization samples that were missed or falsely detected in the initial detection are selected, enabling the optimization process to focus on the weak links in model performance, effectively improving the efficiency and targeting of adjustments.

[0014] Furthermore, S4 also includes: S4-1. The gradient descent method is used to adjust the fusion weights of each downsampling level in order to reduce the error between the abnormal scores corresponding to the key optimized samples and the real defect information. S4-2. Freeze the bottom layer parameters of the backbone network and adjust the intermediate layer parameters of the detectors corresponding to the key optimized samples; S4-3. Based on the industrial material type corresponding to the key optimized sample, dynamically switch the detector allocation strategy and adjust the clustering parameters in the switched detector allocation strategy.

[0015] Beneficial effects: By adjusting the fusion weights of the downsampling level, the fusion effect of multi-scale features was optimized; by freezing the bottom layer parameters of the backbone network and adjusting only the intermediate layer parameters of the detector, the general feature extraction capability learned by the model on a large number of normal samples was preserved, while rapid adaptation to specific defect samples was achieved; by dynamically switching the detector allocation strategy according to the type of industrial material and optimizing the clustering parameters, the model can adapt to the detection requirements of different materials, achieving a balance between accurate improvement of detection performance and optimization cost.

[0016] Furthermore, in S4-1, the fusion weights are adjusted based on the loss function, and their calculation logic is as follows: ; In the formula, To optimize the number of samples, The corresponding real anomaly information is manually labeled. The anomaly score predicted by the high-resolution anomaly detection model; The fusion weights for each downsampling level, This represents the maximum downsampling rate. .

[0017] Beneficial effects: By defining a mean squared error loss function based on key optimized samples, the difference between manually labeled real anomaly information and model-predicted anomaly scores is quantified into an optimizable mathematical objective. This gives the adjustment process of fusion weights a clear optimization direction and convergence basis, ensuring continuous improvement of multi-resolution feature fusion effects and enhancing the model's sensitivity to novel defects and detection accuracy.

[0018] Furthermore, the method also includes a preset differential parameter adaptation scheme, which configures different combinations of core parameters for the high-resolution anomaly detection model according to the type of industrial material; the core parameter combination includes detection resolution, image patch size, downsampling rate set, number of detectors, detector allocation strategy, and multi-resolution fusion weights.

[0019] Beneficial effects: By configuring different core parameter combinations for the high-resolution anomaly detection model according to the type of industrial material, the model can be optimized for the image characteristics of different materials in the initial detection stage. This solves the problem that a single parameter configuration cannot adapt to diverse industrial scenarios, improves the initial detection accuracy and adaptability of the model in different material detection tasks, and lays a good foundation for subsequent closed-loop optimization.

[0020] Furthermore, S5 also includes: The iteration triggering condition of the high-resolution anomaly detection model is set. The triggering condition is that when the accumulated real defect information reaches a preset number, the model adjustment process is started. After the adjusted model passes offline verification, it is updated and deployed.

[0021] Beneficial effects: By setting an iteration trigger threshold based on the amount of real defect information accumulated, model optimization can be automatically started after sufficient supervision signals are accumulated, avoiding the waste of computing resources caused by frequent optimization; at the same time, offline verification ensures that the performance of the optimized model is not lower than that of the unoptimized model, guaranteeing the safety and reliability of model updates and achieving continuous and stable improvement in detection performance.

[0022] Furthermore, a closed-loop optimization system for AOI defect detection in a smart factory includes: The image acquisition module is used to acquire high-resolution industrial images to be inspected; The high-resolution anomaly detection module is used to perform initial defect detection on high-resolution industrial images to be inspected based on a dual-branch architecture of a high-resolution anomaly detection model, and output preliminary detection results. The human interaction module is used to filter high-resolution industrial images that are determined to have defects in the preliminary detection results, and to obtain the actual defect information for the high-resolution industrial images. The data processing module is used to align the real defect information as a supervision signal with the intermediate detection data generated by the high-resolution anomaly detection model during the detection process to generate a supervised training dataset. The model optimization module is used to adjust the core parameters of the high-resolution anomaly detection model based on the supervised training data in order to optimize the detection performance of the model. The storage and iteration module updates and deploys the optimized high-resolution anomaly detection model to the AOI inspection equipment, and repeats S1 to S4 for the next cycle of defect detection and subsequent continuous optimization, forming a closed-loop iterative process of detection-feedback-optimization.

[0023] Beneficial effects: By setting up a high-resolution anomaly detection module, a human interaction module, a data processing module, a model optimization module, and a storage and iteration module, the complete closed-loop process of detection, feedback, optimization, and deployment is realized in a modular form. Each module has a clear function and works in alignment, which not only ensures the technical feasibility of the method, but also improves the implementability and engineering implementation capability of the system, providing a complete integrated hardware and software solution for AOI defect detection in smart factories. Attached Figure Description

[0024] This specification will be further described by way of exemplary embodiments, which will be described in detail with reference to the accompanying drawings. These embodiments are not limiting; in these embodiments, the same reference numerals denote the same structures, wherein: Figure 1 This is an exemplary flowchart of a closed-loop optimization method for AOI defect detection in smart factories; Figure 2 This is an exemplary technical architecture diagram of an intelligent factory AOI defect detection closed-loop optimization system. Detailed Implementation

[0025] To more clearly illustrate the technical solutions of the embodiments in this specification, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are merely some examples or embodiments of this specification. For those skilled in the art, these drawings can be applied to other similar scenarios without creative effort. Unless obvious from the context or otherwise specified, the same reference numerals in the drawings represent the same structures or operations.

[0026] As indicated in this specification and claims, unless the context clearly indicates otherwise, the words "a," "an," "an," and / or "the" do not specifically refer to the singular and may also include the plural. Generally speaking, the terms "comprising" and "including" only indicate the inclusion of expressly identified steps and elements, which do not constitute an exclusive list, and the method or apparatus may also include other steps or elements.

[0027] Flowcharts are used in this specification to illustrate the operations performed by the system according to embodiments of this specification. It should be understood that the preceding or following operations are not necessarily performed in exact order. Instead, the steps can be processed in reverse order or simultaneously. Furthermore, other operations can be added to these processes, or one or more steps can be removed from them.

[0028] The following detailed explanation illustrates the specific implementation methods: Example 1: like Figures 1-2 As shown, a closed-loop optimization method for AOI defect detection in a smart factory includes: S1. Divide the high-resolution industrial image to be detected into multiple image blocks. Based on the dual-branch architecture in the high-resolution anomaly detection model, perform initial defect detection on the high-resolution industrial image to be detected and output preliminary detection results.

[0029] High-resolution industrial images refer to high-definition digital images with a resolution of 2048×2048 pixels or higher, captured by industrial cameras. They can clearly present the fine textures, edge contours, and minute defects on the surface of industrial products.

[0030] In this embodiment, the high-resolution industrial image to be detected can be acquired by a high-resolution industrial camera deployed on the AOI inspection equipment.

[0031] An image patch refers to a high-resolution industrial image divided into several sub-regions according to a preset size. Each image patch serves as the basic processing unit of the high-resolution anomaly detection model, used to extract local features and perform independent detection.

[0032] In this embodiment, the AOI inspection equipment captures real-time images of the PCB board using a high-resolution industrial camera deployed on the production line. After image acquisition, the system transmits the high-resolution industrial image to the high-resolution anomaly detection module. This module divides the entire image into multiple image blocks according to a preset size and step size, serving as the basic unit for subsequent processing. Initial defect detection is then performed based on the dual-branch architecture of the trained high-resolution anomaly detection model.

[0033] The high-resolution anomaly detection model is a deep learning-based anomaly detection model used for defect detection in high-resolution industrial images. This model can use existing convolutional neural networks or visual Transformers as the backbone network, and then optimize it with a dual-branch architecture.

[0034] In this embodiment, the model employs a dual-branch architecture, including a high-resolution branch and a low-resolution branch. During the detection process, the high-resolution branch divides the input high-resolution industrial image into multiple image patches, extracts multi-resolution fusion features, and then performs anomaly detection through a detector pool. The low-resolution branch downsamples the original image, extracts features, and performs anomaly detection. Finally, the results from both branches are fused to obtain the final anomaly map. The anomaly score is calculated based on the Euclidean distance between the multi-resolution fusion features of a pixel and the prototype features in the normal sample feature library. The larger the distance, the higher the anomaly score, indicating a greater likelihood that the pixel belongs to a defect. For details on the calculation of the anomaly score, please refer to the formulas and related descriptions below.

[0035] After calculating the anomaly image, the system judges the image according to a preset NG (Not From Good) threshold (τ=0.89). This threshold is calculated based on the 99.9 quantile (τ0=0.85) of the anomaly score of normal samples in the validation set, with a dynamic adjustment coefficient α=1.05, i.e., τ=1.05×0.85=0.89. If the percentage of pixels with anomaly scores exceeding τ in the image is ≥0.01%, or if there is a continuous 16×16 pixel region where the anomaly scores all exceed τ, then the material is judged as a non-conforming product, i.e., a high-resolution industrial image with defects.

[0036] In this embodiment, the high-resolution anomaly detection model can be trained in the following way: S1-1. Collect normal sample images of the same type of industrial materials; select WideResNet50 as the backbone network and load ImageNet pre-trained weights, and set the learning rate of the AdamW optimizer to [value missing]. This controls the step size for each parameter update; the weight decay is set to... It is used to regularize the network weights during the optimization process to prevent the model from overfitting, and performs 20,000 iterations of training.

[0037] S1-2. During training, normal sample images are divided into 512×512 image blocks. Based on the SCA allocation strategy, these image blocks are assigned to four detectors. Each detector independently learns the feature distribution of normal samples, constructing a normal sample feature library. The training termination condition is when the mean anomaly score of normal samples on the validation set converges, ultimately yielding a trained high-resolution anomaly detection model. The convergence condition is that the fluctuation is ≤0.001 and there is no decrease after 500 consecutive iterations.

[0038] Among them, normal sample images need to be ≥100 per class, with a resolution of 2048×2048, illumination uniformity ≥85%, and noise level ≤10dB.

[0039] WideResNet50 is a classic convolutional neural network architecture. It is an improved version of ResNet (Residual Network), which enhances the model's feature extraction capability by increasing the network's width, i.e., increasing the number of channels in the convolutional layers rather than the depth. While maintaining computational efficiency, it can better capture fine-grained features in images, making it suitable for tasks such as detecting subtle defects in high-resolution industrial images.

[0040] ImageNet pre-trained weights refer to model parameters that have been pre-trained on the large-scale image dataset ImageNet. ImageNet is a large image dataset containing over 14 million images covering more than 20,000 categories, and is one of the most commonly used benchmark datasets in the field of computer vision.

[0041] In this embodiment, during initial defect detection based on the dual-branch architecture in step S1, anomaly scores are calculated to characterize the degree to which a pixel belongs to a defect. The calculation logic for the anomaly score is as follows: ; In the formula, For the first Pixels in an image patch Abnormal scores; This is the multi-resolution fusion feature of the pixel; The first normal sample feature library learned during detector training One prototype feature; The number of prototype features; This indicates the calculation of the Euclidean distance between two vectors.

[0042] Based on the anomaly scores, high-resolution branch anomaly maps and low-resolution branch anomaly maps are generated respectively, and the final anomaly map is obtained by the following fusion logic: ; In the formula, These are the pixel coordinates of the original image; This is a high-resolution branch anomaly graph; This is a low-resolution branch anomaly graph; This is the final anomaly graph.

[0043] Furthermore, in S1, the dual-branch architecture includes a backbone network for feature extraction; the high-resolution anomaly detection model also includes a detector pool and a preset detector allocation strategy; the detector pool contains multiple detectors, each detector corresponds to a set of normal sample feature distributions, and each detector includes a bottom layer, an intermediate layer, and an output layer; the detector allocation strategy is used to assign image blocks to the corresponding detectors for processing, and the detector allocation strategy is configured differently according to the type of industrial material.

[0044] A detector pool is a collection of multiple detectors included in a high-resolution anomaly detection model. Each detector independently maintains a set of feature distributions learned from normal samples, i.e., a normal sample feature library, which is used to calculate anomaly scores for image patches assigned to that detector.

[0045] In this embodiment, the size of the detector pool (i.e. the number of detectors) can be configured according to the type of industrial material and the detection requirements. For example, four detectors can be set for a 2048×2048 image, and eight detectors can be set for a 4096×4096 image.

[0046] The preset detector allocation strategy is a computational rule used to determine which detector in the detector pool to assign each image patch for processing. This strategy is differentiated according to the type of industrial material to adapt to the image characteristics of different materials. Common allocation strategies include: Spatial Clustering Allocation Strategy (SCA, suitable for structured materials with positional calibration such as PCBs, which clusters and allocates based on the spatial location of image patches), Retrieval Allocation Strategy (RA, suitable for materials without positional calibration such as screws, which retrieves and allocates based on the visual feature similarity of image patches), and Neighborhood Allocation Strategy (NA, suitable for homogeneous textured materials such as wood, which allocates based on the texture similarity of adjacent image patches), etc.

[0047] S2. Filter the high-resolution industrial images that are determined to have defects in the preliminary detection results, and obtain the actual defect information for the high-resolution industrial images.

[0048] True defect information refers to the accurate and truthful description of defects marked on high-resolution industrial images that are determined to have defects in S1 through manual review.

[0049] In this embodiment, the information includes the precise location coordinates of the defect in the image, the type label of the defect (such as broken circuit, missing pad, minor scratch, etc.), and metadata such as the annotation time.

[0050] In this embodiment, after completing the initial detection, the high-resolution anomaly detection module pushes the high-resolution industrial image identified as having defects, along with its preliminary detection results, to the human interaction module. Workers can view the pushed image on a high-resolution display and use a stylus to precisely mark the location of the actual defects on the image. The human interaction module supports zooming to clearly present subtle defects.

[0051] In this embodiment, after labeling is completed, the operator selects the corresponding defect type from a pre-defined defect type label library. The system associates the labeling information with the image, generating a standardized labeling file in JSON format. This file contains information such as the image's unique identifier, the pixel coordinates of the defect area, the defect type label, and the labeling time, and is automatically saved to a MySQL database. During a single batch of production, when the cumulative number of valid NG samples labeled reaches 50, the system automatically triggers the subsequent model optimization process.

[0052] S3. Use real defect information as a supervision signal and align it with the intermediate detection data generated by the high-resolution anomaly detection model during the detection process to generate a supervised training dataset.

[0053] In this embodiment, the supervisory signal refers to the actual defect information that has been manually verified and labeled. It serves as a correct guide for adjusting the high-resolution anomaly detection model.

[0054] Intermediate detection data refers to the internal process data generated by the high-resolution anomaly detection model during the initial defect detection process in S1, excluding the final detection results. This includes multi-resolution fusion features of image patches and the final anomaly map.

[0055] Furthermore, S3 also includes: S3-1. Map the pixel coordinates of the real defect information to the image block grid of the high-resolution branch to determine the set of target image blocks corresponding to the real defect.

[0056] S3-2. Extract the multi-resolution fusion features corresponding to the target image patch set, and construct a supervised training dataset with real defect areas as positive samples and non-defect areas as negative samples.

[0057] S3-3. Calculate the overlap rate between the region corresponding to the real defect information and the defect candidate region in the preliminary detection result. Select samples with an overlap rate lower than a preset threshold as key optimization samples and include them in the supervised training dataset for adjusting the core parameters in S4.

[0058] The calculation logic for the overlap rate is as follows: ; In the formula, These are manually marked defect areas; Candidate regions for defects detected by a high-resolution anomaly detection model; The pixel area of ​​the region.

[0059] In this embodiment, if the overlap rate is high, it means that the model detection result is basically consistent with the manual annotation, and the detection effect of the sample is good. If the overlap rate is lower than the preset threshold (e.g., 50%), it means that the model has missed detections (failed to detect defects in the manual annotation) or false detections (the detected area deviates greatly from the manual annotation). Such samples are identified as key optimization samples and need to be included in the supervised training dataset for subsequent targeted optimization of the model's core parameters to improve the model's ability to identify similar defects.

[0060] S4. Based on the supervised training dataset, the core parameters of the high-resolution anomaly detection model are adjusted to optimize the model's detection performance.

[0061] Furthermore, S4 also includes: S4-1. The gradient descent method is used to adjust the fusion weights of each downsampling level in order to reduce the error between the abnormal scores corresponding to the key optimized samples and the real defect information.

[0062] In S4-1, the fusion weights are adjusted based on the loss function, and their calculation logic is as follows: ; In the formula, To optimize the number of samples, The corresponding real anomaly information is manually labeled. The anomaly score predicted by the high-resolution anomaly detection model; The fusion weights for each downsampling level, This represents the maximum downsampling rate. .

[0063] In this embodiment, the mean squared error between the model-predicted anomaly score and the manually labeled real anomaly information for each key optimization sample is calculated, and the average of the errors of all key optimization samples is summed to obtain the loss value for fusion weight adjustment. When adjusting the fusion weights of each downsampling level using the gradient descent method, the optimization objective is to minimize this loss function, so that the anomaly scores predicted by the model continuously approach the real defect information annotated by humans, thereby improving the ability of multi-resolution fusion features to represent defect regions and reducing the risk of missed detections and false detections.

[0064] S4-2. Freeze the bottom layer parameters of the backbone network and adjust the intermediate layer parameters of the detectors corresponding to the key optimization samples.

[0065] In this embodiment, freezing the bottom parameters of the backbone network means setting the weight parameters of the first few layers (such as the first 30% of the bottom convolutional layers) of the backbone network to an untrainable state during the model adjustment process. That is, the gradients of these layers are not calculated and their weight values ​​are not updated during backpropagation.

[0066] In this embodiment, the system can use the PyTorch deep learning framework to implement parameter freezing.

[0067] S4-3. Based on the industrial material type corresponding to the key optimization samples, dynamically switch the detector allocation strategy and adjust the clustering parameters in the switched detector allocation strategy.

[0068] In this embodiment, the system can achieve dynamic switching by pre-setting a correspondence table between industrial material types and detector allocation strategies.

[0069] For example, when the industrial material type of the sample to be optimized is PCB board (structured material with position calibration), the system automatically switches the detector allocation strategy to the spatial clustering allocation strategy (SCA). This strategy is based on the spatial location of image blocks and is suitable for grouping image blocks with fixed locations and similar appearances.

[0070] When the industrial material type of the key optimized sample is screws (materials without position calibration), the system automatically switches the detector allocation strategy to the retrieval allocation strategy (RA). This strategy is based on the visual feature similarity of image patches for retrieval and allocation, and can adapt to parts with no fixed placement position and varied appearance.

[0071] When the industrial material type of the sample to be optimized is wood board (homogeneous texture material), the system automatically switches the detector allocation strategy to the neighborhood allocation strategy (NA). This strategy is based on the texture similarity of adjacent image blocks and is suitable for materials with uniform texture and no complex structure.

[0072] In this embodiment, through this dynamic switching mechanism based on material type, the model can adapt to the detection requirements of different industrial materials, thereby improving the stability and adaptability of detection performance.

[0073] S5. Update and deploy the optimized high-resolution anomaly detection model to the AOI detection equipment, and repeat S1 to S4 for the next cycle of defect detection and subsequent continuous optimization, forming a closed-loop iterative process of detection-feedback-optimization.

[0074] Furthermore, S5 also includes: Set the iteration trigger condition for the high-resolution anomaly detection model. The trigger condition is that when the accumulated real defect information reaches a preset number, the model adjustment process is started. After the adjusted model passes offline verification, it is updated and deployed.

[0075] In this embodiment, the storage and iteration module presets the iteration trigger condition for model optimization, that is, when the system accumulates 50 valid manually labeled samples, the model optimization process is automatically triggered. For scenarios with a fast production line cycle and a fast sample accumulation speed, this threshold can be flexibly adjusted to 100 to balance the optimization frequency and production interruption costs.

[0076] After the optimization process is triggered, the model optimization module adjusts the core parameters of the high-resolution anomaly detection model based on key optimization samples in the supervised training dataset. The default number of iterations per optimization round is set to 1000. In a hardware environment with four Nvidia RTX 4090 GPUs, the optimization time per round is controlled within 2 hours, ensuring that the optimization process does not affect the normal production cycle.

[0077] After optimization, the system performs offline validation of the updated model. The validation set includes historical out-of-memory (NG) samples and normal samples. By comparing the detection accuracy of the model before and after optimization on the validation set, it is ensured that the detection performance of the optimized model is not lower than that of the unoptimized model. If the validation passes, the storage and iteration module overwrites the original model with the optimized model parameters and redeploys it to the AOI detection equipment for production use.

[0078] The above process is continuously repeated as the production line runs: every 50 or 100 valid manually labeled samples are accumulated, triggering a round of model optimization and deployment, thereby achieving continuous iterative improvement in detection performance.

[0079] Example 2: like Figure 2 As shown, a closed-loop optimization system for AOI defect detection in a smart factory includes: The image acquisition module is used to acquire high-resolution industrial images to be inspected.

[0080] The high-resolution anomaly detection module is used to perform initial defect detection on high-resolution industrial images to be inspected based on a dual-branch architecture of a high-resolution anomaly detection model, and output preliminary detection results.

[0081] The human interaction module is used to filter high-resolution industrial images that are determined to have defects in the preliminary detection results, and to obtain the actual defect information for the high-resolution industrial images.

[0082] The data processing module is used to align the real defect information as a supervision signal with the intermediate detection data generated by the high-resolution anomaly detection model during the detection process, and generate a supervised training dataset.

[0083] The model optimization module is used to adjust the core parameters of the high-resolution anomaly detection model based on the supervised training data in order to optimize the model's detection performance.

[0084] The storage and iteration module updates and deploys the optimized high-resolution anomaly detection model to the AOI inspection equipment, and repeats S1 to S4 for the next cycle of defect detection and subsequent continuous optimization, forming a closed-loop iterative process of detection-feedback-optimization.

[0085] In this embodiment, by setting up a high-resolution anomaly detection module, a human interaction module, a data processing module, a model optimization module, and a storage and iteration module, the complete closed-loop process of detection, feedback, optimization, and deployment is implemented in a modular form. Each module has a clear function and works in alignment, which not only ensures the technical feasibility of the method, but also improves the implementability and engineering implementation capability of the system, providing a complete integrated hardware and software solution for AOI defect detection in smart factories.

[0086] Example 3: Based on Embodiment 1 and Embodiment 2, the method further includes a preset differential parameter adaptation scheme. The differential parameter adaptation scheme configures different core parameter combinations for the high-resolution anomaly detection model according to the type of industrial material. The core parameter combinations include detection resolution, image patch size, downsampling rate set, number of detectors, detector allocation strategy, and multi-resolution fusion weights.

[0087] In this embodiment, differentiated high-resolution anomaly detection model parameter configuration schemes were designed based on the characteristics of different types of industrial materials to ensure a balance between detection performance and computational efficiency. Specific parameter configurations are shown in the table below:

[0088] In this embodiment, the model is equipped with optimized adaptability for different materials in the initial detection stage through this differentiated parameter configuration, which improves the accuracy and robustness of the initial detection, lays a good foundation for subsequent closed-loop optimization, reduces the cost of repeated debugging caused by improper parameters, and enhances the versatility and deployment efficiency of the system in diverse industrial scenarios.

[0089] The basic concepts have been described above. Obviously, for those skilled in the art, the detailed disclosure above is merely illustrative and does not constitute a limitation of this specification. Although not explicitly stated herein, those skilled in the art may make various modifications, improvements, and corrections to this specification. Such modifications, improvements, and corrections are suggested in this specification and therefore remain within the spirit and scope of the exemplary embodiments described herein.

[0090] Furthermore, unless expressly stated in the claims, the order of processing elements and sequences, the use of numbers and letters, or other names described in this specification are not intended to limit the order of the processes and methods described herein. Although various examples have been discussed in the foregoing disclosure of some embodiments of the invention that are currently considered useful, it should be understood that such details are for illustrative purposes only, and the appended claims are not limited to the disclosed embodiments; rather, the claims are intended to cover all modifications and equivalent combinations that conform to the spirit and scope of the embodiments described herein. For example, while the system components described above can be implemented using hardware devices, they can also be implemented solely using software solutions, such as installing the described system on existing servers or mobile devices.

[0091] Similarly, it should be noted that, in order to simplify the description disclosed herein and thus aid in the understanding of one or more embodiments of the invention, the foregoing description of embodiments in this specification may sometimes combine multiple features into a single embodiment, drawing, or description thereof. However, this method of disclosure does not imply that the subject matter of this specification requires more features than those mentioned in the claims. In fact, the embodiments contain fewer features than all the features of a single embodiment disclosed above.

[0092] In some embodiments, numbers describing the quantity of components and attributes are used. It should be understood that such numbers used in the description of embodiments are modified in some examples with the terms "approximately," "approximately," or "generally." Unless otherwise stated, "approximately," "approximately," or "generally" indicates that the numbers are allowed to vary by ±20%. Accordingly, in some embodiments, the numerical parameters used in the specification and claims are approximate values, which may be changed depending on the characteristics required by individual embodiments. In some embodiments, numerical parameters should take into account specified significant digits and employ a general method of digit reservation. Although the numerical ranges and parameters used to confirm their breadth of range in some embodiments of this specification are approximate values, in specific embodiments, such values ​​are set as precisely as feasible.

[0093] For each patent, patent application, patent application publication, and other material such as articles, books, specifications, publications, and documents referenced in this specification, the entire contents of which are incorporated herein by reference. This excludes historical application documents that are inconsistent with or conflict with the content of this specification, as well as documents that limit the broadest scope of the claims in this specification (currently or subsequently appended to this specification). It should be noted that in the event of any inconsistency or conflict between the descriptions, definitions, and / or terminology used in the supplementary materials to this specification and the content of this specification, the descriptions, definitions, and / or terminology used in this specification shall prevail.

[0094] Finally, it should be understood that the embodiments described in this specification are merely illustrative of the principles of the embodiments described herein. Other variations may also fall within the scope of this specification. Therefore, alternative configurations of the embodiments described herein are intended to be illustrative rather than limiting, and should be considered consistent with the teachings of this specification. Accordingly, the embodiments described herein are not limited to those explicitly introduced and described herein.

Claims

1. A closed-loop optimization method for AOI defect detection in intelligent factories, characterized in that, include: S1. Divide the high-resolution industrial image to be detected into multiple image blocks, perform initial defect detection on the high-resolution industrial image to be detected based on the dual-branch architecture in the high-resolution anomaly detection model, and output preliminary detection results. S2. Filter the high-resolution industrial images that are determined to have defects in the preliminary detection results, and obtain the actual defect information for the high-resolution industrial images; S3. The real defect information is used as a supervision signal and aligned with the intermediate detection data generated by the high-resolution anomaly detection model during the detection process to generate a supervision training dataset. S4. Based on the supervised training dataset, adjust the core parameters of the high-resolution anomaly detection model to optimize the model's detection performance; S5. Update and deploy the optimized high-resolution anomaly detection model to the AOI inspection equipment, and repeat S1 to S4 for the next cycle of defect detection and subsequent continuous optimization, forming a closed-loop iterative process of detection-feedback-optimization.

2. The closed-loop optimization method for AOI defect detection in a smart factory according to claim 1, characterized in that, In step S1, during initial defect detection based on the dual-branch architecture, anomaly scores are calculated to characterize the degree to which a pixel belongs to a defect. The calculation logic for the anomaly score is as follows: ； In the formula, For the first Pixels in an image patch Abnormal scores; This is the multi-resolution fusion feature of the pixel; The first normal sample feature library learned during detector training One prototype feature; The number of prototype features; This indicates the calculation of the Euclidean distance between two vectors; Based on the anomaly scores, high-resolution branch anomaly maps and low-resolution branch anomaly maps are generated respectively, and the final anomaly map is obtained by the following fusion logic: ； In the formula, These are the pixel coordinates of the original image; This is a high-resolution branch anomaly graph; This is a low-resolution branch anomaly graph; This is the final anomaly graph.

3. The closed-loop optimization method for AOI defect detection in a smart factory according to claim 2, characterized in that, In S1, the dual-branch architecture includes a backbone network for feature extraction; the high-resolution anomaly detection model also includes a detector pool and a preset detector allocation strategy; the detector pool contains multiple detectors, each detector corresponds to a set of normal sample feature distributions, and each detector includes a bottom layer, an intermediate layer, and an output layer; the detector allocation strategy is used to assign image blocks to the corresponding detectors for processing, and the detector allocation strategy is configured differently according to the type of industrial material.

4. The closed-loop optimization method for AOI defect detection in a smart factory according to claim 2, characterized in that, S3 further includes: S3-1. Map the pixel coordinates of the real defect information to the image block division grid of the high-resolution branch to determine the target image block set corresponding to the real defect; S3-2. Extract the multi-resolution fusion features corresponding to the target image patch set, and construct a supervised training dataset with real defect areas as positive samples and non-defect areas as negative samples. S3-3. Calculate the overlap rate between the region corresponding to the real defect information and the defect candidate region in the preliminary detection result. Select samples with an overlap rate lower than a preset threshold as key optimization samples and include the key optimization samples in the supervised training dataset for adjusting the core parameters in S4. The calculation logic for the overlap rate is as follows: ； In the formula, These are manually marked defect areas; Candidate regions for defects detected by a high-resolution anomaly detection model; The pixel area of ​​the region.

5. The closed-loop optimization method for AOI defect detection in a smart factory according to claim 4, characterized in that, S4 further includes: S4-1. The gradient descent method is used to adjust the fusion weights of each downsampling level in order to reduce the error between the abnormal scores corresponding to the key optimized samples and the real defect information. S4-2. Freeze the bottom layer parameters of the backbone network and adjust the intermediate layer parameters of the detectors corresponding to the key optimized samples; S4-3. Based on the industrial material type corresponding to the key optimized sample, dynamically switch the detector allocation strategy and adjust the clustering parameters in the switched detector allocation strategy.

6. The closed-loop optimization method for AOI defect detection in a smart factory according to claim 5, characterized in that, In S4-1, the fusion weights are adjusted based on the loss function, and their calculation logic is as follows: ； In the formula, To optimize the number of samples, The corresponding real anomaly information is manually labeled. The anomaly score predicted by the high-resolution anomaly detection model; The fusion weights for each downsampling level, This represents the maximum downsampling rate. .

7. The closed-loop optimization method for AOI defect detection in a smart factory according to claim 6, characterized in that, The method also includes a preset differential parameter adaptation scheme, which configures different combinations of core parameters for the high-resolution anomaly detection model according to the type of industrial material; the core parameter combination includes detection resolution, image patch size, downsampling rate set, number of detectors, detector allocation strategy, and multi-resolution fusion weight.

8. The closed-loop optimization method for AOI defect detection in a smart factory according to claim 1, characterized in that, The S5 also includes: The iteration triggering condition of the high-resolution anomaly detection model is set. The triggering condition is that when the accumulated real defect information reaches a preset number, the model adjustment process is started. After the adjusted model passes offline verification, it is updated and deployed.

9. A closed-loop optimization system for AOI defect detection in intelligent factories, characterized in that, A method for implementing a closed-loop optimization method for AOI defect detection in a smart factory as described in any one of claims 1-8; comprising: The image acquisition module is used to acquire high-resolution industrial images to be inspected; The high-resolution anomaly detection module is used to perform initial defect detection on high-resolution industrial images to be inspected based on a dual-branch architecture of a high-resolution anomaly detection model, and output preliminary detection results. The human interaction module is used to filter high-resolution industrial images that are determined to have defects in the preliminary detection results, and to obtain the actual defect information for the high-resolution industrial images. The data processing module is used to align the real defect information as a supervision signal with the intermediate detection data generated by the high-resolution anomaly detection model during the detection process to generate a supervised training dataset. The model optimization module is used to adjust the core parameters of the high-resolution anomaly detection model based on the supervised training data in order to optimize the detection performance of the model. The storage and iteration module updates and deploys the optimized high-resolution anomaly detection model to the AOI inspection equipment, and repeats S1 to S4 for the next cycle of defect detection and subsequent continuous optimization, forming a closed-loop iterative process of detection-feedback-optimization.

Images