A multi-region targeting algorithm fusion detection method for FPCA

Abstract

This application relates to the field of flexible printed circuit board (FPCA) inspection technology, and discloses a multi-region targeted algorithm fusion detection method for FPCA. The method includes: acquiring the image to be inspected, extracting process reference points and estimating the nonlinear deformation field, and performing geometric correction on image block features; orthogonally decoupling the corrected features into semantic direction components and interference amplitude components, discarding the interference amplitude components; performing weighted similarity matching between the semantic direction components and a weighted memory stored according to preset weights of functional regions to obtain anomaly scores; fusing multi-scale anomaly scores and comparing them with a threshold to output the detection results. This application eliminates deformation and illumination interference through a dual mechanism of geometric correction and feature decoupling, and improves the detection capability of minute defects by combining a functional region weighted memory with multi-scale fusion, achieving a significant improvement in detection accuracy and robustness.

Description

Technical Field

[0001] This application relates to the field of flexible printed circuit board (FPCB) inspection, and in particular to a multi-region targeted algorithm fusion inspection method for FPCA. Background Technology

[0002] Flexible printed circuit board assemblies (FPCA) are key components in consumer electronics, automotive electronics, and other fields, and the inspection of their appearance defects is a crucial aspect of product quality control. Currently, FPCA appearance defect inspection mainly employs three types of technical solutions: the first is traditional machine vision methods, which use algorithms such as threshold segmentation and template matching to detect specific areas; the second is supervised deep learning detection methods, which train neural network models by labeling a large number of defect samples to achieve automatic defect identification; and the third is unsupervised anomaly detection methods, such as PatchCore, which build a feature memory library of normal samples and use feature distance metrics to achieve anomaly detection. These solutions have already been applied to some extent in the inspection of rigid printed circuit boards (PCBs).

[0003] However, the aforementioned existing technical solutions have the following drawbacks when applied to the inspection of flexible substrates: First, as a flexible substrate, FPCA is prone to nonlinear deformations such as wrinkles, stretching, and bending during the production process, resulting in geometric misalignment between the image under test and the standard template. Traditional methods do not consider the interference of such deformations on the detection results, and direct feature matching is prone to false alarms and missed detections. Second, existing unsupervised inspection methods usually use Euclidean distance to measure feature similarity. This measurement method is highly sensitive to changes in feature amplitude caused by illumination fluctuations and deformation of the flexible board. When the amplitude fluctuates, even if the semantic content remains unchanged, the similarity calculation will still produce a large deviation, resulting in poor detection stability. Third, existing methods treat all inspection areas equally and do not combine the process characteristics of FPCA to give higher detection sensitivity to key areas such as connectors and components. Small defects in key areas are easily drowned out by noise in non-critical areas. Finally, existing methods mostly use single-scale feature extraction, which is insufficient for detecting small defects such as gold finger burrs and pin micro-offsets.

[0004] To address the aforementioned issues, this invention proposes a multi-region targeting algorithm fusion detection method for FPCA. Summary of the Invention

[0005] To address the aforementioned issues, this application provides a multi-region targeting algorithm fusion detection method for FPCA, employing the following technical solution: A multi-region targeting algorithm fusion detection method for FPCA includes the following steps: S1: Acquire an image of the flexible substrate to be detected, automatically locate multiple process reference points in the image of the flexible substrate, construct an actual topology diagram based on the process reference points, and compare the actual topology diagram with a preset standard topology diagram to estimate the nonlinear deformation field. S2: For multiple image block features extracted from the image of the flexible substrate to be detected, their spatial coordinates are transformed according to the nonlinear deformation field to obtain geometrically corrected image block features; S3: Perform orthogonal decoupling on the geometrically corrected image block features to obtain semantic direction components and interference amplitude components, and discard the interference amplitude components; S4: Perform weighted similarity matching between the semantic direction component and the pre-constructed weighted memory to obtain an anomaly score. The feature vector in the weighted memory is associated with the preset weight of its functional region. During matching, the preset weight is applied to the cosine similarity. The weighted similarity is defined as the product of the preset weight and the cosine similarity. S5: Output the detection result based on the comparison between the anomaly score and the preset threshold.

[0006] Preferably, the process reference point includes at least one of the following: connector pin center point, component pad corner point, gold finger endpoint, and reinforcement area boundary point.

[0007] Preferably, estimating the nonlinear deformation field specifically includes: The nonlinear mapping relationship from the standard topology diagram to the actual topology diagram is calculated using thin-plate spline functions or local weighted regression.

[0008] Preferably, in step S2, transforming the spatial coordinates of the image patch features specifically includes: The nonlinear deformation field is applied to the center coordinates of each image block feature, and mapped to the corresponding position under the standard pose to obtain the geometrically corrected image block features.

[0009] Preferably, in step S3, orthogonal decoupling of the geometrically corrected image patch features specifically includes: The geometrically corrected image patch features are normalized to obtain a unit vector as the semantic direction component, and the magnitude of the geometrically corrected image patch features is used as the interference amplitude component.

[0010] Preferably, the weighted memory is constructed through the following steps: Multiple standard pose normal sample images are acquired, image patch features are extracted and orthogonally decoupled to obtain standard semantic direction components; Determine the functional region to which each standard semantic direction component belongs in the flexible substrate image. The functional regions include connector regions, component regions, gold finger regions, reinforcement regions, and general regions. Each functional area is assigned a preset weight, with the connector area having the highest weight, the component area having the second highest weight, and the ordinary area having the lowest weight. Based on the preset weights, the corresponding region weights are associated with the standard semantic direction components to form a weighted memory.

[0011] Preferably, in step S4, performing weighted similarity matching between the semantic direction component and the pre-constructed weighted memory specifically includes: Calculate the cosine similarity between the semantic direction component and each feature vector in the weighted memory, and multiply the similarity by the region weight corresponding to the feature vector to obtain the weighted similarity; Select the feature vector corresponding to the maximum weighted similarity as the nearest neighbor; The maximum weighted similarity is converted into cosine distance and used as the anomaly score of the image patch features corresponding to the semantic direction component.

[0012] Preferably, the anomaly score in step S4 is a fusion result of multi-scale anomaly scores, specifically obtained through the following method: Before performing step S2, image patch features of multiple scales are extracted from the image of the flexible substrate to be detected. Steps S2 to S4 are performed on the image patch features at each scale to obtain the anomaly score at each scale; The anomaly scores from multiple scales are aligned by coordinates and then weighted and fused to obtain the final anomaly score.

[0013] Preferably, the plurality of scales includes a large scale for capturing macroscopic defects, a medium scale for capturing conventional defects, and a small scale for capturing microscopic defects.

[0014] Preferably, in step S5, the output detection result specifically includes: When the anomaly score is greater than or equal to the preset threshold, the defect detection result is output and an anomaly heatmap is generated. The anomaly heatmap is used to mark the abnormal areas in the image.

[0015] In summary, this application includes at least one of the following beneficial technical effects: 1. Effective elimination of flexible substrate deformation and illumination interference through a dual mechanism of geometric correction and feature decoupling: This application first constructs a topological structure diagram by extracting inherent process reference points of FPCA, estimates the nonlinear deformation field, and performs geometric correction on image block features to eliminate geometric misalignment caused by wrinkles, stretching, and bending of the flexible substrate. Then, orthogonal decoupling is performed on the corrected features, separating semantic direction components from interference amplitude components, retaining only the semantic direction components for matching, fundamentally eliminating the interference of illumination fluctuations and contrast changes on feature similarity calculation. The synergistic effect of these dual mechanisms significantly improves the robustness of the detection system, greatly reducing the false alarm rate in deformation scenarios.

[0016] 2. Significantly improve the detection capability of minute defects by using a functional area weighted memory and multi-scale feature fusion: Based on the process characteristics of FPCA, this application divides the image into multiple functional areas such as connectors, components, gold fingers, and reinforcement, and assigns higher weights to key areas to build a weighted memory, ensuring that the abnormal response of major defects such as connector pin offset and component omission is not drowned out by noise in non-critical areas; at the same time, multi-scale feature extraction and weighted fusion are adopted to capture minute defects such as gold finger burrs and pin micro-offsets through small-scale features, and capture macroscopic defects such as overall deformation and offset through large-scale features.

[0017] 3. Achieving rapid deployment and efficient inference, fully adapting to the small-batch, quick-change, and high-precision detection requirements of FPCA production lines: This application adopts an unsupervised learning framework, requiring only a small number of normal samples to construct a weighted memory, eliminating the need to collect a large number of defective samples for annotation and training. During line changes, only 10-20 normal samples are needed to complete deployment, reducing deployment time from several hours to minutes. Simultaneously, lightweight compression of the memory is achieved through core set sampling, combined with a feature-level registration strategy, controlling the inference time per image to within 200ms, meeting production line cycle time requirements. The method in this application significantly outperforms existing technologies in terms of detection rate, false alarm rate, and deployment efficiency, demonstrating significant industrial application value. Attached Figure Description

[0018] Figure 1 This is a flowchart of a multi-region targeting algorithm fusion detection method for FPCA in an embodiment of this application; Figure 2 This is a flowchart of the method for constructing a weighted memory in an embodiment of this application; Figure 3 This is a flowchart of the weighted similarity matching method in the embodiments of this application; Figure 4 This is a flowchart of the method for multi-scale feature fusion in the embodiments of this application. Detailed Implementation

[0019] The following is in conjunction with the appendix Figure 1-4 This application will be described in further detail.

[0020] This application discloses a multi-region targeting algorithm fusion detection method for FPCA. (Refer to...) Figure 1 A multi-region targeting algorithm fusion detection method for FPCA includes the following steps: S1. Deformation field estimation: Acquire an image of the flexible substrate to be inspected, automatically locate multiple process reference points in the image of the flexible substrate, construct an actual topology diagram based on the process reference points, and compare the actual topology diagram with a preset standard topology diagram to estimate the nonlinear deformation field; the process reference points include at least one of the following: connector pin center point, component pad corner point, gold finger endpoint, and reinforcement area boundary point. S2. Geometric correction: For multiple image block features extracted from the image of the flexible substrate to be detected, their spatial coordinates are transformed according to the nonlinear deformation field to obtain geometrically corrected image block features; S3. Feature Orthogonal Decoupling: Orthogonally decouple the geometrically corrected image patch features to obtain semantic direction components and interference amplitude components, and discard the interference amplitude components. S4. Weighted matching and multi-scale fusion: The semantic direction component is matched with a pre-constructed weighted memory using weighted similarity to obtain an anomaly score. The feature vector in the weighted memory is associated with a preset weight of its functional region. During matching, the preset weight is applied to the cosine similarity. The weighted similarity is defined as the product of the preset weight and the cosine similarity. S5. Output detection results: Based on the comparison between the anomaly score and the preset threshold, output the detection results.

[0021] The above method eliminates geometric misalignment of flexible substrates through deformation correction, eliminates illumination amplitude interference through feature decoupling, enhances sensitivity of key areas through weighted memory, and improves the detection capability of minute defects through multi-scale fusion. It effectively solves the problem of false alarms and missed detections caused by deformation and illumination interference in FPCA flexible substrate inspection, significantly improves detection accuracy and robustness, reduces false alarm rate, and enables rapid inference at the edge, which is fully adapted to the small-batch, quick-change, and high-precision inspection requirements of FPCA production lines.

[0022] The specific implementation methods of each of the above steps are explained in detail below.

[0023] Specifically, in step S1 above, FPCA (Flexible Printed Circuit Board Assembly), as a typical representative of flexible substrates, is prone to nonlinear deformations such as wrinkles, stretching, and bending during production and transportation. In order to eliminate the interference of deformation on subsequent detection, this application first estimates the deformation field of the image.

[0024] The process reference points include at least one of the following: connector pin center point, component pad corner point, gold finger endpoint, and reinforcement area boundary point. These reference points have the following characteristics: first, they are inherent process feature points of the FPCA product itself and do not require additional marking; second, they are uniformly distributed and can effectively characterize the overall deformation state of the flexible substrate; and third, they are sensitive to deformation and can accurately reflect local geometric changes.

[0025] In this step, the automatic positioning of process reference points can be achieved in the following ways: For connector pins, the pin contour is extracted through edge detection and morphological analysis, and the coordinates of its center point are calculated; for component pads, the pad corners are extracted through template matching or Harris corner detection; for gold finger endpoints and reinforcement area boundaries, the endpoint coordinates are extracted through edge detection and line fitting.

[0026] In one specific embodiment, the specific parameters of the above-mentioned automatically positioned process reference point are as follows: For connector pins: Canny edge detection is used with a low threshold of 50 and a high threshold of 150, and a Gaussian kernel size of 3×3; morphological analysis uses a closing operation with a 3×3 rectangular structuring element to fill the small holes inside the pin contour; after the pin contour is extracted, the centroid of the contour moments is calculated as the coordinates of the center point.

[0027] For component pads: Harris corner detection (corner response threshold 0.01, window size 3×3, neighborhood nonmaximum suppression distance 10 pixels) is used to extract pad corners; or normalized cross-correlation template matching (template size 32×32 pixels, matching threshold 0.85) is used to locate the pad area and then extract the corners.

[0028] For the endpoints of the gold finger and the boundary of the reinforcement area: Sobel edge detection (threshold range 50-150) is used to extract the edges, and Hough line detection (angle accuracy 1°, distance accuracy 1 pixel, accumulator threshold 20) is used to extract the line segments and obtain the endpoint coordinates.

[0029] The above parameter range is the optimal value determined by orthogonal experiments based on a 2048×2048 resolution image under standard lighting conditions. Those skilled in the art can fine-tune it according to the actual production line image resolution and lighting conditions, and the appropriate parameters can be determined through a limited number of experiments.

[0030] The specific method for constructing the actual topology graph is as follows: the extracted process reference points are used as graph nodes, and the geometric relationships between adjacent reference points (such as Euclidean distance and relative angles) are used as graph edges to form a topology graph representing the current image geometry. Preset standard topology diagram It is constructed in the same way as the standard pose FPCA product during the offline stage, and serves as a reference benchmark for deformation estimation.

[0031] Estimating the nonlinear deformation field specifically includes: using Thin Plate Spline (TPS) or Locally Weighted Regression (LWR) to calculate the nonlinear mapping relationship from the standard topology diagram to the actual topology diagram. The input (x, y) is the coordinates in the standard pose, and the output is... This represents the corresponding position of the coordinates in the image to be measured. This ensures that for each reference point... ,in This corresponds to the actual benchmark point.

[0032] Taking the thin-plate spline function as an example, its mathematical expression is: , in For radial basis functions, For standard reference point coordinates, For the parameters to be solved, minimize the bending energy function. Solve for the regularization parameter. To control the smoothness of the deformation field, this application takes... .

[0033] By constructing a topology and estimating the nonlinear deformation field using process reference points, the nonlinear deformations of FPCA, such as wrinkles, stretching, and bending, can be accurately characterized. This provides a precise deformation basis for subsequent geometric correction, eliminates the interference of geometric deformation on feature extraction and matching, and fundamentally solves the problem of false alarms caused by deformation in traditional methods.

[0034] In step S2 above, multiple image patch features are first extracted from the image to be detected. A pre-trained convolutional neural network is used as the feature encoder, such as a ResNet18 network pre-trained on the ImageNet dataset. The output of its penultimate layer is taken as the feature vector. The input image size is uniformly 224×224 pixels. Image patches are extracted by a sliding window method. The window size can be set as needed. This application will further explain the window size in conjunction with multi-scale feature fusion later.

[0035] When transforming the spatial coordinates of image block features, a nonlinear deformation field is applied to the center coordinates of each image block feature, mapping it to the corresponding position under the standard pose, thus obtaining the geometrically corrected image block features.

[0036] Specifically, assuming the center of the image patch feature has coordinates (x, y) in the original image, then after deformation field mapping, the coordinates (x', y') in the standard pose are obtained. In practical implementation, a reverse mapping strategy is adopted: for each target position in the standard pose, its corresponding position in the original image is calculated, and then feature values ​​are obtained through bilinear interpolation, thereby avoiding holes. The mapped feature vector is denoted as... , representing the feature representation under standard pose.

[0037] This step transforms the spatial coordinates of image patch features based on a nonlinear deformation field, mapping all features to a standard pose to achieve precise correction of geometric distortion and ensure the consistency and accuracy of subsequent feature matching.

[0038] In step S3 above, orthogonal decoupling of the geometrically corrected image patch features specifically includes: normalizing the geometrically corrected image patch features to obtain a unit vector as the semantic direction component, and using the magnitude of the geometrically corrected image patch features as the interference amplitude component.

[0039] For the geometrically corrected eigenvectors Its normalization formula is: ; in Represents the L2 norm. This application takes as a minimal constant This is used to prevent division by zero. The unit vector obtained after normalization. These are called semantic direction components, representing the semantic content of features (such as pin shape, pad outline, and texture features), and are independent of light intensity, contrast, etc.

[0040] At the same time, the magnitude of the feature vector As an interference amplitude component, since factors such as illumination fluctuations and flexible plate deformation mainly affect the amplitude of the feature vector, while having a relatively small impact on its direction, this application completely discards the interference amplitude component in subsequent matching and only uses the semantic direction component for similarity calculation, thereby fundamentally eliminating the interference of illumination and deformation on feature matching.

[0041] By separating semantic direction and interference amplitude components through feature orthogonal decoupling, and retaining only semantic direction for matching, this application can eliminate amplitude interference caused by changes in illumination and contrast, significantly reduce the false alarm rate, and improve the robustness and stability of feature matching.

[0042] In step S4 above, the first step is to construct a weighted memory. (Refer to...) Figure 2 The weighted memory is constructed through the following steps: A1. Normal Sample Feature Extraction: Acquire multiple standard pose normal sample images, extract image patch features, and decouple them through feature orthogonalization to obtain standard semantic direction components. Specifically, collect N (N is 1000 in this application) defect-free FPCA images in standard poses, and perform the above steps S1 to S3 on each image (where the deformation field in step S1 is approximately a unit mapping, i.e., no actual correction is performed), extract all image patch features, and decouple them to obtain a set of standard semantic direction components.

[0043] A2. Functional Region Division: Determine the functional region to which each standard semantic direction component belongs in the flexible substrate image. The functional regions include connector regions, component regions, gold finger regions, reinforcement regions, and general regions. Region division can employ an automatic segmentation method based on template matching: a position template is pre-established for each region, and its region is determined based on the coordinates of the image block's center.

[0044] A3. Regional Weight Allocation: A preset weight is assigned to each functional region, with the connector region having the highest weight, followed by the component region, and the general region having the lowest weight. In this application, the weights are set as follows: connector region weight w1 = 0.4, component region weight w2 = 0.3, gold finger region weight w3 = 0.2, reinforcement region weight w4 = 0.1, and general region weight w5 = 0.05. The weight values ​​can be adjusted according to the actual application scenario, but the basic principle of maintaining the highest weight for the connector region and the lowest weight for the general region remains unchanged.

[0045] A4. Associating and Storing Weights: Based on the preset weights, each standard semantic direction component is associated with the weight coefficient of its corresponding functional region, forming a weighted memory. Each feature vector stores its vector value and its region weight coefficient (i.e., the preset weight) in the memory (preferred in this embodiment: connector 0.4, component 0.3, gold finger 0.2, reinforcement 0.1, ordinary 0.05). This region weight coefficient is the w in the subsequent weighted matching formula. r This is for use in subsequent weighted matching.

[0046] A5. Core Set Sampling: To reduce the storage size and inference time of the weighted memory, the weighted memory is lightweighted and compressed. This application employs a K-center greedy sampling algorithm based on cosine distance, with the specific steps as follows: (1) Randomly select an eigenvector from the weighted memory as the first center point; (2) For each remaining feature vector, calculate its minimum distance to the selected center point. ); (3) Select the feature vector with the largest minimum distance as the next center point; (4) Repeat steps (2)-(3) until the number of selected center points reaches the preset target (e.g., 20% of the original memory bank size).

[0047] The retained center points form a lightweight memory for subsequent nearest neighbor searches. This algorithm can preserve the distribution of the original feature space to the greatest extent, and combined with the FAISS approximate nearest neighbor search library, the inference time for a single image is controlled within 200ms.

[0048] During the detection phase, the semantic direction components obtained from steps S1-S3 of the image to be detected are matched with the weighted memory. (Refer to...) Figure 3 The specific steps of performing weighted similarity matching between the semantic direction components and the pre-built weighted memory include: B1. Calculate the weighted similarity: Calculate the cosine similarity between the semantic direction component and each feature vector in the weighted memory, and then multiply this similarity by the region weight corresponding to the feature vector to obtain the weighted similarity. In this application, the weighted similarity is defined as the product of the region weight (preset weight) and the cosine similarity: ; in, Cosine similarity (value range [-1, 1]). The semantic direction component represents the feature of the image patch to be detected. The feature vectors in the weighted memory, The preset weights for the functional regions to which the feature vector belongs (in this embodiment, the value range is [0.05, 0.4]) are used, so the weighted similarity range is [-0.4, 0.4]. By multiplying by the region weights, the weighted similarity of key regions (e.g., weight 0.4) is amplified, while that of ordinary regions (e.g., weight 0.05) is suppressed. Thus, when the maximum value is taken later, minor anomalies in key regions are more easily detected.

[0049] B2. Selecting the nearest neighbor: Select the feature vector corresponding to the maximum weighted similarity as the nearest neighbor, that is: ; in, It is a weighted memory.

[0050] B3. Obtain the anomaly score: Convert the maximum weighted similarity value into cosine distance, which is used as the anomaly score of the image patch features corresponding to the semantic direction component. , The abnormal score ranges from [0,2]. The larger the value, the greater the difference between the features of the image patch and all normal features in the memory, and the more likely it is to be a defective area.

[0051] Furthermore, the anomaly score in step S4 above is the fusion result of multi-scale anomaly scores. (Refer to...) Figure 4 Specifically, it can be obtained through the following methods: C1. Multi-scale Feature Extraction: Before performing step S2, image patch features at multiple scales are extracted from the image of the flexible substrate to be inspected. These multiple scales include a large scale for capturing macroscopic defects, a medium scale for capturing conventional defects, and a small scale for capturing microscopic defects. In this application, the large-scale image patch is 11×11 pixels in size with a step size of 8 pixels, used to capture macroscopic defects such as overall deformation and offset; the medium-scale image patch is 7×7 pixels in size with a step size of 4 pixels, used to capture conventionally sized defects; and the small-scale image patch is 3×3 pixels in size with a step size of 2 pixels, used to capture minute defects such as gold finger burrs and pin micro-offsets.

[0052] C2. Anomaly score calculation at each scale: Perform steps S2 to S4 for the image patch features at each scale to obtain the anomaly score at each scale. ,in The anomaly score maps for each scale have different resolutions, as follows: Large scale (11×11, step size 8): approximately 28×28 grid; Mesoscale (7×7, step size 4): approximately 55×55 grid; Small scale (3×3, step size 2): approximately 111×111 grid.

[0053] Because the center point coordinate grids of image patches differ at different scales (8 pixels for large scale, 4 pixels for medium scale, and 2 pixels for small scale), direct fusion would lead to spatial misalignment. This application aligns the anomaly score maps at each scale by upsampling them to the original image size (224×224 pixels) using bilinear interpolation, ensuring all scales have the same resolution. Specifically, for the large-scale anomaly score map (size (224-11) / 8+1 ≈ 28×28), each score corresponds to the center position of an image patch in the original image, and bilinear interpolation is used to estimate the anomaly score at each pixel position. The same applies to medium and small scales. After upsampling, all scales of the anomaly score map are 224×224 pixels, pixel-level corresponding, allowing for direct weighted fusion.

[0054] C3. Scale Fusion: Anomaly scores from multiple scales are weighted and fused to obtain the final anomaly score. In this application, a linear weighting method is used for fusion, which involves multiplying the anomaly scores at each scale by their corresponding weights and then summing the results. For each pixel location (u, v), the fusion formula is: ; in For scale weights, the subscript 's' indicates the scale type. These correspond to large-scale, medium-scale, and small-scale, respectively. This is the pixel-level anomaly score map after upsampling. In this application, the importance of each scale for defect detection is set as follows: This is to highlight the detection contribution of minute defects. The final image-level anomaly score is the maximum value or quantile of all fused anomaly scores.

[0055] By fusing the above-mentioned multi-scale features, this application can take into account the detection needs of both macroscopic and microscopic defects, significantly improve the detection rate of minute defects, and effectively solve the problem of serious missed detection of minute defects such as gold finger burrs and pin micro-deviations by traditional single-scale methods.

[0056] This step employs a functional area weighted memory and weighted similarity matching to enhance the abnormal response of key functional areas such as connectors and components. It combines multi-scale feature fusion to take into account both macroscopic and micro-defect detection, thereby comprehensively improving the defect detection rate.

[0057] In step S5 above, the output detection results specifically include: When the anomaly score is greater than or equal to the preset threshold, the defect detection result is output and an anomaly heatmap is generated. The anomaly heatmap is used to mark the abnormal areas in the image.

[0058] The preset threshold is determined using the training set: when constructing the weighted memory, a portion of normal samples is reserved as a validation set. The image-level anomaly score for each validation sample is calculated (taking the maximum value of all image patch anomaly scores). Then, the 99.5th percentile of the validation set anomaly scores is taken as the detection threshold T. When the image-level anomaly score of the image to be tested... If so, it is determined to be a defective product.

[0059] The abnormal heatmap is generated by mapping the abnormal score of each image block back to its position in the original image to form a score matrix with the same size as the image. Then, Gaussian filtering is performed to smooth it. Finally, color mapping (such as blue for normal and red for abnormal) is used to generate a visual heatmap, which makes it easy for operators to quickly locate the defect.

[0060] This step achieves automatic defect identification and visual location by judging the anomaly score threshold and outputting the anomaly heatmap. It provides operators with intuitive defect location information, significantly improves the interpretability of industrial quality inspection and the efficiency of human-machine collaboration, and meets the actual needs of industrial production lines for rapid identification and accurate location.

[0061] The technical solution of this application is verified through specific embodiments below. A normal FPCA image in a standard pose is selected as input, with an image size of 2048×2048 pixels. Processing is performed according to the steps S1-S5 described above: S1: Extract 28 connector pin center points, 36 component pad corner points, 8 gold finger endpoints, and 12 reinforcement area boundary points, for a total of 84 process reference points, construct the actual topology diagram, and estimate the nonlinear deformation field after comparing it with the standard topology diagram. Since the input is a standard pose, the deformation field is approximately a unit mapping. S2: ResNet18 was used to extract multi-scale image patch features, a total of 1024 large-scale features, 4096 medium-scale features, and 16384 small-scale features were extracted. After deformation field correction, the feature coordinates remained basically unchanged. S3: Normalize and decouple each feature vector to obtain semantic direction components; S4: Match the semantic direction components with the weighted memory and calculate the weighted similarity. All image patch anomaly scores are less than the threshold T=0.15. S5: The image-level anomaly score is 0.08, which is less than the threshold, so the output result is normal.

[0062] An FPCA image showing a connector pin offset is selected as input. After processing, the anomaly score at the image block corresponding to the offset pin increases significantly, with the maximum anomaly score reaching 0.42, exceeding the threshold of 0.15, and is judged as a defect. At the same time, an anomaly heatmap is generated, with the red area accurately indicating the position of the offset pin.

[0063] Select an FPCA image with missing components as input. In the area corresponding to the missing components, due to the lack of normal pads and component outlines, the feature direction deviates significantly from the normal features in the memory bank, and the anomaly score reaches 0.38, which is judged as a defect.

[0064] An FPCA image with tiny burrs on the edge of the gold finger was selected as input. Small-scale features (3×3) produced a high anomaly score (0.35) in this area, while large-scale features had a lower anomaly score (0.12). After multi-scale fusion, the final anomaly score was 0.29, and the tiny defect was successfully detected. If only single-scale features are used, the tiny defect may be missed.

[0065] To verify the technical effectiveness of this application, comparative experiments were conducted on a test set containing 500 normal samples and 200 defective samples (including connector pin misalignment, missing components, burrs on gold fingers, and missing reinforcement). The experimental environment consisted of an Intel Core i7-12700K CPU, an NVIDIA RTX 3060 GPU, and Ubuntu 20.04 operating system. The native Patch, Core (Euclidean distance) method achieved an overall detection rate of 86.5%, a minor defect detection rate of 61.2%, a false positive rate of 8.3%, and a single-sample inference time of 320ms. The traditional threshold segmentation method achieved an overall detection rate of 72.3%, a minor defect detection rate of 45.6%, a false positive rate of 15.7%, and a single-sample inference time of 120ms. The method of this application achieved an overall detection rate of 97.8%, a minor defect detection rate of 95.3%, a false positive rate reduced to 2.1%, and a single-sample inference time of 186ms. Experimental results show that the method proposed in this application is significantly superior to the existing technology in key indicators such as overall detection rate, minor defect detection rate, and false alarm rate. At the same time, the inference speed meets the production line cycle time requirements (<200ms / sheet), and has good industrial application value.

[0066] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A multi-region targeting algorithm fusion detection method for FPCA, characterized in that, Includes the following steps: S1: Acquire an image of the flexible substrate to be detected, automatically locate multiple process reference points in the image of the flexible substrate, construct an actual topology diagram based on the process reference points, and compare the actual topology diagram with a preset standard topology diagram to estimate the nonlinear deformation field. S2: For multiple image block features extracted from the image of the flexible substrate to be detected, their spatial coordinates are transformed according to the nonlinear deformation field to obtain geometrically corrected image block features; S3: Perform orthogonal decoupling on the geometrically corrected image block features to obtain semantic direction components and interference amplitude components, and discard the interference amplitude components; S4: Perform weighted similarity matching between the semantic direction component and the pre-constructed weighted memory to obtain an anomaly score. The feature vector in the weighted memory is associated with the preset weight of its functional region. During matching, the preset weight is applied to the cosine similarity. The weighted similarity is defined as the product of the preset weight and the cosine similarity. S5: Output the detection result based on the comparison between the anomaly score and the preset threshold.

2. The multi-region targeting algorithm fusion detection method for FPCA according to claim 1, characterized in that, The process reference points include at least one of the following: connector pin center point, component pad corner point, gold finger endpoint, and reinforcement area boundary point.

3. The multi-region targeting algorithm fusion detection method for FPCA according to claim 1, characterized in that, Estimating the nonlinear deformation field specifically includes: The nonlinear mapping relationship from the standard topology diagram to the actual topology diagram is calculated using thin-plate spline functions or local weighted regression.

4. The multi-region targeting algorithm fusion detection method for FPCA according to claim 1, characterized in that, In step S2, transforming the spatial coordinates of the image patch features specifically includes: The nonlinear deformation field is applied to the center coordinates of each image block feature, and mapped to the corresponding position under the standard pose to obtain the geometrically corrected image block features.

5. The multi-region targeting algorithm fusion detection method for FPCA according to claim 1, characterized in that, Step S3, specifically the orthogonal decoupling of the geometrically corrected image patch features, includes: The geometrically corrected image patch features are normalized to obtain a unit vector as the semantic direction component, and the magnitude of the geometrically corrected image patch features is used as the interference amplitude component.

6. The multi-region targeting algorithm fusion detection method for FPCA according to claim 1, characterized in that, The weighted memory is constructed through the following steps: Multiple standard pose normal sample images are acquired, image patch features are extracted and orthogonally decoupled to obtain standard semantic direction components; Determine the functional region to which each standard semantic direction component belongs in the flexible substrate image. The functional regions include connector regions, component regions, gold finger regions, reinforcement regions, and general regions. Each functional area is assigned a preset weight, with the connector area having the highest weight, the component area having the second highest weight, and the ordinary area having the lowest weight. Based on the preset weights, the corresponding region weights are associated with the standard semantic direction components to form a weighted memory.

7. The multi-region targeting algorithm fusion detection method for FPCA according to claim 6, characterized in that, In step S4, performing weighted similarity matching between the semantic direction component and the pre-built weighted memory specifically includes: Calculate the cosine similarity between the semantic direction component and each feature vector in the weighted memory, and multiply the similarity by the region weight corresponding to the feature vector to obtain the weighted similarity; Select the feature vector corresponding to the maximum weighted similarity as the nearest neighbor; The maximum weighted similarity is converted into cosine distance and used as the anomaly score of the image patch features corresponding to the semantic direction component.

8. The multi-region targeting algorithm fusion detection method for FPCA according to claim 1, characterized in that, The anomaly score mentioned in step S4 is the fusion result of multi-scale anomaly scores, specifically obtained through the following method: Before performing step S2, image patch features of multiple scales are extracted from the image of the flexible substrate to be detected; Steps S2 to S4 are performed on the image patch features at each scale to obtain the anomaly score at each scale; The anomaly scores from multiple scales are aligned by coordinates and then weighted and fused to obtain the final anomaly score.

9. A multi-region targeting algorithm fusion detection method for FPCA according to claim 8, characterized in that, The multiple scales include large scales for capturing macroscopic defects, medium scales for capturing conventional defects, and small scales for capturing microscopic defects.

10. The multi-region targeting algorithm fusion detection method for FPCA according to claim 1, characterized in that, In step S5, the output detection results specifically include: When the anomaly score is greater than or equal to the preset threshold, the defect detection result is output and an anomaly heatmap is generated. The anomaly heatmap is used to mark the abnormal areas in the image.

Images