Flexible circuit board intelligent detection method and system based on machine vision

Abstract

The application discloses a kind of flexible circuit board intelligent detection method and system based on machine vision, it is related to circuit board detection field, first, by global coarse registration step, the rigid transformation such as translation, rotation and scaling of the whole of the FPC image to be detected is corrected quickly, preliminary alignment with standard template is realized.Subsequently, for the nonlinear deformation such as local stretching, wrinkle caused by FPC material characteristics, local fine registration is introduced, to accurately perceive and compensate these subtle, non-uniform local misplacement, realize the accurate matching between the image to be detected and template nearly pixel level.Based on this high-precision alignment, difference attention defect segmentation is carried out again, can effectively eliminate the false difference generated due to deformation registration inaccuracy, to accurately identify real defects, significantly reduce false positive rate, greatly improve the reliability and practicality of detection scheme.

Description

Technical Field

[0001] This application relates to the field of circuit board inspection, and more specifically, to a machine vision-based intelligent inspection method and system for flexible circuit boards. Background Technology

[0002] Flexible printed circuit boards (FPCs), due to their thinness and flexibility, have become an indispensable core component in modern precision electronic products such as smartphones and wearable devices. With the rapid development of electronic products towards high density and miniaturization, extremely stringent requirements have been placed on the wiring accuracy and manufacturing quality of FPCs. Any minor defect, such as a broken circuit, short circuit, or scratch, can lead to the failure of the entire device. Traditional manual visual inspection methods are inefficient, subjective, and prone to fatigue errors, and can no longer meet the needs of large-scale, high-standard production. Therefore, developing automated and intelligent FPC defect detection solutions based on machine vision has become an inevitable trend for improving product quality and production efficiency.

[0003] However, in current machine vision inspection practices, defects are typically identified by comparing the image of the FPC to be inspected with a pre-stored standard template image. The core premise of this method is achieving pixel-level precise registration between the two images. Existing technologies mostly employ alignment based on global transformation models (such as affine or perspective transformations). These methods achieve good results for rigid circuit boards (PCBs), but face significant challenges when applied to FPCs. Due to insufficient registration accuracy, even for qualified FPC products, numerous false differences arise after comparing their images with the template image, leading to serious false alarms in the inspection system and greatly affecting the reliability and practicality of the inspection. The root cause of these problems lies in the non-rigid deformation characteristics of FPCs. The substrate of FPCs (such as polyimide) itself has elasticity and ductility. During the conveying, gripping, and positioning processes of automated production lines, it inevitably experiences uneven stress, resulting in minute, non-uniform stretching, shrinkage, twisting, and even localized bulging or wrinkling. This deformation is local and non-linear, and cannot be completely and accurately described by a single global transformation matrix. Forcibly applying a global registration model often results in aligning region A of the image but potentially exacerbating the misalignment in region B.

[0004] Therefore, how to effectively sense and compensate for the local nonlinear deformation generated by the FPC during the imaging process and achieve high-precision registration under non-rigid conditions has become a key technical bottleneck in solving the problem of high false alarm rate in FPC visual inspection, and an innovative solution is urgently needed. Summary of the Invention

[0005] To address the problems in the prior art, according to one aspect of this application, a machine vision-based intelligent inspection method for flexible circuit boards is provided, comprising:

[0006] In response to the transmission belt in place signal, acquire the original FPC image to be inspected and load the standard template image;

[0007] Global coarse registration is performed between the original FPC image to be inspected and the standard template image to obtain a globally aligned FPC image to be inspected.

[0008] Based on the standard template image, local fine registration is performed on the globally aligned FPC image to be inspected to obtain a finely aligned FPC image to be inspected.

[0009] Based on the standard template image, differential attention defect segmentation is performed on the finely aligned FPC image to be inspected to obtain the defect probability mask.

[0010] Defect analysis is performed based on the defect probability mask and the original FPC image to be inspected to obtain a defect report.

[0011] According to another aspect of this application, a machine vision-based intelligent inspection system for flexible circuit boards is provided, comprising:

[0012] The original standard image acquisition module is used to acquire the original FPC image to be inspected and load the standard template image in response to the transmission belt in place signal;

[0013] The global image alignment module is used to perform global coarse registration between the original FPC image to be inspected and the standard template image to obtain a globally aligned FPC image to be inspected.

[0014] The fine alignment module is used to perform local fine registration on the globally aligned FPC image to obtain a finely aligned FPC image based on a standard template image.

[0015] The defect segmentation module is used to perform differential attention defect segmentation on finely aligned FPC images to obtain a defect probability mask based on standard template images.

[0016] The defect analysis module is used to perform defect analysis based on the defect probability mask and the original FPC image to be inspected in order to obtain a defect report.

[0017] Compared with existing technologies, this application provides a machine vision-based intelligent inspection method and system for flexible printed circuit boards (FPCs). It proposes a scheme combining global coarse registration and local fine registration to fundamentally solve the problem of false alarms caused by non-rigid deformation of FPCs. First, a global coarse registration step quickly corrects rigid transformations such as translation, rotation, and scaling of the overall FPC image under inspection, achieving initial alignment with a standard template. Then, a local fine registration step is introduced to address the non-linear deformations such as local stretching and wrinkling caused by the material properties of FPCs, as mentioned in the background technology. This step can accurately perceive and compensate for these subtle, non-uniform local misalignments, achieving near-pixel-level precise matching between the image under inspection and the template. Based on this high-precision alignment, differential attention defect segmentation is then performed, effectively eliminating spurious differences caused by inaccurate deformation registration, thereby accurately identifying real defects, significantly reducing the false alarm rate, and greatly improving the reliability and practicality of the inspection scheme. Attached Figure Description

[0018] The above and other objects, features and advantages of this application will become more apparent from the more detailed description of the embodiments of this application in conjunction with the accompanying drawings.

[0019] Figure 1 This is a flowchart of a machine vision-based intelligent inspection method for flexible circuit boards according to an embodiment of this application.

[0020] Figure 2 This is a schematic diagram of the data flow in a machine vision-based intelligent inspection method for flexible circuit boards according to an embodiment of this application.

[0021] Figure 3 This is a flowchart of step S3 in the machine vision-based intelligent inspection method for flexible circuit boards according to an embodiment of this application.

[0022] Figure 4 This is a flowchart of step S4 in the machine vision-based intelligent inspection method for flexible circuit boards according to an embodiment of this application.

[0023] Figure 5 This is a block diagram of a machine vision-based intelligent inspection system for flexible circuit boards according to an embodiment of this application. Detailed Implementation

[0024] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. It should be understood that the drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0025] To address the technical challenges mentioned above, this application proposes a machine vision-based intelligent inspection method for flexible circuit boards. Figure 1This is a flowchart of a machine vision-based intelligent inspection method for flexible circuit boards according to an embodiment of this application. Figure 2 This is a schematic diagram of the data flow in a machine vision-based intelligent inspection method for flexible circuit boards according to an embodiment of this application. Figure 1 and Figure 2 As shown, the machine vision-based intelligent inspection method for flexible circuit boards according to an embodiment of this application includes: S1, in response to a conveyor belt in place signal, acquiring an original FPC image to be inspected and loading a standard template image; S2, performing global coarse registration on the original FPC image to be inspected and the standard template image to obtain a globally aligned FPC image to be inspected; S3, based on the standard template image, performing local fine registration on the globally aligned FPC image to be inspected to obtain a finely aligned FPC image to be inspected; S4, based on the standard template image, performing differential attention defect segmentation on the finely aligned FPC image to be inspected to obtain a defect probability mask; S5, performing defect analysis based on the defect probability mask and the original FPC image to be inspected to obtain a defect report.

[0026] In step S1, in response to the conveyor belt arrival signal, the original FPC image to be inspected and the standard template image are acquired. It should be understood that on an automated production line, flexible circuit boards are continuously transported to the inspection station via a conveyor belt. To ensure that each image acquisition is performed when the FPC is in the optimal shooting position, avoiding image quality degradation due to positional offset or motion blur, and to establish a unified benchmark for all subsequent comparison-based analyses, this application first establishes a precise and synchronous data acquisition starting point. By acquiring the real-time image of the FPC to be inspected as the inspection object and simultaneously retrieving a pre-made, flawless standard template image as the comparison benchmark, high-quality, homogeneous input data is provided for subsequent registration and defect analysis stages.

[0027] In an exemplary operation, the specific process of step S1 is as follows: The execution environment of this step is deployed on an FPC automated production line, where an industrial camera is vertically installed above the inspection station and equipped with a highly uniform coaxial ring light source, and a photoelectric sensor is installed next to the designated shooting position of the conveyor belt.

[0028] The specific execution process of this step is as follows: First, when an FPC to be inspected enters the inspection station along the conveyor belt, its leading edge blocks the optical path of the photoelectric sensor, causing the sensor to output a pulse signal that transitions from a low level to a high level. This signal is the conveyor belt arrival signal. This signal is sent to an industrial control computer as a hardware trigger command. Upon receiving the rising edge of this arrival signal, the industrial control computer immediately sends a single acquisition command to the industrial camera. For example, a 20-megapixel monochrome area array industrial camera with a resolution set to 5120×3840 pixels can be used to ensure clear capture of micron-level line details. To prevent image blurring caused by minor vibrations or movement of the conveyor belt, the camera's exposure time is set to an extremely short value, such as 100 microseconds, while the gain is set to 0 dB to obtain a clean image. After the camera completes the exposure, the generated raw image data is transmitted at high speed to the memory of the industrial control computer through the data interface. This unprocessed 8-bit grayscale image with a resolution of 5120×3840 is the raw FPC image to be inspected. It refers to the digital image of the FPC located at the current inspection station, which is captured in real time by an industrial camera.

[0029] Simultaneously with triggering the camera to acquire an image, or during image data transmission, the industrial control computer's inspection program loads a standard template image. The standard template image is a pre-acquired and stored digital image of a gold sample FPC that contains no manufacturing defects; it is created under ideal conditions. The specific creation process is as follows: an FPC, confirmed to be flawless by multiple parties, is selected as the gold sample. Under vibration-free and perfectly uniform lighting conditions, multiple images are taken using the same camera and lens as the online inspection station. These images are then averaged to minimize random noise, ultimately generating a high-quality standard template image. This image has the exact same dimensions as the original FPC image to be inspected, such as 5120×3840 pixels, and a bit depth, such as 8-bit grayscale. Depending on the FPC model being processed on the production line, for example, FX-001, the inspection program reads the corresponding file named FX-001_template.bmp from the template library on the local hard drive and loads its data into memory.

[0030] In step S2, a global coarse registration is performed between the original FPC image to be inspected and the standard template image to obtain a globally aligned FPC image to be inspected. Correspondingly, in automated production processes, the FPC image to be inspected inevitably exhibits global geometric deviations compared to the standard template image, such as overall positional translation, angular rotation, and scale scaling caused by minor changes in object distance. These macroscopic inconsistencies severely interfere with subsequent fine registration steps aimed at handling local nonlinear deformations and may directly lead to the failure of defect analysis. Therefore, this application uses a fast and robust global transformation model to eliminate these major rigid or affine transformation differences before pixel-level detail processing, aligning the image to be inspected globally to the coordinate system of the template image. This provides a reference image with consistent posture and scale for subsequent processing, significantly reducing the difficulty and computational complexity of subsequent local fine registration.

[0031] In an exemplary operation, step S2, which involves globally coarsely registering the original FPC image to be inspected and the standard template image to obtain a globally aligned FPC image to be inspected, includes: S21, extracting feature points from the original FPC image to be inspected and the standard template image to obtain a sample keypoint set and a template keypoint set; S22, performing feature matching and transformation matrix estimation on the sample keypoint set and the template keypoint set to obtain a homography matrix; and S23, performing perspective transformation on the original FPC image to be inspected based on the homography matrix to obtain the globally aligned FPC image to be inspected.

[0032] In the above exemplary operation, the specific process of step S2 is as follows: The first step is to execute S21. To achieve efficient feature point extraction that is robust to rotation and scale changes, this embodiment uses the ORB algorithm. Specifically, an ORB feature detector instance is initialized, and its key parameters are precisely configured to adapt to the fine texture characteristics of the FPC image. For example, the maximum number of feature points to be detected is set to 2000. This value aims to ensure a balance between the textured, densely packed lines on the FPC and the relatively sparsely textured substrate area, so as to obtain enough features for robust matching while controlling the computational load. The scale factor of the image pyramid is set to 1.2, and the number of pyramid layers is set to 8. This configuration constructs a multi-scale image space from the original resolution to progressively downsampled images to ensure that the algorithm can stably detect and match features that are insensitive to scale changes at different shooting distances. The detector first processes the original FPC image to be inspected. It runs the FAST corner detection algorithm independently on each layer of the constructed image pyramid. This algorithm efficiently identifies corner points as initial keypoints by checking whether there are consecutive pixels (e.g., 9 pixels) in the neighborhood of a pixel (a circular area with a radius of 3 containing 16 pixels) whose gray values ​​differ from the center point's gray value simultaneously by a predetermined number or a predetermined threshold (e.g., 20). To further improve the quality of keypoints and prevent them from over-clustering in strong edge or corner areas, non-maximum suppression is applied to the detected initial keypoints. This involves calculating the response score for each corner point and retaining only the corner point with the highest response value in its local neighborhood, resulting in more evenly distributed and accurately located final keypoints. Subsequently, to impart rotational invariance to these detected keypoints, an intensity centroid method is used to calculate a clear principal direction for them. Specifically, within a circular image patch centered on the keypoint, its zeroth-order moment (the sum of the gray values ​​of all pixels within the image patch) and its first-order gray-level moment (the sum of the products of the pixel's gray value and its coordinates) are calculated. The intensity centroid coordinates of the image patch can be accurately calculated using the ratio of the first-order moment to the zeroth-order moment. A vector pointing from the geometric center of the keypoint to its intensity centroid is constructed, and the direction of this vector is uniquely defined as the principal direction of the keypoint. Next, based on the position, scale, and determined principal direction of each keypoint, a rotation-aware BRIEF descriptor is used for feature description. The descriptor first rotates a 31×31 pixel image patch in the neighborhood of the keypoint according to its principal direction, aligning the principal direction of the image patch to a standard direction, thus effectively eliminating the influence of rotation on the descriptor calculation. Then, within this rotation-corrected image patch, according to a fixed sampling pattern of 256 point pairs obtained through optimized learning with high variance and low correlation, the gray values ​​of each pair of sampled points are compared one by one. If the gray value of the first point is greater than that of the second point, the binary result is 1; otherwise, it is 0.These 256 independent comparison results are concatenated in a predetermined order to form a 256-bit binary string, which serves as the unique descriptor for each keypoint. After processing all keypoints, two data structures are obtained: the first is the sample keypoint set, a list containing all detected keypoint objects, each encapsulating its precise floating-point coordinates, pyramid level (representing scale), and principal orientation angle; the second is the sample descriptor subset, a 2000×32-byte numerical matrix where each row serves as an independent 256-bit binary descriptor, corresponding one-to-one with a keypoint in the sample keypoint set. The same process is then performed on the standard template image to obtain the corresponding template keypoint set and template descriptor subset. In particular, the aforementioned preset values, such as the maximum number of feature points to be detected, the scale factor and number of layers of the pyramid, and the FAST corner detection threshold, were all determined through extensive empirical testing and systematic optimization on a large number of representative FPC image sample sets. The purpose is to achieve the best balance between the stability of feature detection, the accuracy of matching, and the real-time performance of computation for a specific FPC detection application.

[0033] The second step is S22. Since the descriptors generated by the ORB algorithm are binary strings, the computationally efficient Hamming distance is used as the similarity metric. This embodiment employs a brute-force matcher for exhaustive feature matching. This matcher is called a brute-force matcher because it performs a comprehensive, exhaustive comparison: it iterates through every descriptor in the sample descriptor set and calculates its Hamming distance to all descriptors in the template descriptor set. This distance is efficiently obtained by XORing two 256-bit binary strings and counting the number of 1s in the result. For each sample descriptor, the matcher finds the template descriptor with the smallest Hamming distance and uses this pair as an initial matching pair. Since the sample keypoint set contains 2000 keypoints, this process generates an initial list of 2000 best matching pairs. However, this initial list inevitably contains erroneous matches caused by duplicate textures or noise. To remove these unreliable matches, a distance thresholding method is used to filter the list. A Hamming distance threshold, such as 64, is set, meaning that for a 256-bit BRIEF descriptor, the allowed difference in bits does not exceed 25%. Only when the Hamming distance of a matching pair is less than this threshold is it considered a reliable match and retained, thus forming a reduced but higher-quality list of matching pairs. Based on this high-quality list of matching pairs, the pixel coordinates of all successfully matched keypoints in their respective images are extracted, constructing a source point set (from the image to be tested) and a target point set (from the template image). To robustly estimate the homography matrix describing the global perspective transformation relationship between the two images from these point pairs, which may still contain a small number of false matches, the Random Sample Consensus (RANSAC) algorithm is used. This threshold, and the parameters in the subsequent RANSAC algorithm, are determined through experimental optimization on a representative FPC image validation set, aiming to maximize the retention rate of correct matches while minimizing the introduction of false matches. The algorithm performs the following operations within a preset number of iterations, such as 1000: In each iteration, four point pairs are randomly selected without replacement from the matching point pairs. This is because a complete perspective transformation has eight degrees of freedom, and each pair of matching points can provide two linear equations. Therefore, four pairs of points are just enough to solve a unique 3×3 homography matrix. Using the coordinates of these four point pairs, a system of linear equations is constructed and solved using the Direct Linear Transformation (DLT) algorithm to obtain a candidate homography matrix. Using this candidate matrix, all points in the source point set are transformed to the coordinate space of the template image, and the Euclidean distance between each transformed point and its corresponding point in the target point set is calculated one by one. This distance is the reprojection error. A reprojection error threshold is set, such as 3.0 pixels. If the reprojection error of a point pair is less than this threshold, it indicates that the point pair matches the current candidate transformation model and is classified as an interior point. The number of all interior points in this iteration is counted.After all iterations are completed, the iteration that yields the most inliers is selected, as it represents the most statistically advantageous global transformation model in the data. To further improve accuracy and stability, the homography matrix is ​​recalculated using the direct linear transformation algorithm, utilizing all inlier pairs identified by this optimal model (a more reliable dataset with far more than four inliers). This final, more accurate 3×3 matrix, incorporating all consistency information, is the final output homography matrix of this step.

[0034] Finally, execute S23. To avoid holes or multiple pixels mapping to the same position in the transformed image, this step uses inverse mapping for perspective transformation, ensuring that each pixel in the output image has a unique and definite source value. First, create a blank 8-bit grayscale target image with the exact same size as the standard template image (5120×3840 pixels). This image is the final globally aligned FPC image to be inspected. Then, iterate through the coordinates (x_a, y_a) of each pixel in the blank target image. For each target pixel coordinate, first represent it as a homogeneous coordinate vector [x_a, y_a, 1]^T. Next, use the inverse of the 3x3 homography matrix calculated in the previous step to multiply the homogeneous coordinate vector on the left, thereby resolving its corresponding homogeneous source coordinates [x_s', y_s', w_s']^T in the coordinate system of the original FPC image to be inspected. The homogeneous source coordinates are transformed back to Cartesian coordinates to obtain the precise floating-point position (x_s, y_s) of the source pixel, where x_s = x_s' / w_s' and y_s = y_s' / w_s'. After the coordinate transformation, a boundary check is performed to determine whether the calculated precise floating-point position (x_s, y_s) of the source pixel is within the valid size range of the original FPC image to be inspected. If the coordinates are out of bounds, a preset background fill value, such as 0 (representing black), is directly assigned to the current pixel (x_a, y_a) in the target image, and then the subsequent interpolation steps are skipped to continue processing the next target pixel. The preset parameters, such as the background fill value, are pre-set according to the sensitivity of the subsequent algorithm to the background area and the visual presentation requirements to ensure that the blank area introduced by the transformation does not interfere with the defect analysis. If the source coordinates are within the valid range, a bilinear interpolation algorithm is used to determine the pixel grayscale value at that position. The algorithm first determines four integer pixel coordinates surrounding the source coordinates (x_s, y_s): the top-left corner (coordinates: (floor(x_s), floor(y_s)), the top-right corner (coordinates: (floor(x_s)+1, floor(y_s)), the bottom-left corner (coordinates: (floor(x_s), floor(y_s)+1), and the bottom-right corner (coordinates: (floor(x_s)+1, floor(y_s)+1)). Simultaneously, it calculates the decimal parts of the source coordinates in both the horizontal and vertical directions as weighting factors. The interpolation process is performed in two steps: First, in the horizontal direction, linear interpolation is performed on the grayscale values ​​of the two points above and below using the horizontal weighting factor, obtaining two intermediate values. Then, in the vertical direction, linear interpolation is performed on the two intermediate values ​​obtained earlier using the vertical weighting factor, ultimately calculating a precise interpolation result. The grayscale value obtained through bilinear interpolation is then assigned to the pixel at coordinates (x_a, y_a) in the target image. Once all pixels in the target image have undergone this assignment, the entire perspective transformation process is complete.The target image generated at this time is the globally aligned FPC image to be inspected, which is aligned with the standard template image in terms of overall pose and scale.

[0035] In step S3, based on the standard template image, a fine local registration is performed on the globally aligned FPC image to obtain a finely aligned FPC image. It is understood that after global coarse registration, the FPC image to be inspected and the standard template are basically aligned in overall pose and scale. However, this does not solve the core problem described in the background art: the local, nonlinear deformation caused by the flexible nature of the FPC material. These minute stretches, twists, or wrinkles still exist as residual misalignments under the global transformation model. If defect comparison is performed directly, these difference areas caused by deformation rather than defects will be incorrectly identified as defects, resulting in a large number of false alarms. Therefore, this application uses a deep learning method capable of accurately modeling pixel-level displacement to calculate a dense, non-rigid motion vector field, i.e., an optical flow field, describing the transition from the globally aligned FPC image to the standard template image. Based on this optical flow field, fine local correction is performed on the image to achieve near-perfect pixel-level alignment between the two images, thereby completely eliminating the spurious differences introduced by non-rigid deformation and laying the foundation for subsequent high-precision defect segmentation.

[0036] In one exemplary operation Figure 3 This is a flowchart of step S3 in the machine vision-based intelligent inspection method for flexible circuit boards according to an embodiment of this application. Figure 3 As shown, step S3, based on the standard template image, performs local fine registration on the globally aligned FPC image to obtain a finely aligned FPC image to be inspected, including: S31, inputting the globally aligned FPC image to be inspected and the standard template image into a pre-trained deep learning optical flow network to obtain an optical flow field; S32, performing optical flow-based image resampling and fine alignment on the globally aligned FPC image to be inspected based on the optical flow field to obtain the finely aligned FPC image to be inspected.

[0037] In the above exemplary operation, the specific process of step S3 is as follows: First, execute S31. In an exemplary operation, step S31, inputting the globally aligned FPC image to be inspected and the standard template image into a pre-trained deep learning optical flow network to obtain the optical flow field, includes: S311, inputting the globally aligned FPC image to be inspected and the standard template image into the convolutional feature encoder of the pre-trained deep learning optical flow network to obtain the sample high-dimensional feature map and the template high-dimensional feature map; S312, calculating the correlation features between the sample high-dimensional feature map and the template high-dimensional feature map; S313, inputting the correlation features into an update operator based on a gated recurrent unit to iterate and optimize the optical flow field to obtain the final optical flow field; S314, upsampling the final optical flow field to obtain the optical flow field. Specifically, this embodiment uses a pre-trained deep learning optical flow network based on a recurrent iterative optimization architecture. The weights and bias parameters of this network are obtained through supervised learning on a massive synthetic optical flow dataset, giving it a powerful generalization ability to accurately estimate pixel-level motion from image pairs.

[0038] First, S311 is executed. The core of this optical flow network is a weighted convolutional feature encoder, its architecture based on a deep residual network structure. This structure effectively learns deep feature representations while mitigating the vanishing gradient problem during deep network training through skip connections. The encoder consists of a series of cascaded residual blocks, each containing two 3x3 convolutional layers. Each convolutional layer is followed by a batch normalization layer and a Corrected Linear Unit (ReLU) activation function to stabilize the training process and introduce non-linearity. The crucial skip connections directly add the input of the residual block to its output, ensuring unimpeded information flow. In the processing flow, downsampling is not performed through traditional pooling layers, but rather by setting the stride of the first convolutional layer in a specific residual block to 2. When a globally aligned 5120×3840 pixel FPC image to be inspected is used as input, it first passes through an initial convolutional layer, e.g., 7x7, with a stride of 2, to quickly reduce the spatial resolution and increase the channel dimension. Subsequently, the image data flows through a series of residual blocks and convolutional layers with a stride of 2. The spatial size is halved at each downsampling stage, while the number of feature channels increases accordingly to encode more abstract features. This hierarchical processing ultimately produces multi-scale feature representations: when the image is downsampled to 1 / 8 of its original size, i.e., 640×480 pixels, the first set of feature maps is extracted, with a channel dimension of 64. The network continues this process, generating feature maps with 128 channels at a scale of 1 / 16, i.e., 320×240 pixels; finally, at a scale of 1 / 32, i.e., 160×120 pixels, a feature map with 256 channels is generated. These sample feature maps constitute the sample high-dimensional feature map. A key aspect of this design is that the exact same encoder, carrying its fixed, pre-trained weights and bias parameters, is independently applied to the globally aligned FPC image to be inspected and the standard template image. This weight-sharing mechanism ensures that the extracted sample high-dimensional feature maps and the template high-dimensional feature maps are within the same unified and directly comparable feature space. Meanwhile, the network architecture also includes a separate context network branch. The initial layers of this branch employ a residual structure similar to the main feature encoder, processing only the globally aligned FPC image to be detected and downsampling it to a 1 / 8 scale. However, subsequent network layers do not perform downsampling and have their own unique set of weights not shared with the main encoder. The sole purpose of this context network is to generate a more informative, single-scale feature map, called the sample context feature, which provides crucial, high-level semantic guidance information for subsequent motion estimation iterations.

[0039] Next, step S312 is executed. To efficiently represent the potential correspondences between all pixels in two images, the core of this step is to construct a structure capable of quantifying global matching information, namely a four-dimensional correlation volume pyramid. This process is executed in a dedicated correlation calculation module, which receives the sample high-dimensional feature map and template high-dimensional feature map output from the previous sub-step as input. For each scale level of the feature pyramid, for example, processing the sample feature map (size 640×480×C, where C is the number of feature channels, such as 256) and the template feature map (also size 640×480×C) at a 1 / 8 scale, this module calculates a four-dimensional correlation volume. The calculation of this volume is conceptually exhaustive: for each spatial location (u,v) in the sample feature map corresponding to a C-dimensional feature vector, the module performs a dot product operation with the C-dimensional feature vector at each spatial location (x,y) in the template feature map. The result of the dot product is a scalar that directly measures the similarity between the two feature vectors. A larger value indicates greater similarity in appearance or structure between the two pixels at the current feature abstraction level. The dot product of a fixed point (u,v) on the sample feature map and all points (x,y) on the template feature map is arranged into a 640×480 two-dimensional plane. This process is repeated for all 640×480 points on the sample feature map, logically constructing a four-dimensional tensor of size 640×480×640×480. Any element in this tensor, with coordinates (u,v,x,y), represents the feature similarity score between the sample image point (u,v) and the template image point (x,y). It is worth noting that due to the large memory footprint of this four-dimensional tensor, in actual hardware implementations, the entire volume is not instantiated and stored all at once. Instead, subsequent update operators perform efficient local neighborhood queries on this conceptual four-dimensional volume based on the current optical flow estimate, calculating relevance values ​​only within specific regions when needed. However, in principle, this complete four-dimensional relevance volume represents a comprehensive, global matching search space. The construction of this relevance volume is independently repeated at each scale of the feature pyramid (i.e., 1 / 8, 1 / 16, and 1 / 32 scales), generating a series of relevance volumes with decreasing size but complete conceptuality. This set of multi-scale relevance volumes collectively constitutes the final output of this sub-step: the relevance features. This structure fully encodes all possible pixel matching information between two images, from coarse to fine.

[0040] Subsequently, S313 is executed. First, at the 1 / 8 scale, an optical flow field of the same size as the feature map at that scale, such as 640×480, is initialized as a zero tensor. Simultaneously, a GRU hidden state of the same size, used to store the loop state, is also initialized as a zero tensor. Next, a fixed number of iterations is executed, for example, 12. This number of iterations is determined during the network training phase to be the optimal value balancing accuracy and computational efficiency. In each iteration, the update operator performs the following operations: First, a correlation query: For the optical flow estimation field of the current iteration, and for each pixel on the sample feature map at the 1 / 8 scale, its predicted corresponding point position in the template feature map is obtained by adding the current optical flow vector to the pixel's coordinates. Using this predicted corresponding point at floating-point coordinates as the center, a local region search is performed within the previously calculated four-dimensional correlation volume. This search operation specifically involves pooling the correlation score within a preset neighborhood (e.g., a region with a radius of 4 pixels), thereby extracting a compact and information-rich local correlation feature vector for each pixel. Simultaneously, the current optical flow field itself is fed into a dedicated motion feature encoder consisting of several convolutional layers, such as two 3x3 kernels, followed by a ReLU activation function, to generate a feature map describing the current motion state, i.e., motion features. Next, the local relevance features obtained through relevance queries, the motion features generated by the motion feature encoder, and the corresponding sample context features extracted from the context network are concatenated along the channel dimension to form a high-dimensional comprehensive feature that includes similarity, motion priors, and image content information. This feature will serve as the input to the gated recurrent unit (GRU) in the current iteration step. The GRU here is a recurrent neural network structure containing a reset gate and an update gate. The reset gate determines the extent to which historical information should be ignored based on the current input and the hidden state of the previous time step; the update gate determines the extent to which historical information should be retained and new information calculated from the current input should be introduced. Specifically, the GRU unit utilizes these gating mechanisms to nonlinearly combine the synthesized features of the input with the hidden state from the previous time step to compute the updated hidden state at the current time step. This updated hidden state contains all the information needed to correct the current optical flow estimate. This hidden state is then fed into a small convolutional network, such as two 1x1 convolutional layers, which is trained to decode the optical flow increment from the hidden state. This computed optical flow increment is a vector field of the same size as the optical flow field, representing the correction amount to the optical flow estimate in this iteration. This increment is added pixel-by-pixel to the current optical flow field, forming the new optical flow field after this iteration. This process is repeated, with the newly generated optical flow field and hidden state serving as the input for the next iteration.In each iteration, the GRU unit refines the optical flow field based on more precise correlation information obtained from corresponding point locations and more accurate motion states, gradually approximating the true optical flow value. After all 12 iterations, this iterative refinement process terminates, and the resulting 1 / 8 scale optical flow field is the final optical flow field that integrates multi-scale feature information and has undergone deep optimization.

[0041] In a preferred exemplary operation, step S313, inputting the correlation features into the update operator based on the gated loop unit for optical flow field iteration and optimization to obtain the final optical flow field, includes: within a preset number of iterations, performing the following operations in a loop: S313-1, querying the correlation features from the correlation features based on the current optical flow field to obtain local correlation features; S313-2, encoding the current optical flow field to obtain motion features; S313-3, applying a context-adaptive gating mechanism to modulate the local correlation features, motion features, and sample context features to obtain modulated local correlation features, modulated motion features, and modulated sample context features; S313-4, concatenating the modulated local correlation features, modulated motion features, and modulated sample context features to obtain comprehensive features; S313-5, inputting the comprehensive features and the hidden state of the previous time step into the gated loop unit to obtain the current hidden state; S313-6, decoding the current hidden state to obtain the optical flow increment; S313-7, updating the current optical flow field based on the optical flow increment, and outputting the final optical flow field after the final iteration. Specifically, the process described in step S313 above is the same after the preset number of iterations and the specific processing of S313-1, S313-2, S313-4, S313-5, S313-6, and S313-7, so it will not be described in detail here. The implementation process of step S313-3 will be explained in detail below.

[0042] It is understandable that during the iterative optimization of optical flow, the input to the gated recurrent unit (GRU) consists of three distinct types of features: local correlation features, motion features, and sample context features. Local correlation features provide the most direct pixel-matching evidence for optical flow estimation, but in weakly textured or repetitive line areas of the FPC image, their signals are filled with noise and ambiguity. Motion features represent the current assumptions about motion smoothness, becoming increasingly reliable in later iterations and effectively combating local noise. Sample context features provide stable and reliable priors about scene structure, such as line edges or substrate areas. If these three are simply concatenated as input, the model will fuse them in a fixed, non-adaptive manner, unable to dynamically adjust the dependence on different features according to the characteristics of different regions on the FPC image (such as clear line areas versus large substrate areas). Therefore, this step aims to introduce a context-adaptive gating mechanism that uses stable and reliable sample context features as a meta-controller to dynamically and explicitly guide the model at each pixel location to either trust direct matching evidence (local correlation features) or rely more on smooth priors of motion (motion features), thereby enabling the model to make optimal feature fusion decisions in various complex scenarios and significantly improve the accuracy and robustness of optical flow estimation.

[0043] S313-3 modulates local correlation features, motion features, and sample context features using a context-adaptive gating mechanism to obtain modulated local correlation features, modulated motion features, and modulated sample context features, including:

[0044] Based on the sample context features, a convolution-based activation-squeezing mechanism is used to generate joint gating to obtain local correlation weight features and motion weight features, i.e.:

[0045]

[0046]

[0047]

[0048]

[0049] in, It is a sample context feature. yes Activation function It is a convolution operation with a 3x3 convolution kernel. It is a convolution operation with a 1x1 kernel. It is an intermediate gating feature. yes Activation function It is a normalized joint gating feature. It is a local correlation weight feature. This refers to motion weight features. It can be understood that this step utilizes sample context features to generate two independent, pixel-level dynamic weight maps—i.e., gating signals—for local correlation features and motion features. This is implemented using a convolution-based activation-squeeze network. First, the sample context features are input into a network consisting of two cascaded 3x3 convolutional layers. The first convolutional layer is followed by a ReLU activation function; this is the activation phase, designed to extract deeper, more abstract semantic information from the sample context features, such as identifying whether the current region belongs to a line edge, a dense wiring area, or a smooth substrate area. Subsequently, its output is fed into a second 3x3 convolutional layer, followed by a Sigmoid activation function; this is the squeezing phase. The Sigmoid function is crucial; it non-linearly maps each element value in the feature map to the interval [0,1], giving it a natural physical meaning as a gating weight. To generate independent gating for local correlation features and motion features respectively, the compressed feature map is further processed through two parallel 1x1 convolutional layers for channel adjustment. The final output consists of two weight maps with the same spatial size as the input feature map but matching the number of channels for local correlation features and motion features, respectively: local correlation weight features and motion weight features. Ultimately, the network autonomously learns and generates two control knobs based on the local content of the FPC image, preparing for subsequent feature modulation.

[0050] Based on the local correlation weight feature and the motion weight feature, the local correlation feature and the motion feature are respectively multiplied and weighted to obtain the modulation local correlation feature and the modulation motion feature, that is: and ,in, It is a dot product by position. It is a feature of local correlation. It is a characteristic of motion. It is a modulation of local correlation characteristics. This involves modulating motion features. Correspondingly, this step applies the gated signal generated in the previous step to the corresponding feature map, achieving adaptive feature modulation. Specifically, the local correlation weight feature is multiplied element-wise with the local correlation feature to obtain the modulated local correlation feature; simultaneously, the motion weight feature is multiplied element-wise with the motion feature to obtain the modulated motion feature. This enables intelligent control of the information flow. For example, in the clear line area of ​​an FPC image, the context features identify rich textures, thus learning to generate a local correlation weight feature with a value close to 1, allowing accurate pixel matching signals to pass without attenuation. In contrast, in large areas of smooth substrate, the context features identify weak textures, generating a local correlation weight feature with a value close to 0 to suppress unreliable matching noise, and potentially generating a higher motion weight feature, allowing the model to rely more on the surrounding smooth motion field for optical flow estimation. In this way, this step significantly improves the model's adaptability to different regions in the FPC image, effectively suppressing incorrect matches and generating a smoother, more reasonable optical flow field.

[0051] Based on modulation local correlation features and modulation motion features, structural space modulation is applied to the sample context features to obtain modulated sample context features. In other words, the sample context features themselves, acting as a controller, are refined to better reflect the scene structure information related to the current motion estimation. This is achieved through structural space modulation. Specifically, the L2 norms (i.e., the magnitudes of the feature vectors) of the modulation local correlation features and modulation motion features are first calculated. These norms quantify the intensity of the matching signal and motion signal at each pixel after gating. Then, the L2 norm of the original sample context features is combined with the aforementioned two norms to calculate a spatial modulation weight. This weight calculation ensures that the weight value is close to 1 when the structural information intensity of the context features themselves is much greater than the intensity of the modulated motion and matching signals, and decreases otherwise. Finally, this spatial modulation weight map is multiplied element-wise with the original sample context features to obtain the final modulated sample context features. In this way, the contextual features are adaptively calibrated, ensuring that the contextual information input to the gated recurrent unit not only includes the original scene structure but also implicitly contains the relative importance of that structure in the current motion estimation state. This provides subsequent recurrent units with more complete and optimized input information. Then, the three independently modulated features are concatenated before being input to the gated recurrent update unit, ensuring that the input to the update unit is a pre-processed feature combination with a higher signal-to-noise ratio and stronger correlation. This avoids the difficulty of processing raw, mixed information in a single fusion stage, making update decisions more accurate and efficient, thus significantly improving the accuracy and robustness of optical flow estimation in complex scenes.

[0052] Finally, sub-step S314 is executed. Traditional bilinear interpolation upsampling methods, due to their use of a fixed, content-independent weighted averaging strategy, often introduce blurring effects when amplifying the optical flow field, especially in motion boundaries and fine-texture regions. They may also produce checkerboard artifacts, all of which significantly affect the accuracy of the final registration. To overcome this deficiency, this embodiment employs a learning-based, content-adaptive convex combination upsampling method. This method inputs the final hidden state, containing rich motion and contextual information, output from the previous GRU update operator, into a small convolutional network, namely the upsampling mask predictor. The specific architecture of this upsampling mask predictor consists of a series of convolutional layers, for example, two consecutive 3x3 convolutional layers with ReLU activation functions, used to further integrate and refine local features in the hidden state, followed by a 1x1 convolutional layer as the output layer. The number of channels in this output layer is precisely set to 576. This value means that, since upsampling is performed from 1 / 8 scale to full resolution, each low-resolution pixel corresponds to an 8x8=64 pixel block in the high-resolution grid; and this method independently predicts a 3x3=9 weighted upsampling kernel for each of these 64 high-resolution pixels. Therefore, a total of 64×9=576 weight values ​​need to be predicted. After outputting the tensors of these 576 channels, they are grouped, and the Softmax activation function is applied to each group of 9 weight values. This operation ensures that the 9 weight values ​​inside each predicted 3x3 upsampling kernel are all non-negative and sum to 1, forming a strictly convex combination. The adaptability of this method is reflected in the fact that, since the input to the upsampling mask predictor is the content-rich hidden state, the predicted upsampling kernel is dynamically generated based on the local image structure and motion complexity. For example, at the edges or corners of lines in an FPC image, the hidden state reflects drastic motion changes. The predictor learns to generate a sharpened kernel with weights concentrated at a few key locations to accurately maintain boundary clarity. In flat substrate areas, the predictor generates a smoother kernel, similar to a Gaussian blur, to suppress noise. In practice, this method processes two paths in parallel. One path is the mask prediction process described above. The other path directly amplifies the low-resolution final optical flow field by a factor of 8 using standard bilinear interpolation to obtain a preliminary high-resolution optical flow field with dimensions of 5120×3840×2. Finally, a fusion calculation is performed: for each pixel location in the final full-resolution optical flow field, the 3x3 upsampling kernel specifically predicted for it is extracted and applied to the corresponding 3x3 neighborhood in the preliminary high-resolution optical flow field. This essentially performs a unique weighted average operation determined by the local image content for each high-resolution pixel. After traversing all high-resolution pixels and completing the calculations, the final full-resolution optical flow field with a size of 5120×3840×2 is obtained.The optical flow field accurately describes the displacement of each pixel in the globally aligned FPC image to the corresponding point in the standard template image.

[0053] The second step is to execute step S32. In an exemplary operation, step S32, which involves performing optical flow-based image resampling and fine alignment on the globally aligned FPC image to be inspected based on the optical flow field to obtain the finely aligned FPC image to be inspected, includes: S321, generating a basic coordinate network based on the image size; S322, performing a scale transformation from pixel displacement to normalized displacement on the optical flow field based on the image size to obtain a normalized optical flow field; S323, calculating a sampling coordinate network based on the basic coordinate network and the normalized optical flow field to obtain a sampling coordinate grid; and S324, performing bilinear interpolation sampling on the globally aligned FPC image to be inspected based on the sampling coordinate grid to obtain the finely aligned FPC image to be inspected.

[0054] First, step S321 creates a regular, normalized coordinate grid that perfectly matches the size of the finely aligned FPC image to be inspected. This grid conceptually defines the standard position of each pixel in an ideal, unchanging target image. The specific processing is as follows: First, the dimensional information of the target image is received as input; in this embodiment, the width W is 5120 pixels and the height H is 3840 pixels. Based on these dimensions, two parallel linear space generators begin operation. The x-axis linear space generator calculates a one-dimensional vector containing 5120 floating-point values, precisely representing the coordinates of all evenly spaced points within a closed interval from -1 to 1, where the value of the i-th point (i from 0 to 5119) is calculated using the formula: -1 + 2*i / (W-1). Simultaneously, the y-axis linear space generator performs a similar operation, calculating a one-dimensional vector containing 3840 floating-point values, representing evenly spaced points on the y-axis from -1 to 1. Subsequently, these two one-dimensional vectors undergo a broadcasting or outer product operation, extending the one-dimensional spatial information to a two-dimensional plane. Specifically, it copies the one-dimensional vector along the x-axis 3840 times vertically, forming a 3840×5120 x-coordinate matrix where each row is identical, representing the original x-axis vector. Simultaneously, it copies the one-dimensional vector along the y-axis 5120 times horizontally, forming a 3840×5120 y-coordinate matrix where each column is identical, representing the original y-axis vector. Finally, these two two-dimensional coordinate matrices are merged along a new channel dimension (the third dimension). The output of this operation is a final tensor of size 3840×5120×2, which forms the basic coordinate network. In this constructed normalized coordinate system, each position (j,i) stores a two-dimensional coordinate, where x comes from the (j,i) position of the x-coordinate matrix, and y comes from the (j,i) position of the y-coordinate matrix. This coordinate system has a clear geometric meaning: the precise geometric center of the image corresponds to coordinates (0,0), the top left vertex corresponds to (-1,-1), and the bottom right vertex corresponds to (1,1).

[0055] Next, execute S322. The displacement vectors in the original optical flow field are in pixels, while the underlying coordinate network is normalized. To perform effective mathematical operations on both, the units of the optical flow field need to be unified to a normalized coordinate system. First, from the optical flow field tensor of size 3840×5120×2, separate the horizontal displacement component (a 3840×5120 matrix) and the vertical displacement component (another 3840×5120 matrix). Then, for each element in the horizontal displacement component matrix, divide it by the number of pixels at half the image width, i.e., (5120-1) / 2 = 2559.5. Similarly, for each element in the vertical displacement component matrix, divide it by the number of pixels at half the image height, i.e., (3840-1) / 2 = 1919.5. After such element-wise division, the two new matrices obtained are the normalized horizontal displacement and the normalized vertical displacement. Finally, the two normalized displacement matrices are stacked again along the channel dimension to form a normalized optical flow tensor with a size of 3840×5120×2. The physical meaning of this operation is to convert the absolute pixel displacement of each pixel into a relative displacement in the [-1,1] coordinate space.

[0056] Next, step S323 is executed. This step determines from which position in the input image the pixel information should be extracted when generating the output image. Specifically, this is implemented by performing element-wise addition on the base coordinate network tensor and the normalized optical flow field tensor. Since the two tensors are identical in size (3840×5120×2), this operation is direct and efficient. Its physical meaning is: for each regular grid point defined by the base coordinate network in the finely aligned FPC image to be inspected, its actual sampling position in the source image (i.e., the globally aligned FPC image to be inspected), which may have deviated from the regular grid due to deformation, is calculated based on the corresponding displacement provided by the normalized optical flow field. The result is a tensor of size 3840×5120×2, i.e., the sampling coordinate grid. Each two-dimensional coordinate value in this grid points to a specific, sub-pixel accurate normalized coordinate position in the source image.

[0057] Finally, step S324 is executed. This step involves actually picking pixel values ​​from the source image (i.e., the globally aligned FPC image to be inspected) and constructing a new image based on the sampling positions calculated in the previous step. A blank 8-bit grayscale image of size 3840×5120 is created as the output canvas. Then, each normalized sampling coordinate (x'_n, y'_n) in the sampling coordinate grid is traversed. For each such coordinate, it is first necessary to transform it from the normalized coordinate system back to the pixel coordinate system of the source image. The inverse transformation formula is: x'_p=(x'_n+1)*(5120-1) / 2, y'_p=(y'_n+1)*(3840-1) / 2. The resulting (x'_p, y'_p) is a pixel coordinate represented by a floating-point number. Since this coordinate usually does not fall on an integer point of the pixel grid, bilinear interpolation is used to estimate the grayscale value at this sub-pixel position. Specifically, first, four integer pixel coordinates around (x'_p, y'_p) are determined, and the grayscale values ​​of these four points in the globally aligned FPC image to be inspected are obtained. Then, using the standard bilinear interpolation formula, a weighted average is calculated based on the relative distances between (x'_p, y'_p) and these four points. For example, linear interpolation is first performed along the x-axis for the two pairs of points above and below, obtaining two intermediate values; then, linear interpolation is performed again along the y-axis for these two intermediate values, finally obtaining a precise interpolated grayscale value. This calculated grayscale value is assigned to the integer pixel position on the output canvas corresponding to the currently traversed sampling coordinate grid position. After all coordinate points in the sampling coordinate grid have undergone the above processing and the output canvas is completely filled, the resulting image is the final finely aligned FPC image to be inspected. The content in this image has been precisely locally reshaped according to the optical flow field, eliminating non-rigid deformations.

[0058] In step S4, based on the standard template image, differential attention defect segmentation is performed on the finely aligned FPC image to be inspected to obtain a defect probability mask. It should be understood that after the high-precision registration in the preceding steps, the image to be inspected and the standard template are geometrically aligned at the pixel level, creating ideal conditions for accurate defect identification. However, traditional comparison methods such as simple pixel value subtraction are extremely sensitive to factors such as slight changes in illumination and sensor noise, and may still cause interference. To achieve robust and accurate defect localization in complex industrial environments, this step utilizes the powerful feature representation and contextual understanding capabilities of deep learning models. Through an innovative differential attention mechanism, the model autonomously learns and focuses on meaningful feature differences between the image to be inspected and the template that are truly caused by defects, while suppressing irrelevant background noise and minor changes. Ultimately, a pixel-level defect probability map is generated, providing an accurate and reliable basis for subsequent defect analysis.

[0059] In one exemplary operation Figure 4This is a flowchart of step S4 in the machine vision-based intelligent inspection method for flexible circuit boards according to an embodiment of this application. Figure 4 As shown, step S4, based on the standard template image, performs differential attention defect segmentation on the finely aligned FPC image to be inspected to obtain a defect probability mask, including: S41, performing dual-stream feature extraction and downsampling on the standard template image and the finely aligned FPC image to be inspected to obtain downsampled sample feature maps, downsampled template feature maps, skip connections of sample features, and skip connections of template features; S42, performing differential attention feature enhancement on the skip connections of sample features and skip connections of template features to obtain attention-enhanced skip connections; S43, fusing decoder upsampling and skip connections on decoder features from deeper layers and attention-enhanced skip connections to obtain decoder block output features, which are used to transmit to shallower decoder blocks. In an exemplary operation, step S4, based on the standard template image, performs differential attention defect segmentation on the finely aligned FPC image to be inspected to obtain a defect probability mask, including: S44, extracting the final decoder output from the shallowest decoder block; S45, performing pointwise convolutional encoding and sigmoid activation on the final decoder output to obtain the defect probability mask.

[0060] In the above exemplary operation, the specific process of step S4 is as follows: This embodiment uses a two-stream semantic segmentation network based on the U-Net architecture and integrating a differential attention module. The weights and bias parameters of this network are obtained through supervised learning training on a large dataset of FPC images with pixel-level defect annotations.

[0061] First, step S41 is executed. The network contains a weighted encoder path consisting of multiple encoder blocks cascaded together. In this embodiment, the encoder contains four layers. In the first encoder block, a finely aligned FPC image to be inspected, with a size of 5120×3840, is first passed through an encoder convolutional block. This convolutional block consists of two consecutive 3×3 convolutional layers, each followed by a ReLU non-linear activation function, with the output channel number set to 64. This process extracts shallow features of the image to be inspected, such as edges, corners, and textures. Due to weight sharing, the standard template image is also passed through the exact same encoder convolutional block, generating a template shallow feature map with the same dimensions (5120×3840×64). These two feature maps are defined as skip connections of the sample features and skip connections of the template features at the current layer, respectively, and they are temporarily stored for subsequent fusion with the decoder path. Subsequently, these two feature maps are downsampled through a 2×2 max-pooling layer, halving their spatial size to 2560×1920 while maintaining the number of channels at 64, resulting in downsampled sample feature maps and downsampled template feature maps. The downsampled feature maps are then used as input to the next, deeper encoder block. This convolutional block + downsampling process is repeated four times. After each downsampling, the number of output channels in the subsequent convolutional block doubles, for example, 64->128->256->512, thereby extracting hierarchical features from low to high levels at different scales. Each encoder block generates a pair of feature maps for skip connections.

[0062] Next, execute S42. Taking the first level as an example, take the sample feature skip connections (5120×3840×64) and template feature skip connections (5120×3840×64) of this level. First, calculate the element-wise difference between the two and take its absolute value to generate a difference feature map. This feature map highlights all inconsistencies between the sample and the template at the feature level. Then, input this difference feature map into an attention generation network. This network is extremely lightweight, consisting of only a 1×1 convolutional layer and a sigmoid activation function. The 1×1 convolutional layer reduces the dimensionality of the 64-channel difference feature map and merges it into a single-channel feature map. The subsequent sigmoid function maps each value in this single-channel map to the [0,1] interval, thereby generating an attention weight map of the same size as the feature map. On this weight map, the region with a value close to 1 corresponds to the region where the sample and template differ significantly (which is likely a defect), while the region with a value close to 0 corresponds to the region where the two are consistent. Next, this attention weight map is multiplied element-wise with the original sample feature skip connections. This operation is equivalent to using the attention learned from the differences to weight the sample features, thus enhancing the feature signals of suspected defective regions while suppressing the features of background regions. To prevent information loss and promote gradient flow, this attention-enhanced feature map is added element-wise with the original sample feature skip connections. The final output is the attention-enhanced skip connections, which replace the original sample skip connections and are used for fusion in subsequent decoder paths.

[0063] Then, S43 is executed. The decoder path is symmetrical to the encoder path and consists of four decoder blocks, restoring resolution step by step from the deepest layer (smallest size). In the deepest decoder block, its input is the feature map output from the deepest layer of the encoder path. This decoder block first performs an upsampling operation on the input feature map, for example, using a 2×2 transposed convolution with a stride of 2 to double its spatial size and halve the number of channels to match the size of the feature map output from the previous level of the encoder. Then, this upsampled feature map is concatenated with the skip concatenation from the corresponding encoder level, which has undergone differential attention enhancement, in the channel dimension. For example, in the penultimate decoder layer, the upsampled feature from the deepest decoder, with a size of 320×240×256, is concatenated with the attention-enhanced skip concatenation generated by the third encoder layer, resulting in a feature map with a size of 320×240×256. The concatenated feature map is then passed through a decoder convolutional block (also composed of two 3×3 convolutions + ReLU). This block is responsible for fusing the concatenated information and restoring the number of channels to the standard value for that level, for example, reducing 512 channels back to 256. The output features of this decoder block will serve as the upsampled input to a shallower decoder block. This upsampling -> concatenation -> convolution process is repeated level by level in the decoder path until the resolution is restored to the same as the original input image. After the last level of the decoder path, i.e., the shallowest decoder block, is processed, S44 is executed to extract the final decoder output from this block. At this point, the output feature map size has been restored to 5120×3840, with 64 channels. This final decoder output feature map has highly fused semantic information from different levels and attention-enhanced fine spatial details.

[0064] Finally, step S45 is executed. The final decoder output feature map of 5120×3840×64 obtained in the previous step is input into a 1×1 convolutional layer. The number of output channels of this convolutional layer is set to 1. This operation linearly combines the feature vectors of the 64 channels at each pixel location into a single scalar value, which can be interpreted as the logical score of the pixel belonging to the defect category. Subsequently, this tensor of logical values ​​of size 5120×3840×1 is passed pixel-by-pixel through a Sigmoid activation function. The Sigmoid function maps any real number range of input to the open interval (0,1). After processing by this function, the output tensor is the final defect probability mask. The mask has a size of 5120×3840×1, where the value of each pixel represents the probability that the location is a defect. For example, a pixel with a value of 0.98 means that the model has a 98% confidence that the point is a defect.

[0065] In step S5, defect analysis is performed based on the defect probability mask and the original FPC image to be inspected to obtain a defect report. That is, the previous steps have successfully generated a high-precision defect probability mask, revealing the location of potential defects in pixel-level probability form. However, this raw probability map is not yet comprehensive or intuitive enough for automated quality control processes. The production line needs clear conclusions about each individual defect, such as the total number of defects, the specific type of each defect (e.g., open circuit or short circuit), precise location, and size. Therefore, the probability mask needs to be finally parsed and quantified, transforming it from a continuous probability distribution map into a structured final report containing detailed attributes of all defects. This provides a direct and actionable basis for subsequent automated sorting, manual re-inspection, or process improvement decisions.

[0066] In an exemplary operation, step S5 is as follows: First, the input defect probability mask is binarized. Each pixel value in the defect probability mask is between 0 and 1, representing the confidence level that the point is a defect. To clearly distinguish between defects and background, a probability threshold needs to be set. The setting of this threshold has a direct impact on the detection sensitivity and specificity. Based on a large amount of experimental data, an optimal value that balances the false negative rate and false positive rate is determined by plotting the receiver operating characteristic curve. In this embodiment, the probability threshold is set to 0.5. Each pixel in the defect probability mask is iterated. If its value is greater than 0.5, the pixel value at the corresponding position is set to 255 (representing a defect) in a newly generated image of the same size; if its value is not greater than 0.5, the pixel value at the corresponding position is set to 0 (representing background). After this operation, an 8-bit single-channel binary image called the defect probability binarized image is obtained, which clearly marks all high-probability defect areas.

[0067] Next, connected component analysis is performed on the binary image of the defect probability. The purpose of this analysis is to aggregate spatially connected defect pixels in the binary image into independent, discrete defect entities, thereby achieving a transformation from pixel-level description to object-level description. This embodiment uses an 8-neighborhood connectivity-based labeling algorithm, meaning that if two pixels with a value of 255 are adjacent in the horizontal, vertical, or diagonal direction, they are considered to belong to the same connected component. The algorithm traverses the entire binary image once, assigning a unique integer label starting from 1 to each discovered independent connected region, and updating the values ​​of all pixels within that region to this label. After execution, a data structure of a defect region list is obtained, where each element in the list represents an independent defect patch and contains, in the form of a data structure, the precise coordinates of all pixels constituting that patch.

[0068] Subsequently, detailed feature extraction and classification are performed on each defect patch in the defect area list. This process is a cyclical operation, performing the following analysis for each identified independent defect: First, its geometric features are calculated. Based on the pixel coordinate set constituting the defect patch, its area (i.e., the total number of pixels) and centroid position (the average of all pixel coordinates) can be directly calculated. Its contour can be traced using an edge detection algorithm, thereby calculating the perimeter. Simultaneously, the smallest bounding rectangle that can completely enclose the defect patch can be determined, obtaining its position (top-left corner coordinates) and dimensions (width and height). Second, its appearance features are extracted. Using the calculated bounding rectangle or the precise pixel coordinate set, the corresponding region is located on the original FPC image to be inspected, and the image patch of that region is extracted. For this image patch, its average grayscale, grayscale standard deviation, and other color (grayscale) features, as well as texture features describing its surface characteristics, can be calculated.

[0069] After obtaining the geometric and appearance feature vectors of each defect, a precise type classification is performed. This embodiment uses a pre-trained lightweight classifier model based on a fully connected neural network. The specific architecture of this model is as follows: an input layer with the same number of neurons as the dimension of the feature vector (e.g., if it includes 10 features such as area, perimeter, aspect ratio, average gray level, gray level standard deviation, and texture contrast, then the input layer has 10 neurons); two hidden layers, the first hidden layer contains 32 neurons and the second hidden layer contains 16 neurons, with each hidden layer's neurons using ReLU as the activation function to introduce non-linear expressive power; and an output layer with the number of neurons equal to the total number of predefined defect categories, such as 4 neurons for open circuits, short circuits, copper slag, and scratches, using the Softmax activation function. The weights and bias parameters of this classifier are obtained through supervised learning training using the cross-entropy loss function and the Adam optimizer on a dataset containing tens of thousands of defect samples whose categories have been accurately labeled by experts. During classification, a 10-dimensional feature vector of a defect is fed into the input layer. After forward propagation through the network, a 4-dimensional probability distribution vector is obtained at the output layer, where each element represents the probability that the defect belongs to the corresponding category. The category represented by the neuron with the highest probability value is selected as the final classification result for the defect. For example, an open-circuit defect, due to its elongated shape and grayscale similarity to the substrate, will receive the highest probability value at the output neuron corresponding to the open-circuit category after its feature vector is input into the model.

[0070] Finally, after all defect patches have been analyzed and classified, the results are summarized to generate a final structured defect report. This report uses a JSON format that is easy for both machines to parse and humans to read. The top level of the report is a list, where each object represents a detected defect. Each defect object contains multiple key-value pairs that describe its attributes in detail. For example, a complete report generated for an FPC with two detected defects can be clearly described in a structured form as follows: The report records the first defect (defect ID 1), which is classified as an open circuit. The top-left corner of its bounding rectangle is located at (1024, 2048), and its dimensions are 150 pixels wide and 5 pixels high. Further quantification information shows that its total area is 735 pixels and its perimeter is 312.5 pixels. The report also identified a second defect (defect ID 2), identified as copper slag. Its bounding rectangle is located at coordinates (3500, 1820), with dimensions of 12 pixels wide and 15 pixels high, resulting in a calculated area of ​​160 pixels and a perimeter of 58.2 pixels. This generated JSON file can be sent in real-time to the production management execution unit to trigger audible and visual alarms or control a robotic arm to remove the slag. It can also be stored in the quality management database for long-term quality traceability and statistical process control.

[0071] In summary, a machine vision-based intelligent inspection method for flexible printed circuit boards (FPCs) based on embodiments of this application has been clarified. It proposes a scheme combining global coarse registration and local fine registration to fundamentally solve the problem of false alarms caused by non-rigid deformation of FPCs. First, a global coarse registration step quickly corrects rigid transformations such as translation, rotation, and scaling of the entire FPC image under inspection, achieving initial alignment with a standard template. Subsequently, a local fine registration step is introduced to address the nonlinear deformations such as local stretching and wrinkling caused by the material properties of FPCs, as mentioned in the background art. This step can accurately perceive and compensate for these subtle, non-uniform local misalignments, achieving near-pixel-level precise matching between the image under inspection and the template. Based on this high-precision alignment, differential attention defect segmentation is then performed, effectively eliminating false differences caused by inaccurate deformation registration, thereby accurately identifying real defects, significantly reducing the false alarm rate, and greatly improving the reliability and practicality of the inspection scheme.

[0072] Figure 5 This is a block diagram of a machine vision-based intelligent inspection system for flexible circuit boards according to an embodiment of this application. Figure 5As shown, the machine vision-based intelligent inspection system 100 for flexible circuit boards according to an embodiment of this application includes: a raw standard image acquisition module 110, used to acquire a raw FPC image to be inspected and a loaded standard template image in response to a conveyor belt in place signal; an image global alignment module 120, used to perform global coarse registration of the raw FPC image to be inspected and the standard template image to obtain a globally aligned FPC image to be inspected; a fine alignment module 130, used to perform local fine registration of the globally aligned FPC image to be inspected based on the standard template image to obtain a finely aligned FPC image to be inspected; a defect segmentation module 140, used to perform differential attention defect segmentation of the finely aligned FPC image to be inspected based on the standard template image to obtain a defect probability mask; and a defect analysis module 150, used to perform defect analysis based on the defect probability mask and the raw FPC image to be inspected to obtain a defect report.

[0073] Here, those skilled in the art will understand that the specific operations of each step in the above-described machine vision-based intelligent inspection system for flexible circuit boards have been referenced above. Figures 1 to 4 The method for intelligent inspection of flexible circuit boards based on machine vision has been described in detail, and therefore, its repeated description will be omitted.

Claims

1. A machine vision-based intelligent inspection method for flexible circuit boards, characterized in that, include: In response to the transmission belt in place signal, acquire the original FPC image to be inspected and load the standard template image; Global coarse registration is performed between the original FPC image to be inspected and the standard template image to obtain a globally aligned FPC image to be inspected. Based on a standard template image, a fine-grained local registration is performed on a globally aligned FPC image to obtain a finely aligned FPC image. This includes: inputting the globally aligned FPC image and the standard template image into a pre-trained deep learning optical flow network to obtain local correlation features, motion features, and sample context features; during the update iteration of the gated recurrent unit of the deep learning optical flow network, modulating the local correlation features, motion features, and sample context features using a context-adaptive gating mechanism to obtain modulated local correlation features, modulated motion features, and modulated sample context features; concatenating the modulated local correlation features, modulated motion features, and modulated sample context features and inputting them into the gated recurrent unit for optical flow field iteration to obtain an optical flow field; and performing optical flow-based image resampling and fine alignment on the globally aligned FPC image to obtain the finely aligned FPC image. Based on the standard template image, differential attention defect segmentation is performed on the finely aligned FPC image to be inspected to obtain the defect probability mask. Defect analysis is performed based on the defect probability mask and the original FPC image to be inspected to obtain a defect report; The modulation of local correlation features, motion features, and sample context features using a context-adaptive gating mechanism includes: For the sample context features, a joint gating mechanism based on convolution is used to generate joint gating to obtain local correlation weight features and motion weight features respectively; Based on the local correlation weight feature and the motion weight feature, the local correlation feature and the motion feature are respectively multiplied and weighted to obtain the modulated local correlation feature and the modulated motion feature; Based on the modulation local correlation features and modulation motion features, the sample context features are subjected to structure space modulation to obtain the modulation sample context features.

2. The intelligent inspection method for flexible circuit boards based on machine vision according to claim 1, characterized in that, Global coarse registration is performed between the original FPC image to be inspected and the standard template image to obtain a globally aligned FPC image to be inspected, including: Feature points are extracted from the original FPC image to be inspected and the standard template image to obtain the sample key point set and the template key point set; Feature matching and transformation matrix estimation are performed on the sample keypoint set and the template keypoint set to obtain the homography matrix; A perspective transformation is performed on the original FPC image to be inspected based on the homography matrix to obtain the globally aligned FPC image to be inspected.

3. The intelligent inspection method for flexible circuit boards based on machine vision according to claim 2, characterized in that, Based on the optical flow field, the globally aligned FPC image to be inspected is resampled and finely aligned to obtain the finely aligned FPC image to be inspected, including: Generate a basic coordinate network based on image dimensions; The normalized optical flow field is obtained by scaling the optical flow field from pixel displacement to normalized displacement based on the image size. The sampling coordinate network is calculated based on the basic coordinate network and the normalized optical flow field to obtain the sampling coordinate grid; The finely aligned FPC image to be inspected is obtained by bilinear interpolation sampling based on the sampling coordinate grid.

4. The intelligent inspection method for flexible circuit boards based on machine vision according to claim 1, characterized in that, Based on a standard template image, differential attention defect segmentation is performed on the finely aligned FPC image to be inspected to obtain a defect probability mask, including: Dual-stream feature extraction and downsampling are performed on the standard template image and the finely aligned FPC image to be inspected to obtain the downsampled sample feature map, the downsampled template feature map, the skip connections of the sample features, and the skip connections of the template features. Differential attention feature enhancement is performed on the skip connections of sample features and template features to obtain attention-enhanced skip connections; Decoder upsampling and skip connection fusion are performed on the decoder features from deeper layers and attention-enhanced skip connections to obtain decoder block output features, which are then used to transmit to shallower decoder blocks.

5. The intelligent inspection method for flexible circuit boards based on machine vision according to claim 4, characterized in that, Based on a standard template image, differential attention defect segmentation is performed on the finely aligned FPC image to be inspected to obtain a defect probability mask, including: Extract the final decoder output from the shallowest decoder block; The final decoder output is subjected to pointwise convolutional encoding and Sigmoid activation to obtain the defect probability mask.

6. A machine vision-based intelligent inspection system for flexible circuit boards, characterized in that, include: The original standard image acquisition module is used to acquire the original FPC image to be inspected and load the standard template image in response to the transmission belt in place signal; The global image alignment module is used to perform global coarse registration between the original FPC image to be inspected and the standard template image to obtain a globally aligned FPC image to be inspected. A fine alignment module is used to perform local fine registration on a globally aligned FPC image to obtain a finely aligned FPC image based on a standard template image. This includes: inputting the globally aligned FPC image and the standard template image into a pre-trained deep learning optical flow network to obtain local correlation features, motion features, and sample context features; in the update iteration of the gated recurrent unit of the deep learning optical flow network, modulating the local correlation features, motion features, and sample context features using a context-adaptive gating mechanism to obtain modulated local correlation features, modulated motion features, and modulated sample context features; concatenating the modulated local correlation features, modulated motion features, and modulated sample context features and inputting them into the gated recurrent unit for optical flow field iteration to obtain an optical flow field; and performing optical flow-based image resampling and fine alignment on the globally aligned FPC image to obtain the finely aligned FPC image. The defect segmentation module is used to perform differential attention defect segmentation on finely aligned FPC images to obtain a defect probability mask based on standard template images. The defect analysis module is used to perform defect analysis based on the defect probability mask and the original FPC image to be inspected in order to obtain a defect report. The modulation of local correlation features, motion features, and sample context features using a context-adaptive gating mechanism includes: For the sample context features, a joint gating mechanism based on convolution is used to generate joint gating to obtain local correlation weight features and motion weight features respectively; Based on the local correlation weight feature and the motion weight feature, the local correlation feature and the motion feature are respectively multiplied and weighted to obtain the modulated local correlation feature and the modulated motion feature; Based on the modulation local correlation features and modulation motion features, the sample context features are subjected to structure space modulation to obtain the modulation sample context features.

Images