Machine vision-based ffs roll film surface defect detection method and system

Abstract

The application discloses a kind of FFS roll film surface defect detection method and system based on machine vision, it is first to the original sensor data stream by filtering and correction strategy reorganization enhancement, eliminate noise and optimize contrast, construct high-quality data basis.Subsequently, the texture structure information of image is captured using multi-scale feature extraction technology, drive the non-rigid elastic registration model based on grid, dynamically compensate the nonlinear deformation generated in high-speed transmission of flexible roll film, realize the spatial accurate alignment of the standard template of the image to be detected and template.Furthermore, calculate high signal-to-noise ratio residual and combine adaptive saliency segmentation technology, efficiently peel out potential defect area from complex background.Finally, multi-dimensional feature classification and logic judgment are carried out on candidate object, accurately distinguish real defects and deformation artifacts, and output rejection instructions.In this way, the false alarm problem caused by the deformation of flexible material can be effectively overcome, and the accuracy and robustness of the detection system are improved.

Description

Technical Field

[0001] This application relates to the field of defect detection technology, and more specifically, to a machine vision-based method and system for detecting defects on the surface of FFS roll film. Background Technology

[0002] With the rapid development of the modern packaging industry towards high speed and intelligence, Form-Fill-Seal (FFS) roll film technology, with its highly efficient integrated processing advantages, has been widely used in the food, pharmaceutical, and fine chemical industries. As a key barrier to ensure the quality and safety of the contents, the surface quality of FFS roll film directly affects the product's sealing performance, shelf life, and brand image. Tiny holes, foreign matter inclusions, or printing defects, if not intercepted in time, can easily lead to leakage or spoilage of the contents, causing serious economic losses and a crisis of trust. Therefore, achieving 100% inspection of the roll film surface quality on high-speed production lines has become a core aspect of quality control for manufacturing enterprises. Traditional offline sampling inspections or methods relying on manual strobe observation are limited by the physiological limits of the human eye and subjective fatigue, making them difficult to adapt to the current production pace of hundreds of meters per minute, resulting in a very high risk of missed inspections. Therefore, introducing high-precision machine vision technology to replace human eyes for non-contact real-time monitoring has become an inevitable trend for improving production yield.

[0003] While existing technologies have demonstrated mature application results in defect identification of rigid plates or discrete components, traditional detection methods still have significant technical limitations when dealing with special flexible materials like FFS rolls. Most mainstream detection methods are based on standard template comparison techniques, assuming only rigid displacement exists between the image to be tested and a standard defect-free template, and extracting defects through differential operations. However, FFS rolls are inherently flexible and easily deformable materials. During high-speed transport and mechanical traction, they are highly susceptible to nonlinear elastic deformation due to tension fluctuations, roller friction, and material anisotropy, including non-directional stretching, twisting, or local geometric distortion. This non-rigid deformation prevents strict pixel alignment between the real-time acquired image and the standard template in spatial position, especially at pattern edges and high-frequency texture areas. Even minor misalignments can generate numerous high-grayscale artifacts in the differential image. These deformation-induced artifacts are often misidentified as real defects by the system, leading to a very high false alarm rate. To reduce false alarms, existing systems are often forced to lower the detection threshold, but this inevitably leads to missed detections of minor real defects.

[0004] Therefore, there is an urgent need for an optimized method and system for detecting defects on the surface of FFS roll film. Summary of the Invention

[0005] This application is made in order to solve the above-mentioned technical problems.

[0006] According to one aspect of this application, a machine vision-based method for detecting surface defects on FFS roll film is provided, comprising: S1: Based on the Gaussian smoothing filter and gamma correction parameters, the original sensor data stream containing the line scanning grayscale matrix is ​​reconstructed and enhanced to obtain the real-time image to be inspected. S2: Perform multi-scale pyramid feature extraction on the preset standard defect-free template image and the real-time image to be inspected to obtain the template feature set and the image feature set to be inspected; S3: Based on the template feature set and the feature set of the image to be inspected, perform grid-based non-rigid elastic registration on the real-time image to be inspected to obtain the elastically registered image; S4: Calculate the high signal-to-noise ratio residual map between the elastically registered image and the standard defect-free template image; S5: Perform adaptive saliency segmentation on the high signal-to-noise ratio residual map to extract a list of candidate defect connected components; S6: Perform multidimensional feature classification and decision output for each object in the candidate defect connected component list to obtain the elimination control instruction.

[0007] According to another aspect of this application, a machine vision-based FFS roll film surface defect detection system is provided, comprising: The image reconstruction and enhancement module is used to perform image reconstruction and enhancement processing on the original sensor data stream containing the line scanning grayscale matrix based on the Gaussian smoothing filter and gamma correction parameters to obtain the real-time image to be inspected. The multi-scale feature extraction module is used to perform multi-scale pyramid feature extraction on the preset standard defect-free template image and the real-time image to be inspected to obtain the template feature set and the image feature set to be inspected. The non-rigid elastic registration module is used to perform grid-based non-rigid elastic registration on the real-time image to be inspected based on the template feature set and the feature set of the image to be inspected in order to obtain the elastically registered image. The high signal-to-noise ratio residual calculation module is used to calculate the high signal-to-noise ratio residual map between the elastically registered image and the standard defect-free template image; The adaptive saliency segmentation module is used to perform adaptive saliency segmentation on high signal-to-noise ratio residual maps to extract a list of candidate defect connected components. The multidimensional classification decision module is used to perform multidimensional feature classification and decision output for each object in the candidate defect connected domain list to obtain elimination control instructions.

[0008] Compared with existing technologies, this application provides a machine vision-based method and system for detecting surface defects on FFS roll-to-film. First, it reconstructs and enhances the original sensor data stream through filtering and correction strategies, eliminating noise and optimizing contrast to build a high-quality data foundation. Then, it uses multi-scale feature extraction technology to capture the texture structure information of the image, driving a grid-based non-rigid elastic registration model to dynamically compensate for the nonlinear deformation of the flexible roll-to-film during high-speed transmission, achieving precise spatial alignment between the image to be inspected and the standard template. Based on this, it calculates high signal-to-noise ratio residuals and combines them with adaptive saliency segmentation technology to efficiently extract potential defect regions from complex backgrounds. Finally, it performs multi-dimensional feature classification and logical judgment on candidate objects to accurately distinguish between real defects and deformation artifacts, outputting rejection instructions. This effectively overcomes the problem of false alarms caused by deformation of flexible materials, improving the accuracy and robustness of the detection system. Attached Figure Description

[0009] The above and other objects, features, and advantages of this application will become more apparent from the more detailed description of the embodiments of this application in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this application and form part of the specification. They are used together with the embodiments of this application to explain this application and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.

[0010] Figure 1 This is a flowchart of a machine vision-based FFS roll film surface defect detection method according to an embodiment of this application.

[0011] Figure 2 This is a data flow diagram of a machine vision-based FFS roll film surface defect detection method according to an embodiment of this application.

[0012] Figure 3 This is a flowchart of sub-step S1 of the machine vision-based FFS roll film surface defect detection method according to an embodiment of this application.

[0013] Figure 4 This is a flowchart of sub-step S2 of the machine vision-based FFS roll film surface defect detection method according to an embodiment of this application.

[0014] Figure 5 This is a flowchart of sub-step S3 of the machine vision-based FFS roll film surface defect detection method according to an embodiment of this application.

[0015] Figure 6 This is a flowchart of sub-step S4 of the machine vision-based FFS roll film surface defect detection method according to an embodiment of this application.

[0016] Figure 7This is a flowchart of sub-step S5 of the machine vision-based FFS roll film surface defect detection method according to an embodiment of this application.

[0017] Figure 8 This is a flowchart of sub-step S6 of the machine vision-based FFS roll film surface defect detection method according to an embodiment of this application.

[0018] Figure 9 This is a block diagram of a machine vision-based FFS roll film surface defect detection system according to an embodiment of this application. Detailed Implementation

[0019] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0020] To address the problems mentioned above in the background technology, this application proposes a machine vision-based method for detecting surface defects in FFS roll film. Figure 1 This is a flowchart of a machine vision-based FFS roll film surface defect detection method according to an embodiment of this application. Figure 2 This is a data flow diagram of a machine vision-based FFS roll film surface defect detection method according to an embodiment of this application. Figure 1 and Figure 2 As shown, the machine vision-based FFS roll film surface defect detection method includes the following steps: S1, performing image reconstruction and enhancement processing on the original sensor data stream containing the row scanning grayscale matrix based on Gaussian smoothing filter and gamma correction parameters to obtain the real-time image to be inspected; S2, performing multi-scale pyramid feature extraction on the preset standard defect-free template image and the real-time image to be inspected to obtain the template feature set and the image feature set to be inspected; S3, performing non-rigid elastic registration based on grid on the real-time image to be inspected based on the template feature set and the image feature set to be inspected to obtain the elastically registered image; S4, calculating the high signal-to-noise ratio residual map between the elastically registered image and the standard defect-free template image; S5, performing adaptive saliency segmentation on the high signal-to-noise ratio residual map to extract the candidate defect connected component list; S6, performing multi-dimensional feature classification and decision output on each object in the candidate defect connected component list to obtain the elimination control command.

[0021] In the aforementioned machine vision-based FFS roll film surface defect detection method, step S1 involves image reconstruction and enhancement processing of the original sensor data stream containing the line scan grayscale matrix based on a Gaussian smoothing filter and gamma correction parameters to obtain the real-time image to be inspected. It should be understood that since FFS roll film production lines typically operate at high speeds, the original signal acquired by the line scan camera is a continuous and unstructured one-dimensional data stream, which contains thermal noise generated by high-frequency electronic components and limited grayscale dynamic range due to uneven ambient lighting. Directly using such raw data for detection would result in an extremely high false alarm rate. Therefore, this application further utilizes a Gaussian smoothing filter and gamma correction parameters to perform image reconstruction and enhancement processing on the original sensor data stream containing the line scan grayscale matrix. This reconstructs a two-dimensional image frame containing complete physical texture information from the time-series signal, while suppressing thermal noise generated by the sensor in high-gain mode and correcting the nonlinear response of the grayscale dynamics. This provides a high-quality data foundation for subsequent feature extraction and defect identification, ensuring that the detection system can maintain a high signal-to-noise ratio imaging capability even in complex industrial lighting environments, thereby effectively improving the sensitivity to capture minute defects.

[0022] In particular, in one specific embodiment, Figure 3 This is a flowchart of sub-step S1 of the machine vision-based FFS roll film surface defect detection method according to an embodiment of this application. Figure 3 As shown, step S1 includes: S11, based on the encoder trigger signal contained in the original sensor data stream, extracting linear array data in time sequence and stacking and filling it into the matrix buffer to obtain an initial grayscale frame; S12, performing Gaussian convolution noise reduction processing on the initial grayscale frame based on a Gaussian smoothing filter to obtain a denoised grayscale image; S13, performing nonlinear mapping processing on the grayscale distribution of the denoised grayscale image based on a preset gamma correction coefficient to obtain the real-time image to be inspected.

[0023] Specifically, in step S11, based on the encoder trigger signal included in the original sensor data stream, linear array data is extracted in a time sequence and stacked to fill the matrix buffer to obtain an initial grayscale frame. It should be understood that due to the mechanical traction during transmission of the FFS roll film, its actual movement speed will fluctuate non-periodically. If data is collected only at fixed time intervals, the generated image will be geometrically stretched or compressed, severely compressing the physical fidelity of the surface texture. Therefore, this application further extracts linear array data in a time sequence based on the encoder trigger signal included in the original sensor data stream and stacks it to fill the matrix buffer. This strictly synchronizes the image acquisition frequency with the physical movement distance of the roll film, ensuring that the physical length represented by each row of pixels remains constant regardless of changes in the production line speed. This eliminates longitudinal geometric distortion caused by speed fluctuations, generating an initial grayscale frame that truly reflects the surface morphology of the roll film, providing a geometrically consistent reference plane for subsequent standard template comparison and defect localization.

[0024] Specifically, in one possible embodiment, the data acquisition interface physically connects the line scan camera to a rotary encoder mounted on the traction roller. The encoder generates a precisely corresponding electrical pulse signal for each fixed, minute distance the film advances. The acquisition card monitors this pulse signal in real time and synchronously triggers the camera's exposure and data readout operations when the rising edge of the pulse arrives. The read single-line pixel data is written sequentially based on the line index into a pre-allocated two-dimensional memory address space. When the accumulated number of lines reaches a preset height threshold corresponding to a single pattern cycle of the film, the current writing operation is terminated, and the data in the buffer is encapsulated into an initial grayscale frame object, thereby completing the precise conversion from a one-dimensional temporal stream to a two-dimensional spatial image.

[0025] Specifically, in step S12, the initial grayscale frame is subjected to Gaussian convolution denoising based on a Gaussian smoothing filter to obtain a denoised grayscale image. It should be understood that industrial-grade line scanning sensors inevitably generate thermal noise and photon shot noise under high-frequency exposure conditions. These high-frequency discrete noise points appear as salt-and-pepper clutter in the image, easily interfering with edge feature extraction and even being misjudged as point defects by subsequent differential algorithms. Therefore, this application further applies Gaussian convolution denoising based on a Gaussian smoothing filter to the initial grayscale frame. This utilizes the weight distribution characteristics of the Gaussian function to perform a weighted average calculation on the target pixel and its neighboring pixels, smoothing high-frequency abrupt changes while preserving the low-frequency structural information of the main pattern. This effectively improves the signal-to-noise ratio of the image, preventing deviations in subsequent image registration and residual calculation due to noise interference, and preparing a clean data source for high-precision edge detection.

[0026] Specifically, in one possible embodiment, a two-dimensional convolutional kernel conforming to a Gaussian distribution is constructed, with a kernel matrix size of 5×5 and a standard deviation parameter of 1.2. This convolutional kernel slides pixel-by-pixel across the entire initial grayscale frame. At each sliding position, the kernel performs a multiplication and accumulation operation with the grayscale values ​​of the pixels currently covered by the window. The calculated weighted average is used to replace the original grayscale value of the center pixel of the window, and this process is repeated until all pixels in the image have been traversed. Through this spatial domain filtering operation, the intensity of isolated noise points is suppressed by the information of surrounding pixels, while the jagged edges originally present in the image are smoothly transitioned, generating a smooth and clear denoised grayscale image.

[0027] Specifically, in step S13, the grayscale distribution of the denoised grayscale image is nonlinearly mapped based on a preset gamma correction coefficient to obtain the real-time image to be inspected. It should be understood that, due to the potential for high reflectivity or unclear dark textures on the surface of the FFS roll film, and the difficulty of distinguishing subtle grayscale differences in low-light areas by the camera's linear response, defects such as scratches and oil stains located in shadows or against dark backgrounds are easily missed. Therefore, this application further performs nonlinear mapping on the grayscale distribution of the denoised grayscale image based on a preset gamma correction coefficient to remap the input grayscale levels, nonlinearly expanding the dynamic range of dark areas to enhance detail contrast in shadows, while compressing the grayscale levels in highlight areas to prevent overexposure. This optimizes the overall visual quality of the image, making the optical features of potential defects in different brightness areas more prominent, thereby improving the detection algorithm's capture rate of low-contrast defects.

[0028] Specifically, in one possible embodiment, and more specifically in a particular example of this application, a specific gamma coefficient value adapted to the material properties is pre-configured, and a grayscale mapping lookup table is generated based on this coefficient to establish a one-to-one mapping relationship between the input grayscale level and the output grayscale level. The pixel matrix of the denoised grayscale image is traversed, and the grayscale value of each pixel is quickly replaced using the lookup table. This mapping transformation stretches the histogram distribution of the low grayscale range, causing dark defects that were originally blended into the background to exhibit greater brightness differences in the output image. Through this pixel-level point operation, the image contrast is adjusted to the optimal state, generating the final real-time image to be inspected for subsequent registration and analysis.

[0029] In the aforementioned machine vision-based FFS roll film surface defect detection method, step S2 involves performing multi-scale pyramid feature extraction on a preset standard defect-free template image and the real-time image to be inspected to obtain a template feature set and an image feature set to be inspected. It should be understood that due to the complexity of the printed patterns on the FFS roll film surface and the uncertainty of the nonlinear deformation generated during high-speed transmission, directly performing pixel-level matching on high-resolution images would result in extremely high computational load and be susceptible to interference from local printing dot noise. Therefore, this application further utilizes multi-scale feature extraction technology to perform layered processing on the preset standard defect-free template image and the real-time image to be inspected. This constructs an image pyramid containing different resolution levels, achieving decoupling of macroscopic structural information and microscopic texture details, and extracting robust feature anchor points from the low-resolution levels. This provides noise-resistant and computationally efficient reference data for subsequent non-rigid elastic registration, ensuring high-precision alignment even under large deformations.

[0030] In particular, in one specific embodiment, Figure 4 This is a flowchart of sub-step S2 of the machine vision-based FFS roll film surface defect detection method according to an embodiment of this application. Figure 4 As shown, step S2 includes: S21, performing iterative Gaussian low-pass filtering and intermittent downsampling on the real-time image to be inspected and the standard defect-free template image respectively to obtain the image pyramid to be inspected and the template image pyramid; S22, performing top-level FAST key point detection on the image pyramid to be inspected and the template image pyramid to obtain the key point set to be inspected and the template key point set; S23, calculating the gray centroid direction of each key point in the key point set to be inspected and the brightness comparison result of the neighboring pixel pairs to obtain the feature set of the image to be inspected and the feature set of the template.

[0031] Specifically, in step S21, iterative Gaussian low-pass filtering and intermittent downsampling are performed on the real-time image to be inspected and the standard defect-free template image, respectively, to obtain the image pyramid to be inspected and the template image pyramid. It should be understood that the original high-resolution image is filled with high-frequency information such as halftone dots and surface rough textures. This information constitutes interference noise in the macroscopic registration stage, and if not suppressed, it will seriously affect the stability of feature points. Therefore, this application further performs iterative Gaussian low-pass filtering and intermittent downsampling on the real-time image to be inspected and the standard defect-free template image, respectively. This smooths the high-frequency spatial variations in the image and gradually reduces the image size, establishing a multi-resolution scale space from fine to coarse. This ensures that the top layer image of the pyramid retains only the main low-frequency structural features, making subsequent feature extraction operations insensitive to minor texture noise, while significantly reducing the amount of data processing.

[0032] Specifically, in one possible embodiment, the original resolution image is used as the base layer of the pyramid. A two-dimensional Gaussian convolution kernel is applied to perform a full-frame smoothing convolution operation on the current layer image, and a weighted average is used to remove high-frequency components to prevent aliasing. Next, pixels are extracted from the smoothed image in both the row and column directions with a fixed stride to generate the next layer image with reduced resolution. This downsampled image is used as input, and the above filtering and downsampling process is repeated until the number of generated image layers reaches the preset total number of pyramid layers, thereby completing the construction of the complete pyramid data structure.

[0033] Specifically, in step S22, top-level FAST keypoint detection is performed on the pyramid of the image to be detected and the pyramid of the template image to obtain the keypoint set to be detected and the template keypoint set. It should be understood that although the top-level image of the pyramid has the lowest resolution, it contains the most stable global topological structure of the roll pattern, and the downsampling process naturally filters out most of the random noise, making it an ideal carrier for finding correspondences under large deformations. Therefore, this application further performs top-level FAST keypoint detection on the pyramid of the image to be detected and the pyramid of the template image. This focuses on identifying salient points in the image with the strongest corner responses, such as the intersections of pattern edges or obvious geometric turning points. This allows for rapid identification of rigid skeleton features in the image, avoiding misjudging unstable background textures as feature points, and providing high-quality geometric constraints for global homography estimation.

[0034] Specifically, in one possible embodiment, the top-level image with the lowest resolution in the pyramid structure is selected as the detection target. A sliding window is used to traverse each pixel in the image, examining a set of neighboring pixels on a circle centered on the current pixel. The grayscale values ​​of consecutive pixel segments on the circle are compared with the grayscale value of the center pixel. If multiple consecutive pixels have grayscale values ​​significantly brighter or darker than the center pixel, the center pixel is marked as a candidate corner point. Finally, non-maximum suppression is performed on all candidate corner points, eliminating locally dense responses and retaining only the keypoints with the largest response values ​​as the set of keypoints to be detected and the template keypoint set.

[0035] Specifically, in step S23, the gray-level centroid direction of each key point in the key point set to be inspected and the template key point set are compared with the brightness of neighboring pixel pairs to obtain the feature set of the image to be inspected and the template feature set. It should be understood that since the position coordinates of key points alone cannot describe their local appearance features, and the roll film may rotate slightly during transmission, simple coordinate matching lacks rotation invariance and is prone to mismatches. Therefore, this application further calculates the gray-level centroid direction of each key point in the key point set to be inspected and the template key point set with the brightness of neighboring pixel pairs. This assigns a principal direction to each key point to correct rotational deviations and encodes local texture information into a compact binary string. This generates rotation-invariant feature descriptors, supports fast matching based on Hamming distance, and ensures accurate feature correspondences can still be established under illumination fluctuations or angular deviations.

[0036] Specifically, in one possible embodiment, the gray-level moment in the neighborhood of each keypoint is calculated, and the principal direction of the keypoint is determined based on the offset vector of the gray-level centroid relative to the geometric center. Subsequently, the neighborhood coordinate system is rotated to this principal direction, and point pairs of a predetermined pattern are selected within the rotated region. The gray-level values ​​of each pair of pixels are compared, and a binary value is generated based on the comparison result. The comparison results of all point pairs are then concatenated into a binary vector. This vector, along with the coordinates and principal direction information of the keypoints, is encapsulated into a final data structure, thereby forming a feature set of the image to be examined and a template feature set containing rich geometric and texture information.

[0037] In the aforementioned machine vision-based FFS roll film surface defect detection method, step S3 involves performing non-rigid elastic registration of the real-time image to be inspected based on a grid, using a template feature set and the image to be inspected feature set, to obtain an elastically registered image. It should be understood that, as a flexible material, FFS roll film inevitably undergoes non-linear elastic deformation due to tension fluctuations and mechanical friction during high-speed traction. This deformation leads to complex local geometric differences between the real-time acquired image and the rigid template, making pixel-level precise alignment impossible using only traditional rigid registration. Therefore, this application further performs non-rigid elastic registration of the real-time image to be inspected based on a grid, using a template feature set and the image to be inspected feature set, to obtain an elastically registered image. This introduces a grid deformation mechanism with degrees of freedom to dynamically compensate for the stretching or twisting components in various local areas of the image, thereby correcting the spatial coordinate system of the image to be inspected to be strictly consistent with the standard template. This eliminates artifact noise generated in subsequent differential stages due to physical deformation, ensuring that only real surface defects are retained, thus significantly improving the signal-to-noise ratio and robustness of the detection system under complex working conditions.

[0038] In particular, in one specific embodiment, Figure 5This is a flowchart of sub-step S3 of the machine vision-based FFS roll film surface defect detection method according to an embodiment of this application. Figure 5 As shown, step S3 includes: S31, performing feature matching and global homography estimation on the feature set of the image to be inspected and the template feature set to obtain a global homography matrix and a list of matching feature point pairs; S32, initializing the grid vertex state based on the global homography matrix, and performing grid energy minimization and deformation field solution on the list of matching feature point pairs to obtain a deformed grid field; S33, performing image reshaping based on bilinear interpolation on the real-time image to be inspected based on the deformed grid field to obtain an elastically registered image.

[0039] Specifically, step S31 involves performing feature matching and global homography estimation on the feature set of the image to be inspected and the template feature set to obtain a global homography matrix and a list of matched feature point pairs. It should be understood that since the original feature point set may contain erroneous matching points due to repetitive textures or background noise, and there is a significant initial viewpoint and positional deviation between the image to be inspected and the template image, directly performing fine-grained mesh deformation can easily lead to local optima or topological errors. Therefore, this application further performs feature matching and global homography estimation on the feature set of the image to be inspected and the template feature set to obtain a global homography matrix and a list of matched feature point pairs. This establishes a global perspective projection relationship between the images, and statistical methods are used to remove abnormal feature point pairs, selecting a high-quality set of interior points. This provides a stable and near-realistic initial alignment reference for subsequent non-rigid registration, significantly reducing the search space for mesh deformation and ensuring the convergence and accuracy of the deformation process.

[0040] Specifically, in one possible embodiment, brute-force matching is performed on two sets of binary feature descriptors based on Hamming distance, and a cross-validation strategy is applied to select bidirectionally consistent initial matching point pairs. Subsequently, a random sample consensus algorithm is used for iterative computation. In each iteration, the minimum number of matching point pairs is randomly selected to calculate the hypothetical homography matrix, and the number of interior points that conform to the matrix model is counted. After multiple iterations, the model with the most interior points is selected as the optimal solution, and the matrix parameters are refined and reconstructed using the least squares method with all interior points. Finally, the homography matrix describing the global geometric transformation relationship is output, along with a list of matching feature point pairs that have been rigorously screened and have had mismatched points removed.

[0041] Specifically, in step S32, the grid vertex state is initialized based on the global homography matrix, and the grid energy is minimized and the deformation field is solved for the list of matching feature point pairs to obtain the deformed grid field. It should be understood that since the global homography matrix can only correct the overall rigidity and perspective transformation, it cannot characterize the local anisotropic stretching of the film caused by uneven stress. Therefore, it is necessary to fit this complex physical deformation field through fine grid node displacements. Thus, this application further initializes the grid vertex state based on the global homography matrix and performs grid energy minimization and deformation field solution for the list of matching feature point pairs to obtain the deformed grid field. This constructs a mathematical optimization model that balances feature point alignment accuracy and grid topology smoothness, calculating the optimal offset of each control point under the premise of satisfying physical constraints. This allows for accurate simulation of the nonlinear elastic behavior of the film surface, achieving sub-pixel-level local alignment and preventing grid flipping or image tearing caused by overfitting.

[0042] Specifically, in one possible embodiment, a preset uniform grid vertices are projected onto the coordinate system of the image to be inspected using a global homography matrix to form an initial grid state. A data item is then constructed. With smoothing terms global energy function Among them, data items It is defined as the sum of squared L2 norm Euclidean distances between all pairs of matched feature points and their corresponding points in the interpolated grid cell; To achieve the optimal balance between feature alignment accuracy and mesh topology rigidity, the smoothing weight coefficient is set to 15.0 in this embodiment; the smoothing term... A Laplace regularization constraint is introduced, and the second-order difference between each mesh vertex and the centroid of its neighboring vertices is calculated to penalize local non-affine deformation of the mesh. This energy function is then applied to the coordinate vectors of all mesh vertices. By taking the partial derivatives and setting them to zero, the nonlinear optimization problem is transformed into solving a large-scale sparse linear system of equations. ,in It is a sparse coefficient matrix. Let be the vector to be solved. The constraint vectors are used. The system of equations is solved using the Choreski decomposition or the conjugate gradient method to obtain the final coordinate vectors of all grid vertices. This set of coordinate vectors constitutes a deformable grid field describing the local deformation patterns of different regions of the image.

[0043] Here, when performing feature matching and global homography estimation on the feature set of the image to be examined and the template feature set, a global homography matrix (coarse alignment) and a list of matching feature point pairs are provided. However, despite RANSAC filtering, a small number of outliers with large positional deviations may still remain in the feature point pairs, and their distribution is uneven. If the mesh deformation of the deformed mesh field solved based on the mesh deformation energy function is not smooth or is stretched by noise, then interpolating and resampling the image will cause distortion artifacts in the final image, interfering with subsequent residual calculations.

[0044] Therefore, in a preferred embodiment, this application introduces a robust M-estimation and an anisotropic physical constraint energy model. That is, if the energy function only uses the standard L2 norm (squared Euclidean distance) as the data term and uses simple Laplace smoothing as the regularization term, since the L2 norm penalizes the error quadratically, it means that if there is even one mismatched point with a large deviation in the feature point pair, in order to minimize the total energy, the mesh will be forcibly pulled toward the wrong point, resulting in severe local distortion.

[0045] In other words, if we assume that the smoothing constraint weights of the mesh are the same in the horizontal and vertical directions, but FFS roll film is obviously a typical orthotropic material, on the production line, the roll film bears huge physical traction tension along the machine direction, with large deformation stiffness, while it is relatively relaxed in the lateral direction. Therefore, isotropic constraints do not conform to the mechanical and physical properties of the material. Specifically, step S32 includes: constructing a total energy function containing robust data terms and anisotropic physical regularization terms; wherein, the robust data terms are obtained by calculating the residuals for each pair of matched feature points and introducing a robust kernel function; the anisotropic physical regularization terms are obtained by calculating the longitudinal second-order difference and the lateral second-order difference of each vertex in the mesh, and assigning a machine direction stiffness coefficient to the longitudinal second-order difference and a lateral stiffness coefficient to the lateral second-order difference respectively; solving for the minimum value of the total energy function to determine the optimal coordinate state of the mesh vertices, and thus generating a deformable mesh field.

[0046] Specifically, to improve robustness in industrial settings, a total energy function is constructed that includes robust data terms and anisotropic physical regularization terms. In the energy model of the introduced robust M-estimation and anisotropic physical constraints, a robust kernel function is used to replace the L2 norm with the Cauchy loss function or Huber Loss. When the projection error is small, it approximates L2 (ensuring accuracy), while when the error exceeds a certain threshold, the growth rate slows down (approximately L1 or logarithmic growth), thereby automatically suppressing the influence of anomalous feature points on the mesh. Simultaneously, anisotropic second-order regularization is introduced to decompose the smoothing constraints into independent second-order difference terms along the machine direction (row direction) and laterally (column direction), and assigning different stiffness coefficients to simulate the physical behavior of the thin film under tension, allowing moderate elastic stretching laterally while strictly controlling abrupt bending in the longitudinal direction.

[0047] Thus, the improved total energy function From data items and physical regularization terms Composition, wherein the robust data term is determined by matching each feature point pair Calculate residuals And a robust kernel function, namely the Cauchy kernel function, is introduced. Obtain, among which Represented as: ;in, It is a robust scaling factor used to control sensitivity to outliers, i.e., the error is greater than a certain threshold. Points such as 1.5 are considered outliers and are therefore suppressed; It is the residual of the feature point pair.

[0048] Furthermore, the anisotropic physical regularization term is calculated for each vertex in the mesh. (in For row index, The second-order difference of the column index is obtained by the vertical and horizontal second-order differences, and the vertical second-order differences are respectively... Assigning directional stiffness coefficients to the machine, and applying the second-order lateral difference. Assign a lateral stiffness coefficient.

[0049] The improved robust anisotropy total energy is expressed as: Here, by solving the total energy function To determine the optimal coordinate state of the mesh vertices, a deformable mesh field is generated, wherein... It is the total number of matching feature point pairs. Feature points In the current grid Bilinear interpolation mapping coordinates under the action, These are the coordinates of the corresponding real feature points in the target image. It is the first in the grid Line 1 The vertex coordinate vector of the column. This is the machine directional stiffness coefficient, corresponding to the film traction direction. Due to the high tension, this value should be set relatively large (e.g., 50) to suppress severe local shaking along the travel direction. It is the lateral stiffness coefficient, corresponding to the direction perpendicular to the traction. This value is set relatively small (e.g., 10) to allow the mesh to adapt to Poisson's shrinkage or slight lateral drift of the material. It is a second-order difference term in the longitudinal direction, used to describe the curvature (degree of bending) of the mesh in the longitudinal direction.

[0050] In this way, by introducing M-estimation, even if RANSAC misses a small number of incorrect matching points (such as mismatching similar textures in the background), the energy function will not be biased by these points, which improves noise resistance, ensures the local accuracy of the image after elastic registration, avoids the generation of bright pseudo-differences in subsequent steps due to registration distortion, and thus significantly reduces the false alarm rate in subsequent steps.

[0051] At the same time, through and The differentiated settings make the mesh deformation field follow the mechanical transmission law of FFS film, which is consistent with physical reality. This makes the solution of this application more stable on high-speed production lines with a speed of up to 600ppm, and can distinguish between normal tension fluctuations and abnormal local distortions.

[0052] Specifically, in step S33, based on the deformed mesh field, image reshaping based on bilinear interpolation is performed on the real-time image to be inspected to obtain an elastically registered image. It should be understood that since the deformed mesh field only defines the coordinate mapping relationship, the pixel grayscale values ​​of the image to be inspected are still stored on the original distorted coordinate grid points. Mathematical resampling techniques must be used to accurately transfer the grayscale information to the corrected coordinate positions. Therefore, this application further performs image reshaping based on bilinear interpolation on the real-time image to be inspected based on the deformed mesh field to obtain an elastically registered image. This is used to perform inverse texture mapping, tracing back to the corresponding positions in the original image according to the new mesh coordinates, and calculating the grayscale intensity at sub-pixel positions. In this way, a new image can be generated that highly matches the standard template geometrically, with natural texture transitions and no jagged edges or void effects, providing standardized data input for subsequent high-precision residual analysis.

[0053] Specifically, in one possible embodiment, each pixel coordinate of the target output image is traversed, and the floating-point coordinate position of the pixel in the original image to be inspected is calculated based on the mapping relationship determined by the deformed mesh field. Four adjacent integer pixels surrounding the floating-point coordinate are identified, and their grayscale values ​​are read. A weighting coefficient is calculated based on the fractional part of the floating-point coordinate, and a weighted average is performed on the grayscale values ​​of these four adjacent pixels to obtain the final grayscale value of the target pixel, which is then filled into the result image. By performing the above operations on all pixels of the image, the original distorted image texture is smoothly mapped to the corrected geometric framework, thereby outputting a clear and geometrically distortion-free elastically registered image.

[0054] In the aforementioned machine vision-based FFS roll film surface defect detection method, step S4 involves calculating a high signal-to-noise ratio (SNR) residual image between the elastically registered image and the standard defect-free template image. It should be understood that due to the nonlinear deformation and micro-fluctuations in illumination during high-speed transmission of the FFS roll film, simple image subtraction generates a large amount of artifact noise at high-frequency texture edges and fails to effectively highlight low-contrast structural defects, resulting in a low SNR in the detection results. Therefore, this application further calculates a high SNR residual image between the elastically registered image and the standard defect-free template image to comprehensively utilize grayscale intensity differences and local structural similarity information, constructing a composite evidence image that can suppress edge alignment errors and highlight true defect features. This eliminates interference signals caused by registration residual errors, ensuring that subsequent defect segmentation steps focus only on real physical damage or foreign objects, thereby significantly reducing the system's false alarm rate.

[0055] In particular, in one specific embodiment, Figure 6 This is a flowchart of sub-step S4 of the machine vision-based FFS roll film surface defect detection method according to an embodiment of this application. Figure 6 As shown, step S4 includes: S41, calculating the absolute difference map between the elastically registered image and the standard defect-free template image; S42, calculating the structural difference map between the elastically registered image and the standard defect-free template image; S43, performing spatial weighted linear fusion of the absolute difference map and the structural difference map based on a preset non-printing area weight mask to obtain a high signal-to-noise ratio residual map.

[0056] Specifically, step S41 involves calculating the absolute difference map between the elastically registered image and the standard defect-free template image. It should be understood that common defects such as ink stains, material black spots, or obvious foreign matter inclusions exhibit significant intensity jumps in the grayscale space, resulting in a large direct difference between their pixel values ​​and the surrounding background. Therefore, this application further calculates the absolute difference map between the elastically registered image and the standard defect-free template image to quantify the direct L1 distance in pixel brightness between the image to be inspected and the standard template, quickly capturing high-contrast abnormal areas. This provides a basic grayscale difference component for the composite residual map, ensuring a sensitive initial response to various dark or bright defects and providing solid data support for the detection of such obvious defects.

[0057] Specifically, in one possible embodiment, each pixel coordinate point in the image domain is traversed, and the grayscale value of the elastically registered image and the standard defect-free template image at the current coordinates is read. The absolute value of the difference between these two grayscale values ​​is calculated, which directly reflects the degree of light intensity difference at that location. The calculated absolute difference value is filled into a matrix of the same size as the original image. By performing the above difference operation on all pixels in the entire image, a complete absolute difference map is generated, which visually shows the distribution of regions with inconsistent light intensity between the images.

[0058] Specifically, step S42 involves calculating a structural difference map between the elastically registered image and the standard defect-free template image. It should be understood that defects such as shallow scratches, micro-wrinkles, or gel points on transparent films often have low grayscale contrast, but they disrupt the local texture topology of the material surface, making it difficult to distinguish them from background noise using only grayscale difference. Therefore, this application further calculates a structural difference map between the elastically registered image and the standard defect-free template image to assess the correlation changes in brightness, contrast, and structure of the image signal within the local neighborhood, utilizing the high sensitivity of the structural similarity index to structural damage to capture subtle defects. This compensates for the shortcomings of single grayscale difference and effectively detects low-contrast structural damage that is difficult for the human eye to perceive.

[0059] Specifically, in one possible embodiment, a Gaussian sliding window of a preset size is defined and used to slide pixel-by-pixel across the image. At each window position, the local mean, variance, and covariance between the test image and the template image are statistically analyzed. Based on these statistics, a mean comparison term reflecting brightness similarity, a variance comparison term reflecting contrast similarity, and a covariance comparison term reflecting structural similarity are constructed, and these three components are multiplied to obtain a local structural similarity index. Subsequently, this index is subtracted from the original value to convert the similarity into a difference. The calculated structural difference values ​​are mapped to the corresponding pixels in the output image, ultimately generating a structural difference map reflecting the degree of change in texture structure.

[0060] Specifically, in step S43, based on a preset non-printed area weight mask, the absolute difference map and the structural difference map are spatially weighted linearly fused to obtain a high signal-to-noise ratio residual map. It should be understood that since the edge regions of printed patterns are most prone to high-frequency noise due to small registration residuals, while flat non-printed areas are high-incidence areas of defects and are more sensitive to noise, direct global fusion would lead to false alarms at the edges. Therefore, this application further uses a preset non-printed area weight mask to spatially weight linearly fuse the absolute difference map and the structural difference map to obtain a high signal-to-noise ratio residual map, thereby introducing a spatial attention mechanism to dynamically reduce the detection sensitivity of edge regions and increase the weight of flat regions. This maximizes the suppression of artifact interference from printed edges while enhancing the signal strength of small defects in blank areas, achieving high-precision detection across the entire area.

[0061] Specifically, in one possible embodiment, a pre-defined non-printed area weight mask is read. This mask defines the detection sensitivity weights for different regions in the image, with lower weights for edge regions and higher weights for blank regions. Simultaneously, a balancing coefficient is set to adjust the proportion of intensity differences and structural differences. To eliminate differences in data dimensions, the pixel grayscale values ​​of the absolute difference map are first normalized to the [0, 1] interval to match the numerical range of the structural difference map. Subsequently, for each pixel in the image, the normalized absolute difference value and the structural difference value are weighted and summed according to the balancing coefficient, and this sum is multiplied by the weight value at the corresponding position in the mask. Through this pixel-by-pixel weighted fusion calculation, a high signal-to-noise ratio residual map that suppresses edge noise and enhances defect features is finally output.

[0062] In the aforementioned machine vision-based FFS roll film surface defect detection method, step S5 involves adaptive saliency segmentation of the high signal-to-noise ratio residual image to extract a list of candidate defect connected components. It should be understood that because the ambient light intensity at the FFS roll film production site slowly drifts over time, and different batches of film materials exhibit slight differences in optical reflectivity, relying solely on a fixed grayscale threshold for residual image segmentation can easily lead to severe missed detections or false alarms due to fluctuations in the background baseline. Therefore, this application further performs adaptive saliency segmentation on the high signal-to-noise ratio residual image, combining local contrast enhancement with dynamic threshold calculation based on image statistical characteristics to transform the continuously distributed grayscale residual field into discrete candidate defect objects. This ensures that the segmentation algorithm automatically adapts to changes in the production environment's lighting and the material's texture fluctuations, accurately extracting potential defect regions from a complex and non-uniform background, providing independent targets with clear boundaries for subsequent feature classification, thereby significantly improving the adaptability and stability of the detection system under dynamic operating conditions.

[0063] In particular, in one specific embodiment, Figure 7 This is a flowchart of sub-step S5 of the machine vision-based FFS roll film surface defect detection method according to an embodiment of this application. Figure 7 As shown, step S5 includes: S51, performing local saliency enhancement on the high signal-to-noise ratio residual map to obtain a saliency map; S52, performing adaptive threshold segmentation and morphological denoising on the saliency map to obtain a fine binary mask; S53, extracting connected components from the fine binary mask and removing tiny spots with areas smaller than a preset value to obtain a candidate defect connected component list.

[0064] Specifically, in step S51, local saliency enhancement is performed on the high signal-to-noise ratio residual image to obtain a saliency map. It should be understood that since the high signal-to-noise ratio residual image may still retain low-frequency background brightness variations caused by uneven film thickness or differences in illumination angle, these macroscopic background fluctuations often mask the grayscale peaks of minute defects, making direct segmentation difficult to distinguish between the background and the target. Therefore, this application further performs local saliency enhancement on the high signal-to-noise ratio residual image to obtain a saliency map, thereby simulating the sensitivity of the human visual system to center-periphery contrast. Differential calculations using filters of different scales are used to highlight isolated signals with significant abrupt changes relative to the local neighborhood. This effectively suppresses slowly changing background inhomogeneities while significantly enhancing the signal-to-noise ratio of minute defects such as mosquito spots and crystal spots within a local range, making potential defects appear as bright peak areas in the saliency map, thus reducing the difficulty of subsequent threshold segmentation.

[0065] Specifically, in one possible embodiment, the input residual image is convolved using two mean filters with different preset radii, and the radius of the inner scale filter is set. =3 pixels (corresponding to a 7×7 convolution kernel) is used to capture local detail features; set the radius of the outer scale filter. =9 pixels (corresponding to a 19×19 convolution kernel) are used to estimate local background brightness. The absolute value of the difference between the two filtering results at the corresponding pixel positions is calculated to generate a grayscale image reflecting the local contrast intensity. This process transforms weak defect signals submerged in the background gradient in the original image into high-response regions in the saliency map, while large flat background areas are suppressed to low grayscale values ​​close to zero due to the minimal difference between the internal and external scale means, thus outputting a saliency map with a clean background and prominent target.

[0066] Specifically, step S52 involves adaptive thresholding and morphological denoising of the saliency map to obtain a refined binary mask. It should be understood that since the absolute intensity of the defect signal in the saliency map is not constant but dynamically distributed according to the defect type and severity, and the thermal noise of the sensor itself forms discrete salt-and-pepper clutter in the image, using a single fixed threshold cannot simultaneously ensure the complete extraction of defects with different contrasts. Therefore, this application further performs adaptive thresholding and morphological denoising on the saliency map to obtain a refined binary mask. This automatically finds the optimal segmentation threshold based on the statistical grayscale distribution characteristics of the image and uses morphological structural elements to cut off subtle noise connections. This maximizes the preservation of the integrity of the defect subject while filtering out isolated single-pixel noise and irregular edge spikes, generating a binary image containing only high-confidence potential defect regions, providing mask data with a clear topological structure for subsequent connected component analysis.

[0067] Specifically, in one possible embodiment, the gray-level histogram distribution of the statistical saliency map is analyzed, and the maximum inter-class variance algorithm is applied to traverse all possible gray levels to find the optimal threshold that maximizes the variance between the foreground pixel class and the background pixel class. This dynamically calculated threshold is then used to binarize the saliency map, marking pixels above the threshold as foreground and pixels below the threshold as background. Subsequently, morphological opening is performed on the generated binary image. This involves first performing erosion using structuring elements of a preset size (e.g., a 3×3 rectangle) to eliminate isolated noise, followed by dilation to restore the shape of the main target, thereby outputting a fine binary mask with smooth boundaries and complete internal filling.

[0068] Specifically, in step S53, connected component extraction is performed on the refined binary mask, and tiny spots with areas smaller than a preset value are removed to obtain a list of candidate defect connected components. It should be understood that since the binary mask consists only of discrete pixel arrays, it lacks structured information describing the location, size, and shape of defects. Furthermore, the image may still contain non-fatal micro-dust or minor imperfections within acceptable limits. Directly performing subsequent high-level feature classification on all pixel clusters would result in a huge waste of computational resources. Therefore, this application further performs connected component extraction on the refined binary mask and removes tiny spots with areas smaller than a preset value to obtain a list of candidate defect connected components. This aggregates adjacent foreground pixels into independent geometric objects and performs preliminary size screening of the target based on industrial inspection standards. In this way, meaningless pixel data can be transformed into physical objects with physical properties, effectively eliminating tiny interference sources that have no impact on product quality, focusing on truly potentially risky candidate defects, thereby significantly reducing the computational load of subsequent classification algorithms.

[0069] Specifically, in one possible embodiment, an eight-neighbor connected component labeling algorithm is performed on the fine binary mask. This traverses the image matrix and merges spatially adjacent pixels into the same connected component object, assigning a unique index identifier to each object. For each extracted connected component object, the total number of pixels it contains is counted to calculate its area attribute, and this area value is compared with a pre-set minimum defect area (e.g., 10 pixels, corresponding to a physical area of ​​approximately 0.04 square millimeters). Connected components with an area greater than or equal to this preset value are retained, and their bounding box coordinates and centroid positions, among other geometric metadata, are calculated. This data is encapsulated into structured objects and stored in a list, ultimately outputting a list of candidate defect connected components containing all valid candidate targets.

[0070] In the aforementioned machine vision-based FFS roll film surface defect detection method, step S6 involves performing multi-dimensional feature classification and decision output for each object in the candidate defect connected domain list to obtain a rejection control command. It should be understood that since the candidate defect connected domains extracted in the preceding steps may still contain water droplets, alignment residual noise, or non-critical defects, relying solely on geometric size screening cannot meet the stringent requirements of zero missed detections and zero false alarms for critical defects in industrial settings. Therefore, this application further performs multi-dimensional feature classification and decision output for each object in the candidate defect connected domain list to comprehensively analyze the object's geometric shape, grayscale statistics, and texture details, and uses classification algorithms to accurately identify true and false defects. This effectively eliminates benign interference sources, ensuring that the final output control command targets only confirmed physical damage, thereby improving the intelligent decision-making capability of the detection system under complex working conditions.

[0071] In particular, in one specific embodiment, Figure 8 This is a flowchart of sub-step S6 of the machine vision-based FFS roll film surface defect detection method according to an embodiment of this application. Figure 8 As shown, step S6 includes: S61, using the object coordinates in the candidate defect connected component list to extract the region of interest from the real-time image to be inspected, and calculating the geometric circularity, gray-level mean statistics, and texture entropy of the region of interest to obtain a composite feature vector set; S62, inputting the composite feature vector set into a support vector machine model based on the radial basis kernel function to obtain a list of confirmed defects; S63, based on the vertical pixel coordinates in the list of confirmed defects and the real-time encoder pulse count, calculating the number of rejection delay pulses to generate rejection control instructions.

[0072] Specifically, in step S61, the region of interest (ROI) is extracted from the real-time image under inspection using the object coordinates in the candidate defect connected component list, and the geometric circularity, gray-level mean statistic, and texture entropy of the ROI are calculated to obtain a composite feature vector set. It should be understood that since binary masks only provide contour positions and lack depth information describing the optical properties and microstructure of defect surfaces, it is difficult to distinguish between defect types with similar shapes but different properties, such as dark oil stains and light-colored gels. Therefore, this application further utilizes the object coordinates in the candidate defect connected component list to extract the ROI from the real-time image under inspection and calculates the geometric circularity, gray-level mean statistic, and texture entropy of the ROI. This constructs a multi-dimensional feature space covering shape regularity, brightness contrast intensity, and surface texture complexity. In this way, visual features can be transformed into numerical vectors, providing highly discriminative data support for subsequent high-precision classification.

[0073] Specifically, in one possible embodiment, based on the bounding box coordinates of the connected component object, a corresponding rectangular pixel region is located and cropped as the region of interest in the real-time image to be inspected. The ratio of perimeter to area is calculated based on a binary mask of this region to determine roundness, quantifying the compactness of the shape, and the average and standard deviation of the grayscale values ​​of the pixels within the region are statistically analyzed to quantify its brightness distribution characteristics. Simultaneously, a gray-level co-occurrence matrix is ​​constructed and its entropy value is calculated to quantify the randomness of the texture. Finally, statistical parameters (mean and standard deviation) saved during the offline training phase are called, and the calculated values ​​are Z-score standardized and concatenated in a predetermined order to generate a composite feature vector set describing the specific object.

[0074] Specifically, in step S62, the composite feature vector set is input into a support vector machine model based on a radial basis function kernel to obtain a list of confirmed defects. It should be understood that because different types of defects and background noise often exhibit a non-linear interleaved distribution in the low-dimensional feature space, traditional linear classifiers struggle to find ideal segmentation boundaries, easily leading to misclassification or missed detection of complex defects. Therefore, this application further inputs the composite feature vector set into a support vector machine model based on a radial basis function kernel to obtain a list of confirmed defects. This utilizes kernel tricks to map low-dimensional features to a high-dimensional space and searches for the optimal hyperplane in this high-dimensional space that maximizes the class distance. This significantly improves the classification model's ability to handle non-linearly separable data, greatly reducing the false positive rate while maintaining high recall, and generating a high-confidence list of confirmed defects.

[0075] Specifically, in one possible embodiment, the system pre-executes an offline training process: collecting 1000 positive samples containing various real defects (such as oil stains, crystal points, and insects) and 1000 negative samples containing deformation artifacts and alignment noise, and performing Z-score standardization on their feature vectors. Using a radial basis function as the kernel function, and employing a 5-fold cross-validation and grid search strategy, the optimal penalty coefficient (e.g., 10.0) and kernel parameter (e.g., 0.1) are optimized in logarithmic space to generate a highly generalizable SVM model file. During online detection, this model file is loaded, and the composite feature vectors to be classified are transmitted to the model as input data. The model calculates the kernel distance between the input vector and the support vectors, and performs a weighted summation using Lagrange multipliers and bias terms. Based on the output sign of the decision function, the feature vectors are divided into positive or negative classes, with the positive class corresponding to real defects. Based on this classification result, the original list is filtered, negative class objects are removed, and all positive class objects are encapsulated into a confirmed defect list.

[0076] Specifically, in step S63, the rejection delay pulse count is calculated based on the longitudinal pixel coordinates in the confirmed defect list and the real-time encoder pulse count to generate a rejection control command. It should be understood that because there is a physical distance between the visual inspection station and the downstream rejection mechanism, and the film conveying speed may fluctuate, directly outputting a signal would cause the action to be out of sync with the defect position, resulting in wasted good products or defect escape. Therefore, this application further calculates the rejection delay pulse count based on the longitudinal pixel coordinates in the confirmed defect list and the real-time encoder pulse count to generate a rejection control command. This establishes a precise spatiotemporal mapping between image coordinates and physical conveying distance, converting spatial position into a trigger delay based on pulse count. This ensures that the rejection mechanism acts precisely at the moment the defect's physical location reaches the execution point, achieving accurate online quality control.

[0077] Specifically, in one possible embodiment, if the confirmed defect list is not empty, the longitudinal center coordinates of each defect object in the list in the image coordinate system are extracted. The encoder reference pulse count value latched at the time of image acquisition for that frame is retrieved. To accurately synchronize the rejection action, the longitudinal pixel coordinates of the defect are first multiplied by the longitudinal resolution of the image (mm / pixel) to obtain the physical offset of the defect point relative to the imaging center. Subsequently, this offset is subtracted from the total physical distance from the camera optical center to the rejection mechanism execution point to obtain the remaining physical travel distance for the defect to reach the rejection position. Finally, this remaining physical travel distance is divided by the encoder pulse equivalent (mm / pulse) to determine the incremental pulse count required for the defect to reach the rejection station. This incremental pulse count is added to the reference pulse count value at the time of acquisition to obtain the target trigger count value (thus automatically compensating for time delays during image processing), and it is encapsulated as a control command and sent to the programmable logic controller. When the encoder count reaches the target value, the actuator is driven to move.

[0078] In summary, the machine vision-based FFS roll film surface defect detection method based on the embodiments of this application is explained. First, the original sensor data stream is reconstructed and enhanced through filtering and correction strategies to eliminate noise and optimize contrast, thus constructing a high-quality data foundation. Then, multi-scale feature extraction technology is used to capture the texture structure information of the image, driving a grid-based non-rigid elastic registration model to dynamically compensate for the nonlinear deformation of the flexible roll film during high-speed transmission, achieving precise spatial alignment between the image to be inspected and the standard template. Based on this, a high signal-to-noise ratio residual is calculated and combined with adaptive saliency segmentation technology to efficiently extract potential defect regions from complex backgrounds. Finally, multi-dimensional feature classification and logical judgment are performed on candidate objects to accurately distinguish between real defects and deformation artifacts, outputting rejection instructions. This effectively overcomes the false alarm problem caused by the deformation of flexible materials, improving the accuracy and robustness of the detection system.

[0079] Figure 9 This is a block diagram of a machine vision-based FFS roll film surface defect detection system according to an embodiment of this application. Figure 9 As shown, the machine vision-based FFS roll film surface defect detection system 100 according to an embodiment of this application includes: an image reconstruction and enhancement module 110, used to perform image reconstruction and enhancement processing on the original sensor data stream containing a row scanning grayscale matrix based on a Gaussian smoothing filter and gamma correction parameters to obtain a real-time image to be inspected; a multi-scale feature extraction module 120, used to perform multi-scale pyramid feature extraction on a preset standard defect-free template image and the real-time image to be inspected to obtain a template feature set and an image feature set to be inspected; and a non-rigid elastic registration module 130, used to perform multi-scale pyramid feature extraction on a template image and the real-time image to be inspected to obtain a template feature set and an image feature set to be inspected; and a non-rigid elastic registration module 130, used to perform model... The system consists of a plate feature set and a target image feature set. A grid-based non-rigid elastic registration is performed on the real-time image to obtain an elastically registered image. A high signal-to-noise ratio (SNR) residual calculation module 140 calculates the high SNR residual map between the elastically registered image and the standard defect-free template image. An adaptive saliency segmentation module 150 performs adaptive saliency segmentation on the high SNR residual map to extract a list of candidate defect connected components. A multidimensional classification decision module 160 performs multidimensional feature classification and decision output for each object in the candidate defect connected component list to obtain elimination control instructions.

[0080] As described above, the machine vision-based FFS roll film surface defect detection system 100 according to the embodiments of this application can be implemented in various wireless terminals, such as servers with machine vision-based FFS roll film surface defect detection algorithms. In one possible implementation, the machine vision-based FFS roll film surface defect detection system 100 according to the embodiments of this application can be integrated into the wireless terminal as a software module and / or hardware module. For example, the machine vision-based FFS roll film surface defect detection system 100 can be a software module in the operating system of the wireless terminal, or it can be an application developed for the wireless terminal; of course, the machine vision-based FFS roll film surface defect detection system 100 can also be one of many hardware modules of the wireless terminal.

[0081] Alternatively, in another example, the machine vision-based FFS roll film surface defect detection system 100 and the wireless terminal can also be separate devices, and the machine vision-based FFS roll film surface defect detection system 100 can be connected to the wireless terminal via wired and / or wireless networks, and transmit interactive information in accordance with an agreed data format.

[0082] Here, those skilled in the art will understand that the specific operations of each step in the machine vision-based FFS roll film surface defect detection system described above have been referenced. Figures 1 to 8 The method for detecting surface defects of FFS roll film based on machine vision has been described in detail, and therefore, its repeated description will be omitted.

Claims

1. A machine vision-based method for detecting surface defects in FFS roll film, characterized in that, include: S1: Based on the Gaussian smoothing filter and gamma correction parameters, the original sensor data stream containing the line scanning grayscale matrix is ​​reconstructed and enhanced to obtain the real-time image to be inspected. S2: Perform multi-scale pyramid feature extraction on the preset standard defect-free template image and the real-time image to be inspected to obtain the template feature set and the image feature set to be inspected; S3: Based on the template feature set and the feature set of the image to be inspected, perform grid-based non-rigid elastic registration on the real-time image to be inspected to obtain the elastically registered image; S4: Calculate the high signal-to-noise ratio residual map between the elastically registered image and the standard defect-free template image; S5: Perform adaptive saliency segmentation on the high signal-to-noise ratio residual map to extract a list of candidate defect connected components; S6: Perform multidimensional feature classification and decision output for each object in the candidate defect connected component list to obtain the elimination control instruction.

2. The machine vision-based FFS roll film surface defect detection method according to claim 1, characterized in that, Step S1 includes: Based on the encoder trigger signal contained in the original sensor data stream, linear array data is extracted in time series and stacked to fill the matrix buffer to obtain the initial grayscale frame; Gaussian convolution denoising is performed on the initial grayscale frame based on a Gaussian smoothing filter to obtain a denoised grayscale image. The grayscale distribution of the denoised grayscale image is nonlinearly mapped based on a preset gamma correction coefficient to obtain the real-time image to be inspected.

3. The machine vision-based FFS roll film surface defect detection method according to claim 2, characterized in that, Step S2 includes: Iterative Gaussian low-pass filtering and intermittent downsampling are performed on the real-time image to be inspected and the standard defect-free template image respectively to obtain the image pyramid to be inspected and the template image pyramid. The top-level FAST keypoint detection is performed on the pyramid of the image to be detected and the pyramid of the template image to obtain the keypoint set to be detected and the template keypoint set. The gray-level centroid direction of each key point in the key point set to be inspected and the brightness of the neighboring pixel pairs are compared to obtain the feature set of the image to be inspected and the feature set of the template.

4. The machine vision-based FFS roll film surface defect detection method according to claim 3, characterized in that, Step S3 includes: Feature matching and global homography estimation are performed on the feature set of the image to be inspected and the feature set of the template to obtain the global homography matrix and the list of matching feature point pairs; The mesh vertex state is initialized based on the global homography matrix, and the mesh energy is minimized and the deformation field is solved for the list of matching feature point pairs to obtain the deformed mesh field; Based on the deformable mesh field, image reshaping based on bilinear interpolation is performed on the real-time image to be inspected to obtain an elastically registered image.

5. The machine vision-based FFS roll film surface defect detection method according to claim 4, characterized in that, The mesh vertex states are initialized based on the global homography matrix, and the deformed mesh field is obtained by minimizing the mesh energy and solving the deformation field on the list of matching feature point pairs, including: Construct a total energy function that includes a robust data term and anisotropic physical canonical terms; Among them, the robust data term is obtained by calculating the residual for each pair of matched feature points and introducing a robust kernel function; The anisotropic physical regularization term is obtained by calculating the longitudinal second-order difference and the transverse second-order difference of each vertex in the mesh, and the machine direction stiffness coefficient is assigned to the longitudinal second-order difference, and the transverse stiffness coefficient is assigned to the transverse second-order difference. The minimum value of the total energy function is found to determine the optimal coordinate state of the grid vertices, thereby generating a deformable grid field.

6. The machine vision-based FFS roll film surface defect detection method according to claim 1, characterized in that, Step S4 includes: Calculate the absolute difference map between the image after elastic registration and the standard defect-free template image; Calculate the structural differences between the image after elastic registration and the standard defect-free template image; Based on a preset non-printed region weight mask, the absolute difference map and the structural difference map are spatially weighted and linearly fused to obtain a high signal-to-noise ratio residual map.

7. The machine vision-based FFS roll film surface defect detection method according to claim 6, characterized in that, Step S5 includes: Local significance enhancement is performed on the high signal-to-noise ratio residual plot to obtain a significance plot; Adaptive thresholding and morphological denoising are performed on the saliency map to obtain a fine binary mask; Connected components are extracted from the fine binary mask, and tiny spots with areas smaller than a preset value are removed to obtain a list of candidate defect connected components.

8. The machine vision-based FFS roll film surface defect detection method according to claim 1, characterized in that, Step S6 includes: The region of interest is extracted from the real-time image to be inspected using the object coordinates in the candidate defect connected component list, and the geometric circularity, gray-level mean statistics and texture entropy of the region of interest are calculated to obtain a composite feature vector set. The composite feature vector set is input into a support vector machine model based on radial basis kernel function to obtain a list of confirmed defects; Based on the vertical pixel coordinates in the confirmed defect list and the real-time encoder pulse count, the number of rejection delay pulses is calculated to generate rejection control commands.

9. A machine vision-based FFS roll film surface defect detection system, characterized in that, include: The image reconstruction and enhancement module is used to perform image reconstruction and enhancement processing on the original sensor data stream containing the line scanning grayscale matrix based on the Gaussian smoothing filter and gamma correction parameters to obtain the real-time image to be inspected. The multi-scale feature extraction module is used to perform multi-scale pyramid feature extraction on the preset standard defect-free template image and the real-time image to be inspected to obtain the template feature set and the image feature set to be inspected. The non-rigid elastic registration module is used to perform grid-based non-rigid elastic registration on the real-time image to be inspected based on the template feature set and the feature set of the image to be inspected in order to obtain the elastically registered image. The high signal-to-noise ratio residual calculation module is used to calculate the high signal-to-noise ratio residual map between the elastically registered image and the standard defect-free template image; The adaptive saliency segmentation module is used to perform adaptive saliency segmentation on high signal-to-noise ratio residual maps to extract a list of candidate defect connected components. The multidimensional classification decision module is used to perform multidimensional feature classification and decision output for each object in the candidate defect connected domain list to obtain elimination control instructions.

Images