Industrial spliced image defect detection and duplicate removal method based on dynamic overlapping region

Abstract

The invention discloses an industrial spliced image defect detection and deduplication method based on a dynamic overlapping region, and relates to the technical field of image processing, and the method comprises the steps: carrying out the weighted fusion of a motion parameter, a historical correction error parameter and a defect feature parameter which are collected in real time, calculating the overlapping region, and carrying out the detection and deduplication of an industrial spliced image. The method comprises the following steps: adjusting the size of an overlapping region in a closed-loop verification mode to enable the overlapping region to adapt to the motion state and defect change condition of a target object, then performing pre-correction on the overlapping region according to an inclination angle and an offset, and finally performing sub-pixel alignment, image splicing and defect de-weighting; according to the technical scheme, multi-dimensional information can be fused into dynamic calculation of the overlapping area so as to adapt to the motion state and defect distribution difference of the target object in real time, splicing errors caused by parameter changes are effectively reduced, the method can be combined with a process closed-loop linkage mechanism, and the splicing efficiency is improved. The problems of missing detection, poor adaptability and calculation redundancy of defect segmentation are solved, and high-precision image splicing and defect detection are realized.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more specifically, to a method for defect detection and deduplication of industrial stitched images based on dynamic overlapping regions. Background Technology

[0002] Current image stitching processing mainly employs fixed overlapping area stitching, adaptive feature stitching, and coating defect rating methods.

[0003] Fixed Overlapping Region Stitching: This method uses a pre-set, fixed-height overlapping region for image stitching. After defect detection, simple deduplication is performed based on coordinate range. This method is a common industry technique, but its drawback is its inability to flexibly adapt to changes in defect morphology, such as scratch length and pinhole density. Adaptive Feature Stitching (CN115082314A): This method uses the SIFT (Scale Invariant Feature Transform) algorithm to extract feature points from the image, labels regions according to the number of feature points, and then stitches them together. For defective regions, feature matching is used for stitching, while non-defective regions are processed using coordinate stitching. This method can achieve automated feature point matching to a certain extent, but it has shortcomings in dynamic adjustment, especially in adapting to electrode motion parameters (such as speed fluctuations and tension shifts), failing to fully respond to dynamic changes in image defect morphology. Coating Defect Rating Method (CN120612470A): This method uses segmented image stitching, extracts defect masks using a semantic segmentation model, and performs defect rating. The stitching process relies on improved corner detection and fixed-parameter feature matching. The advantage of this method is that it can classify and rate coating defects, but it still has limitations in dynamic overlap adjustment and real-time identification and repair of defects.

[0004] While existing technologies provide fundamental methods for image stitching and defect detection, they also reveal some significant shortcomings, mainly in the following aspects:

[0005] 1) Poor adaptability of overlapping areas: Existing fixed overlapping area stitching methods cannot respond to dynamic changes in defect morphology (e.g., scratch length and pinhole density) and changes in the state of the electrode during movement (e.g., speed fluctuations and tension shifts). For example, the original scheme only considers defect density, resulting in a relatively singular adaptation dimension; while CN115082314A uses an adaptive feature stitching method, it does not incorporate motion parameters for adaptive adjustment, thus failing to adequately address various changes during image stitching; CN120612470A does not provide a mechanism for dynamically adjusting overlapping areas, leading to inaccurate alignment of overlapping areas between frames. 2) Weak synergy between stitching and detection: In existing technologies, the setting of overlapping areas is disconnected from subsequent correction and deduplication processes. For example, feature point extraction in CN115082314A is not combined with prior defect distribution prediction, resulting in some invalid calculations; while the original scheme does not consider the impact of correction errors on overlapping areas, failing to achieve more accurate image stitching and defect detection. 3) Limited adjustment precision: Current technologies generally employ discrete adjustment levels (such as the original scheme's "two levels of size" and CN115082314A's "two-classification with or without features"). This discrete adjustment method cannot achieve continuous dynamic optimization and is difficult to adapt to micrometer-level detection requirements. Therefore, existing technologies struggle to meet the detail processing requirements in high-precision image stitching and defect detection, especially when dealing with complex defects. Summary of the Invention

[0006] The purpose of this invention is to address the problems of limited accuracy, single dimension of dynamic overlapping region adjustment, insufficient coordination with subsequent processes, and limited precision in existing technologies. This invention provides an optimized scheme that includes multi-parameter coupled dynamic calculation and closed-loop process linkage, achieving high-precision, high-adaptability, and high-efficiency image stitching and defect detection. This scheme dynamically generates overlapping region parameters by fusing parameters such as defect features, motion state, and correction error, and establishes a linkage mechanism with stitching correction and sub-pixel alignment. This solves problems such as missed defect segmentation, poor adaptability, and computational redundancy, thereby improving the accuracy and robustness of high-speed electrode detection.

[0007] The technical solution of this invention is: to provide a method for defect detection and deduplication of industrial stitched images based on dynamic overlapping regions, the method comprising:

[0008] Step 1: Real-time acquisition of motion parameters of the target object, and acquisition of defect feature parameters corresponding to various defects in the current frame image. Based on motion parameters, historical correction error parameters and defect feature parameters, the height and width of the overlapping area between the previous frame image and the current frame image are calculated by weighted fusion.

[0009] Step 2: Divide the feature point verification area in the overlapping area, and extract the first feature points of the previous frame image and the current frame image in the verification area respectively. Calculate the matching rate of the first feature points of the two frames. If the matching rate is less than or equal to the preset matching threshold, iteratively adjust the weight coefficient and update the height and width of the overlapping area until the matching rate is greater than the preset threshold.

[0010] Step 3: For the current frame image, extract n second feature points at predetermined positions on the edge of the target object, calculate the tilt angle and offset of the current frame image based on the coordinates of the n second feature points, and perform pre-correction on the overlapping area according to the tilt angle and offset.

[0011] Step 4: Extract the third feature points in the overlapping area of ​​the previous frame image and the current frame image respectively, and fit the transformation matrix from the overlapping area pixels of the current frame to the overlapping area pixels of the previous frame using the least squares method.

[0012] Step 5: Based on the transformation matrix and the height of the overlapping region, align the pixels of the current frame's overlapping region to the previous frame's overlapping region, and perform feature matching on the current frame's defects in the previous frame's overlapping region. If the matching is successful, mark the current frame's defects as duplicate defects; if the matching fails, do not mark them, and finally delete the duplicate defects.

[0013] Furthermore, in step 1, the motion parameters include velocity V and tension fluctuation value F, and the historical correction error parameters include tilt angle. And the offset Δx, the step of obtaining the defect feature parameters corresponding to various defects in the current frame image specifically includes:

[0014] The Canny operator is used to extract all defects in the current frame image. Then, a pre-trained defect classifier is used to identify the type of each defect to obtain the type T corresponding to each defect. The area S and density D corresponding to each type of defect are calculated. For a single type of defect, its type T, area S and density D are summarized to form the defect feature parameters.

[0015] Furthermore, step 1 also includes: setting fusion weight coefficients a, b, c, d, e, and f, based on the currently real-time acquired velocity V and tension fluctuation value F, and historical tilt angle. The offset Δx, along with the type T, area S, and density D corresponding to each type of defect, are used to calculate the height and width values ​​using a weighted fusion method. The maximum values ​​are then taken as the height H and width W of the overlapping region, respectively, as follows:

[0016] H={H o (a i ·S i +b·D i +c i ·F)} max

[0017] W={W o (d i ·T i +ei·θ+f i ·V)} max

[0018] In the formula, H o W serves as the reference parameter for the height of the overlapping region. o Here, is the baseline parameter for the height and width of the overlapping region, i is the index of different types of defects, and {·} max This represents taking the maximum value.

[0019] Furthermore, the fusion weight coefficients a, b, c, d, e, and f are obtained through a pre-trained MLP regression model, specifically including:

[0020] An MLP regression model is constructed, using historically collected motion parameters, historical correction error parameters, defect feature parameters, and the optimal fusion weight coefficients obtained from historical calibration as training samples. The model is trained by regression, and the model parameters are updated using the gradient backpropagation algorithm. Training stops when the loss converges to a preset range. The currently acquired motion parameters, historical correction error parameters, and defect feature parameters are then input into the MLP regression model to obtain the fusion weight coefficients.

[0021] Further, step 2 specifically includes: dividing the overlapping regions of the previous frame image and the current frame image into feature point verification regions with an area ratio of 10%, and ensuring that the verification regions in the two frames correspond to each other; extracting the first feature points from the verification regions of the previous frame image and the current frame image respectively; and calculating the matching rate of the first feature points corresponding to the two frames, expressed as:

[0022]

[0023] In the formula, P m Let M be the matching rate of the first feature point, M be the number of successfully matched feature points, and N1 be the number of first feature points extracted from the verification area of ​​the previous frame image. This represents the number of feature points extracted from the verification area of ​​the current frame image.

[0024] Furthermore, step 2 also includes: using the motion parameters generated in the current production process, historical correction error parameters, defect feature parameters, and fusion weight coefficients as new training samples, incorporating them into the existing historical training samples, using the updated training samples to retrain the MLP regression model, and using the gradient backpropagation algorithm to update the model parameters. When the loss converges to a preset range, training is stopped. The currently acquired motion parameters, historical correction error parameters, and defect feature parameters are then input into the MLP regression model again to obtain the adjusted weight coefficients. The height and width of the overlapping area are recalculated using the adjusted weight coefficients, and the matching rate of the first feature points corresponding to the two frames is recalculated. The above process is repeated until the feature point matching rate is greater than the preset matching threshold.

[0025] Further, step 3 specifically includes: for the pixel region where the target object is located in the current frame image, extract the coordinates of the two pixels on the left and right sides at 1 / 4 of its height, denoted as (x... L1 ,y L1 ), (x R1 ,y R1 ), and extract the coordinates of the two pixels on the left and right sides at 3 / 4 of the height, denoted as (x L2 ,y L2 ), (x R2 ,y R2 Based on the coordinates of these four second feature points, the tilt angle and offset of the current frame image are calculated, as follows:

[0026]

[0027]

[0028] In the formula, X is the tilt angle, Δx is the offset, and X is the tilt angle. c X represents the average edge position of the detected target object in the current frame image. r The known theoretical edge position of the target object corresponding to the correction reference frame or template frame.

[0029] Furthermore, in step 3, the overlapping area is pre-corrected based on the tilt angle and offset, specifically including:

[0030] When the absolute value of the tilt angle |θ| is greater than the preset angle threshold, the width W of the overlapping area is automatically expanded to 1.2 times. When the offset is greater than the preset number of pixels, offset compensation bands with a width of Δx are set on both sides of the overlapping area. Pre-correction calculation process:

[0031]

[0032] In the formula, W' is the width of the overlapping area after pre-correction, the preset angle threshold is 0.3°, and the preset number of pixels is 2.

[0033] Further, step 5 specifically includes: based on the transformation matrix A, mapping each pixel in the current frame's overlapping region to the previous frame's overlapping region, and combining the height of the overlapping region, precisely aligning the pixels in the current frame's overlapping region to the coordinate system of the previous frame's overlapping region, as shown below:

[0034]

[0035] In the formula, (x,y) are the pixel coordinates in the overlapping region of the current frame, and (x',y') are the pixel coordinates after alignment to the overlapping region of the previous frame.

[0036] Furthermore, in step 5, feature matching is performed on defects in the current frame within the overlapping region of the previous frame, specifically including:

[0037] For each defect in the overlapping region of the current frame, search for defects in the overlapping region of the previous frame within its neighborhood. Then, calculate the Euclidean distance, Hu moment similarity, gray-level variance ratio, and defect texture entropy between the current frame defect and each found defect in the previous frame. Weight the Euclidean distance, Hu moment similarity, gray-level variance ratio, and defect texture entropy to obtain a comprehensive similarity. If the comprehensive similarity is greater than a preset similarity threshold, the two defects are determined to match, and the current frame defect is marked as a duplicate defect. If the comprehensive similarity is less than or equal to the preset similarity threshold, the two defects are determined to not match, and the current frame defect is not a duplicate defect.

[0038] The beneficial effects of this invention are:

[0039] First, the technical solution of this invention utilizes real-time acquired motion parameters, historical correction errors, and defect feature parameters to calculate the overlapping region through weighted fusion. A feature point verification area is set within the overlapping region, and the size of the overlapping region is adjusted through closed-loop verification. This allows the overlapping region to flexibly adapt to the motion state (speed, tension fluctuations) and defect changes of the target object. Simultaneously, continuous optimization of the feature point matching rate ensures accurate alignment of the two images after stitching, effectively avoiding misalignment and error accumulation. The technical solution of this invention can integrate multi-dimensional information such as motion parameters (e.g., speed, tension fluctuations), historical correction errors (e.g., tilt angle and offset), and defect feature parameters (e.g., defect type, area, and density) into the dynamic calculation of the overlapping region through weighted fusion. This allows for real-time adaptation to changes in the target object's state during high-speed movement, addressing differences in defect distribution, and effectively reducing stitching errors caused by parameter changes. This ensures accurate adaptation of the overlapping region between consecutive frames. Compared to existing technologies, this invention, through adaptive dynamic calculation, not only avoids the adaptability problems of fixed overlapping region methods but also improves stitching accuracy and stability under high-speed and high-fluctuation conditions.

[0040] Secondly, the technical solution in this invention achieves high-precision image stitching and defect detection by combining multi-parameter coupled dynamic calculation with a closed-loop linkage mechanism. This solution comprehensively considers multiple factors such as motion parameters, historical correction errors, and defect characteristics to dynamically generate overlapping regions and establishes a linkage mechanism with stitching correction and sub-pixel alignment. This effectively solves problems such as missed defect segmentation, poor adaptability, and computational redundancy, significantly improving the accuracy and robustness of high-speed electrode detection and optimizing the stitching process. Compared with traditional methods, this invention provides higher adaptability and robustness, solving the problems of single-dimensional dynamic overlapping region adjustment, insufficient coordination with subsequent processes, and limited accuracy. Especially under dynamic working conditions, it can flexibly cope with the high-speed movement and tension fluctuations of the electrode, making defect detection more accurate and reliable.

[0041] Third, the technical solution of this invention uses the aforementioned pre-correction step to initially adjust the overlapping area of ​​the current frame image, ensuring the accuracy of feature point matching. In the subsequent sub-pixel alignment, the transformation matrix is ​​directly calculated using the least squares method, skipping the relatively time-consuming RANSAC algorithm, controlling the overall processing time to 5-8 milliseconds, saving computation time, and solving the problem of computational redundancy in traditional methods. Compared with traditional methods, after coarse alignment, this invention directly uses these feature points for least squares fitting, without relying on the RANSAC algorithm to select feature points. In this way, this invention can quickly calculate the final fine registration transformation matrix while minimizing registration error, achieving more efficient and accurate image registration, significantly improving processing speed and accuracy, and is especially suitable for detection scenarios with dynamic changes and high-speed motion. Attached Figure Description

[0042] The advantages of the above and additional aspects of the present invention will become apparent and readily understood in the description of the embodiments in conjunction with the following drawings, wherein:

[0043] Figure 1 This is a schematic flowchart of an industrial image stitching defect detection and deduplication method based on dynamic overlapping regions according to an embodiment of the present invention;

[0044] Figure 2 This is the image generated in Example 1 using the existing fixed overlapping region stitching method;

[0045] Figure 3 The image is generated using the method of this invention in Example 1;

[0046] Figure 4 This is the first set of images generated using the method of the present invention in Example 2;

[0047] Figure 5 This is the second set of images generated using the method of the present invention in Example 2;

[0048] Figure 6 These are images captured by the linear scan camera in Example 3;

[0049] Figure 7 yes Figure 6 Image of the area where the defect is located;

[0050] Figure 8 Example 3 uses the method of the present invention to... Figure 6 The image obtained by stitching together the images in the image;

[0051] Figure 9 yes Figure 8 Image of the area where the defect is located. Detailed Implementation

[0052] To better understand the above-mentioned objectives, features, and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments of the present invention and the features thereof can be combined with each other.

[0053] In the following description, many specific details are set forth in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and therefore the scope of protection of the invention is not limited to the specific embodiments disclosed below.

[0054] like Figure 1 As shown, this embodiment provides a method for defect detection and deduplication of industrial stitched images based on dynamic overlapping regions. The method includes:

[0055] Step 1: Real-time acquisition of motion parameters of the target object, and acquisition of historical correction error parameters and defect feature parameters corresponding to various defects in the current frame image. Based on the motion parameters, historical correction error parameters and defect feature parameters corresponding to various defects, the height and width of the overlapping area between the previous frame image and the current frame image are calculated by weighted fusion.

[0056] Specifically, the motion parameters of the target object (usually located on the conveyor) are collected in real time using a data acquisition device, including velocity V and tension fluctuation value F. Historical correction error parameters, including tilt angle, are directly retrieved from existing historical processing results. The system uses the offset Δx and the Canny operator to extract all defects in the current frame image. Then, it uses a pre-trained defect classifier to identify the type of each defect, obtaining the type T corresponding to each defect. The area S and density D corresponding to each type of defect are calculated. For a single type of defect, its type T, area S and density D are summarized to form defect feature parameters. Before entering the current frame image processing flow, the system has pre-recorded and saved the correction error parameters of the previous frame or the previous few frames, which can be directly used for the estimation of the overlapping area of ​​the current frame.

[0057] Set the fusion weighting coefficients a, b, c, d, e, and f, based on the currently real-time collected velocity V and tension fluctuation value F, and the historical tilt angle. The offset Δx, along with the type T, area S, and density D corresponding to each type of defect, are used to calculate the height and width values ​​using a weighted fusion method. The maximum values ​​are then taken as the height H and width W of the overlapping region, respectively, as follows:

[0058] H={H o (a i ·S i +b·D i +c i ·F)}max

[0059] W={W o (d i ·T i +ei·θ+f i ·V)} max

[0060] In the formula, H o W serves as the reference parameter for the height of the overlapping region. o Here, is the baseline parameter for the height and width of the overlapping region, i is the index of different types of defects, and {·} max This represents taking the maximum value.

[0061] It should be noted that the values ​​of the fusion weight coefficients a, b, c, d, e, and f are different for different types of defects. For example, the weight coefficient d = 1.5 for scratch defects and d = 1.0 for pinhole scratch defects. The fusion weight coefficients for each type of defect need to be obtained through a pre-trained MLP regression model, specifically including:

[0062] A small multilayer perceptron (MLP) regression model is constructed, using historically acquired motion parameters (velocity V, tension fluctuation value F) and historical correction error parameters (tilt angle). The model is trained using the following samples: offset Δx, defect feature parameters (type T, defect area S, defect density D), and optimal fusion weight coefficients (a, b, c, d, e, f) obtained from historical calibration. The model is then trained using regression (least squares loss function), and the model parameters are updated using gradient backpropagation. As training iterates, when the loss converges to a preset range, the MLP regression model learns the mapping relationship between input features and fusion weight coefficients. In actual operation, the MLP can use the currently collected V, F, and... The system outputs a set of optimal initial weight coefficients a, b, c, d, e, and f in real time, which are used for subsequent calculations of the height and width of the overlapping area.

[0063] In this embodiment, the speed V can be acquired by an encoder or laser velocimeter installed on the conveying device, and the tension fluctuation value F can be obtained by the feedback signal of the tension sensor or tension control system. The Canny operator can perform edge detection on the image and extract all suspected defective regions; the pre-trained defect classifier can identify the type of each defective region and output the type parameter. In this embodiment, the defect classifier can use a traditional machine learning classifier (such as SVM, KNN, etc.) or a convolutional neural network (CNN), and its output type parameter T is a specific value, such as T=1.5 representing a scratch defect and T=1.0 representing a pinhole defect. The values ​​corresponding to different types of defects can be defined based on actual needs.

[0064] Step 2: Divide the overlapping area into a feature point verification area, and extract the first feature point of the previous frame image and the current frame image in the verification area respectively. Calculate the matching rate of the first feature point of the two frames. If the matching rate is less than or equal to the preset matching threshold, update the height and width of the overlapping area by iteratively adjusting the weight coefficient until the feature point matching rate is greater than the preset threshold.

[0065] Specifically, feature point verification regions with an area ratio of 10% are defined in the overlapping regions of the previous and current frames, and the verification regions in the two frames correspond to each other. The first feature point is extracted from the verification regions of the previous and current frames, and the matching rate of the first feature points corresponding to the two frames is calculated, expressed as:

[0066]

[0067] In the formula, P m Let M be the matching rate of the first feature point, M be the number of successfully matched feature points (i.e., the number of successfully paired feature points in the two frames), and N1 be the number of the first feature points extracted from the verification area of ​​the previous frame image. This represents the number of feature points extracted from the verification area of ​​the current frame image.

[0068] If the matching rate is less than or equal to the preset matching threshold, the height and width of the overlapping region are updated by iteratively adjusting the weight coefficients until the feature point matching rate is greater than the preset matching threshold. The specific steps include the following:

[0069] The motion parameters (velocity V, tension fluctuation value F) generated in the current production process and the historical correction error parameters (tilt angle) are used to determine the motion parameters (velocity V, tension fluctuation value F) generated in the current production process. The offset Δx), defect feature parameters (type T, defect area S, defect density D), and fusion weight coefficients (a, b, c, d, e, f) are used as new training samples and incorporated into the existing historical training samples. The new training samples include positive and negative samples. The MLP regression model is trained again using the updated training samples, and the model parameters are updated using the gradient backpropagation algorithm. Training stops when the loss converges to a preset range. The MLP regression model can better learn the mapping relationship between input features and fusion weight coefficients. The currently acquired motion parameters, historical correction error parameters, and defect feature parameters are input into the MLP regression model again to obtain the adjusted weight coefficients. The height and width of the overlapping area are recalculated using the adjusted weight coefficients, and the matching rate of the first feature point corresponding to the two frames is recalculated. The above process is repeated until the feature point matching rate is greater than the preset matching threshold. When the feature point matching rate is greater than the preset threshold, the next step of the operation is performed.

[0070] In this embodiment, the preset matching threshold can be set to 0.9, which means that the matching rate of the first feature point is required to reach 90%.

[0071] In this embodiment, feature points can be extracted from the image using methods such as SIFT (Scale-Invariant Feature Transform) and Harris Corner; corresponding matching of the first feature points in two frames can be performed using methods such as Brute-Force Matching and KNN Matching (K-Nearest Neighbors).

[0072] Step 3: For the current frame image, extract n second feature points at predetermined positions on the edge of the target object, calculate the tilt angle and offset of the current frame image based on the coordinates of the n feature points, and perform pre-correction on the overlapping area according to the tilt angle and offset.

[0073] Specifically, for the pixel region where the target object is located in the current frame image, extract the coordinates of the two pixels on the left and right sides at 1 / 4 of its height (i.e., the intersection of the horizontal line at 1 / 4 of the height and the left and right boundaries of the region where the target object is located), denoted as (x... L1 ,y L1 ), (x R1 ,y R1 ), and extract the coordinates of the two pixels on the left and right sides at 3 / 4 height (i.e., the intersection of the horizontal line at three-quarters height and the left and right boundaries of the target object area), denoted as (x L2 ,y L2 ), (x R2 ,y R2Based on the coordinates of these four second feature points, the tilt angle and offset of the current frame image are calculated, as follows:

[0074]

[0075]

[0076] In the formula, X is the tilt angle, Δx is the offset, and X is the tilt angle. c X represents the average edge position of the detected target object in the current frame image. r The known theoretical edge position of the target object corresponding to the correction reference frame or template frame.

[0077] It should be noted that the method of the present invention can be used to stitch together polarimetric images. When extracting the second feature points from a single frame polarimetric image, four pixels at a predetermined position on the coating edge in the image can be extracted as the second feature points.

[0078] Pre-correction of overlapping areas is performed based on tilt angle and offset, specifically including:

[0079] When the absolute value of the tilt angle |θ| is greater than the preset angle threshold, the width W of the overlapping area is automatically expanded to 1.2 times to compensate for the risk of defect segmentation caused by rotation; when the offset is greater than the preset number of pixels, an offset compensation band with a width of Δx is set on both sides of the overlapping area to avoid missing edge defects. The preset angle threshold is 0.3° and the preset number of pixels is 2. The pre-correction calculation process is as follows:

[0080]

[0081] In the formula, W' is the width of the overlapping region after pre-correction.

[0082] In this embodiment, when performing image processing, the actual movement direction of the target object is defined as the height direction of the image (i.e., the up-down direction), and the left and right sides of the target object perpendicular to this direction correspond to the horizontal direction of the image (i.e., the left-right direction).

[0083] Step 4: Extract the third feature points in the overlapping areas of the previous frame image and the current frame image after pre-correction, and calculate the transformation matrix from the overlapping area pixels of the current frame to the overlapping area pixels of the previous frame by least squares fitting, thereby achieving sub-pixel alignment of the overlapping areas of the two frames.

[0084] Specifically, M third feature points are extracted from the overlapping regions of the previous frame and the current frame, respectively denoted as (x 2,j ,y 2,j ) and (x 1,j ,y 1,j), where j is the index of the third feature point, j=1,2,...,M, where the number of third feature points extracted in high-density defect areas is greater than that in low-density areas. For example, 30-40 feature points can be extracted in high-density defect areas and 20-25 feature points can be extracted in low-density areas within overlapping regions; (x 2,j ,y 2,j ) and (x 1,j ,y 1,j The mapping relationship between them is expressed as follows:

[0085]

[0086] In the formula, A is the transformation matrix, a 11 and a 12 To control the parameters of rotation and scaling in the x-direction, a 21 and a 22 To control the parameters of rotation and scaling in the y-direction, t x and t y These are parameters used to control displacement in the x and y directions.

[0087] The transformation matrix is ​​calculated using the least squares method, and an error function is constructed. For each pair of matching feature points in the overlapping regions of the previous and current frames, the parameters in the transformation matrix A are continuously adjusted to minimize the error function, ultimately obtaining the transformation matrix A. The error function is expressed as:

[0088]

[0089] In the formula, E is the error function.

[0090] It should be noted that during feature matching, extracting 30-40 feature points from high-density defect areas and 20-25 feature points from low-density defect areas allows for a more accurate estimation of the transformation matrix of the overlapping region, while keeping the computation time within 5-8 milliseconds. By selecting a reasonable feature point density, a balance between computational accuracy and processing speed can be effectively achieved, avoiding computational overload caused by too many feature points. Using the least squares method to fit the transformations between these feature points enables high-precision sub-pixel alignment, ensuring the accuracy of image stitching or alignment.

[0091] Step 5: Based on the transformation matrix and the height of the overlapping region, align the pixels of the current frame's overlapping region to the previous frame's overlapping region, and perform feature matching on the current frame's defects in the previous frame's overlapping region. If there is a defect in the previous frame's overlapping region that matches the current frame's defect, then mark the current frame's defect as a duplicate defect. If there is no defect in the previous frame's overlapping region that matches the current frame's defect, then do not mark it, and finally delete the duplicate defects.

[0092] Specifically, based on the transformation matrix A, each pixel in the overlapping region of the current frame is mapped to the overlapping region of the previous frame. Combined with the height of the overlapping region, the pixels in the overlapping region of the current frame are precisely aligned to the coordinate system of the overlapping region of the previous frame, achieving accurate alignment of the overlapping regions of the two images. This is represented as:

[0093]

[0094] In the formula, (x,y) are the pixel coordinates in the overlapping region of the current frame, and (x',y') are the pixel coordinates after alignment to the overlapping region of the previous frame.

[0095] Feature matching is performed on defects in the current frame within the overlapping region of the previous frame, specifically including:

[0096] For each defect in the overlapping region of the current frame, search for defects in the overlapping region of the previous frame within its neighborhood. Then, calculate the Euclidean distance (representing position), Hu moment similarity (representing shape), gray-level variance ratio (representing gray level), and defect texture entropy (representing texture) between the current frame defect and each found defect in the previous frame. Weighted summation of the Euclidean distance, Hu moment similarity, gray-level variance ratio, and defect texture entropy yields a comprehensive similarity score. If the comprehensive similarity score is greater than a preset similarity threshold, the two defects are considered a match, and the current frame defect is marked as a duplicate defect. If the comprehensive similarity score is less than or equal to the preset similarity threshold, the two defects are considered a mismatch, and the current frame defect is not a duplicate defect. The comprehensive similarity score is expressed as:

[0097]

[0098] In the formula, For positional weights, For shape weights, For grayscale weights, For texture weights, S com To assess overall similarity, S pos S is the Euclidean distance. shape For Hu moment similarity, S gray S represents the grayscale variance ratio. texture This represents the entropy of the defect texture.

[0099] In this embodiment, the position weight can be set to 0.4, the shape weight to 0.3, the grayscale weight to 0.2, the texture weight to 0.1, and the preset similarity threshold to 0.8. After the overlapping areas of the two consecutive frames are accurately aligned, the two images can be stitched together using methods such as linear weighted average or Laplacian pyramid fusion.

[0100] It should be noted that Euclidean distance is the straight-line distance between two points (usually the center point of a region), a common metric used to compare the location of defects in an image; Hu moments are a classic method for describing the shape features of an image, using the moments of the image to describe the shape and compare similarities; gray-level variance ratio is used to evaluate the variation of pixel gray levels within an image region, and is widely used for image comparison and matching; texture entropy is an indicator of the complexity of image texture, often used for texture analysis and matching of image features.

[0101] Example 1:

[0102] The existing fixed overlapping area stitching method and the method of the present invention were used respectively to stitch together the images of the electrode taken during the movement, so as to verify the effectiveness and robustness of the method of the present invention under dynamic working conditions. Specifically, an 8192×3000 resolution line scan camera with a frame rate of 500fps was used to simultaneously acquire the electrode speed (encoder) and tension (tension sensor) signals.

[0103] After completing the splicing, you get Figure 2 and Figure 3 , Figure 2 For images generated using existing fixed overlap region stitching methods, Figure 3 As can be seen from the image generated by the method of this invention, misalignment occurs in the image generated by the existing fixed overlapping area stitching method. This is because the coating area of ​​the second image has shifted, and the tilt angle and offset exceed the preset range. Direct stitching causes some pixels in the image to be misaligned, making it impossible to completely restore the true shape and structural features of the electrode. However, no misalignment occurs in the image generated by the method of this invention, and the two images are smoothly stitched together with a natural transition and clear details. The continuity and structural integrity of the coating area of ​​the electrode are completely preserved. Therefore, it can be seen that the method of this invention can effectively adapt to the positional changes and deformations of the electrode during movement, has stronger dynamic stitching ability and adaptability, significantly improves the registration accuracy and stitching robustness of images under complex motion conditions, and effectively overcomes the interference caused by motion and deformation.

[0104] Example 2:

[0105] The method of this invention is used to stitch together images of the electrode taken during the motion process. An 8192×3000 resolution line scan camera with a frame rate of 500fps is used to simultaneously acquire the electrode speed (encoder) and tension (tension sensor) signals.

[0106] After splicing, you get Figure 4 and Figure 5 Two sets of images, Figure 4 (a) in the image is the previous frame before stitching. Figure 4 (b) in the image is the stitched image of the previous frame. Figure 4 (c) in the image is the highly compressed version of the previous frame displayed on the device. Figure 5 (a) in the image is the previous frame before stitching. Figure 5 (b) in the image is the stitched image of the previous frame. Figure 5 Image (c) shows the highly compressed image of the previous frame displayed on the device. The gray area represents the region covered by the target electrode (corresponding to the green line in the image), and the electrode region contains defects (corresponding to the red line in the image). Comparing the two sets of images, it can be seen that the size of the overlapping area is jointly determined by motion parameters, correction error parameters, and defect feature parameters. Since the motion parameters of the two sets of images are consistent with the historical correction error parameters, the difference in the overlapping area directly reflects the dynamic response to defect features (area S and density D). Figure 4 The overlapping area is small. Figure 5 The overlapping area is relatively large, and the overlapping area in both cases can just cover the defects, ensuring the complete preservation and seamless integration of defects during the splicing process and avoiding information loss. Therefore, this invention can provide a complete and reliable image basis for subsequent accurate defect identification and quantitative analysis, and significantly improve the accuracy and reliability of electrode quality detection.

[0107] Example 3:

[0108] The method of this invention is used to stitch together images with incomplete defects, such as... Figures 6 to 7 As shown, Figure 6 This is the previous frame image captured by the line scan camera. Figure 7 This refers to the region where the defect is located in the previous frame image. Below this region, there are incomplete defects formed by segmentation. In subsequent defect detection, these incomplete defects formed by segmentation are prone to being missed or detected repeatedly.

[0109] After splicing is completed, as follows Figures 8 to 9 As shown, Figure 8 This is the previous frame image after stitching. Figure 9 The image shows the area where the defect is located in the stitched image. It can be seen that the polarimetric coating area of ​​the stitched image is complete and continuous, with a natural transition and clear details. In the area where the defect is located, the incomplete defect at the bottom caused by segmentation has been repaired and can be fully presented without any misalignment or distortion. Therefore, it can be seen that the method of the present invention can effectively restore the image defects caused by segmentation, improve the accuracy of defect detection, and avoid the risk of missed detection or repeated detection.

[0110] It should be noted that experimental tests show that the multi-parameter coupling model used in this invention reduces the defect segmentation false negative rate by more than 60%, avoiding the risk of missed or repeated detections. It can adapt to dynamic scenarios with electrode speeds ranging from 0.5-5 m / s and tension fluctuations of ±5 N, while the industry standard for electrode detection speed is typically 0.83-2 m / s. Compared with existing technologies, this invention's method has stronger adaptability and robustness, maintaining high-precision defect detection even under high-speed movement and changing tension conditions. Compared to the adaptive feature stitching method, this invention does not require full-image feature extraction, significantly reducing the number of feature points and reducing the stitching error to <1 pixel, meeting the requirements for micron-level defect detection. This invention reduces unnecessary computation through a dynamic weight adjustment mechanism and, combined with an on-demand feature point extraction strategy in overlapping areas, reduces the computation time of the stitching process compared to the adaptive feature stitching method. Furthermore, this invention improves the success rate of feature point matching through a closed-loop verification mechanism, solving the problem of missed detection of small defects caused by stitching misalignment.

[0111] The steps in this invention can be adjusted, combined, or deleted according to actual needs.

[0112] The units in the device of the present invention can be merged, divided, or reduced according to actual needs.

[0113] In this invention, the terms "installation," "connection," "linking," and "fixing" should be interpreted broadly. For example, "connection" can be a fixed connection, a detachable connection, or an integral connection; "linking" can be a direct connection or an indirect connection through an intermediate medium. Those skilled in the art can understand the specific meaning of these terms in this invention according to the specific circumstances.

[0114] The shapes of the components in the accompanying drawings are schematic and may differ from their actual shapes. The drawings are only used to illustrate the principles of the present invention and are not intended to limit the present invention.

[0115] Although the invention has been disclosed in detail with reference to the accompanying drawings, it should be understood that these descriptions are merely exemplary and not intended to limit the application of the invention. The scope of protection of the invention is defined by the appended claims and may include various modifications, alterations, and equivalents made to the invention without departing from the scope and spirit of the invention.

Claims

1. A method for defect detection and deduplication in industrial stitched images based on dynamic overlapping regions, characterized in that, The method includes: Step 1: Real-time acquisition of motion parameters of the target object, and acquisition of defect feature parameters corresponding to various defects in the current frame image. Based on motion parameters, historical correction error parameters and defect feature parameters, the height and width of the overlapping area between the previous frame image and the current frame image are calculated by weighted fusion. Step 2: Divide the feature point verification area in the overlapping area, and extract the first feature points of the previous frame image and the current frame image in the verification area respectively. Calculate the matching rate of the first feature points of the two frames. If the matching rate is less than or equal to the preset matching threshold, iteratively adjust the weight coefficient and update the height and width of the overlapping area until the matching rate is greater than the preset threshold. Step 3: For the current frame image, extract n second feature points at predetermined positions on the edge of the target object, calculate the tilt angle and offset of the current frame image based on the coordinates of the n second feature points, and perform pre-correction on the overlapping area according to the tilt angle and offset. Step 4: Extract the third feature points in the overlapping area of ​​the previous frame image and the current frame image respectively, and fit the transformation matrix from the overlapping area pixels of the current frame to the overlapping area pixels of the previous frame using the least squares method. Step 5: Based on the transformation matrix and the height of the overlapping region, align the pixels of the current frame's overlapping region to the previous frame's overlapping region, and perform feature matching on the current frame's defects in the previous frame's overlapping region. If the matching is successful, mark the current frame's defects as duplicate defects; if the matching fails, do not mark them, and finally delete the duplicate defects.

2. The method for defect detection and deduplication of industrial stitched images based on dynamic overlapping regions as described in claim 1, characterized in that, In step 1, the motion parameters include velocity V and tension fluctuation value F, and the historical correction error parameters include tilt angle. And the offset Δx, the step of obtaining the defect feature parameters corresponding to various defects in the current frame image specifically includes: The Canny operator is used to extract all defects in the current frame image. Then, a pre-trained defect classifier is used to identify the type of each defect to obtain the type T corresponding to each defect. The area S and density D corresponding to each type of defect are calculated. For a single type of defect, its type T, area S and density D are summarized to form the defect feature parameters.

3. The method for defect detection and deduplication of industrial stitched images based on dynamic overlapping regions as described in claim 2, characterized in that, Step 1 further includes: Set the fusion weighting coefficients a, b, c, d, e, and f, based on the currently real-time collected velocity V and tension fluctuation value F, and the historical tilt angle. The offset Δx, along with the type T, area S, and density D corresponding to each type of defect, are used to calculate the height and width values ​​using a weighted fusion method. The maximum values ​​are then taken as the height H and width W of the overlapping region, respectively, as follows: H={H o (a i ·S i +b·D i +c i ·F)} max ; W={W o (d i ·T i +ei·θ+f i ·V)} max ; In the formula, H o W serves as the reference parameter for the height of the overlapping region. o Here, is the baseline parameter for the height and width of the overlapping region, i is the index of different types of defects, and {·} max This represents taking the maximum value.

4. The method for defect detection and deduplication of industrial stitched images based on dynamic overlapping regions as described in claim 3, characterized in that, The fusion weight coefficients a, b, c, d, e, and f are obtained through a pre-trained MLP regression model, specifically including: An MLP regression model is constructed, using historically collected motion parameters, historical correction error parameters, defect feature parameters, and the optimal fusion weight coefficients obtained from historical calibration as training samples. The model is trained by regression, and the model parameters are updated using the gradient backpropagation algorithm. Training stops when the loss converges to a preset range. The currently acquired motion parameters, historical correction error parameters, and defect feature parameters are then input into the MLP regression model to obtain the fusion weight coefficients.

5. The method for defect detection and deduplication of industrial stitched images based on dynamic overlapping regions as described in claim 4, characterized in that, Step 2 specifically includes: Feature point verification regions with an area ratio of 10% are defined in the overlapping regions of the previous frame and the current frame, respectively, and the verification regions in the two frames correspond to each other. The first feature point is extracted from the verification regions of the previous frame and the current frame, respectively, and the matching rate of the first feature points corresponding to the two frames is calculated, expressed as: ； In the formula, P m Let M be the matching rate of the first feature point, M be the number of successfully matched feature points, and N1 be the number of first feature points extracted from the verification area of ​​the previous frame image. This represents the number of feature points extracted from the verification area of ​​the current frame image.

6. The method for defect detection and deduplication of industrial stitched images based on dynamic overlapping regions as described in claim 5, characterized in that, Step 2 also includes: The motion parameters, historical correction error parameters, defect feature parameters, and fusion weight coefficients generated in the current production process are used as new training samples and incorporated into the existing historical training samples. The updated training samples are then used to retrain the MLP regression model, and the model parameters are updated using the gradient backpropagation algorithm. Training stops when the loss converges to a preset range. The currently acquired motion parameters, historical correction error parameters, and defect feature parameters are then input into the MLP regression model again to obtain the adjusted weight coefficients. The height and width of the overlapping region are then recalculated using the adjusted weight coefficients, and the matching rate of the first feature points corresponding to the two frames is recalculated. The above process is repeated until the feature point matching rate is greater than the preset matching threshold.

7. The method for defect detection and deduplication of industrial stitched images based on dynamic overlapping regions as described in claim 3, characterized in that, Step 3 specifically includes: For the pixel region where the target object is located in the current frame image, extract the coordinates of the two pixels to the left and right of its 1 / 4 height, denoted as (x... L1 ,y L1 ), (x R1 ,y R1 ), and extract the coordinates of the two pixels on the left and right sides at 3 / 4 of the height, denoted as (x L2 ,y L2 ), (x R2 ,y R2 Based on the coordinates of these four second feature points, the tilt angle and offset of the current frame image are calculated, as follows: ； ； In the formula, X is the tilt angle, Δx is the offset, and X is the tilt angle. c X represents the average edge position of the detected target object in the current frame image. r The known theoretical edge position of the target object corresponding to the correction reference frame or template frame.

8. The method for defect detection and deduplication of industrial stitched images based on dynamic overlapping regions as described in claim 7, characterized in that, In step 3, the overlapping area is pre-corrected based on the tilt angle and offset, specifically including: When the absolute value of the tilt angle |θ| is greater than the preset angle threshold, the width W of the overlapping area is automatically expanded to 1.2 times. When the offset is greater than the preset number of pixels, offset compensation bands with a width of Δx are set on both sides of the overlapping area. Pre-correction calculation process: ； In the formula, W' is the width of the overlapping area after pre-correction, the preset angle threshold is 0.3°, and the preset number of pixels is 2.

9. The method for defect detection and deduplication of industrial stitched images based on dynamic overlapping regions as described in claim 3, characterized in that, Step 5 specifically includes: Based on the transformation matrix A, each pixel in the overlapping region of the current frame is mapped to the overlapping region of the previous frame. Combined with the height of the overlapping region, the pixels in the overlapping region of the current frame are precisely aligned to the coordinate system of the overlapping region of the previous frame, as shown below: ； In the formula, (x,y) are the pixel coordinates in the overlapping region of the current frame, and (x',y') are the pixel coordinates after alignment to the overlapping region of the previous frame.

10. The method for defect detection and deduplication of industrial stitched images based on dynamic overlapping regions as described in claim 1, characterized in that, In step 5, feature matching is performed on defects in the current frame within the overlapping region of the previous frame, specifically including: For each defect in the overlapping region of the current frame, search for defects in the overlapping region of the previous frame within its neighborhood. Then, calculate the Euclidean distance, Hu moment similarity, gray-level variance ratio, and defect texture entropy between the current frame defect and each found defect in the previous frame. Weight the Euclidean distance, Hu moment similarity, gray-level variance ratio, and defect texture entropy to obtain a comprehensive similarity. If the comprehensive similarity is greater than a preset similarity threshold, the two defects are determined to match, and the current frame defect is marked as a duplicate defect. If the comprehensive similarity is less than or equal to the preset similarity threshold, the two defects are determined to not match, and the current frame defect is not a duplicate defect.

Images