An adaptive stitching method and system for scratch images

Abstract

The present application relates to the technical field of image processing, and discloses a kind of self-adapting splicing method and system for scratch image.The method includes by comparing the matching score of overlapping area in different directions, automatically select the optimal splicing direction;Along the direction, overlapping area is divided into multiple calculation windows, and the vertical offset of each window is obtained using hybrid registration method, and the global offset is obtained by weighted fusion;In continuous splicing, the difference between current offset and historical value is monitored in real time, and if it exceeds the threshold, trigger the template matching recalculation and correction based on global area to block error accumulation;Finally, geometric alignment and gradual fusion are performed according to the corrected offset.The method realizes the adaptive determination of splicing direction and real-time error correction of splicing process, and improves the robustness and overall accuracy of scratch image splicing under complex conditions.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, specifically to an adaptive stitching method and system for scratch images. Background Technology

[0002] In fields such as materials science, surface engineering, and coating adhesion research, scratch testing is a crucial quantitative evaluation method. This test involves scratching the sample surface with a continuously loaded diamond indenter while simultaneously observing the surface morphology using an integrated optical microscope.

[0003] In practice, due to the limited field of view of high-resolution optical microscopes, a single microscope image cannot completely cover the entire scratch trajectory from start to finish. Therefore, to achieve panoramic observation of the scratch, a series of local high-resolution images acquired along the scratch trajectory during the test need to be seamlessly stitched into a complete panoramic image. Existing image stitching techniques typically pre-set a fixed stitching direction. This method works when the device movement trajectory strictly conforms to the preset direction and the relative relationships between images are simple and stable. However, in practice, due to the uncertainty of device movement or changes in sample placement angle, the actual direction of the acquired sequence may not match the preset direction, causing the fixed-direction-based matching algorithm to fail or produce large registration errors, resulting in stitching failure or distorted results.

[0004] Conventional image stitching processes, after obtaining the offset parameters for a single stitch, directly apply them to image transformation and fusion before proceeding to the next frame. When processing long sequences of images continuously, the registration error of each frame propagates and accumulates throughout the stitching process. If a subtle, undetectable deviation exists in the stitching of a previous frame, subsequent frames will use this erroneous position as a reference for registration, causing the error to amplify and ultimately resulting in a significant deviation of the overall stitching path from reality, leading to obvious misalignment or discontinuities. This is particularly problematic for long, continuous defects like scratches, where accumulated errors disrupt the continuity of their shape and measurement accuracy. Therefore, adapting the stitching direction to the actual acquisition conditions and suppressing error propagation during the stitching process in real time are key challenges for improving the robustness and accuracy of scratch image stitching. Summary of the Invention

[0005] The purpose of this invention is to provide an adaptive stitching method and system for scratch images, so as to solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides an adaptive stitching method for scratch images, the method comprising:

[0007] Collect the overlapping area of ​​adjacent images in the scratch image sequence;

[0008] Based on the image matching score of the overlapping region, the matching score comparison results of each possible splicing direction are obtained;

[0009] Based on the matching score comparison results, the optimal splicing direction is automatically determined and selected;

[0010] Based on the optimal stitching direction, the overlapping region is divided into multiple calculation windows in the vertical direction; for each calculation window, the vertical offset of each calculation window is obtained based on the hybrid registration of feature matching and template matching.

[0011] Based on the matching quality weight of each calculation window, the vertical offset of each calculation window is weighted and fused to obtain the global vertical offset.

[0012] During continuous stitching, the difference between the current global vertical offset and the historical global vertical offset is monitored in real time; when the difference exceeds a preset threshold, template matching is triggered to recalculate and correct the global overlapping area.

[0013] Based on the global vertical offset, geometric alignment is performed by adding pixel borders;

[0014] In the overlapping area of ​​the aligned images, pixel-level weighted average fusion is performed using a linearly gradient weight mask to complete the stitching.

[0015] Preferably, the step of obtaining the matching score comparison results for each possible stitching direction based on the image matching score of the overlapping region includes:

[0016] For adjacent images in a scratch image sequence, calculate the image matching score of the overlapping region in the stitching direction from left to right, and the image matching score of the overlapping region in the stitching direction from right to left.

[0017] The matching score of the left-to-right splicing direction is compared with the matching score of the right-to-left splicing direction, and the direction with the higher matching score is selected as the optimal direction candidate in the matching score comparison results.

[0018] Preferably, the step of automatically determining and selecting the optimal splicing direction based on the matching score comparison result includes:

[0019] When the matching score of the left-to-right splicing direction is greater than the matching score of the right-to-left splicing direction, the left-to-right splicing direction is selected as the optimal splicing direction.

[0020] Otherwise, choose the right-to-left splicing direction as the optimal splicing direction.

[0021] Preferably, dividing the overlapping region into multiple calculation windows in the vertical direction includes:

[0022] The overlapping area is evenly divided into multiple rectangular windows at a preset height, with each window covering the entire width of the overlapping area.

[0023] Preferably, the hybrid registration based on feature matching and template matching, to obtain the vertical offset of each calculation window, includes:

[0024] For each calculation window, the number of feature matching point pairs and the vertical offset based on feature matching are obtained through a feature matching algorithm;

[0025] For each calculation window, the similarity score of the template match and the vertical offset based on the template match are obtained through the template matching algorithm;

[0026] Based on the number of feature matching point pairs and the similarity score of the template matching, the vertical offset based on feature matching or the vertical offset based on template matching is selected as the vertical offset of each calculation window.

[0027] Preferably, selecting the vertical offset based on feature matching or the vertical offset based on template matching based on the number of feature matching point pairs and the similarity score of template matching includes:

[0028] When the number of feature matching point pairs is greater than or equal to a preset threshold, the vertical offset based on feature matching is used as the vertical offset of the calculation window.

[0029] Otherwise, use the template-match-based vertical offset as the vertical offset for calculating the window.

[0030] Preferably, the step of weighted fusion of the vertical offsets of each calculation window based on the matching quality weights of each calculation window to obtain the global vertical offset includes:

[0031] For each calculation window, the number of matching points extracted by the feature matching algorithm is used as the matching quality weight for each calculation window;

[0032] The sum of the products of the vertical offset of each calculation window and the corresponding matching quality weight is divided by the sum of all matching quality weights to obtain the global vertical offset.

[0033] Preferably, the triggering of recalculation and correction using template matching in the globally overlapping region includes:

[0034] When the difference between the current global vertical offset and the historical global vertical offset exceeds a preset threshold, a template matching algorithm is executed over the entire overlapping area to calculate a new vertical offset.

[0035] Replace the current global vertical offset with the new vertical offset for geometric alignment.

[0036] Preferably, the geometric alignment by adding pixel borders includes:

[0037] Based on the sign and magnitude of the global vertical offset, add a black pixel border above or below an image to align the two images in the overlapping area.

[0038] The pixel-level weighted average fusion using a linearly gradient weight mask includes:

[0039] In the overlapping region, the first image is assigned a weight that decreases linearly from one to zero, and the second image is assigned a weight that increases linearly from zero to one.

[0040] For each pixel, a weighted average is calculated based on the weights to obtain the merged pixel value.

[0041] Preferably, the present invention also includes an adaptive stitching system for scratch images, the system including a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor, when executing the computer program, implements the steps of the adaptive stitching method for scratch images as described above.

[0042] Compared with the prior art, the beneficial effects of the present invention are:

[0043] This method compares the matching scores of overlapping areas in different possible stitching directions and automatically selects the direction with the highest score as the optimal stitching direction. This process eliminates the need for pre-fixed directions, allowing the stitching direction to be entirely determined by the image content itself. Regardless of whether the acquisition device moves horizontally, vertically, or diagonally, as long as there is sufficient overlap between adjacent images, this method can adaptively identify the correct image continuation relationship. This solves the registration failure problem caused by the mismatch between the preset direction and the actual movement direction, improves the generalization ability and flexibility of the stitching system for different acquisition scenarios, and avoids fundamental stitching errors caused by misjudgment of direction.

[0044] During continuous stitching, the difference between the currently acquired global vertical offset and the historical offset sequence is calculated and monitored in real time. When this difference exceeds a preset threshold, a global recalculation and correction process is immediately triggered: the current window fusion result, which may be affected by local errors, is discarded, and the vertical offset is recalculated using a template matching method across the entire overlapping area of ​​the image. This mechanism is equivalent to setting a dynamic quality checkpoint and error correction link in the stitching pipeline. It can promptly capture and interrupt abrupt shifts in offset caused by accidental matching errors, local image quality changes, or previously accumulated errors, resetting the error state through a global re-registration. This prevents the propagation and amplification of local errors in subsequent stitching steps, ensuring the overall consistency and geometric accuracy of the long-sequence stitching results. Attached Figure Description

[0045] Figure 1 This is a schematic diagram illustrating the working principle of the adaptive stitching method for scratch images described in this invention.

[0046] Figure 2 A flowchart for comparing splicing direction matching scores;

[0047] Figure 3 A comparison chart of the vertical offsets of feature matching and template matching under different calculation windows;

[0048] Figure 4 A flowchart for weighted fusion of global vertical offsets;

[0049] Figure 5 A graph showing the relationship between the number of matching points in the calculation window and the weighted fusion weights;

[0050] Figure 6 This is a schematic diagram of the illumination distribution of the original 1920×1080 resolution image.

[0051] Figure 7 This is a schematic diagram of a 1080×1080 resolution image after automatic cropping processing according to the present invention;

[0052] Figure 8 Add an example image to the ruler based on pixel-to-physical size conversion;

[0053] Figure 9 This is a diagram of the scene data collection and parameter setting interface of an automatic chip removal device in the existing technology.

[0054] Figure 10 This is a schematic diagram illustrating the poor image stitching effect of existing technology for scratches.

[0055] Figure 11 This is an overall effect image of the scratch images after being stitched together using the method described in this embodiment.

[0056] Figure 12 This is a magnified comparison image of the overlapping area;

[0057] Figure 13 A panoramic image stitched together from a long sequence of images. Detailed Implementation

[0058] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0059] Please see Figure 1 This invention provides an adaptive stitching method for scratch images. The method includes: acquiring adjacent scratch images with overlapping regions sequentially obtained by an image acquisition device, and performing subsequent processing based on the overlapping regions. The method calculates image matching scores for the overlapping regions under different possible stitching directions, compares these scores, and automatically determines and selects the optimal stitching direction based on the comparison results. Under the determined optimal stitching direction, the overlapping regions of the images are vertically divided into multiple independent calculation windows covering the entire width of the overlapping regions. For each calculation window, a hybrid registration method combining feature matching and template matching strategies is used to calculate the local vertical offset of that window. After obtaining the vertical offsets of all calculation windows, these local vertical offsets are weighted and fused according to the matching quality weights obtained by each window during the registration process to calculate a global vertical offset. During the continuous stitching of multiple images, the system monitors in real time the difference between the currently calculated global vertical offset and the historical global vertical offset. If the difference exceeds a set threshold, a correction mechanism is triggered, and template matching is recalculated on the global overlapping region, and the new result is used to correct the current global vertical offset. Based on the final determined global vertical offset, geometric alignment is achieved by adding pixel borders above or below one of the images, aligning the two images in the overlapping area. On the geometrically aligned image pair, a linearly gradient weight mask is applied to the overlapping area, and the corresponding pixel values ​​in the two images are then weighted and merged at the pixel level, ultimately generating a seamless stitched image.

[0060] Example 1: See Figure 2For two adjacent images to be stitched in a scratch image sequence, the image matching score of their overlapping region is calculated in both the left-to-right and right-to-left stitching directions. These two matching scores are calculated using the same image similarity metric. Then, the matching scores from left to right and right to left are compared, and the direction with the higher score is selected as the optimal candidate. If the matching score from left to right is greater than that from right to left, the left-to-right direction is chosen as the optimal stitching direction; otherwise, the right-to-left direction is chosen. This determination process achieves adaptive selection of the stitching direction. In specific implementation, the adaptive stitching method for scratch images performs stitching direction determination for two adjacent images to be stitched in the image sequence. After acquiring the overlapping regions of adjacent images, the system calculates the image matching score for the overlapping regions in both the "left-to-right stitching direction" and the "right-to-left stitching direction." Calculating the score for the "left-to-right stitching direction" involves registering and comparing the overlapping portions of the right and left images; calculating the score for the "right-to-left stitching direction" involves registering and comparing the overlapping portions of the left and right images. In some embodiments, the matching score can be calculated using a normalized cross-correlation method, which is robust to changes in illumination and effectively reflects the similarity between two image patches.

[0061] Next, the system compares the two calculated matching scores. The direct purpose of this comparison is to determine which stitching direction yields a higher image alignment match. This comparison is performed through a specific numerical comparison operation, comparing the matching score of the "left-to-right stitching direction" with the matching score of the "right-to-left stitching direction." The direction with the higher matching score is recorded as the optimal candidate direction in the matching score comparison results. In some embodiments, the comparison can be performed using a simple subtraction operation, i.e., calculating the difference between the matching score of the "left-to-right stitching direction" and the matching score of the "right-to-left stitching direction." Optionally, the ratio of the two scores can also be calculated. This comparison process can be expressed by the formula:

[0062]

[0063] in: Image matching score representing the overlapping region in the "left-to-right stitching direction". Image matching score representing the overlapping region in the "right-to-left stitching direction". This represents the difference between two scores. When When the "left-to-right splicing direction" matches, the score is higher; when When the score is higher, it indicates that the "right-to-left splicing direction" has a higher matching score.

[0064] Based on the matching score comparison results, the system automatically determines and selects the optimal stitching direction. The logic for this determination is straightforward and deterministic. When the comparison result shows that the matching score for the "left-to-right stitching direction" is greater than that for the "right-to-left stitching direction," the system selects the "left-to-right stitching direction" as the optimal stitching direction for this image stitching operation. Otherwise, the system selects the "right-to-left stitching direction" as the optimal stitching direction for this image stitching operation. Optionally, in the difference... When the absolute value of is very close to zero, a very small threshold can be set. When the value is less than this threshold, either "left-to-right splicing direction" or "right-to-left splicing direction" can be selected as the default direction to ensure the uniqueness of the judgment and the stability of the process.

[0065] Example 2: After determining the optimal stitching direction, the overlapping area is uniformly divided into multiple rectangular calculation windows along the vertical direction at a preset height. The width of each window is consistent with the width of the overlapping area, thus ensuring that the window covers the entire width of the overlapping area. For each calculation window obtained after segmentation, a feature matching algorithm and a template matching algorithm are executed respectively. The feature matching algorithm extracts and matches feature points within the window, outputting the number of successfully matched feature point pairs and the vertical offset calculated based on these matched point pairs. The template matching algorithm calculates within a set search range, outputting the similarity score of the best matching position and the vertical offset calculated based on the template matching position. Each calculation window will obtain matching results from both algorithms. The specific implementation is as follows: the template matching algorithm uses the normalized cross-correlation method, which evaluates similarity by calculating the correlation of gray values ​​of two image regions. For each calculation window, firstly, an image block of the window is extracted from the overlapping area of ​​the first image as a template. The size of the template is consistent with the calculation window, covering the entire window width and preset height. Then, a search range is set within the corresponding overlapping area of ​​the second image. This search range is predefined based on historical offsets or image size, for example, extending vertically by a certain number of pixels from the current window position to cover possible offset variations. The algorithm slides the template pixel by pixel within the search range, calculating the normalized cross-correlation score between each position and the template. The score formula is the ratio of the product of the covariance of the two image patches to their respective standard deviations. The score ranges from -1 to +1, with higher scores indicating better matching. The algorithm outputs the position with the highest score within the search range as the optimal matching position and records the similarity score at that position. Based on the optimal matching position, the vertical offset is calculated, which is the difference between the ordinate of the center point of the template in the first image and the ordinate of the optimal matching point in the second image. This difference is the vertical offset based on template matching. Each calculation window performs feature matching algorithms, such as accelerated robust feature algorithms, while performing template matching, extracting and matching key points to obtain two independent matching results, providing a data foundation for subsequent hybrid decision-making.

[0066] In its implementation, the normalized cross-correlation algorithm serves as the core of template matching. Its input data is the grayscale matrix of the image patches within the calculation window, and its output data includes a similarity score scalar and the coordinates of the optimal matching position. The algorithm is integrated with scratch image stitching scenarios, focusing on image grayscale distribution rather than feature points to compensate for the registration needs in areas lacking features. The input image patches originate from the local texture of the scratches, the output score directly reflects the alignment quality, and the offset calculation reflects the geometric relationship between images through coordinate differences. The search range for template matching needs to be set in conjunction with the movement characteristics of the scratch image acquisition device; for example, the vertical search range is set to tens of pixels based on the camera resolution to ensure coverage of the actual offset without excessively increasing the computational burden. Finally, the template matching results and feature matching results for each calculation window are output in parallel, including the vertical offset and confidence index, for subsequent weighted fusion or decision selection.

[0067] In practical implementation, based on the determined optimal stitching direction, the overlapping area of ​​two adjacent scratch images is segmented. The segmentation operation is performed vertically, and the overlapping area is evenly divided into multiple rectangular calculation windows with a preset height value. The width of each calculation window is exactly the same as the width of the overlapping area, thus ensuring that each calculation window covers the entire width of the overlapping area. The preset height value is a configurable parameter, and its specific value is set according to the resolution of the image acquisition system and the typical size of the scratch. In some embodiments, the preset height value can be set to a fixed value, such as 100 pixels, but the system allows adjustment of this value according to the actual application scenario. Optionally, the preset height value can also be dynamically calculated based on the total height of the overlapping area, for example, by determining the number of calculation windows through a formula.

[0068]

[0069] in: Represents the vertical height of the overlapping area. Represents the preset window height. Represents the number of calculation windows. This indicates the rounding function to ensure that the entire overlapping area is completely covered.

[0070] In practice, for each computation window, a hybrid registration method combining feature matching and template matching is performed to obtain the vertical offset. Feature matching is applied first, for example, using a scale-invariant feature transform algorithm to extract and match feature points within the computation window. The number of successfully matched feature point pairs and the vertical offset calculated based on these matching point pairs using a geometric model are output. Template matching is then applied, for example, using a normalized cross-correlation method. The computation window is cropped from the first image as a template, and a sliding comparison is performed within the corresponding search region of the second image. The similarity score of the optimal matching position and the vertical offset calculated based on the positional difference are output. In some embodiments, the feature matching algorithm can be an accelerated robust feature algorithm, and the search range for template matching can be pre-defined based on prior knowledge to optimize computational efficiency. In practical implementation, when the feature matching algorithm selects the accelerated robust feature algorithm, the algorithm first detects key points in the image by constructing a Hessian matrix within each calculation window. These key points correspond to areas with significant gray-level changes in the scratch image, such as scratch edges or texture features. Then, the algorithm calculates the distribution of Haar wavelet responses in the neighborhood of each key point, generating a multi-dimensional feature descriptor that can effectively represent the local image information of the key point. Next, by comparing the Euclidean distance between the feature descriptors of key points in adjacent image calculation windows, the nearest neighbor search method is used to match feature points, thereby identifying corresponding feature point pairs in the two images. After matching, the system calculates the offset based on the vertical coordinate position of these matched point pairs, and excludes abnormal matches through statistical methods or robust estimation techniques. Finally, the vertical offset based on feature matching is output for subsequent weighted fusion.

[0071] In practical implementation, based on the number of feature matching point pairs and the similarity score of template matching, the system selects the final vertical offset for each calculation window. The selection process directly compares the reliability of feature matching and the confidence of template matching. For example, when the number of feature matching point pairs is high, the system tends to select the vertical offset based on feature matching, while when the number of feature matching point pairs is low, the system selects the vertical offset based on template matching with reference to the similarity score of template matching. Optionally, the selection logic can be implemented through a simple decision rule, such as setting a threshold for the number of feature matching point pairs, but the specific threshold definition falls within the scope of system parameter configuration. In some embodiments, the similarity score of template matching can also be used as an auxiliary decision indicator. When the number of feature matching point pairs is insufficient but the similarity score of template matching is high, the vertical offset based on template matching is preferentially selected.

[0072] See Figure 3This figure is a core data visualization result of the hybrid registration strategy, focusing on the acquisition of the vertical offset of the calculation window: the horizontal axis represents the calculation window number, and the vertical axis represents the vertical offset. The two curves correspond to the feature matching offset and the template matching offset, respectively. The value of this figure lies in verifying the rationality of the feature + template hybrid registration: the difference in offset between the two methods is minimal, which not only ensures the accuracy of local window registration but also provides stable basic data for subsequent weighted fusion of global vertical offsets. This is one of the key steps in improving the robustness of scratch image stitching.

[0073] Example 3: See Figure 4 For each computation window, after obtaining the number of feature matching point pairs and the similarity score of template matching, a vertical offset from either algorithm is selected as the final vertical offset for that window. When the number of feature matching point pairs is greater than or equal to a preset threshold, it indicates that the feature matching result is reliable, and the vertical offset based on feature matching is used as the vertical offset for that computation window. If the number of feature matching point pairs does not reach the preset threshold, the vertical offset based on template matching is used instead. After obtaining the vertical offsets of all computation windows, appropriate weights are assigned to the vertical offsets of each window for fusion. The matching quality weight of each computation window is set to the number of matching point pairs successfully extracted by the feature matching algorithm within that window. The global vertical offset is calculated by weighted summing of the vertical offsets of each computation window and their corresponding matching quality weights, and then dividing this sum by the sum of the matching quality weights of all computation windows.

[0074] In practice, for each calculation window, the system determines the final vertical offset based on the number of feature matching point pairs and the similarity score of template matching. The decision logic is executed according to preset rules. When the number of feature point pairs extracted and successfully matched by the feature matching algorithm within a certain calculation window is greater than or equal to a preset threshold, the system determines that the vertical offset based on feature matching is more reliable, and then uses the vertical offset based on feature matching as the final vertical offset of this calculation window. When the number of feature matching point pairs is less than the preset threshold, it indicates that the reliability of the feature matching result is insufficient, and the system switches to using the vertical offset based on template matching as the final vertical offset of this calculation window.

[0075] In practical implementation, after obtaining the vertical offsets of all calculation windows, the system performs weighted fusion of these vertical offsets based on the matching quality weights of each calculation window to obtain the global vertical offset. The matching quality weight is defined as the number of matching point pairs successfully extracted by the feature matching algorithm within each calculation window; the more matching point pairs, the greater the weight. The weighted fusion process is accomplished by calculating the sum of the products of the vertical offset of each calculation window and its corresponding matching quality weight, and then dividing by the sum of the matching quality weights of all calculation windows. The calculation relationship can be expressed as follows:

[0076]

[0077] in: This represents the total number of calculation windows. Representing the The vertical offset of each calculation window. Representing the The matching quality weights for each calculation window. This represents the calculated global vertical offset. In some embodiments, if a calculated window ultimately uses a template-match-based vertical offset, its matching quality weight... Still using the number of feature matching point pairs may result in lower weight values ​​and thus smaller contributions to the fusion. Optionally, for computation windows that rely entirely on template matching results, a weight derived from the template matching similarity score can also be assigned.

[0078] See Figure 5 This figure visualizes the global vertical offset weighted fusion strategy: the horizontal axis represents the calculation window number, the left vertical axis represents the number of feature matching points in each window, and the right vertical axis represents the weight calculated based on the number of matching points. The core value of this figure lies in its intuitive verification of the logical rationality of the weighted fusion: by allocating weights based on the number of matching points, the accurate offsets of high-quality windows dominate the global result while mitigating the error impact of low-quality windows. This effectively improves the reliability of the global vertical offset and is a key step in suppressing stitching errors and ensuring the stitching accuracy of scratched images.

[0079] Example 4: When stitching consecutive image sequences, the system monitors and calculates in real time the difference between the newly obtained current global vertical offset and the previously determined historical global vertical offset. When this difference exceeds a preset tolerance threshold, a correction mechanism is triggered. The correction process involves re-executing the template matching algorithm over the overlapping area of ​​the entire image pair, rather than within a single calculation window, to calculate a new vertical offset. Subsequently, this new vertical offset, recalculated through global template matching, replaces the previous current global vertical offset obtained through window-weighted fusion, and this new offset is used in subsequent geometric alignment steps.

[0080] In practical implementation, the adaptive stitching method performs real-time monitoring and correction during the continuous stitching of scratch image sequences. After calculating the global vertical offset for the current image pair to be stitched, the system immediately compares the current global vertical offset with a historical global vertical offset. The historical global vertical offset typically refers to the global vertical offset calculated and used during the stitching of the previous frame. The core of real-time monitoring is calculating the difference between the current global vertical offset and the historical global vertical offset. This difference can be calculated using absolute value operations, expressed by the formula:

[0081]

[0082] in: This represents the current global vertical offset calculated for the current image pair. Represents the historical global vertical offset. This represents the absolute value of the difference between the two. In some embodiments, the historical global vertical offset can be stored in a circular buffer or variable and updated after each successful stitching. Optionally, for the first pair of images at the beginning of the sequence, the historical global vertical offset can be initialized to zero or a preset baseline value.

[0083] The monitoring logic will calculate the difference. With a preset threshold Compare and set a threshold. This is a positive integer set according to image resolution and stitching accuracy requirements. Refer to Table 1, which shows monitoring data that may occur during continuous stitching, including offset monitoring data for five pairs of consecutive images. When the difference... Exceeding the preset threshold When the system determines that the current global vertical offset obtained through weighted fusion of calculation windows may be abnormal, it triggers a recalculation and correction process. In some embodiments, a preset threshold is used. It can be set to 5 pixels, but the specific value should be adjusted according to the tolerance for splicing continuity in actual application.

[0084] Table 1: Offset Monitoring and Correction Trigger Table during Continuous Stitching

[0085] Image pair sequence Current global vertical offset (pixels) Historical global vertical offset (pixels) Difference (pixels) Preset threshold (pixels) Trigger calibration? Image pair 1 +2 0 2 5 no Image pair 2 +3 +2 1 5 no Image pair 3 +9 +3 6 5 yes Image pair 4 +4 +9 5 5 No (or depends on the boundary conditions). Image pair 5 +5 +4 1 5 no

[0086] In practice, once the triggering condition is met, the correction process begins immediately. The system recalculates using a template matching algorithm over the globally overlapping region. This means that the overlapping region is no longer divided into calculation windows; instead, the entire overlapping region is treated as a single search area, and a new vertical offset is directly calculated using the template matching algorithm. This step aims to bypass the windowed fusion process, which may introduce errors due to missing or incorrect local features, and directly relies on a wide range of image grayscale correlations for global registration. After calculation, the newly calculated vertical offset directly replaces the current global vertical offset generated by the weighted fusion. Optionally, the replacement operation is atomic; the new vertical offset is immediately used in subsequent geometric alignment steps, and the stored historical global vertical offsets are updated simultaneously, providing a new benchmark for monitoring subsequent image pairs. In some embodiments, the search range of the global template matching can be set wider to accommodate greater offset correction needs.

[0087] Example 5: Reference Figure 6 and Figure 7 Before acquiring the overlapping areas of adjacent images in the scratch image sequence, an automatic cropping preprocessing is performed on each individual scratch image. To address the issue of uneven brightness after stitching caused by a bright center and dark edges in a single image, a cropping method from the image center is adopted. By default, the original 1920×1080 resolution image is cropped into a 1080×1080 square image. Flexible cropping parameters are supported, allowing customization of the cropping width and height based on actual image resolution and lighting distribution differences. Setting it to 0 indicates no cropping, ensuring that the effective area of ​​the cropped image is preserved intact and preventing areas with uneven brightness at the edges from participating in subsequent stitching processes. The cropped image is then used for subsequent overlapping area acquisition and stitching operations. Figure 6 This is a schematic diagram of the illumination distribution of the original 1920×1080 resolution image. It can be seen that the brightness is uniform in the middle area of ​​the image, while the brightness of the two edge areas is significantly reduced, presenting the characteristic of "bright in the middle and dark on both sides". This characteristic will lead to the defect of alternating brightness and darkness after direct stitching. Figure 7This is a schematic diagram of a 1080×1080 resolution image after automatic cropping processing according to this invention. By cropping from the center of the image, the core area with uniform brightness is preserved, while the unevenly lit edges on both sides are removed. This provides high-quality image data for subsequent stitching processes and avoids the problem of alternating brightness after stitching. The cropping diagram is only for illustrating the principle. In practical applications, the cropping size can be flexibly adjusted by configuring cropping parameters according to the original image resolution and differences in illumination distribution to ensure that the core area of ​​the scratch is completely preserved. Based on the sign and absolute value of the final determined global vertical offset, a black pixel border is added above or below one of the images to be stitched. The pixel width of the added border is equal to the absolute value of the global vertical offset. This operation enables the two images to achieve geometric alignment in the overlapping area. In the overlapping area of ​​the two geometrically aligned images, a linearly gradient weight mask is applied for pixel-level fusion. The weight value assigned to the first image is one at the starting boundary of the overlapping area and decreases linearly along the stitching direction, reaching zero at the ending boundary of the overlapping area. Accordingly, the weight assigned to the second image is zero at the beginning boundary of the overlapping region and increases linearly along the stitching direction, reaching one at the end boundary of the overlapping region. For each pixel position within the overlapping region, the pixel value of the first image is multiplied by the weight of the first image at that position, and then the pixel value of the second image is multiplied by the weight of the second image at that position. The calculated weighted average is used as the pixel value of the fused image at that position.

[0088] In practice, the geometric alignment step is performed based on a determined global vertical offset. Two images are aligned in the overlapping area by adding pixel borders. The position and width of the border are determined by the sign and magnitude of the global vertical offset. When the global vertical offset is positive, it indicates that the second image needs to be aligned downwards relative to the first image. In this case, a black pixel border with a height equal to the absolute value of the global vertical offset is added below the first image, or equivalently, a border of the same height is added above the second image. When the global vertical offset is negative, it indicates that the second image needs to be aligned upwards relative to the first image, and a border is added above the first image. In some embodiments, adding the border is achieved by creating a new, expanded image and copying the original image data to the corresponding position. The border area not covered by the original image is filled with pure black. It can be understood that this method achieves vertical alignment by expanding the canvas rather than cropping the image content, thus preserving all image information.

[0089] In specific implementation, after geometric alignment, the two images are fused using a pixel-level weighted average. The fusion operation is limited to the overlapping region of the two images. A linearly gradient weight mask is used to assign weights in the overlapping region. The weight assigned to the first image is 1 at the starting boundary of the overlapping region and linearly decreases along the image stitching direction to 0 at the ending boundary of the overlapping region. Simultaneously, the weight assigned to the second image is 0 at the starting boundary of the overlapping region and linearly increases along the image stitching direction to 1 at the ending boundary of the overlapping region. In some embodiments, for a left-to-right stitching direction, the starting boundary of the overlapping region is the left boundary, and the ending boundary is the right boundary, with the linear gradient proceeding from left to right; for a right-to-left stitching direction, the opposite is true. Optionally, the weight mask can be pre-calculated and stored as a one-dimensional array or a two-dimensional mapping table to improve processing efficiency. and Satisfying Relationships .

[0090] In practical implementation, for each pixel position within the overlapping area... The fused pixel value is calculated by weighting the pixel values ​​at that location in both images and their corresponding weight values, using the following formula:

[0091]

[0092] in: Indicates the position of the fused image pixel values, Indicates the position of the first image. pixel values, Indicates the position of the second image. pixel values, Indicates the location Linear gradient weights for the first image, Indicates the location The linear gradient weights in the second image. This is understandable. and The value is only related to the horizontal coordinate of the pixel. This relates to the linear gradient along the stitching direction. Optionally, for color images, this calculation is performed independently on the R, G, and B color channels. In some embodiments, the linear gradient function can employ the simplest linear interpolation, or other functions with smooth transition properties can be used to generate weight values, but the core is to ensure that the weights change continuously and monotonically from 1 to 0.

[0093] refer to Figure 8After image stitching is completed, the system automatically adds a scale bar to the stitched result to facilitate quantitative analysis of scratch dimensions. The scale bar is generated based on camera calibration parameters, such as a ratio of "250 pixels corresponding to 0.1 mm" measured using dedicated vision software. The scale bar is divided into large divisions of 1000 pixels and small divisions of 100 pixels, and the actual physical dimensions (e.g., millimeters) are marked. The scale bar can be flexibly positioned below or to the side of the scratch area to avoid obscuring key features.

[0094] Example 6: Reference Figure 9 and Figure 10 Currently, most scratch image stitching technologies on the market employ simple image stacking or overlapping according to a predetermined template. This approach suffers from the following problems: First, simple image stacking results in the stitched image merging data from multiple high-resolution sub-images, leading to a massive data volume. This massive image data significantly challenges the time and space complexity of the stitching algorithm, directly impacting the system's real-time processing capabilities and scalability. Second, stacking involves continuously acquiring new images at their end positions, and the camera's lighting conditions are typically brighter in the center and slightly darker at the edges. Under poor lighting conditions, the stitched result becomes increasingly darker. Furthermore, simply overlapping images according to a predetermined template results in stitched images with obvious boundaries between each image, poor feature point overlap, and even noticeable misalignment. The stitching effect is unsatisfactory. Figure 9 The presentation showcases commonly used automatic chip removal devices in existing technologies, including their data acquisition scenarios and parameter setting interfaces. However, during the acquisition process, limitations such as fixed equipment parameters and insufficient supplementary lighting conditions can lead to defects in the basic data for image stitching. Figure 10 This is a schematic diagram illustrating the undesirable effects of splicing existing technology, clearly showing the obvious boundary and misalignment issues between the sub-images.

[0095] refer to Figures 11 to 13 The image stitching effect after processing with a pixel-level weighted average fusion method. Figure 11 The overall effect of stitching together scratch images using the method of this embodiment is shown, demonstrating that the scratches are continuous and without misalignment; Figure 12 The magnified comparison of the overlapping area clearly shows that the fusion based on the linear gradient weight mask achieves a smooth transition with no visible seams. Figure 13 The panoramic results obtained by stitching together a long sequence of images demonstrate the effectiveness and consistency of the method in this embodiment in continuous scenes.

[0096] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. An adaptive stitching method for scratch images, characterized in that, The method includes the following steps: Collect the overlapping area of ​​adjacent images in the scratch image sequence; Based on the image matching score of the overlapping region, the matching score comparison results of each possible splicing direction are obtained; Based on the matching score comparison results, the optimal splicing direction is automatically determined and selected; Based on the optimal stitching direction, the overlapping region is divided into multiple calculation windows in the vertical direction; for each calculation window, the vertical offset of each calculation window is obtained based on the hybrid registration of feature matching and template matching. Based on the matching quality weight of each calculation window, the vertical offset of each calculation window is weighted and fused to obtain the global vertical offset. During continuous stitching, the difference between the current global vertical offset and the historical global vertical offset is monitored in real time; when the difference exceeds a preset threshold, template matching is triggered to recalculate and correct the global overlapping area. Based on the global vertical offset, geometric alignment is performed by adding pixel borders; In the overlapping area of ​​the aligned images, pixel-level weighted average fusion is performed using a linearly gradient weight mask to complete the stitching.

2. The adaptive stitching method for scratch images as described in claim 1, characterized in that, The process of obtaining matching score comparison results for each possible stitching direction based on the image matching score of the overlapping region includes: For adjacent images in a scratch image sequence, calculate the image matching score of the overlapping region in the stitching direction from left to right, and the image matching score of the overlapping region in the stitching direction from right to left. The matching score of the left-to-right splicing direction is compared with the matching score of the right-to-left splicing direction, and the direction with the higher matching score is selected as the optimal direction candidate in the matching score comparison results.

3. The adaptive stitching method for scratch images as described in claim 2, characterized in that, The automatic determination and selection of the optimal splicing direction based on the matching score comparison result includes: When the matching score of the left-to-right splicing direction is greater than the matching score of the right-to-left splicing direction, the left-to-right splicing direction is selected as the optimal splicing direction. Otherwise, choose the right-to-left splicing direction as the optimal splicing direction.

4. The adaptive stitching method for scratch images as described in claim 1, characterized in that, The step of dividing the overlapping region into multiple calculation windows in the vertical direction includes: The overlapping area is evenly divided into multiple rectangular windows at a preset height, with each window covering the entire width of the overlapping area.

5. The adaptive stitching method for scratch images as described in claim 4, characterized in that, The hybrid registration based on feature matching and template matching obtains the vertical offset of each calculation window, including: For each calculation window, the number of feature matching point pairs and the vertical offset based on feature matching are obtained through a feature matching algorithm; For each calculation window, the similarity score of the template match and the vertical offset based on the template match are obtained through the template matching algorithm; Based on the number of feature matching point pairs and the similarity score of the template matching, the vertical offset based on feature matching or the vertical offset based on template matching is selected as the vertical offset of each calculation window.

6. The adaptive stitching method for scratch images as described in claim 5, characterized in that, The step of selecting either the vertical offset based on feature matching or the vertical offset based on template matching based on the number of feature matching point pairs and the similarity score of template matching includes: When the number of feature matching point pairs is greater than or equal to a preset threshold, the vertical offset based on feature matching is used as the vertical offset of the calculation window. Otherwise, use the template-match-based vertical offset as the vertical offset for calculating the window.

7. The adaptive stitching method for scratch images as described in claim 1, characterized in that, The process of weighting and fusing the vertical offsets of each calculation window based on the matching quality weights of each window to obtain the global vertical offset includes: For each calculation window, the number of matching points extracted by the feature matching algorithm is used as the matching quality weight for each calculation window; The sum of the products of the vertical offset of each calculation window and the corresponding matching quality weight is divided by the sum of all matching quality weights to obtain the global vertical offset.

8. The adaptive stitching method for scratch images as described in claim 1, characterized in that, The triggering process involves recalculating and correcting the global overlap region using template matching, including: When the difference between the current global vertical offset and the historical global vertical offset exceeds a preset threshold, a template matching algorithm is executed over the entire overlapping area to calculate a new vertical offset. Replace the current global vertical offset with the new vertical offset for geometric alignment.

9. The adaptive stitching method for scratch images as described in claim 1, characterized in that, The geometric alignment achieved by adding pixel borders includes: Based on the sign and magnitude of the global vertical offset, add a black pixel border above or below an image to align the two images in the overlapping area. The pixel-level weighted average fusion using a linearly gradient weight mask includes: In the overlapping region, the first image is assigned a weight that decreases linearly from one to zero, and the second image is assigned a weight that increases linearly from zero to one. For each pixel, a weighted average is calculated based on the weights to obtain the merged pixel value.

10. An adaptive stitching system for scratch images, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the adaptive stitching method for scratch images as described in any one of claims 1 to 9.

Images