A linear array camera image matching processing method based on adaptive sliding window

Abstract

This invention discloses an image matching processing method for a linear array camera based on an adaptive sliding window, comprising: acquiring a template image and a patrol image; setting a base image and two equal-height windows; sliding the two windows synchronously to the right from their respective left boundaries; calculating the mapping relationship between the window pairs; transforming the patrol sub-image to generate a transformed image 1, and transforming the mask image to generate a transformed image 2; determining the left boundary of the target pixel in transformed image 1; if the left boundary is greater than the stitching left boundary when it is not the first window pair; determining the right boundary of the target pixel in transformed image 1; extracting an effective region from transformed image 1 and embedding it into the base image; calculating the next movement step size of the two windows and updating the stitching left boundary; repeating until both windows reach their respective image right boundaries, at which point the base image is the matched stitched image. This invention transforms the high-resolution image matching problem into multiple local high-precision matching sub-problems, which can eliminate the discontinuities after matching and stitching, preserve the original details of the image, and avoid image distortion after matching.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more specifically to an image matching processing method for a linear scan camera based on an adaptive sliding window. Background Technology

[0002] In the rail transit sector, during train inspections, line-scan cameras are typically installed in maintenance depots to scan the train surface and acquire full-vehicle images. In subsequent anomaly detection, it's often necessary to compare template images (reference images acquired under standard conditions) with inspection images (images actually taken during inspection) to determine if equipment components are intact, misaligned, or if there are any missing or damaged parts. Before comparison, aligning the template and inspection images pixel-wise (i.e., image matching algorithms) is a crucial upstream task.

[0003] Existing image matching algorithms typically employ the following strategies when processing images:

[0004] 1. Direct Global Computation: The entire high-resolution image is directly input into the deep learning algorithm model.

[0005] 2. Global scaling preprocessing: Due to video memory limitations, it is often necessary to first downsample the large-size original image, calculate the matching relationship, and then upsample the result to restore it.

[0006] 3. Fixed-size segmentation and traditional overlapping sliding window: The large image is cut into fixed-size sub-images, processed separately, and then stitched together.

[0007] While the aforementioned existing strategies have addressed the problem of processing large images to some extent, significant technical bottlenecks still exist when dealing with ultra-large images output by line scan cameras:

[0008] 1. Direct global computation places extremely high demands on video memory in high-resolution image scenarios, especially since images output by line scan cameras contain hundreds of millions of pixels and cannot be directly input into deep learning algorithm models for computation.

[0009] 2. Global scaling preprocessing can easily lead to image detail distortion and information loss. When the template image and the inspection image have extremely high resolution, if a global scaling strategy is used, small features in the image (such as fine cracks, small scratches, pins of precision components, etc.) become blurred or even completely lost during the downsampling process. This will also cause the matching algorithm to lose texture and other information, making it impossible to align the inspection image with the template image.

[0010] 3. Fixed-size segmentation and traditional overlapping sliding window preprocessing have the same problem: they cannot ensure the continuity of the image at the stitching point. When dealing with images from a line scan camera, image matching and stitching cannot be completed.

[0011] For example, Chinese patent application CN111598177A discloses an adaptive maximum sliding window matching method for low-overlap image matching. Its technical solution is as follows: 1) Find the maximum overlapping region, set a search range, move the sliding image pixel by pixel, and calculate the similarity of the current overlapping region each time (e.g., normalized cross-correlation); 2) Select the position with the highest similarity as the optimal matching position. The "adaptive" aspect of this solution involves calculating the size and similarity of the overlapping region between two images at each possible position within a fixed search range, and selecting the optimal one. Its adaptability lies in the fact that the shape of the overlapping region changes with the search position, but the search strategy itself is exhaustive and fixed, without feedback based on intermediate results. The effect of this technical solution is to find the global translation relationship between two images (i.e., the optimal matching position), essentially a global image registration method. However, this method does not consider the local matching quality differences in image content and cannot solve the problem of discontinuity in block matching stitching. Summary of the Invention

[0012] To overcome the shortcomings of the above-mentioned technologies, this invention provides an image matching processing method for linear scan cameras based on an adaptive sliding window. This invention abandons the traditional global scaling approach and transforms the high-resolution image matching problem into multiple local high-precision matching sub-problems by dynamically cropping images within a suitable sliding window from template and inspection images. Furthermore, it eliminates discontinuities after matching and stitching, preserves original image details, and avoids image distortion after matching.

[0013] The technical solution adopted by this invention to overcome its technical problems is:

[0014] This invention discloses an image matching processing method for a linear scan camera based on an adaptive sliding window, comprising the following steps:

[0015] S1. Obtain the template diagram and the inspection diagram, set the base image to be the same size as the template diagram, set the first sliding window on the template diagram and the second sliding window on the inspection diagram, the template diagram and the inspection diagram are the same height, the two windows are the same size and the height is equal to the height of the template diagram;

[0016] S2. Make the two windows slide horizontally to the right synchronously from the left edge of their respective images, and perform the following operations on the window pairs obtained by each slide;

[0017] S3. Calculate the mapping relationship between the template subgraph in the current sliding window one and the inspection subgraph in the current sliding window two;

[0018] S4. Based on the mapping relationship, transform the current inspection subgraph to generate the transformation. Figure 1 Create a mask image and perform the same transformation to generate the transformation. Figure 2 ;

[0019] S5. Analysis and Transformation Figure 2 Determine the transformation Figure 1 The left boundary of the target pixel;

[0020] S6. If the first window pair is processed, proceed with the next steps; otherwise, compare the current transformation. Figure 1 The left boundary of the target pixel and the left boundary of the last updated stitching: if the former is greater than the latter, adjust the position of the current sliding window 2 and return to execute S3; otherwise, continue to execute the subsequent steps.

[0021] S7. Analysis and Transformation Figure 2 Determine the transformation Figure 1 The right boundary of the target pixel;

[0022] S8. From the transformation Figure 1 Extract the valid area from the middle, and embed the valid area into the base image according to the current position of the sliding window;

[0023] S9. According to the transformation Figure 1 The target pixel's right boundary and the preset baseline overlap width are used to calculate the next movement step of the sliding window. The next movement step of the sliding window is calculated based on the baseline overlap width, and the stitching left boundary is updated.

[0024] S10. Process each window pair sequentially until both sliding window one and sliding window two reach the right boundary of their respective images. At this point, the base image is used as the matching stitched image.

[0025] Furthermore, step S1 specifically includes:

[0026] S11. Obtain a defect-free image scanned by a fixed-position linear scan camera as a template image, and obtain a target image scanned by the same linear scan camera during the inspection process as an inspection image. The template image and the inspection image have the same height.

[0027] S12. Set a base image with the same size as the template image, and initialize all pixel values ​​of the base image to preset values;

[0028] S13. Set sliding window one on the template diagram and sliding window two on the inspection diagram. Both sliding window one and sliding window two are rectangular frames with the same size and height equal to the height of the template diagram.

[0029] Furthermore, step S2 specifically includes:

[0030] S21. Establish global coordinate systems for the template diagram, inspection diagram, and base diagram, respectively, with the top left corner as the origin; establish local coordinate systems for the sliding window one and sliding window two, respectively, with the top left corner as the origin; for all coordinate systems, the horizontal axis is positive to the right and the vertical axis is positive downward.

[0031] S22. Set the initial x-coordinate of the upper left corner of sliding window one and sliding window two to zero. Make sliding window one slide horizontally to the right simultaneously from the left boundary of the template image and sliding window two slide horizontally from the left boundary of the inspection image, respectively. The horizontal sliding range of sliding window one is from zero to the width of the template image minus the width of sliding window one, and the horizontal sliding range of sliding window two is from zero to the width of the inspection image minus the width of sliding window two.

[0032] Furthermore, step S3 specifically includes:

[0033] S31. Use the image in sliding window one as the template sub-image and the image in sliding window two as the inspection sub-image;

[0034] S32. Based on the image matching algorithm, calculate the mapping function between the template sub-image and the inspection sub-image in the local coordinate system as the mapping relationship.

[0035] Furthermore, step S4 specifically includes:

[0036] S41. Based on the mapping relationship described in step S3, the current inspection subgraph is resampled using a bilinear interpolation algorithm to generate a transformed image. Figure 1 ;

[0037] S42. Create a mask image with the same size as the sliding window, and initialize the pixel values ​​of all pixels in the mask image to preset valid label values, as the initial fully valid mask image;

[0038] S43. Transform the initial fully effective mask image using the same mapping relationship as in step S41 to generate the transformed image. Figure 2 .

[0039] Furthermore, step S5 specifically includes:

[0040] S51. In the transformation Figure 2 In the process, multiple simply connected regions composed of all valid pixels are extracted, and the simply connected region with the largest area is selected as the target connected region.

[0041] S52. Calculate the circumscribed rectangle or convex hull of the target connected region, and determine the transformation based on the left and right boundaries of the circumscribed rectangle or convex hull. Figure 1 The initial left and initial right boundaries;

[0042] S53. From transformation Figure 1 The transformation begins by scanning to the right from the initial left boundary. Figure 2 The process continues until the maximum length of the line segment intersecting the current vertical scan line and the target connected region is greater than or equal to a preset length threshold. The abscissa of this vertical scan line is then used as the transformation... Figure 1The left boundary of the target pixel; if the condition is not met even after scanning to the preset upper limit position, then the x-coordinate of the maximum line segment length during the scanning process is taken as the transformation. Figure 1 The left boundary of the target pixel.

[0043] Furthermore, in step S6, adjusting the position of the current sliding window two on the inspection map specifically involves subtracting the current transformation value from the x-coordinate of the upper left corner of the current sliding window two. Figure 1 The value of the left boundary of the target pixel causes the sliding window two to move to the left.

[0044] Furthermore, step S7 specifically includes:

[0045] The transformation determined in step S52 Figure 1 The transformation begins by scanning to the left from the initial right boundary. Figure 2 The process continues until the maximum length of the line segment intersecting the current vertical scan line and the target connected region is greater than or equal to a preset length threshold. The abscissa of this vertical scan line is then used as the transformation... Figure 1 The target pixel's right boundary; if the condition is not met even after scanning to the preset upper limit position, then the x-coordinate of the maximum line segment length during the scanning process is taken as the transformation. Figure 1 The right boundary of the target pixel.

[0046] Furthermore, step S8 specifically includes:

[0047] S81. Calculate the current transformation Figure 1 The average of the horizontal coordinates of the left boundary of the target pixel and the left boundary of the stitching;

[0048] S82. From the transformation Figure 1 Extract the horizontal axis from the average value of the x-coordinate described in step S81 to the transformed value. Figure 1 The area on the right boundary is considered the valid area.

[0049] S83. Place the upper left corner of the effective area in the global coordinate system with the horizontal coordinate of the current horizontal coordinate of sliding window one plus the average horizontal coordinate of the horizontal coordinate described in step S81, and the vertical coordinate is the same as the vertical coordinate of sliding window one, and embed it into the base map.

[0050] Furthermore, step S9 specifically includes:

[0051] S91. A baseline overlap width is preset to control the minimum horizontal overlap area between sliding window one and sliding window two;

[0052] S92. If the width of the current sliding window one is reduced by the transformation... Figure 1 If the difference between the right boundaries of the target pixels is greater than the baseline overlap width, then the next starting x-coordinate of the sliding window is set to the current x-coordinate and the transformation. Figure 1The sum of the right boundaries of the target pixels; otherwise, set the next starting x-coordinate of sliding window one to the current x-coordinate plus the width of sliding window one minus the baseline overlap width.

[0053] S93. Set the next starting x-coordinate of sliding window two to the current x-coordinate plus the width of sliding window two minus the baseline overlap width;

[0054] S94. Update the left boundary of the stitching in the next slide to the current transformation. Figure 1 The target pixel's right boundary is calculated by subtracting the x-coordinate of the next sliding window and adding the x-coordinate of the current sliding window.

[0055] Furthermore, step S10 specifically includes:

[0056] Each window pair is processed sequentially, and after each processing step, it is determined whether sliding window one has reached the right boundary of the template diagram and whether sliding window two has reached the right boundary of the inspection diagram, as detailed below:

[0057] Case 1: When sliding window 1 and sliding window 2 reach the right boundary of their respective images at the same time, the base image is used as the matching stitched image.

[0058] Scenario 2: When sliding window 1 reaches the right boundary of the template image first, fix sliding window 1 at the right boundary of the template image, continue to slide sliding window 2 but do not update the position of sliding window 1 for the next time, until sliding window 2 reaches the right boundary of the inspection image. At this time, the base image is used as the matching and stitching image.

[0059] Scenario 3: When sliding window 2 reaches the right boundary of the inspection image first, sliding window 1 continues to slide and the position of sliding window 2 is updated until both sliding window 1 and sliding window 2 reach the right boundary of their respective images. At this time, the base image is used as the matching stitched image.

[0060] The beneficial effects of this invention are:

[0061] 1. Eliminate stitching gaps and achieve smooth and continuous transition at stitching points: This invention dynamically adjusts the size of the overlapping area between adjacent sliding windows based on the local matching quality of the image, effectively solving the stitching gap and artifact problems caused by the traditional fixed overlapping area, and ensuring the seamless continuity of the matched and stitched image.

[0062] 2. Preserving original image details and avoiding global scaling distortion: This invention uses an adaptive overlapping region sliding window to extract a local portion of the original image for calculation, avoiding the downsampling of the entire large image as required by existing technologies to adapt to computing power. Because the image within the window maintains its original clarity, minute defect features (such as fine cracks and scratches) and high-frequency texture information are fully preserved and input into the matching algorithm. This not only solves the distortion problem in large image matching but also significantly improves the sensitivity and positioning accuracy of subsequent defect detection.

[0063] 3. No limit on input image size, suitable for line scan camera characteristics: This invention achieves image matching and stitching based on an adaptive overlapping region sliding window. Specifically, the sliding position of the next window and the overlapping area between adjacent result blocks are determined based on the real-time feedback of the local matching quality of the previous window. This adjustment is based on the real-time feedback of the matching results. This adaptive matching can highly match the characteristics of the output image of the line scan camera, enabling this invention to process ultra-large images of any size, effectively solving the problem that it is difficult to achieve lossless matching and stitching due to the large output image size of the line scan camera.

[0064] 4. Accurately extract the boundaries of effective pixel areas and improve stitching accuracy: This invention extracts the largest connected region in the transformed mask image and calculates its circumscribed rectangle or convex hull. This can quickly filter out noise interference and obtain a reliable boundary scan start reference line. The circumscribed rectangle is simple to implement and has a small computational load, while the convex hull can better fit the real contour of irregular effective areas. Both can be flexibly selected according to the actual matching quality to ensure stitching accuracy.

[0065] 5. Provides a reliable foundation for subsequent defect detection and status assessment: By precisely aligning the template image and the inspection image at their original clarity, this invention provides a high-confidence input for subsequent advanced tasks such as pixel-level comparison, abnormal area location, and status change analysis, thereby improving the closed-loop capability and reliability of the entire intelligent inspection system.

[0066] 6. Wide applicability, compatible with various matching algorithms and application scenarios: This invention can be applied to matching algorithms based on generating dense displacement fields as well as matching algorithms based on generating matching feature points. Furthermore, this invention is not only suitable for static industrial equipment inspection, but can also be extended to scenarios requiring high-resolution template-inspection image matching, such as power line inspection, rail transit, photovoltaic array monitoring, and warehouse inventory management. Attached Figure Description

[0067] Figure 1 This is a schematic flowchart of an image matching processing method for a linear scan camera based on an adaptive sliding window, as described in an embodiment of the present invention.

[0068] Figure 2This is a schematic diagram of the sliding process of the sliding window according to an embodiment of the present invention.

[0069] Figure 3 The current transformation described in the embodiments of the present invention Figure 2 Transformation Figure 1 A diagram illustrating when the left boundary of the target pixel is greater than the left boundary of the stitching.

[0070] Figure 4 The current transformation described in the embodiments of the present invention Figure 2 Transformation Figure 1 A diagram illustrating when the left boundary of the target pixel is less than or equal to the left boundary of the stitching.

[0071] Figure 5 The first embodiment of the present invention A diagram illustrating the stitching process during the next slide. Detailed Implementation

[0072] To facilitate a better understanding of the present invention by those skilled in the art, exemplary embodiments of the present invention are described in detail below with reference to the accompanying drawings. These are merely exemplary embodiments of the present invention; however, it should be understood that the present invention can be implemented in various forms and is not limited to the embodiments described herein. These embodiments are provided to enable those skilled in the art to gain a clearer and more thorough understanding of the present invention.

[0073] This embodiment discloses an image matching processing method for a linear scan camera based on an adaptive sliding window, such as... Figure 1 and Figure 5 As shown, it includes the following steps:

[0074] S1. Obtain the template image and the inspection image, set the base image to be the same size as the template image, set sliding window one on the template image and sliding window two on the inspection image, the template image and the inspection image are the same height, and the two windows are the same size and the height is equal to the height of the template image.

[0075] Specifically, in this embodiment, step S1 includes:

[0076] S11. Obtain a defect-free image scanned by a fixed-position linear scan camera as a template image, and obtain a target image scanned by the same linear scan camera during the inspection process as an inspection image. The template image and the inspection image have the same height.

[0077] In one alternative implementation, obtaining Defect-free images scanned by a fixed-position linear scan camera are used as template image A to acquire images from the same linear scan camera during the inspection process. The target image scanned at any given time is used as inspection image B, where... The template diagram and the inspection diagram have the same height, both being... pixels, width is respectively Pixels and Pixels. In this preferred embodiment, Pixels Pixels Pixel.

[0078] S12. Set a base image with the same size as the template image, and initialize all pixel values ​​of the base image to preset values.

[0079] In one optional implementation, a base image S with the same size as the template image A is set, and all pixel values ​​of the base image are initialized to preset values. In this embodiment, the RGB values ​​are preferably assigned as (0, 0, 0). This pixel value is only a preferred assignment and can be set to other values. After subsequent steps, the base image S will be the result image after the inspection image B is matched and stitched to the template image A. Since the base image S has the same size as the template image A, that is, in this embodiment, the width of the base image S is 306930 pixels and the height is 2048 pixels.

[0080] S13. Set sliding window one on the template diagram and sliding window two on the inspection diagram. Both sliding window one and sliding window two are rectangular frames with the same size and height equal to the height of the template diagram.

[0081] In one optional implementation, a rectangular sliding window 1 on template diagram A is set as... Set the second rectangular sliding window on inspection diagram B as... Among them, sliding window one And sliding window 2 The width is Pixels, height are both Pixels. In this preferred embodiment, Pixels Pixel.

[0082] S2. Make the two windows slide horizontally to the right synchronously from the left edge of their respective images, and perform the following operations on the windows obtained from each slide.

[0083] Specifically, in this embodiment, step S2 includes:

[0084] S21. Establish global coordinate systems for the template diagram, inspection diagram, and base diagram, respectively, with the top left corner as the origin; establish local coordinate systems for the sliding window one and sliding window two, respectively, with the top left corner as the origin; for all coordinate systems, the horizontal axis is positive to the right and the vertical axis is positive downward.

[0085] In one optional implementation, the upper left corner of template diagram A, inspection diagram B, and base diagram S are respectively taken as the origin of the Cartesian coordinate system, with the horizontal coordinate system extending to the right as... The positive direction of the axis, vertically downwards is Establish their respective global coordinate systems along the positive axis, denoted as . Each using a sliding window And sliding window 2 The top left corner is the origin of the Cartesian coordinate system, and the horizontal direction to the right is... The positive direction of the axis, vertically downwards is Establish local coordinate systems for each axis in the positive direction, denoted as . .

[0086] S22. Set the initial x-coordinate of the upper left corner of sliding window one and sliding window two to zero. Make sliding window one slide horizontally to the right simultaneously from the left boundary of the template image and sliding window two slide horizontally from the left boundary of the inspection image, respectively. The horizontal sliding range of sliding window one is from zero to the width of the template image minus the width of sliding window one, and the horizontal sliding range of sliding window two is from zero to the width of the inspection image minus the width of sliding window two.

[0087] In one alternative implementation, let the sliding window... And sliding window 2 In their respective In coordinate system The position begins to slide horizontally to the right in sync, where the sliding window one The lateral sliding range is Sliding window two The lateral sliding range is Sliding window one and sliding window two move Each is recorded as follows: and The initial position is The window movement is illustrated as follows: Figure 2 As shown.

[0088] S3. Calculate the mapping relationship between the template subgraph in the current sliding window one and the inspection subgraph in the current sliding window two.

[0089] Specifically, in this embodiment, step S3 includes:

[0090] S31. Use the image in sliding window one as the template sub-image and the image in sliding window two as the inspection sub-image.

[0091] Specifically, the sliding window one The image inside is used as a template sub-image Slide window two The image within is used as an inspection sub-image. .

[0092] S32. Based on the image matching algorithm, calculate the mapping function between the template sub-image and the inspection sub-image in the local coordinate system as the mapping relationship.

[0093] In one alternative implementation, based on an image matching algorithm, in a local coordinate system Calculate the template subgraph Inspection Sub-diagram Mapping function between and As a mapping relationship, the image matching algorithm is either an algorithm based on generating a dense displacement field or an algorithm based on generating matching feature points; the algorithm based on generating a dense displacement field includes any one of DGC-Net, Farneback optical flow, PWC-Net, and FlowNet2; the algorithm based on generating matching feature points includes any one of SIFT, ORB, SuperPoint+LightGlue, and ROMA.

[0094] S4. Based on the mapping relationship, transform the current inspection subgraph to generate the transformation. Figure 1 Create a mask image and perform the same transformation to generate the transformation. Figure 2 .

[0095] Specifically, in this embodiment, step S4 includes:

[0096] S41. Based on the mapping relationship described in step S3, the current inspection subgraph is resampled using a bilinear interpolation algorithm to generate a transformed image. Figure 1 .

[0097] In one alternative implementation, a bilinear interpolation algorithm is used to inspect the subgraph. Perform resampling transformation and inspect subgraphs The image generated after mapping is a transformation. Figure 1 , set as ,but ,Right now Each location Pixel values ​​come from of The position pixel value, if Exceeding the inspection sub-map The boundary, then That is, the position is an invalid pixel; if No more than the inspection sub-map The boundary, then From the inspection sub-graph The pixel value obtained is the valid pixel at that location.

[0098] S42. Create a mask image with the same size as the sliding window, and initialize the pixel values ​​of all pixels in the mask image to preset valid label values, as the initial fully valid mask image.

[0099] In one alternative implementation, in order to determine the inspection subgraph Create a sliding window within the effective pixel area. A mask of the same size, set as That is, the width is pixels, height is The pixel values ​​of all pixels in the mask are preferably initialized to a valid marker value of 1 (this value is an empirical value) as an initial fully valid mask. The purpose of creating the mask is to assist subsequent algorithm processing.

[0100] S43. Transform the initial fully effective mask image using the same mapping relationship as in step S41 to generate the transformed image. Figure 2 .

[0101] In one alternative implementation, the mask image The transformation is performed using the same mapping relationship as in step S41 to generate the transformation. Figure 2 ,as follows:

[0102]

[0103] like Beyond the mask image The boundary, then Due to the transformation Figure 2 and transformation Figure 1 The same mapping relationship was used, therefore the transformation Figure 2 The position corresponding to the value of 0 is transformed Figure 1 The invalid pixel positions can be determined based on the transformation. Figure 2 Value determination transformation Figure 1 The location of invalid pixels and the location of valid pixels.

[0104] S5. Analysis and Transformation Figure 2 Determine the transformation Figure 1 The left boundary of the target pixel.

[0105] Specifically, in this embodiment, step S5 includes:

[0106] S51. In the transformation Figure 2 In the process, multiple simply connected regions composed of all valid pixels are extracted, and the simply connected region with the largest area is selected as the target connected region.

[0107] In one alternative implementation, the transformation Figure 2 In the process, extract multiple simply connected regions formed by all valid pixel positions (those with a value of 1), and select the set of simply connected region positions with the largest area. As the target connected region.

[0108] S52. Calculate the circumscribed rectangle or convex hull of the target connected region, and determine the transformation based on the left and right boundaries of the circumscribed rectangle or convex hull. Figure 1 The initial left and initial right boundaries.

[0109] In one alternative implementation, the target connected region is calculated. The circumscribed rectangle or convex hull. When calculating the target connected region. When defining the bounding rectangle, obtain the vertical line of the left boundary of the bounding rectangle. and the vertical line of the right boundary These are the initial left and right boundaries of the circumscribed rectangle. When calculating the target connected region... When constructing the convex hull, obtain the two points with the minimum and maximum x-coordinates in the convex hull point set. The vertical line passing through the point with the minimum x-coordinate is... The vertical line passing through the point with the maximum x-coordinate is... , which are the initial left and initial right boundaries of the convex hull.

[0110] S53. From transformation Figure 1 The transformation begins by scanning to the right from the initial left boundary. Figure 2 The process continues until the maximum length of the line segment intersecting the current vertical scan line and the target connected region is greater than or equal to a preset length threshold. The abscissa of this vertical scan line is then used as the transformation... Figure 1 The left boundary of the target pixel; if the condition is not met even after scanning to the preset upper limit position, then the x-coordinate of the maximum line segment length during the scanning process is taken as the transformation. Figure 1 The left boundary of the target pixel.

[0111] In one alternative implementation, in a local coordinate system Below, from the initial left boundary Start to the right ( The transformation is scanned in the positive direction of the axis. Figure 2 Until the current vertical scan line connects with the target region. The maximum length of the line segment formed by the intersection points is greater than or equal to a preset length threshold. Stop at time (this length threshold is set based on empirical values; this embodiment is preferred). (in pixels), the horizontal coordinate of the vertical scan line As a transformation Figure 1The left boundary of the target pixel; if the vertical scan line scans to the preset upper limit position of the scan (in this embodiment, the preset upper limit position of the scan is preferably the mask image) If the position still does not meet the condition, then take the x-coordinate of the point where the line segment length reaches its maximum value during the scanning process. As a transformation Figure 1 The target pixel's left boundary. In this embodiment, the vertical scan line is a vertical straight line.

[0112] S6. If the first window pair is processed, proceed with the next steps; otherwise, compare the current transformation. Figure 1 The left boundary of the target pixel is compared with the left boundary of the last stitching: if the former is greater than the latter, adjust the position of the current sliding window 2 and return to execute S3; otherwise, continue to execute the subsequent steps.

[0113] In this embodiment, in order to determine the first After the first slide, the sliding window is one. And sliding window 2 Location And determine whether to update the current sliding window 2. To ensure seamless stitching of matched images without gaps or other issues, it is necessary to calculate the current transform. Figure 1 left boundary of target pixel and right boundary and splicing the left boundary Among them, splicing the left side It will be in the The initial value is calculated when the window moves, and the value is calculated at the second time. Use the second window when sliding. The value of the second time is calculated, and the value of the third time is calculated. The value when the window is slid.

[0114] Specifically, in one optional implementation, step S6 includes:

[0115] When processing the first window pair, i.e. At that time, the subsequent steps are executed, namely, S7.

[0116] When processing non-first window pairs, i.e. When comparing the current transformation Figure 1 left boundary of target pixel Left border of the splicing compared to the last update There are two cases: Case 1, if the current transformation Figure 1 The left boundary of the target pixel is greater than the left boundary of the stitching, that is... ,like Figure 3 As shown, the position of the current sliding window two will be adjusted, specifically, in the global coordinate system. The current sliding window 2 Subtract the current transformation from the x-coordinate of the top left corner Figure 1 The value of the left boundary of the target pixel causes the sliding window two to move to the left, i.e. And return to execute S3. Case 2, if the current transformation Figure 1 The left boundary of the target pixel is less than or equal to the left boundary of the concatenation, that is... ,like Figure 4 As shown, no update will be performed. And continue with the subsequent steps. Indicates the first The top left corner of the second sliding window before it was updated during the second slide. Axis coordinate values.

[0117] S7. Analysis and Transformation Figure 2 Determine the transformation Figure 1 The right boundary of the target pixel.

[0118] Specifically, in this embodiment, step S7 includes:

[0119] The transformation determined in step S52 Figure 1 The transformation begins by scanning to the left from the initial right boundary. Figure 2 The process continues until the maximum length of the line segment intersecting the current vertical scan line and the target connected region is greater than or equal to a preset length threshold. The abscissa of this vertical scan line is then used as the transformation... Figure 1 The target pixel's right boundary; if the condition is not met even after scanning to the preset upper limit position, then the x-coordinate of the maximum line segment length during the scanning process is taken as the transformation. Figure 1 The right boundary of the target pixel.

[0120] In one alternative implementation, in a local coordinate system Below, from transformation Figure 1 initial right boundary Start to the left ( The transformation is scanned in the negative direction of the axis. Figure 2 Until the current vertical scan line connects with the target region. The maximum length of the line segment formed by the intersection points is greater than or equal to a preset length threshold. Stop at time (this length threshold is set based on empirical values; this embodiment is preferred). (in pixels), the horizontal coordinate of the vertical scan line As a transformation Figure 1 The right boundary of the target pixel; if the vertical scan line scans to the preset upper limit position of the scan (in this embodiment, the preset upper limit position of the scan is preferably the mask image). If the position still does not meet the condition, then take the x-coordinate of the point where the line segment length reaches its maximum value during the scanning process. As a transformation Figure 1 The right boundary of the target pixel.

[0121] S8. From the transformation Figure 1 Extract the valid area from the middle, and embed the valid area into the base map according to the current position of the sliding window.

[0122] Specifically, in this embodiment, step S8 includes:

[0123] S81. Calculate the current transformation Figure 1 The average of the x-coordinates of the left boundary of the target pixel and the left boundary of the stitching, i.e. .

[0124] S82. From the transformation Figure 1 Extract the horizontal axis from the average value of the x-coordinate described in step S81 to the transformed value. Figure 1 The right boundary region, i.e., in the local coordinate system Below, truncation transformation Figure 1 The image range is , , as the effective area.

[0125] S83. Place the upper left corner of the effective area described in step S82 at a position in the global coordinate system where the horizontal coordinate is the current horizontal coordinate of sliding window one plus the average horizontal coordinate described in step S81, and the vertical coordinate is the same as the vertical coordinate of sliding window one, and embed it into the base map. Specifically, in the local coordinate system... Below, the coordinates of the upper left corner of the effective region embedded in the base map S are: In the position, among them, Indicates the first The sliding window is one during the next slide. top left corner Axis coordinate values.

[0126] S9. According to the transformation Figure 1 The target pixel's right boundary and the preset baseline overlap width are used to calculate the next movement step of the sliding window. The next movement step of the sliding window is calculated based on the baseline overlap width, and the stitching left boundary is updated.

[0127] Specifically, in this embodiment, step S9 includes:

[0128] S91. A baseline overlap width is preset to control the minimum horizontal overlap area between sliding window one and sliding window two. In this embodiment, using... Indicates the baseline overlap width.

[0129] S92. If the width of the current sliding window one is reduced by the transformation... Figure 1If the difference between the right boundaries of the target pixels is greater than the baseline overlap width, then the next starting x-coordinate of the sliding window is set to the current x-coordinate and the transformation. Figure 1 The sum of the right boundaries of the target pixels; otherwise, set the next starting x-coordinate of sliding window one to the current x-coordinate plus the width of sliding window one minus the baseline overlap width.

[0130] In one alternative implementation, the purpose of this step is to, in the global coordinate system The calculation window is calculated after each horizontal slide. Top left corner Axis coordinates . Specifically, if Then set the next starting x-coordinate of the sliding window one as the current x-coordinate and the transformation. Figure 1 The sum of the right boundaries of the target pixels, i.e. Otherwise, that is, Set the next starting x-coordinate of sliding window one to the current x-coordinate plus the width of sliding window one minus the baseline overlap width. .

[0131] S93. Set the next starting horizontal coordinate of sliding window two to the current horizontal coordinate plus the width of sliding window two minus the baseline overlap width.

[0132] In one alternative implementation, the purpose of this step is to, in the global coordinate system Next calculation of the sliding window, the second horizontal sliding... Top left corner Axis coordinates Specifically, the next starting horizontal coordinate of sliding window two is set to the current horizontal coordinate plus the width of sliding window two minus the baseline overlap width. Since the position of sliding window two has already been updated in step S6, it is only necessary to update it based on the baseline overlap width here. Calculate the second sliding window The next position is fine.

[0133] S94. Update the left boundary of the stitching in the next slide to the current transformation. Figure 1 The target pixel's right boundary is subtracted from the x-coordinate of the next sliding window and then added to the x-coordinate of the current sliding window. Specifically, the sub-image is inspected during the next sliding. The vertical line of the left boundary of the splicing is updated to .

[0134] S10. Process each window pair sequentially until both sliding window one and sliding window two reach the right boundary of their respective images. At this point, the base image is used as the matching stitched image.

[0135] Specifically, in this embodiment, step S10 includes:

[0136] Each window pair is processed sequentially, and after each processing step, it is determined whether sliding window one has reached the right boundary of the template diagram and whether sliding window two has reached the right boundary of the inspection diagram, as detailed below:

[0137] Scenario 1: When the sliding window is... And sliding window 2 When they simultaneously reach the right boundary of their respective images, let ,make At this point, the base image S serves as the matching stitching image.

[0138] Scenario 2: When the sliding window is... When the right boundary of template A is reached, the sliding window 1 is fixed at the right boundary position of template A, that is, it is set to... Continue sliding window two but do not update the position of sliding window one for the next time. Until the second sliding window reaches the right boundary of the inspection map B, the base map S is used as the matching stitching map.

[0139] Scenario 3: When sliding window two When the right boundary of inspection diagram B is reached first, let Continue sliding window one, if Then update the position of the top left corner of the second sliding window to... Otherwise, no update. The process continues until both sliding window one and sliding window two reach the right boundary of their respective images. At this point, the base image S serves as the matching stitched image.

[0140] The above description only outlines the basic principles and preferred embodiments of the present invention. Those skilled in the art can make many changes and modifications based on the above description, and these changes and modifications should fall within the protection scope of the present invention.

Claims

1. A method for image matching processing of a linear array camera based on adaptive sliding window, characterized in that, Includes the following steps: S1. Obtain the template diagram and the inspection diagram, set the base image to be the same size as the template diagram, set the first sliding window on the template diagram and the second sliding window on the inspection diagram, the template diagram and the inspection diagram are the same height, the two windows are the same size and the height is equal to the height of the template diagram; S2. Make the two windows slide horizontally to the right synchronously from the left edge of their respective images, and perform the following operations on the window pairs obtained by each slide; S3. Calculate the mapping relationship between the template subgraph in the current sliding window one and the inspection subgraph in the current sliding window two; S4. Based on the mapping relationship, transform the current inspection sub-graph to generate Transformation Graph 1; create a mask graph and perform the same transformation to generate Transformation Graph 2; S5. Analyze the second transformation image to determine the left boundary of the target pixel in the first transformation image; S6. When processing the first window pair, execute the subsequent steps; otherwise, compare the left boundary of the target pixel of the current transformed image one with the last updated stitching left boundary: if the former is greater than the latter, adjust the position of the current sliding window two and return to execute S3; otherwise, continue to execute the subsequent steps. S7. Analyze the second transformation image to determine the right boundary of the target pixel in the first transformation image; S8. Extract the effective region from the transformed image 1, and embed the effective region into the base image according to the current position of the sliding window 1; S9. Calculate the next movement step of the sliding window based on the right boundary of the target pixel in the transformed image 1 and the preset baseline overlap width. Calculate the next movement step of the sliding window 2 based on the baseline overlap width and update the stitching left boundary. S10. Process each window pair sequentially until both sliding window one and sliding window two reach the right boundary of their respective images. At this point, the base image is used as the matching stitched image.

2. The method of claim 1, wherein, Step S1 specifically includes: S11. Obtain a defect-free image scanned by a fixed-position linear scan camera as a template image, and obtain a target image scanned by the same linear scan camera during the inspection process as an inspection image. The template image and the inspection image have the same height. S12. Set a base image with the same size as the template image, and initialize all pixel values ​​of the base image to preset values; S13. Set sliding window one on the template diagram and sliding window two on the inspection diagram. Both sliding window one and sliding window two are rectangular frames with the same size and height equal to the height of the template diagram.

3. The method of claim 1, wherein, Step S2 specifically includes: S21. Establish global coordinate systems for the template diagram, inspection diagram, and base diagram, respectively, with the top left corner as the origin; establish local coordinate systems for the sliding window one and sliding window two, respectively, with the top left corner as the origin; for all coordinate systems, the horizontal axis is positive to the right and the vertical axis is positive downward. S22. Set the initial x-coordinate of the upper left corner of sliding window one and sliding window two to zero. Make sliding window one slide horizontally to the right simultaneously from the left boundary of the template image and sliding window two slide horizontally from the left boundary of the inspection image, respectively. The horizontal sliding range of sliding window one is from zero to the width of the template image minus the width of sliding window one, and the horizontal sliding range of sliding window two is from zero to the width of the inspection image minus the width of sliding window two.

4. The method of claim 1, wherein, Step S3 specifically includes: S31. Use the image in sliding window one as the template sub-image and the image in sliding window two as the inspection sub-image; S32. Based on the image matching algorithm, calculate the mapping function between the template sub-image and the inspection sub-image in the local coordinate system as the mapping relationship.

5. The method of claim 1, wherein, Step S4 specifically includes: S41. Based on the mapping relationship described in step S3, the current inspection subgraph is resampled and transformed using a bilinear interpolation algorithm to generate Transformed Graph 1; S42. Create a mask image with the same size as the sliding window, and initialize the pixel values ​​of all pixels in the mask image to preset valid label values, as the initial fully valid mask image; S43. Transform the initial fully effective mask image using the same mapping relationship as in step S41 to generate the transformed image 2.

6. The method of claim 1, wherein, Step S5 specifically includes: S51. In the transformed graph 2, extract multiple simply connected regions composed of all valid pixels, and select the single connected region with the largest area as the target connected region. S52. Calculate the circumscribed rectangle or convex hull of the target connected region, and determine the initial left and initial right boundaries of the transformed graph 1 based on the left and right boundaries of the circumscribed rectangle or convex hull. S53. Starting from the initial left boundary of Transformation 1, scan Transformation 2 to the right until the maximum length of the line segment intersecting the current vertical scan line and the target connected region is greater than or equal to a preset length threshold. Use the horizontal coordinate of the vertical scan line as the left boundary of the target pixel of Transformation 1. If the condition is not met even after scanning to the preset upper limit position, take the horizontal coordinate of the maximum line segment length during the scanning process as the left boundary of the target pixel of Transformation 1.

7. The method of claim 1, wherein, In step S6, adjusting the position of the current sliding window two on the inspection map specifically involves subtracting the value of the left boundary of the target pixel in the current transformation map one from the horizontal coordinate of the upper left corner of the current sliding window two, thereby moving the sliding window two to the left.

8. The method of claim 6, wherein, Step S7 specifically includes: Starting from the initial right boundary of the first transformation image determined in step S52, the second transformation image is scanned to the left until the maximum length of the line segment intersecting the current vertical scan line and the target connected region is greater than or equal to a preset length threshold. The horizontal coordinate of the vertical scan line is taken as the right boundary of the target pixel of the first transformation image. If the condition is not met even after scanning to the preset upper limit position, the horizontal coordinate of the maximum line segment length during the scanning process is taken as the right boundary of the target pixel of the first transformation image.

9. The method of claim 1, wherein, Step S8 specifically includes: S81. Calculate the average abscissa of the left boundary of the target pixel and the left boundary of the stitching in the current transformed image 1; S82. Extract the region from the first transformation diagram, from the average horizontal coordinate value described in step S81 to the right boundary of the first transformation diagram, as the effective region; S83. Place the upper left corner of the effective area in the global coordinate system with the horizontal coordinate of the current horizontal coordinate of sliding window one plus the average horizontal coordinate of the horizontal coordinate described in step S81, and the vertical coordinate is the same as the vertical coordinate of sliding window one, and embed it into the base map.

10. The method of claim 1, wherein, Step S9 specifically includes: S91. A baseline overlap width is preset to control the minimum horizontal overlap area between sliding window one and sliding window two; S92. If the difference between the width of the current sliding window 1 and the right boundary of the target pixel of the transformation image 1 is greater than the reference overlap width, then set the next starting horizontal coordinate of the sliding window 1 to the sum of the current horizontal coordinate and the right boundary of the target pixel of the transformation image 1; otherwise, set the next starting horizontal coordinate of the sliding window 1 to the current horizontal coordinate plus the width of the sliding window 1 minus the reference overlap width. S93. Set the next starting x-coordinate of sliding window two to the current x-coordinate plus the width of sliding window two minus the baseline overlap width; S94. Update the left boundary of the stitching at the next slide to the right boundary of the target pixel of the current transformed image, minus the x-coordinate of the next sliding window, plus the x-coordinate of the current sliding window.

11. The method of claim 1, wherein, Step S10 specifically includes: Each window pair is processed sequentially, and after each processing step, it is determined whether sliding window one has reached the right boundary of the template diagram and whether sliding window two has reached the right boundary of the inspection diagram, as detailed below: Case 1: When sliding window 1 and sliding window 2 reach the right boundary of their respective images at the same time, the base image is used as the matching stitched image. Scenario 2: When sliding window 1 reaches the right boundary of the template image first, fix sliding window 1 at the right boundary of the template image, continue to slide sliding window 2 but do not update the position of sliding window 1 for the next time, until sliding window 2 reaches the right boundary of the inspection image. At this time, the base image is used as the matching and stitching image. Scenario 3: When sliding window 2 reaches the right boundary of the inspection image first, sliding window 1 continues to slide and the position of sliding window 2 is updated until both sliding window 1 and sliding window 2 reach the right boundary of their respective images. At this time, the base image is used as the matching stitched image.

Images