A multi-image array splicing method for large cement precast components

Abstract

The application provides a multi-image array splicing method for large cement prefabricated components, and the splicing method comprises the following steps: step one: collecting original images of the cement prefabricated components, dividing the original images into regions, and adjusting the gray scale of each region to obtain preprocessed images; step two: searching for feature points between every two adjacent images in each row in the images, identifying and extracting key feature points in the images through a SIFT algorithm, and then matching the key feature points of the adjacent images through a KNN algorithm to determine the relative position relationship between the key feature points; step three: searching for feature points between every two adjacent images in each column and matching the feature points; and step four: calculating the boundary positions of each image in the splicing area, and then splicing. The application realizes high-precision splicing of the cement prefabricated components, can effectively improve the shape detection precision of subsequent cement components, and improves the product detection quality.

Description

Technical Field

[0001] This invention relates to the fields of machine vision technology and image stitching technology, and more specifically, to a system for image acquisition and stitching of large precast concrete slabs using machine vision image stitching technology. Background Technology

[0002] Precast concrete components, as a product of modern industrialized construction, refer to reinforced concrete components prefabricated in specialized prefabrication plants or on construction sites. To ensure that precast components meet the expected quality standards before installation, they must be inspected for dimensional accuracy, surface flatness, strength, and the positional accuracy of embedded parts before being transported to the construction site, thereby guaranteeing the quality and safety of the entire construction project. Traditional methods for inspecting the shape characteristics of precast slabs mainly rely on manual inspection and some simple mechanical equipment. This is inefficient. Using machine vision systems greatly improves inspection efficiency and accuracy. Concrete components can reach sizes of 3m × 5m. To achieve a 1mm accuracy on-site, multiple cameras must be used to acquire multiple sets of images and then stitch them together. The accuracy and efficiency of image stitching are the main goals of researchers. Environmental complexity, complex backgrounds, and a lack of rich texture information often affect the stitching results.

[0003] When using a single camera for shooting, the existing detection methods are difficult to meet the detection accuracy requirements. Under the condition of ensuring high detection accuracy, it is difficult for a single camera with a field of view of 700mm×900mm to cover such a large area at once. Therefore, the image acquisition module and system detection method need to be designed specifically. In addition, the system light source illumination scheme also needs to be carefully considered.

[0004] Therefore, this invention focuses on large precast cement components and studies an image stitching method based on machine vision. This method optimizes the stitching effect and speed, thereby improving the accuracy, stability, and efficiency of the stitching of the test components. Summary of the Invention

[0005] In order to address the aforementioned issues in the prior art, the present invention aims to provide a multi-image stitching method to achieve high-precision and accurate stitching of large precast cement components.

[0006] This invention provides a method for stitching together multiple images of large precast cement components, the method comprising the following steps:

[0007] Step 1: Acquire the original images of the precast cement components. The original images are arranged in a matrix. Divide the original images into regions and adjust the grayscale of each region to obtain the preprocessed image.

[0008] Step 2: In the preprocessed image, find the feature points between each pair of adjacent images in each row. First, use the Scale Invariant Feature Transform (SIFT) algorithm to identify and extract the key feature points in the image. Then, use the KNN feature point matching algorithm to match the key feature points of adjacent images and determine the relative positional relationship between the key feature points.

[0009] Step 3: Using the method in Step 2, find the feature points between adjacent pairs of images in each column of the preprocessed image and perform matching;

[0010] Step 4: By analyzing the matched feature points, calculate the boundary position of each image in the splicing area, and then splice the images to obtain a complete image of a large precast concrete component.

[0011] Furthermore, the stitching method also includes step five: using Poisson fusion or multi-band fusion image fusion technology to fuse the stitching area of ​​the images, smooth the transition at the seam, and reduce stitching marks.

[0012] Furthermore, in step one, original images of X×Y precast cement components are acquired, and the original images cover the entire area to be spliced.

[0013] Furthermore, in step one, the original image is divided into regions, and the grayscale of each region is adjusted to obtain the preprocessed image as follows:

[0014] The cement portion of each image is divided into regions, and a fixed grayscale value is set. During image acquisition, the exposure of this fixed region is adjusted to ensure that the grayscale values ​​of the stitched portions of each image are nearly identical. The images are then converted to grayscale and stored in a two-dimensional array. The grayscale adjustment expression is as follows:

[0015]

[0016] Where I(i,j) is the pixel value at position (i,j) in the image, and M and N are the number of rows and columns in the image, respectively; the average brightness of the image is adjusted from μ to a new grayscale value μ′ by adjusting factor α:

[0017]

[0018] I'(i,j)=α·I(i,j).

[0019] Furthermore, the stitching method employs a single-row linear guide rail for moving and shooting.

[0020] Furthermore, step two specifically involves: traversing the two-dimensional image array and using a scale-invariant feature transform (SIFT) detector to find feature points between two horizontally adjacent images; for horizontally stitched images, the left side is the training image and the right side is the query image; feature points on the right half are found in the left image, and feature points on the left half are found in the right image; for horizontal stitching, a 1×4 long image is first generated, and the stitching edge position of each image is calculated based on the stitching distance, and the pixels of each image are copied sequentially to the 1×4 long image;

[0021] For a vertically stitched image, the upper side is the training image and the lower side is the query image. Feature points in the lower half are found in the upper image. The feature point matching algorithm KNN is used in the lower half to match feature descriptors and obtain the average value of the stitching distance in the x and y directions. Distances greater than the average value are filtered from the two-dimensional array, and then the average value of these filtered distances is calculated.

[0022] For vertical stitching, first generate a 4×4 matrix image, and then copy the 1×4 long image that was stitched horizontally to the long image according to the calculated stitching edge positions.

[0023] Furthermore, the scale-invariant feature transform SIFT specifically refers to:

[0024] First, by constructing the Difference of Gaussian Pyramid (DoG), we find extreme points in space at different scales. Then, based on the DoG approximation of the Laplacian operator (LoG), we identify scale- and rotation-invariant key points to obtain candidate key points.

[0025] For each candidate keypoint detected, the SIFT algorithm determines its location and scale by fitting a fine-grained model and selects stable keypoints.

[0026] For each keypoint, the gradient magnitude and direction of the region image centered at that point with a radius of 3 × 1.5σ are calculated, where σ is the scale provided in the SIFT algorithm, representing the size or influence range of the keypoint. The gradient magnitude m(x,y) and direction θ(x,y) of each point are calculated using the following formulas:

[0027]

[0028] Where L(x+i,y+j) is the image brightness information in Gaussian scale space, i={0,-1,+1},j={0,-1,+1};

[0029] The histogram of gradient direction θ(x,y) is used to statistically analyze the gradient direction and magnitude of pixels in the neighborhood of a feature point, thereby obtaining the main direction of the feature point. Based on the three pieces of information for each feature point: position x,y, scale σ, and direction θ, a SIFT feature region is determined.

[0030] Furthermore, the KNN algorithm for feature descriptor matching is specifically performed as follows:

[0031] Feature descriptors are used to describe the mathematical representation of features. A matching relationship is established between a feature point in the test image and a feature point in the training image to form a matching pair. The distance between the two feature descriptors of the matching pair is calculated, and the Euclidean distance D between the two feature descriptors d1 and d2 is calculated.

[0032] The expression is:

[0033]

[0034] Where n is the dimension of the descriptor, d 1i and d 2i These are the values ​​of d1 and d2 in the i-th dimension, respectively;

[0035] For each feature point in the image, the nearest neighbor search algorithm is used to find the K feature points in the training image that are closest to its descriptor;

[0036] The ratio test is used to filter matches. For each match pair, the ratio R of its distance to its nearest neighbor is calculated.

[0037]

[0038] If R is less than the set threshold, the match is considered valid.

[0039] Where D(dquery, d1st, nearest) is the feature point d query Matching the nearest neighbor point d 1st,nearest The distance between them; D(dquery, d2st, nearest) is the distance between feature points d. query Matching point d with the second nearest neighbor 2st,nearest The distance between them.

[0040] Furthermore, the weight of each pixel in the stitching region of the two images being stitched is proportional to the distance of the current processing point from the left boundary of the overlapping region.

[0041] Beneficial effects:

[0042] 1. The single-row camera image acquisition method used in this invention can minimize the offset and distortion caused by the camera itself and its installation position to image acquisition.

[0043] 2. This invention uses the ROI division method to process the image with fixed gray values, so that the brightness of the splicing area is basically consistent.

[0044] 3. The feature point finding method of this invention adopts a half-image finding method, which can improve the speed and accuracy of feature point matching.

[0045] 4. When calculating the splicing and fusion distance, this invention uses a larger distance as the basis for calculation. The purpose is to use the upper surface of the cement board as the splicing reference and the upper surface of the cement board as the basis for subsequent testing. This method can ignore the influence of the size ratio caused by the height difference. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1 The splicing effect diagram of the cement board prefabricated component is the research object of this invention patent;

[0048] Figure 2 This is a schematic diagram of an image acquisition device for acquiring images of precast cement components according to an embodiment of the present invention;

[0049] Figure 3 This is a flowchart of the image stitching process of this invention patent;

[0050] In the diagram, 201—camera lens group; 202—bar light source. Detailed Implementation

[0051] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0052] Figure 1 The diagram shows the splicing effect of the precast cement board component, which is the subject of this invention patent.

[0053] Figure 2 A schematic diagram of an image acquisition device for acquiring images of precast cement components according to an embodiment of the present invention is shown. The image acquisition device for recognizing the appearance features of precast slabs provided in this application includes:

[0054] A camera group with multiple cameras in a row: 4 cameras arranged horizontally;

[0055] Multiple rows of light sources are arranged within the camera's field of view: two rows and four columns of strip light sources are arranged in front and behind, and the illumination angle of the light sources can be adjusted to ensure uniform brightness of the acquired image.

[0056] Brightness adjustment is performed on a fixed area when acquiring images. Only the cement slab part to be stitched is divided into ROI regions. After calculating the average gray value of the ROI region, the brightness of the entire image is adjusted.

[0057] When searching for feature points, feature points are searched for on the right half of the left image and on the left half of the right image. This can improve the speed and accuracy of feature point matching.

[0058] When calculating the splicing distance, the center distance greater than the average distance is selected as the splicing reference. Since there are height differences in the cement boards, the upper surface of the cement board, which is more important, is selected as the splicing reference.

[0059] To achieve better stitching results, the first image is completely copied during the stitching process, and the remaining images are merged according to the stitching distance. This means that the gray value of the current processing point is proportional to the distance from the left boundary of the overlapping area, which makes the transition of the stitched area smoother.

[0060] Camera lens group 101, installation height 1000mm, camera field of view 900mm×700mm, camera spacing 550mm, adjacent image overlap area 150mm;

[0061] The bar light source 102 consists of two rows of horizontally arranged bar light sources with adjustable illumination angles to ensure uniform brightness on the detection surface.

[0062] Figure 3 The image stitching flowchart of this invention patent is shown.

[0063] Based on the above-described device, this embodiment provides a high-precision image stitching method for precast cement components, such as... Figure 2 The implementation steps are as follows:

[0064] Step 1: Acquire the original images of the precast cement components. The original images are arranged in a matrix. Divide the original images into regions and adjust the grayscale of each region to obtain the preprocessed image. In this embodiment, the image acquisition device moves 750mm at a time to acquire multiple rows of images and uploads them to the host computer via gigabit network.

[0065] The cement portion of each image is divided into regions, and a fixed grayscale value is set. During image acquisition, the exposure of this fixed region is adjusted to ensure that the grayscale values ​​of the stitched portions of each image are nearly identical. The images are then converted to grayscale and stored in a two-dimensional array. The formula for grayscale adjustment is as follows:

[0066]

[0067] Where I(i,j) is the pixel value at position (i,j) in the image, and M and N are the number of rows and columns in the image, respectively. If the average brightness of the image is adjusted from μ to a new value μ′, an adjustment factor α can be calculated:

[0068]

[0069] I'(i,j)=α·I(i,j).

[0070] Step 2: In the preprocessed image, find the feature points between each pair of adjacent images in each row. First, use the Scale Invariant Feature Transform (SIFT) algorithm to identify and extract key feature points in the image. Then, use the KNN algorithm to match the key feature points of adjacent images and determine the relative positional relationship between the key feature points. Specifically:

[0071] The process iterates through a two-dimensional array, using a Scale-Invariant Feature Transform (SIFT) detector to find feature points between horizontally adjacent images. For horizontally stitched images, the left side is the training image, and the right side is the query image. Feature points are searched for on the right half of the left image and on the left half of the right image. The KNN algorithm is used to match feature descriptors, obtaining the average stitching distance in the x and y directions. Distances greater than the average are selected from each set of data, and then the average of these selected distances is calculated. This step aims to use the top layer of the cement slab plane as the stitching reference. A 1×4 long image is generated, completely copying the first image from the left to the long image. The remaining images are then fused and stitched according to the stitching distance. Pixels from each image are copied sequentially to the 1×4 long image, meaning the gray value of the current processing point is proportional to the distance from the left boundary of the overlapping area, resulting in better stitching performance.

[0072] For a vertically stitched image, the upper side is the training image and the lower side is the query image. Feature points in the lower half are found in the upper image. The feature point matching algorithm KNN is used in the lower half to match feature descriptors and obtain the average value of the stitching distance in the x and y directions. Distances greater than the average value are filtered from the two-dimensional array, and then the average value of these filtered distances is calculated.

[0073] For vertical stitching, first generate a 4×4 matrix image, and then copy the 1×4 long image that was stitched horizontally to the long image according to the calculated stitching edge positions.

[0074] The SIFT (Scale Invariant Feature Transform) detector mentioned in this embodiment mainly involves the following steps:

[0075] 1. Scale-space extremum detection: The SIFT algorithm first searches for extrema, i.e., keypoints, in space at different scales. This is achieved by constructing a Difference of Gaussian (DoG) pyramid, which approximates the Laplacian of Gaussian (LoG) operator and is used to identify scale- and rotation-invariant keypoints.

[0076] 2. Keypoint localization: For each detected candidate keypoint, the SIFT algorithm examines each pixel's eight neighboring points in the current scale layer and its adjacent scale layers. If the point is a local extremum (maximum or minimum value) among these 26 points, it is considered a candidate keypoint.

[0077] 3. Feature Point Orientation Estimation: For each keypoint, the SIFT algorithm calculates the gradient magnitude and direction of the image region centered at that point with a radius of 3 × 1.5σ. The gradient magnitude m(x,y) and direction θ(x,y) of each point can be calculated using the following formula:

[0078]

[0079] Where L(x+i,y+j) is the image brightness information in Gaussian scale space, i={0,-1,+1},j={0,-1,+1}; this also shows that if the brightness difference between two images is large, the direction estimation of feature points will be biased.

[0080] Generating a gradient histogram: A histogram of gradient directions is used to statistically analyze the gradient direction and magnitude of pixels within the neighborhood of a feature point. The horizontal axis of the histogram represents the angle of the gradient direction (0 to 360 degrees, typically divided into 10 bars), and the vertical axis represents the sum of the gradient magnitudes. The peak value represents the dominant direction of the feature point. The SIFT algorithm also considers that a feature point may have multiple directions, which increases the descriptive power of the feature point. If there exists a bar value that corresponds to 80% of the energy of the dominant peak, this direction can also be considered as an auxiliary direction for the feature point. After obtaining the dominant direction of the feature point, three pieces of information can be obtained for each feature point (position x, y, scale σ, and direction θ), thereby determining a SIFT feature region.

[0081] The KNN feature point matching algorithm mentioned above has the following steps:

[0082] 1. Distance Metrics: Calculates the distance between two feature descriptors. A feature descriptor is a mathematical representation used to describe a feature (such as a keypoint in an image). It transforms the local information of a feature into a feature vector, which contains detailed information about the region surrounding the feature, making the feature uniquely identifiable and matchable in different images or scenes. For example, the SIFT descriptor generates a 128-dimensional feature vector by calculating the gradient magnitude and direction of the region surrounding the keypoint. Commonly used distance metrics include Euclidean distance and Hamming distance. For two feature descriptors d1 and d2, the Euclidean distance is calculated as follows:

[0083]

[0084] Where n is the dimension of the descriptor, d 1i and d 2i These are the values ​​of d1 and d2 in the i-th dimension, respectively.

[0085] 2. Nearest Neighbor Search: For each feature point in the query image, the nearest neighbor search algorithm is used to find the K feature points in the training image that are closest to its descriptor.

[0086] 3. Match Filtering: Use a ratio test to filter matches. For each match pair, calculate the ratio of its distance to its nearest neighbor:

[0087]

[0088] A match is considered valid if R is less than a certain threshold (usually 0.75). In this case, K is set to 2, and the threshold for R is 0.7.

[0089] Where D(dquery, d1st, nearest) is the feature point d query Matching the nearest neighbor point d 1st,nearest The distance between them; D(dquery, d2st, nearest) is the distance between feature points d. query Matching point d with the second nearest neighbor 2st,nearest The distance between them.

[0090] Generate a 4×4 long image. Completely copy the top horizontal image group from step three into the long image. The remaining image groups are merged and stitched according to the stitching distance, that is, the gray value of the current processing point is proportional to the distance of the left boundary of the overlapping area, which can obtain a better stitching effect.

[0091] Step 3: Using the method in Step 2, find the feature points between adjacent pairs of images in each column of the preprocessed image and perform matching;

[0092] Step 4: By analyzing the matched feature points, calculate the boundary position of each image in the splicing area, and then splice the images to obtain a complete image of a large precast concrete component.

[0093] Finally, crop the image and remove the black borders at the edges of the stitched image.

[0094] By employing the above-described technical process, the overall image stitching task of large precast concrete components can be completed with high accuracy and efficiency. The image acquisition system ensures the acquisition of high-resolution images with high linearity; while the stitching technology can accurately stitch images onto the surfaces of precast concrete components, thereby improving the efficiency of the stitching process.

[0095] Regarding the limitation of the scope of protection of this invention, those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solution of this invention are still within the scope of protection of this invention.

Claims

1. A method for stitching together multiple images in a large precast concrete component, characterized in that, The splicing method includes the following process: Step 1: Acquire the original images of the precast cement components. The original images are arranged in a matrix. Divide the original images into regions and adjust the grayscale of each region to obtain the preprocessed image. Divide the cement part of each image, set a fixed grayscale value, and set the exposure of the fixed region when acquiring the image so that the grayscale value of the stitched part of each image is close to the same. Then convert the image into a grayscale image and store it in a two-dimensional array. Step 2: In the preprocessed image, find the feature points between adjacent images in each row. First, use the Scale Invariant Feature Transform (SIFT) algorithm to identify and extract key feature points in the image. Then, use the KNN feature point matching algorithm to match the key feature points of adjacent images and determine the relative positional relationship between the key feature points. Specifically, Step 2 involves: traversing the two-dimensional array and using the SIFT detector to find feature points between horizontally adjacent images. For horizontally stitched images, the left side is the training image and the right side is the query image. Find the feature points of the right half in the left image and the feature points of the left half in the right image. For horizontal stitching, first generate a 1×4 long image, calculate the stitching edge position of each image based on the stitching distance, and copy the pixels of each image to the 1×4 long image in sequence. For a vertically stitched image, the upper side is the training image and the lower side is the query image. Feature points in the lower half are found in the upper image. The feature point matching algorithm KNN is used to match feature descriptors in the lower half to obtain the average value of the stitching distance in the x and y directions. Distances greater than the average value are filtered from the two-dimensional array, and then the average value of these filtered distances is calculated. For vertical stitching, first generate a 4×4 matrix image, and then copy the 1×4 long image that has been stitched horizontally to the long image according to the calculated stitching edge positions. Step 3: Using the method in Step 2, find the feature points between adjacent pairs of images in each column of the preprocessed image and perform matching; Step 4: By analyzing the matched feature points, calculate the boundary position of each image in the splicing area, and then splice the images to obtain a complete image of a large precast concrete component. The stitching method further includes step five: using Poisson fusion or multi-band fusion image fusion technology to fuse the stitching area of ​​the images, smooth the transition at the seam, and reduce stitching marks.

2. The multi-image array stitching method for large precast cement components according to claim 1, characterized in that, In step one, original images of X×Y precast cement components are acquired, and these original images cover the entire area to be spliced.

3. The multi-image array stitching method for large precast cement components according to claim 1, characterized in that, In step one, the expression for grayscale adjustment is as follows: ； in, Is the image in position The pixel values ​​are M and N, respectively, the number of rows and columns of the image; the average brightness of the image is adjusted from μ to a new grayscale value μ′ by adjusting factor α: ； 。 4. The multi-image array stitching method for large precast cement components according to claim 1, characterized in that, The stitching method uses a single-row linear guide rail for moving and shooting.

5. The multi-image array stitching method for large precast cement components according to claim 1, characterized in that, The scale-invariant feature transform SIFT specifically refers to: First, by constructing the Difference of Gaussian Pyramid (DoG), we find extreme points in space at different scales. Then, based on the DoG approximation of the Laplacian operator (LoG), we identify scale- and rotation-invariant key points to obtain candidate key points. For each candidate keypoint detected, the SIFT algorithm determines its location and scale by fitting a fine-grained model and selects stable keypoints. For each keypoint, the gradient magnitude and direction of the region image centered at that point with a radius of 3 × 1.5σ, where σ is the scale provided in the SIFT algorithm, representing the size or influence range of the keypoint; the gradient magnitude of each point... and direction It is calculated using the following formula: ； in It represents the image brightness information in Gaussian scale space, i={0,-1,+1},j={0,-1,+1}; gradient direction The histogram is used to statistically analyze the gradient direction and magnitude of pixels in the neighborhood of a feature point, thereby obtaining the main direction of the feature point. Based on three pieces of information for each feature point: position x, y, scale σ, and direction θ, a SIFT feature region is determined.

6. The multi-image array stitching method for large precast cement components according to claim 1, characterized in that, The specific steps for matching feature descriptors using the KNN feature point matching algorithm are as follows: Feature descriptors are used to describe the mathematical representation of features. A matching relationship is established between a feature point in the test image and a feature point in the training image, forming a matching pair. The distance between the two feature descriptors of the matching pair is calculated. and The Euclidean distance D between them is expressed as: ； Where n is the dimension of the descriptor. and They are and The value in the i-th dimension; For each feature point in the image, the nearest neighbor search algorithm is used to find the K feature points in the training image that are closest to its descriptor; The ratio test is used to filter matches. For each match pair, the ratio R of its distance to its nearest neighbor is calculated. ； If R is less than the set threshold, the match is considered valid; in Feature points Matching points with nearest neighbors The distance between them; Feature points Matching points with the next nearest neighbor The distance between them.

7. The multi-image array stitching method for large precast cement components according to claim 1, characterized in that, The weight of each pixel in the stitching region of the two images being stitched is proportional to the distance of the current processing point from the left boundary of the overlapping region.

Images