Multi-image array splicing method for large cement prefabricated part

Abstract

The invention provides a multi-image array splicing method for a large cement prefabricated part, and the method comprises the following steps: 1, collecting an original image of the cement prefabricated part, carrying out the region division of the original image, carrying out the gray adjustment of each region, and obtaining a preprocessed image; 2, searching feature points between every two adjacent images in each row in the images, identifying and extracting key feature points in the images through an SIFT algorithm, matching the key feature points of the adjacent images through a feature point matching algorithm KNN algorithm, and determining a relative position relationship between the key feature points; 3, searching feature points between every two adjacent images in each column, and matching the feature points; and 4, calculating the boundary position of each image in the splicing area, and then splicing. According to the invention, high-precision splicing of the cement prefabricated components is realized, the appearance detection precision of the subsequent cement components can be effectively improved, and the product detection quality is improved.

Description

Technical Field

[0001] The present invention relates to the technical fields of machine vision and image splicing, and more specifically to an image acquisition and splicing system for large cement prefabricated panels using machine vision image splicing technology. Background Art

[0002] As a product of modern building industrialization, precast concrete components refer to reinforced concrete components that are pre-processed and manufactured in a special precast plant or construction site. In order to ensure that precast components meet the expected quality standards before installation, the dimensional accuracy, surface flatness, strength and position accuracy of precast components must be tested before being transported to the construction site to ensure the quality and safety of the entire construction project. The traditional method of detecting the appearance characteristics of precast panels mainly relies on manual inspection and some simple mechanical equipment. The efficiency is low, and the use of machine vision systems greatly improves the detection efficiency and accuracy. The size of concrete components can reach 3m×5m. In order to achieve 1mm accuracy on site, multiple cameras must be used to collect multiple sets of images and stitch them together. The accuracy and efficiency of image stitching are the main goals of researchers. Problems such as environmental complexity, complex background, and lack of rich texture information often affect the stitching effect.

[0003] When using a single camera for shooting, the existing detection methods are difficult to meet the detection accuracy requirements. Under the condition of ensuring high detection accuracy, the single camera field of view of 700mm×900mm is difficult to cover such a large range at one time, so it is necessary to make targeted designs for the image acquisition module and system detection method. In addition, the system light source lighting solution must also be considered.

[0004] Therefore, the present invention aims at large-scale cement prefabricated components and studies an image splicing method based on machine vision, which will optimize the splicing effect and splicing speed and improve the accuracy, stability and efficiency of the splicing of the detection parts. Summary of the invention

[0005] In order to solve the above-mentioned problems in the prior art, the purpose of the present invention is to provide a multi-image stitching method to achieve high-precision and accurate stitching of large-scale prefabricated cement components.

[0006] The present invention provides a multi-image array splicing method for a large-scale cement prefabricated component, and the splicing method comprises the following processes:

[0007] Step 1: collecting an original image of a cement prefabricated component, wherein the original image is in a matrix form, dividing the original image into regions, and adjusting the grayscale of each region to obtain a preprocessed image;

[0008] Step 2: Find the feature points between two adjacent images in each row in the preprocessed image, first identify and extract the key feature points in the image through the scale-invariant feature transform SIFT algorithm, and then match the key feature points of adjacent images through the feature point matching algorithm KNN algorithm to determine the relative position relationship between the key feature points;

[0009] Step 3: Using the method in step 2, find feature points between two adjacent images in each column in the preprocessed image and perform matching;

[0010] Step 4: By analyzing the matched feature points, the boundary position of each image in the stitching area is calculated, and then the images are stitched to obtain a complete large-scale prefabricated cement component image.

[0011] Furthermore, the stitching method also includes step five: fusing the stitching of the images through Poisson fusion or multi-band fusion image fusion technology to smooth the transition at the seams and reduce stitching marks.

[0012] Furthermore, in the step 1, X×Y original images of cement prefabricated components are collected, and the original images cover the entire area to be spliced.

[0013] Furthermore, in step 1, the original image is divided into regions, and the grayscale of each region is adjusted to obtain a preprocessed image:

[0014] Divide the cement area of ​​each image, set a fixed gray value, and perform exposure setting for the fixed area when collecting images, so that the gray value of the spliced ​​part of each image is close to the same, and convert the image into a grayscale image and store it in a two-dimensional array; the expression for grayscale adjustment is as follows:

[0015]

[0016] Where I(i,j) is the pixel value of the image at position (i,j), M and N are the number of rows and columns of the image respectively; the average brightness of the image is adjusted from μ to a new grayscale value μ′ by adjusting the factor α:

[0017]

[0018] I'(i,j)=α·I(i,j).

[0019] Furthermore, the stitching method uses a single row of following linear guide rails for mobile shooting.

[0020] Furthermore, the step 2 is specifically as follows: traverse the two-dimensional image array, use the scale-invariant feature transform SIFT detector to find feature points for two horizontally adjacent images; for the horizontally spliced ​​image, the left side is the training image and the right side is the query image; find the feature points of the right half in the left image, and find the feature points of the left half in the right image; for horizontal splicing, first generate a 1×4 long image, calculate the splicing edge position of each image according to the splicing distance, and copy the pixels of each image to the 1×4 long image in turn;

[0021] For the vertical stitching image, the upper side is the training image, and the lower side is the query image; find the feature points of the lower half in the upper image, use the feature point matching algorithm KNN algorithm to match the feature descriptor in the lower volume, obtain the average value of the stitching distance in the x direction and the y direction, filter out the distance greater than the average value from the two-dimensional array, and then calculate the average value of these filtered distances;

[0022] For vertical splicing, a 4×4 matrix image is first generated, and the 1×4 long image spliced ​​horizontally is copied to the long image in sequence according to the calculated splicing edge positions.

[0023] Furthermore, the scale-invariant feature transform SIFT is specifically:

[0024] Firstly, we construct a Gaussian difference pyramid DoG to find extreme points in the space of different scales, and then identify the scale and rotation invariant key points based on the approximation of DoG to the Laplace operator LoG to obtain candidate key points.

[0025] For each candidate key point detected, the SIFT algorithm determines its position and scale by fitting a sophisticated model and selecting stable key points;

[0026] For each key point, the gradient magnitude and direction of the regional image with the point as the center and a radius of 3×1.5σ, where σ is the scale provided in the SIFT algorithm, indicating the size or influence range of the key point; the gradient modulus m(x, y) and direction θ(x, y) of each point are calculated by the following formula:

[0027]

[0028] Where L(x+i,y+j) is the image brightness information in Gaussian scale space, i = {0, -1, +1}, j = {0, -1, +1};

[0029] The histogram of the gradient direction θ(x, y) is used to count the gradient direction and amplitude corresponding to the pixels in the neighborhood of the feature point to obtain the main direction of the feature point. A SIFT feature area is determined based on the three information of each feature point: position x, y, scale σ, and direction θ.

[0030] Furthermore, the feature point matching algorithm KNN algorithm is used to match the feature descriptor as follows:

[0031] Use feature descriptors to describe the mathematical representation of features, establish a matching relationship between a feature point in the test image and a feature point in the training image to form a matching pair, calculate the distance between the two feature descriptors of the matching pair, and calculate the Euclidean distance D between the two feature descriptors d1 and d2.

[0032] The expression is:

[0033]

[0034] Among them, n is the dimension of the descriptor, d 1i and d 2i are the values ​​of d1 and d2 in the i-th dimension respectively;

[0035] For each feature point in the image, use the nearest neighbor search algorithm to find the K feature points in the training image that are closest to its descriptor;

[0036] Use the ratio test to screen matches. For each matching pair, calculate the ratio R of its distance to the nearest neighbor:

[0037]

[0038] If R is less than the set threshold, the match is considered valid.

[0039] Where D(dquery,d1st,nearest) is the feature point d query Match point d with the nearest neighbor 1st,nearest The distance between them; D(dquery,d2st,nearest) is the distance between feature points d query Match point d with the next nearest neighbor 2st,nearest The distance between.

[0040] Furthermore, the weight of each pixel in the stitching area of ​​the two images to be stitched is proportional to the distance between the current processing point and the left border of the overlapping area.

[0041] Beneficial effects:

[0042] 1. The single-row camera image acquisition method adopted in the present invention can avoid the deviation and distortion caused by the camera itself and the installation position to the image acquisition to the greatest extent.

[0043] 2. The present invention adopts the ROI division method to perform fixed gray value processing on the image, so that the brightness of the splicing area is basically consistent.

[0044] 3. The method for finding feature points of the present invention adopts a half-graph finding method, which can improve the speed and accuracy of feature point matching.

[0045] 4. When calculating the splicing and fusion distance, the present invention uses the larger distance as the calculation basis, with the purpose of using the upper surface of the cement board as the splicing reference and the upper surface of the cement board as the basis for subsequent detection. This method can ignore the influence of the size ratio caused by the height difference. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0047] Figure 1 The splicing effect diagram of cement board prefabricated components, which is the research object of the present invention;

[0048] Figure 2 Schematic diagram of an image acquisition device for acquiring images of cement precast components according to an embodiment of the present invention;

[0049] Figure 3 This is the image stitching flow chart of the patent of this invention;

[0050] In the figure, 201 is a camera lens group; 202 is a bar light source. DETAILED DESCRIPTION

[0051] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0052] Figure 1 The splicing effect diagram of the cement board prefabricated components which is the research object of the patent of this invention is shown.

[0053] Figure 2 A schematic diagram of an image acquisition device for acquiring an image of a cement prefabricated component according to an embodiment of the present invention is shown. The image acquisition device for recognizing the appearance features of a prefabricated board provided in the present application comprises:

[0054] A camera group with multiple cameras in a row: 4 cameras are arranged horizontally;

[0055] Arrange multiple rows of light source groups within the camera's field of view: 2 rows and 4 columns of strip light sources are arranged in front and behind. The light source can adjust the illumination angle to make the brightness of the collected image uniform.

[0056] When acquiring images, the brightness of a fixed area is adjusted. Only the cement board part to be spliced ​​is divided into a ROI area. After the average gray value of the ROI area is solved, the brightness of the entire image is adjusted.

[0057] When searching for feature points, the left image searches for feature points on the right half, and the right image searches for feature points on the left half, which can improve the speed and accuracy of feature point matching.

[0058] When calculating the splicing distance, select the center distance that is greater than the average distance as the splicing reference. Because the cement board has height differences, select the more important upper surface of the cement board as the splicing reference.

[0059] When stitching, in order to achieve better stitching effect, the first image is completely copied, and the remaining images are fused and stitched according to the stitching distance, that is, the gray value of the current processing point is proportional to the distance from the left boundary of the overlapping area, which can make the transition of the stitching area smoother.

[0060] Camera lens group 101, installation height 1000mm, camera field of view size 900mm×700mm, camera spacing 550mm, adjacent image overlap area 150mm;

[0061] The strip light source 102 is arranged in two rows horizontally, and the illumination angle can be adjusted to make the brightness of the detection surface uniform;

[0062] Figure 3 The image stitching flow chart of the patent of the present invention is shown.

[0063] Based on the above device, this embodiment provides a high-precision image stitching method for cement prefabricated components, such as Figure 2 The implementation steps are as follows:

[0064] Step 1: Collect the original image of the cement precast component, the original image is in a matrix form, the original image is divided into regions, and the grayscale of each region is adjusted to obtain a preprocessed image; in this embodiment, the image acquisition device moves 750mm at a time to collect multiple lines of images, and uploads them to the host computer via the Gigabit network;

[0065] Divide the cement area of ​​each image, set a fixed grayscale value, and perform exposure setting for the fixed area when collecting images, so that the grayscale value of the spliced ​​part of each image is close to the same, and convert the image into a grayscale image and store it in a two-dimensional array; the principle formula of grayscale adjustment is as follows:

[0066]

[0067] Where I(i,j) is the pixel value of the image at position (i,j), M and N are the number of rows and columns of the image respectively. If the average brightness of the image is adjusted from μ to a new value μ′, an adjustment factor α can be calculated:

[0068]

[0069] I'(i,j)=α·I(i,j).

[0070] Step 2: Find the feature points between two adjacent images in each row in the preprocessed image, first identify and extract the key feature points in the image through the scale-invariant feature transform SIFT algorithm, and then match the key feature points of adjacent images through the feature point matching algorithm KNN algorithm to determine the relative position relationship between the key feature points; specifically:

[0071] Traverse the two-dimensional array and use the scale-invariant feature transform SIFT detector to find feature points for two adjacent horizontal images; for horizontal splicing images, the left side is the training image, the left side is the training image, and the right side is the query image; the left image finds the feature points of the right half, and the right image finds the feature points of the left half. Use the KNN algorithm to match the feature descriptors, obtain the average value of the splicing distance in the x and y directions, filter out the distance greater than the average value from each set of data, and then calculate the average value of these filtered distances. The purpose of this step is to use the top cement board plane as the splicing reference. Generate a 1×4 long image, completely copy the first image on the left to the long image, and merge and splice the remaining images according to the splicing distance. Copy the pixels of each image to the 1×4 long image in turn, that is, the gray value of the current processing point is proportional to the distance from the left boundary of the overlapping area, which can obtain a better splicing effect.

[0072] For the vertical stitching image, the upper side is the training image, and the lower side is the query image; find the feature points of the lower half in the upper image, use the feature point matching algorithm KNN algorithm to match the feature descriptor in the lower volume, obtain the average value of the stitching distance in the x direction and the y direction, filter out the distance greater than the average value from the two-dimensional array, and then calculate the average value of these filtered distances;

[0073] For vertical splicing, a 4×4 matrix image is first generated, and the 1×4 long image spliced ​​horizontally is copied to the long image in sequence according to the calculated splicing edge positions.

[0074] The SIFT (Scale Invariant Feature Transform) detector mentioned in this embodiment mainly involves the following steps:

[0075] 1. Scale space extreme value detection: The SIFT algorithm first searches for extreme points, i.e. key points, in spaces of different scales. This is achieved by constructing a Difference of Gaussian (DoG) pyramid, which is similar to the Laplacian of Gaussian (LoG) and is used to identify scale and rotation invariant key points.

[0076] 2. Key point positioning: For each candidate key point detected, the SIFT algorithm checks the 8 neighborhood points of each pixel in the current scale layer and its adjacent scale layers. If the point is a local extreme point (maximum or minimum value) among these 26 points, it is considered to be a candidate key point.

[0077] 3. Feature point direction estimation: For each key point, the SIFT algorithm calculates the gradient magnitude and direction of the image with the point as the center and a radius of 3×1.5σ. The gradient magnitude m(x, y) and direction θ(x, y) of each point can be calculated by the following formula:

[0078]

[0079] Where L(x+i,y+j) is the image brightness information in the Gaussian scale space, i={0,-1,+1}, j={0,-1,+1}; this also means that if the brightness difference between the two images is large, the direction estimation of the feature point will be biased.

[0080] Generate a gradient histogram: Use the histogram of the gradient direction to count the gradient direction and amplitude corresponding to the pixels in the neighborhood of the feature point. The horizontal axis of the histogram is the angle of the gradient direction (0 to 360 degrees, usually divided into 10 columns), and the vertical axis is the accumulation of the gradient amplitude. The peak represents the main direction of the feature point. The SIFT algorithm also takes into account that feature points may have multiple directions, which increases the descriptive ability of feature points. If there is a column value equivalent to 80% of the energy of the main peak, this direction can also be considered as the auxiliary direction of the feature point. After obtaining the main direction of the feature point, three information (position x, y, scale σ, direction θ) can be obtained for each feature point, thereby determining a SIFT feature area.

[0081] The above-mentioned KNN feature point matching algorithm has the following steps:

[0082] 1. Distance metric: Calculate the distance between two feature descriptors. A feature descriptor is a mathematical representation used to describe a feature (such as a key point in an image) in detail. It converts the local information of a feature into a feature vector, which contains detailed information about the area surrounding the feature, so that the feature can be uniquely identified and matched in different images or scenes. For example, the SIFT descriptor: generates a 128-dimensional feature vector by calculating the gradient amplitude and direction of the area around the key point. Commonly used distance metrics include Euclidean distance and Hamming distance. For two feature descriptors d1 and d2, the Euclidean distance calculation formula is:

[0083]

[0084] Among them, n is the dimension of the descriptor, d 1i and d 2i are the values ​​of d1 and d2 in the i-th dimension respectively.

[0085] 2. Nearest neighbor search: For each feature point in the query image, use the nearest neighbor search algorithm to find the K feature points in the training image that are closest to its descriptor.

[0086] 3. Match screening: Use ratio test to screen matches. For each matching pair, calculate the distance ratio between it and the nearest neighbor:

[0087]

[0088] If R is less than a certain threshold (usually 0.75), the match is considered valid. In this case, the K value is set to 2 and the R threshold is 0.7.

[0089] Where D(dquery,d1st,nearest) is the feature point d query Match point d with the nearest neighbor 1st,nearest The distance between them; D(dquery,d2st,nearest) is the distance between feature points d query Match point d with the next nearest neighbor 2st,nearest The distance between.

[0090] Generate a 4×4 long image, completely copy the top set of horizontal images in step 3 to the long image, and merge and stitch the remaining image groups according to the stitching distance, that is, the grayscale value of the current processing point is proportional to the distance from the left boundary of the overlapping area, which can obtain a better stitching effect.

[0091] Step 3: Using the method in step 2, find feature points between two adjacent images in each column in the preprocessed image and perform matching;

[0092] Step 4: By analyzing the matched feature points, the boundary position of each image in the stitching area is calculated, and then the images are stitched to obtain a complete large-scale prefabricated cement component image.

[0093] Finally, crop the image to remove the black edges at the edges of the stitched image.

[0094] The above technical process can complete the overall image stitching task of large precast cement components with high accuracy and efficiency. The image acquisition system can ensure the acquisition of high-resolution images with high linearity; while the stitching technology can accurately stitch the surface of the precast cement components and improve the efficiency of the stitching process.

[0095] Regarding the limitation of the protection scope of the present invention, technicians in the relevant field should understand that various modifications or variations that can be made by those skilled in the art without creative work on the basis of the technical solution of the present invention are still within the protection scope of the present invention.

Claims

1. A multi-image array splicing method for large-scale cement prefabricated components, characterized in that: The splicing method includes the following processes: Step 1: collecting an original image of a cement prefabricated component, wherein the original image is in a matrix form, dividing the original image into regions, and adjusting the grayscale of each region to obtain a preprocessed image; Step 2: Find the feature points between two adjacent images in each row in the preprocessed image, first identify and extract the key feature points in the image through the scale-invariant feature transform SIFT algorithm, and then match the key feature points of adjacent images through the feature point matching algorithm KNN algorithm to determine the relative position relationship between the key feature points; Step 3: Using the method in step 2, find feature points between two adjacent images in each column in the preprocessed image and perform matching; Step 4: By analyzing the matched feature points, the boundary position of each image in the stitching area is calculated, and then the images are stitched to obtain a complete large-scale prefabricated cement component image.

2. The multi-image array splicing method for large-scale cement prefabricated components according to claim 1 is characterized in that: The stitching method further comprises step five: fusing the stitching parts of the images by using the image fusion technology of Poisson fusion or multi-band fusion to smooth the transition at the seams and reduce stitching marks.

3. The multi-image array splicing method for large-scale cement prefabricated components according to claim 1 is characterized in that: In the step 1, X×Y original images of cement prefabricated components are collected, and the original images cover the entire area to be spliced.

4. The multi-image array splicing method for large-scale cement prefabricated components according to claim 1, characterized in that: In step 1, the original image is divided into regions, and the grayscale of each region is adjusted to obtain a preprocessed image: Divide the cement area of ​​each image, set a fixed gray value, and perform exposure setting for the fixed area when collecting images, so that the gray value of the spliced ​​part of each image is close to the same, and convert the image into a grayscale image and store it in a two-dimensional array; the expression for grayscale adjustment is as follows: Where I(i,j) is the pixel value of the image at position (i,j), M and N are the number of rows and columns of the image respectively; the average brightness of the image is adjusted from μ to a new grayscale value μ′ by adjusting the factor α: I'(i,j)=α·I(i,j).

5. The multi-image array splicing method of large-scale cement prefabricated components according to claim 1 is characterized in that: The splicing method adopts a single row of accompanying linear guide rails for mobile shooting.

6. The multi-image array splicing method of large-scale cement prefabricated components according to claim 4, characterized in that: The step 2 is specifically as follows: traverse the two-dimensional image array, use the scale-invariant feature transform SIFT detector to find feature points for two horizontally adjacent images; for the horizontal splicing image, the left side is the training image and the right side is the query image; find the feature points of the right half in the left image, and find the feature points of the left half in the right image; for the horizontal splicing, first generate a 1×4 long image, calculate the splicing edge position of each image according to the splicing distance, and copy the pixels of each image to the 1×4 long image in turn; For the vertical stitching image, the upper side is the training image, and the lower side is the query image; find the feature points of the lower half in the upper image, use the feature point matching algorithm KNN algorithm to match the feature descriptor in the lower volume, obtain the average value of the stitching distance in the x direction and the y direction, filter out the distance greater than the average value from the two-dimensional array, and then calculate the average value of these filtered distances; For vertical splicing, a 4×4 matrix image is first generated, and the 1×4 long image spliced ​​horizontally is copied to the long image in sequence according to the calculated splicing edge positions.

7. The multi-image array splicing method of large-scale cement prefabricated components according to claim 6, characterized in that: The scale invariant feature transform SIFT is specifically: Firstly, we construct a Gaussian difference pyramid DoG to find extreme points in the space of different scales, and then identify the scale and rotation invariant key points based on the approximation of DoG to the Laplace operator LoG to obtain candidate key points. For each candidate key point detected, the SIFT algorithm determines its position and scale by fitting a sophisticated model and selecting stable key points; For each key point, the gradient magnitude and direction of the regional image with the point as the center and a radius of 3×1.5σ, where σ is the scale provided in the SIFT algorithm, indicating the size or influence range of the key point; the gradient modulus m(x, y) and direction θ(x, y) of each point are calculated by the following formula: Where L(x+i,y+j) is the image brightness information in Gaussian scale space, i = {0, -1, +1}, j = {0, -1, +1}; The histogram of the gradient direction θ(x, y) is used to count the gradient direction and amplitude corresponding to the pixels in the neighborhood of the feature point to obtain the main direction of the feature point. A SIFT feature area is determined based on the three information of each feature point: position x, y, scale σ, and direction θ.

8. The multi-image array splicing method for large-scale cement prefabricated components according to claim 6, characterized in that: The feature point matching algorithm KNN algorithm is used to match the feature descriptor as follows: Use feature descriptors to describe the mathematical representation of features, establish a matching relationship between a feature point in the test image and a feature point in the training image, form a matching pair, calculate the distance between the two feature descriptors of the matching pair, and calculate the Euclidean distance D between the two feature descriptors d1 and d2. The expression is: Among them, n is the dimension of the descriptor, d 1i and d 2i are the values ​​of d1 and d2 in the i-th dimension respectively; For each feature point in the image, use the nearest neighbor search algorithm to find the K feature points in the training image that are closest to its descriptor; Use the ratio test to screen matches. For each matching pair, calculate the ratio R of its distance to the nearest neighbor: If R is less than the set threshold, the match is considered valid. Where D(dquery,d1st,nearest) is the feature point d query Match point d with the nearest neighbor 1st,nearest The distance between them; D(dquery,d2st,nearest) is the distance between feature points d query Match point d with the next nearest neighbor 2st,nearest The distance between.

9. The multi-image array splicing method for large-scale cement prefabricated components according to claim 1, characterized in that: The weight of each pixel in the stitching area of ​​the two images to be stitched is proportional to the distance between the current processing point and the left border of the overlapping area.

Images