Image correction and shooting path-based stitching method

Abstract

The present invention discloses a splicing method based on image correction and shooting path, including: First, calculate the shooting step length according to the pre-determined shooting angle, shooting distance, cropping ratio, and camera parameters, set the shooting path, and number the photos taken on-site; then calculate the correction coefficient, and successively correct and then crop each photo according to the cropping ratio; finally, splice the cropped photos according to the photo numbers to obtain a complete image. The splicing method based on image correction and shooting path of the present invention can be used for image splicing without feature points, involves less computational amount and method steps, is easy to implement, can reduce the consumption of computing resources, improve the image splicing efficiency, and ensure accurate image information in subsequent analysis and processing processes.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and particularly to a stitching method based on image correction and shooting path. Background Art

[0002] At present, both at home and abroad, attempts have been made to use intelligent robots to replace manual labor for bridge maintenance and inspection. Obtaining high-quality bridge surface images through computer vision technology for disease identification is a current research hotspot. However, for large-scale bridges, a single photo cannot cover the entire component or cannot fully cover a disease. It is necessary to stitch the detection images of the component to comprehensively restore the true situation of the component surface and diseases, so as to further complete the identification and scale evaluation of component diseases.

[0003] Image stitching technology needs to comprehensively consider multiple factors, including geometry, color, illumination, etc., to obtain high-quality stitching results. Currently, most stitching algorithms are based on different data sources such as feature point matching, optical flow estimation, depth information, etc. to achieve image stitching. These methods have problems such as inaccurate image alignment, discontinuous or uneven stitching edge regions, and image deformation caused by the perspective distortion of the camera, and cannot achieve efficient stitching of bridge surface disease images. Summary of the Invention

[0004] Aiming at the deficiencies of the existing technology, the present invention proposes a stitching method based on image correction and shooting path, which can significantly improve the quality of image stitching and present a more seamless and realistic image stitching effect. The specific technical solutions are as follows:

[0005] A stitching method based on image correction and shooting path is provided. In the first implementable manner, it includes:

[0006] Calculating the shooting step according to the pre-determined shooting angle, shooting distance, cropping ratio, and camera parameters;

[0007] Setting the shooting path according to the actual situation of the object to be photographed, and numbering the photos taken on site in combination with the shooting step;

[0008] Calculating the correction coefficient according to the shooting angle and camera parameters, and correcting the photos one by one using the photo numbers;

[0009] Cropping the corrected photos one by one according to the cropping ratio using the photo numbers;

[0010] Stitching the cropped photos according to the photo numbers to obtain a complete image.

[0011] In the first implementation manner, in the second implementation manner, the correction coefficient includes a Y-axis correction coefficient that matches the Y-axis direction. The specific calculation formula for the Y-axis correction coefficient is as follows:

[0012]

[0013]

[0014] Wherein, is the Y-axis magnification factor, is the Y-axis reduction factor, and θ y is the beam viewing angle in the Y-axis direction, and α y is the shooting angle in the Y-axis direction.

[0015] In the first implementation manner, in the third implementation manner, the correction coefficient includes an X-axis correction coefficient that matches the X-axis direction. The specific calculation formula for the X-axis correction coefficient is as follows:

[0016]

[0017]

[0018] Wherein, is the X-axis magnification factor, is the X-axis reduction factor, and θ x is the beam viewing angle in the X-axis direction, and α x is the shooting angle in the X-axis direction.

[0019] In the first implementation manner, in the fourth implementation manner, according to the position sorting of pixel units in the horizontal and vertical directions, the camera field of view FOV x and FOV y are equally divided to obtain the beam viewing angles θ y and θ x . The specific calculation formulas for and corresponding to the (m, n)-th pixel unit are as follows:

[0020]

[0021] The values of m and n are as follows:

[0022] m = -M, ···, -1, 1, ···, M

[0023] n = -N, ···, -1, 1, ···, N

[0024] Wherein, the camera field of view FOV x and FOV y can be determined by the camera lens parameters, and M and N can be determined by the camera pixel parameters.

[0025] In the fifth implementation manner in combination with the fourth implementation manner, correcting the vertex coordinates of the pixel unit by the correction coefficient includes:

[0026] Determine the corresponding X-axis and Y-axis correction coefficients according to the shooting angles in the X-axis and Y-axis directions and the positions of the pixel units, and multiply the vertex coordinates of each pixel unit by the corresponding X-axis and Y-axis correction coefficients to obtain the corrected vertex coordinates.

[0027] In the sixth implementation manner in combination with the first implementation manner, determining the pixel size of the corrected image includes determining the pixel size of the corrected image by using the following calculation formula:

[0028]

[0029]

[0030] where L is the shooting distance, l is the pixel length of the corrected image, and w is the pixel width of the corrected image.

[0031] In the seventh implementation manner in combination with the first implementation manner, determining the maximum number of columns and the maximum number of rows of the pixels after the corrected image is cropped includes calculating the maximum number of columns and the maximum number of rows of the pixels after the corrected image is cropped by using the floor function:

[0032]

[0033]

[0034] where INT is the floor function, δ is the cropping ratio, J is the maximum number of columns of the pixels after the corrected image is cropped, and K is the maximum number of rows of the pixels after the corrected image is cropped.

[0035] In the eighth implementation manner in combination with the first implementation manner, according to the pixel size of the corrected image and the maximum number of columns and the maximum number of rows of the pixels after cropping, including determining the matching moving steps in the X-axis and Y-axis directions before correction by using the following calculation formula:

[0036] or

[0037] or

[0038] where λ x 、λ y are the matching moving steps in the X-axis and Y-axis directions before correction respectively.

[0039] Beneficial effects: Based on the image correction and shooting path stitching method, the present invention can be used for image stitching without feature points, involves less computational complexity and fewer method steps, is easy to implement, can reduce computational resource consumption, improve image stitching efficiency, and ensure accurate image information is obtained during subsequent analysis and processing. Description of the Drawings

[0040] In order to more clearly illustrate the specific embodiments of the present invention, the following will briefly introduce the drawings required for the specific embodiments. In all the drawings, the components or parts are not necessarily drawn to actual scale.

[0041] Figure 1 It is a flowchart of the stitching method based on image correction and shooting path provided by an embodiment of the present invention;

[0042] Figure 2 It is a schematic diagram of the shooting numbers of multiple consecutive frame images;

[0043] Figure 3 It is a schematic diagram for calculating the non-orthogonal image correction coefficient in the vertical direction;

[0044] Figure 4 It is a schematic diagram for calculating the non-orthogonal image correction coefficient in the horizontal direction;

[0045] Figure 5 It is for the tangent curve at Schematic diagram of linearization processing;

[0046] Figure 6 It is a schematic diagram for calculating the beam viewing angle in the vertical direction;

[0047] Figure 7 It is a schematic diagram for calculating the beam viewing angle in the horizontal direction;

[0048] Figure 8 It is a schematic diagram of non-orthogonal image correction;

[0049] Figure 9 It is a schematic diagram for calculating the moving step size;

[0050] Figure 10 It is a schematic diagram of the stitching order;

[0051] Figure 11 It is a schematic diagram of the stitching result. Detailed Embodiments

[0052] The following will describe in detail the embodiments of the technical solutions of the present invention with reference to the drawings. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention, so they are only examples and cannot be used to limit the protection scope of the present invention.

[0053] Specifically, the present invention first calculates the shooting step length according to the pre-determined shooting angle, shooting distance, cropping ratio, and camera parameters, sets the shooting path, and numbers the photos taken on-site; then calculates the correction coefficient, and corrects and then crops each photo according to the cropping ratio; finally, stitches the cropped photos according to the photo numbers to obtain a complete image.

[0054] As Figure 1 shown in the flowchart of the stitching method based on image correction and shooting path, the stitching method includes:

[0055] Step 1: Calculate the shooting step length according to the pre-determined shooting angle, shooting distance, cropping ratio, and camera parameters;

[0056] Step 2: Set the shooting path according to the actual situation of the object to be photographed, and number the photos taken on-site in the order of shooting in combination with the shooting step length;

[0057] Step 3: Calculate the correction coefficient according to the shooting angle and camera parameters, and correct each photo one by one using the photo number;

[0058] Step 4: Crop the corrected photos one by one according to the cropping ratio using the photo number;

[0059] Step 5: Stitch the cropped photos according to the photo number to obtain a complete image.

[0060] In this embodiment, according to the known shooting angle α y and α x , shooting distance L, cropping ratio δ, camera parameters M, N, FOV x , FOV y , using the formula

[0061] or

[0062] or calculate the matching moving step lengths λ x , λ y in the X and Y axis directions before correction respectively. In the formula, the detailed derivation processes of the calculation formulas of parameters such as λ x , λ y , l, w, K, J will be discussed later.

[0063] In this embodiment, further, as Figure 2 shown, set the shooting path according to the actual situation of the object to be photographed, and number the photos taken on-site in the order of shooting in combination with the shooting step length.

[0064] In this embodiment, further, according to the known shooting angle α y and α x 、camera parameters M, N, FOV x 、FOV y calculate the correction coefficient and correct the photos one by one using the photo numbers.

[0065] As Figure 3 shown, denote the beam viewing angle ∠BOC as θ y , the shooting angle ∠AOO' as α y , and the shooting distance OA as L.

[0066] The calculation formula for the imaging height BC before image correction is as follows:

[0067]

[0068]

[0069] The calculation formula for the imaging height B'C' after image correction is as follows:

[0070]

[0071] Denote the magnification coefficient of O'B' relative to O'B as Denote the reduction coefficient of O'C' relative to O'C as Then the specific calculation formula for the Y-axis correction coefficient is:

[0072]

[0073]

[0074] Among them, is the Y-axis magnification coefficient, is the Y-axis reduction coefficient, θ y is the beam viewing angle in the Y-axis direction, α y is the shooting angle in the Y-axis direction.

[0075] As Figure 4 shown, denote the camera viewing angle ∠BOC as θ x , the shooting angle ∠AOO' as α x , and the shooting distance OA as L. Similarly, the specific calculation formula for the X-axis correction coefficient can be obtained as:

[0076]

[0077]

[0078] Among them, is the X-axis magnification coefficient, is the X-axis reduction coefficient, θ x is the beam viewing angle in the X-axis direction, α x is the shooting angle in the X-axis direction.

[0079] Regardless of whether it is orthographic shooting or non-orthographic shooting, the beam viewing angle θ y is invariant. Therefore, the beam viewing angle θ can be calculated under orthographic shooting conditions y . The field of view (FOV) of a conventional camera generally does not exceed 90°. As Figure 5 shown, the function image of the tangent function within is approximately a straight line. To simplify the algorithm and improve the calculation efficiency, a method of equally dividing the camera field of view (FOV) according to the positions of pixel units in the horizontal and vertical directions is adopted to calculate the beam viewing angles in the horizontal and vertical directions. As Figure 6 shown, the calculation formula corresponding to the pixel unit in the nth row is:

[0080]

[0081] Similarly, as Figure 7 shown, the calculation formula corresponding to the pixel unit in the mth column is:

[0082]

[0083] Among them, the camera field of view (FOV) x , FOV y can be determined by the camera lens parameters, and M and N can be determined by the camera pixel parameters. Thus, the Y-axis correction coefficients and X-axis correction coefficients

[0084] of each pixel unit can be calculated respectively. Figure 8 shown, assuming that the pixel coordinates in the first to fourth quadrants before correction are (x1, y1), (x2, y2), (x3, y3), (x4, y4), then the pixel coordinates in each quadrant after correction are

[0085] In this embodiment, further, according to the cropping ratio, the corrected photos are cropped one by one using the photo numbers.

[0086] As Figure 3 , Figure 4 , Figure 9 shown, the length and width of the corrected orthographic image are respectively:

[0087]

[0088] The lengths and widths of the pixels of the orthographic image after correction are respectively:

[0089]

[0090] Wherein, a is the length of the orthographic image after correction, b is the width of the orthographic image after correction; l is the pixel length of the orthographic image after correction, and w is the pixel width of the orthographic image after correction.

[0091] The lengths and widths of the orthographic image after cropping are respectively:

[0092]

[0093] Wherein, c is the length of the orthographic image after cropping, d is the width of the orthographic image after cropping, and δ is the cropping ratio;

[0094] Then the maximum number of columns and the maximum number of rows of the pixels of the orthographic image after cropping are respectively:

[0095]

[0096]

[0097] Wherein, INT is the floor function, J is the maximum number of columns of the pixels of the corrected image after cropping, and K is the maximum number of rows of the pixels of the corrected image after cropping.

[0098] From this, the length l, width w, maximum number of columns J, and maximum number of rows K of the pixels of the corrected image after cropping can be obtained. Select any row of pixels (the number of pixels in this row is J), and divide the length l of each pixel in this row in the orthographic state by its corresponding X-axis correction coefficient or And sum them up to obtain the actual length of the orthographic cropped image before correction, that is, the moving step λ matching the X-axis direction x . Similarly, select any column of pixels (the number of pixels in this column is K), and divide the width w of each pixel in this column in the orthographic state by its corresponding Y-axis correction coefficient or And sum them up to obtain the actual width of the orthographic cropped image before correction, that is, the moving step λ matching the Y-axis direction y . The specific calculation formulas are respectively:

[0099] or

[0100] or

[0101] In this embodiment, further, as Figure 10 and Figure 11As shown, the cropped photos are spliced according to the photo numbers to obtain a complete image.

[0102] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention, and they should all be covered by the scope of the claims and the specification of the present invention.

Claims

1. A stitching method based on image correction and shooting path, characterized in that: include: Calculate the shooting step length based on the pre-determined shooting angle, shooting distance, cropping ratio, and camera parameters; Set the shooting path according to the actual situation of the object being photographed, and number the photos taken on site in combination with the shooting step length; Calculate the correction coefficient according to the shooting angle and the beam angle, and correct the photos one by one using the photo numbers; The corrected photos are cropped one by one using the photo numbers according to the cropping ratio; The cropped photos are stitched together according to the photo numbers to obtain a complete image; The correction coefficient includes a Y-axis correction coefficient that matches the Y-axis direction. The specific calculation formula of the Y-axis correction coefficient is: ； ； in, is the Y-axis magnification factor, is the Y-axis reduction coefficient, is the beam viewing angle in the Y-axis direction, is the shooting angle in the Y-axis direction.

2. The stitching method based on image correction and shooting path according to claim 1, characterized in that: The correction coefficient includes an X-axis correction coefficient that matches the X-axis direction. The specific calculation formula of the X-axis correction coefficient is: ； ； in, is the X-axis magnification factor, is the X-axis reduction factor, is the beam viewing angle in the X-axis direction, is the shooting angle in the X-axis direction.

3. The stitching method based on image correction and shooting path according to claim 1, characterized in that: The camera field of view FOV is sorted according to the horizontal and vertical positions of the pixel units. x FOV y The beam viewing angle is obtained by dividing the beam into equal parts. and , the pixel unit (m, n) corresponds to and The specific calculation formula is: , The values ​​of m and n are as follows: Among them, the camera field of view FOV x FOV y It can be determined by the camera lens parameters, and M and N can be determined by the camera pixel parameters.

4. The stitching method based on image correction and shooting path according to claim 3, characterized in that: Correcting the vertex coordinates of the pixel unit by using the correction coefficient includes: The corresponding X and Y axis correction coefficients are determined according to the shooting angles in the X and Y axis directions and the position of each pixel unit, and the vertex coordinates of each pixel unit are multiplied by the corresponding X and Y axis correction coefficients to obtain the corrected vertex coordinates.

5. The stitching method based on image correction and shooting path according to claim 3, characterized in that: Determining the pixel size of the rectified image includes determining the pixel size of the rectified image using the following calculation formula: ； ； in, is the shooting distance, is the pixel length of the rectified image, is the pixel width of the rectified image.

6. The stitching method based on image correction and shooting path according to claim 3, characterized in that: Determine the maximum number of columns and rows of pixels after cropping the rectified image, including using the floor function to calculate the maximum number of columns and rows of pixels after cropping the rectified image: in, is the floor function, is the crop ratio, is the maximum number of columns of pixels after the rectified image is cropped, The maximum number of rows of pixels after cropping the rectified image.

Images