A method for image stitching and calibration of camera arrays with long focal length and large field of view

Abstract

The present invention relates to the field of computer vision technology, and specifically discloses a camera array image stitching calibration method for long focal length and large field of view, the calibration method comprising: calibrating a panoramic camera to obtain a calibrated panoramic camera; using multiple detail cameras and the calibrated panoramic camera to respectively shoot a target object to obtain multiple detail images and a panoramic image; projecting multiple detail images into the panoramic image to obtain a projected panoramic image; adjusting each detail image in the projected panoramic image to obtain an adjusted panoramic image. The camera array image stitching calibration method for long focal length and large field of view provided by the present invention does not rely on a calibration plate to calibrate images for large scenes, reduces errors caused by non-calibration plate calibration by iteration, reduces calibration costs, and ensures calibration accuracy.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a camera array image stitching calibration method for a long focal length and large field of view. Background Art

[0002] With the rapid development of computer and semiconductor technology, the requirements for security are getting higher and higher. When the resolution improvement of a single camera requires a high investment to achieve a small improvement, a camera array with spatially related fields of view has the opportunity to appear. After the camera array takes a group of images, the homography matrix required for image registration and projection to an image plane is obtained through the previously calculated camera internal and external parameters. After projecting these images to the same image plane, they are fused and color adjusted to finally obtain a high-resolution image. This technology has its place in many usage scenarios: such as virtual reality, remote sensing image processing, medical image analysis, imaging measurement, security and other fields all need such technology.

[0003] Currently, in order to obtain the spatial relationship between camera arrays, it is necessary to find out the internal and external parameters of the camera. The more mainstream method is to use Zhang Zhengyou's calibration method for calibration calculations. This algorithm uses the chessboard as the key to provide corner points and can accurately calculate the internal and external parameters. Because the focus needs to be fixed before calibration so that the chessboard appears clearly in the field of view, the chessboard required for long-focal-length lenses requires a huge amount of money and may not be able to build a scene that meets the requirements. Therefore, the current main method for stitching camera arrays with long focal lengths is to select landscapes with rich textures for calibration, and extract feature points from the texture of the overlapping area of ​​the cameras to calculate the relative position and posture between the cameras.

[0004] In order to ensure high-resolution imaging of all details at a large field of view, an array of spatially related cameras is used to obtain a field of view far exceeding that of a single camera and ultra-high resolution for wide-area monitoring perception. This is an application of image stitching technology that has emerged in recent years. In order to seamlessly stitch these images together, they need to be calibrated to obtain the spatial relationship between these cameras and imaging. The camera matrix composed of short-focal-length cameras requires a calibration plate with special textures drawn on it to help improve the accuracy of the calibration algorithm. At the same time, an open field is required to meet the requirements of calibration plates in different spatial relationships during calibration. When calibrating the stitched images of a long-focal-length and large-field-of-view camera array, it is very expensive to make a calibration plate with special textures that meets the long-focal-length calibration requirements, and it is difficult to prepare an environment that meets the standard calibration process. Summary of the invention

[0005] In view of the shortcomings of the prior art, the present invention provides an image stitching and calibration method for a camera array with a long focal length and a large field of view, so as to solve the problems in the prior art that it is very expensive to make a calibration plate and it is difficult to prepare an environment that meets the standard calibration process.

[0006] As a first aspect of the present invention, a method for calibrating a camera array image stitching with a long focal length and a large field of view is provided, and the method for calibrating a camera array image stitching with a long focal length and a large field of view comprises:

[0007] Step S1: calibrating the panoramic camera to obtain a calibrated panoramic camera;

[0008] Step S2: using a plurality of detail cameras and the calibrated panoramic camera to respectively photograph the target object to obtain a plurality of detail images and a panoramic image;

[0009] Step S3: projecting the plurality of detail images into the panoramic image to obtain a projected panoramic image;

[0010] Step S4: adjusting each detail image in the projected panoramic image to obtain an adjusted panoramic image.

[0011] Furthermore, the step of calibrating the panoramic camera to obtain a calibrated panoramic camera further includes:

[0012] Obtain the internal parameter M and distortion parameter d of the panoramic camera;

[0013] ,

[0014] in, represents the focal length, dX It represents the physical length of a pixel in the X direction on the photosensitive plate of the panoramic camera. d Y Indicates the physical length of a pixel in the Y direction on the photosensitive plate of the panoramic camera. Indicates the X coordinate of the center of the panoramic camera's photosensitive plate in the pixel coordinate system. Indicates the Y coordinate of the center of the panoramic camera's photosensitive plate in the pixel coordinate system. Indicates the angle between the horizontal and vertical sides of the panoramic camera's photosensitive plate;

[0015] The radial distortion of the panoramic camera is corrected using a radial distortion model formula, wherein the radial distortion model formula is as follows:

[0016] ,

[0017] The tangential distortion of the panoramic camera is corrected using a tangential distortion model formula, wherein the tangential distortion model formula is as follows:

[0018] ,

[0019] in, is the current pixel coordinate of the original image, is the current pixel coordinate of the image after distortion correction, are radial distortion parameters, are tangential distortion parameters, The center pixel coordinates of the original image and the current pixel coordinates The distance between.

[0020] Furthermore, the projecting of the plurality of detail images into the panoramic image to obtain a projected panoramic image further includes:

[0021] Using the SIFT feature point extraction and matching algorithm, all feature points in the panoramic image and all feature points in each detail image are extracted respectively, and using the RANSAC algorithm to calculate four pairs of key feature points between each detail image and the panoramic image;

[0022] According to the four pairs of key feature points between each detail image and the panoramic image, the homography matrix H between each detail image and the panoramic image is calculated. The calculation formula of the homography matrix H is as follows:

[0023] ,

[0024] in,( ) is the coordinate of the key feature point in the detail image, ( ) is the coordinate of the key feature point in the panoramic image, is an element in the homography matrix H;

[0025] According to the homography matrix H between each detail image and the panoramic image, each detail image is projected into the panoramic image, and the projected panoramic image is obtained after the multiple detail images are projected; and the original coordinates of the four corner points of the upper left, lower left, upper right, and lower right in each detail image before projection and the projection coordinates of the four corner points projected into the panoramic image are recorded, totaling 4 pairs of corner point coordinates, wherein the calculation formula of the projection coordinates of the four corner points projected into the panoramic image is as follows:

[0026] ,

[0027] in, is an element in the homography matrix H, are the original coordinates of the corner points in the detail image before projection, It is the projection coordinate of the corner point in the detail image in the panoramic image after projection.

[0028] Further, the adjusting each detail image in the projected panoramic image to obtain an adjusted panoramic image further includes:

[0029] Step S41: taking any unadjusted detail image in the projected panoramic image as a current detail image;

[0030] Step S42: searching for all feature point pairs in the overlapping area between the current detail image and its adjacent detail image in the projected panoramic image, and selecting matching feature point pairs that can be matched together from all feature point pairs in the overlapping area;

[0031] Step S43: The matching feature point pairs that can be matched together form an evaluation point set, and the initial reprojection error value of the evaluation point set is calculated. ;

[0032] Step S44: Use the gradient descent method to adjust the projection coordinates of each corner point in the current detail image, and calculate the current reprojection error value of the evaluation point set during the corner point projection coordinate adjustment process. , when the current reprojection error value of the evaluation point set Less than the termination iteration value , stop adjusting the projection coordinates of each corner point in the current detail image, and then use the projection coordinates of each corner point in the current detail image at this time as the optimal projection coordinates;

[0033] Step S45: Calculate the best homography matrix between the current detail image and the panoramic image based on the original coordinates of each corner point in the current detail image before projection and the best projection coordinates of each corner point in the current detail image after projection, and reproject each current detail image into the panoramic image based on the best homography matrix between each current detail image and the panoramic image to obtain the adjusted panoramic image.

[0034] Furthermore, the initial reprojection error value of the evaluation point set is The calculation formula is as follows:

[0035] ,

[0036] n is the number of matching feature point pairs in the evaluation point set, ( ) is the number of pixels in the current detail image after projection. i The coordinates of the matching feature points, ( ) is the neighboring detail image of the current detail image after projection. i The coordinates of the matched feature points.

[0037] Furthermore, the step S44 further includes:

[0038] Step S441: Set a termination iteration value , and taking any corner point in the current detail image whose projection coordinates have not been adjusted as the current corner point;

[0039] Step S442: Calculate the current reprojection error value of the evaluation point set after the initial projection coordinates of the current corner point in the current detail image are moved one pixel in each of the four directions of upward, downward, left and right. ;

[0040] If the current reprojection error values ​​in the four directions are all less than the termination iteration value , then stop moving the projection coordinates of the current corner point in the current detail image, and then use the initial projection coordinates of the current corner point in the current detail image as the optimal projection coordinates of the current corner point in the current detail image;

[0041] On the contrary, if there is a current reprojection error value Greater than the termination iteration value direction, select from the four directions: up, down, left, and right The direction with the largest value is moved to the direction where the initial projection coordinates of the current corner point in the current detail image are The moving direction of the maximum value of is moved by one pixel to obtain the moved projection coordinates of the current corner point in the current detail image, and then return to execute the step S442 to calculate the current reprojection error value of the evaluation point set after the moved projection coordinates of the current corner point in the current detail image are moved one pixel in each of the four directions of upward, downward, left, and right. , until the current reprojection error values ​​in the four directions are all less than the termination iteration value .

[0042] The image stitching calibration method for a camera array with a long focal length and a large field of view provided by the present invention has the following beneficial effects: it does not rely on a calibration plate to calibrate images of a large scene, reduces the error caused by calibration without a calibration plate through iteration, reduces the calibration cost, and ensures the calibration accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. Together with the following specific embodiments, they are used to explain the present invention, but do not constitute a limitation of the present invention.

[0044] Figure 1 A flow chart of the camera array image stitching calibration method for long focal length and large field of view provided by the present invention.

[0045] Figure 2 A flowchart of a specific implementation of the method for image stitching and calibration of a camera array with a long focal length and a large field of view provided by the present invention.

[0046] Figure 3 A schematic diagram of the spatial relationship between the field of view of the detail camera array and the panoramic camera provided by the present invention.

[0047] Figure 4 A schematic diagram of the distortion correction effect of the panoramic camera provided by the present invention.

[0048] Figure 5 A schematic diagram of a set of evaluation points provided by the present invention.

[0049] Figure 6 This is a schematic diagram of the projection of four corner points in the detail image provided by the present invention.

[0050] Figure 7 A schematic diagram of adjusting the projection coordinates of corner points in a detail image provided by the present invention. DETAILED DESCRIPTION

[0051] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0052] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiment of the present invention. Obviously, the described embodiment is only a part of the embodiment of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work should fall within the scope of protection of the present invention.

[0053] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so as to describe the embodiments of the present invention described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0054] In this embodiment, a method for calibrating camera array images with long focal length and large field of view is provided. Figure 1As shown, the camera array image stitching calibration method for long focal length and large field of view includes:

[0055] Step S1: calibrating a non-telephoto panoramic camera to obtain a calibrated panoramic camera;

[0056] Preferably, the step of calibrating the panoramic camera to obtain a calibrated panoramic camera further includes:

[0057] Use Zhang Zhengyou calibration method or other calibration methods to obtain the internal parameter M and distortion parameter d of the panoramic camera;

[0058] ,

[0059] in, represents the focal length, dX It represents the physical length of a pixel in the X direction on the photosensitive plate of the panoramic camera. d Y Indicates the physical length of a pixel in the Y direction on the photosensitive plate of the panoramic camera. Indicates the X coordinate of the center of the panoramic camera's photosensitive plate in the pixel coordinate system. Indicates the Y coordinate of the center of the panoramic camera's photosensitive plate in the pixel coordinate system. Indicates the angle between the horizontal and vertical edges of the panoramic camera's photosensitive plate (if 90° means no error);

[0060] The radial distortion of the panoramic camera is corrected using a radial distortion model formula, wherein the radial distortion model formula is as follows:

[0061] ,

[0062] The tangential distortion of the panoramic camera is corrected using a tangential distortion model formula, wherein the tangential distortion model formula is as follows:

[0063] ,

[0064] in, is the current pixel coordinate of the original image, is the current pixel coordinate of the image after distortion correction, are radial distortion parameters, are tangential distortion parameters, The center pixel coordinates of the original image and the current pixel coordinates The distance between.

[0065] It should be noted that after the obtained intrinsic parameter M and distortion parameter d are used to calibrate the panoramic camera, the following can be obtained: Figure 4 The effect shown.

[0066] Step S2: using a plurality of detail cameras and the calibrated panoramic camera to respectively photograph the target object to obtain a plurality of detail images and a panoramic image;

[0067] Specifically, a position is found where a panoramic image with rich textures and texture filling can be captured, and the calibrated panoramic camera is used to start capturing a panoramic image of the target object at the target imaging distance and multiple detail cameras are used to capture multiple detail images of the target object.

[0068] It should be noted that if Figure 3 As shown, a camera array including multiple detail cameras is used to photograph the target object, wherein the fields of view of the detail cameras can be arranged and stacked arbitrarily with a certain overlap range. In addition, a calibrated panoramic camera is used to photograph a panoramic image including all details of the target object.

[0069] Step S3: projecting the plurality of detail images into the panoramic image to obtain a projected panoramic image;

[0070] Preferably, projecting the plurality of detail images into the panoramic image to obtain a projected panoramic image further comprises:

[0071] All feature points in the panoramic image and all feature points in each detail image are extracted using a SIFT feature point extraction and matching algorithm, and four pairs of key feature points between each detail image and the panoramic image are calculated using a RANSAC algorithm; wherein the reprojection error value of the four pairs of key feature points is the smallest;

[0072] According to the four pairs of key feature points between each detail image and the panoramic image, the homography matrix H between each detail image and the panoramic image is calculated. The calculation formula of the homography matrix H is as follows:

[0073] ,

[0074] in,( ) is the coordinate of the key feature point in the detail image, ( ) is the coordinate of the key feature points in the panoramic image, is an element in the homography matrix H;

[0075] According to the homography matrix H between each detail image and the panoramic image, each detail image is projected into the panoramic image, and the projected panoramic image is obtained after the multiple detail images are projected; and the original coordinates of the four corner points of the upper left, lower left, upper right, and lower right in each detail image before projection and the projection coordinates of the four corner points projected into the panoramic image are recorded, totaling 4 pairs of corner point coordinates, wherein the calculation formula of the projection coordinates of the four corner points projected into the panoramic image is as follows:

[0076] ,

[0077] in, is an element in the homography matrix H, are the original coordinates of the corner points in the detail image before projection, It is the projection coordinate of the corner point in the detail image in the panoramic image after projection.

[0078] like Figure 6 As shown, the coordinates are the original coordinates of the four corner points in the detail image before projection, which become .

[0079] Step S4: adjusting each detail image in the projected panoramic image to obtain an adjusted panoramic image.

[0080] Preferably, if Figure 2 As shown, the step of adjusting each detail image in the projected panoramic image to obtain an adjusted panoramic image further includes:

[0081] Step S41: taking any unadjusted detail image in the projected panoramic image as the current detail image, and finding all adjacent detail images that have overlapping areas with the current detail image;

[0082] Step S42: searching for all feature point pairs in the overlapping area between the current detail image and its adjacent detail image in the projected panoramic image, and selecting matching feature point pairs that can be matched together from all feature point pairs in the overlapping area;

[0083] Specifically, Figure 5 As shown, detail image 1 is the current detail image, detail image 2 is the adjacent detail image of the current detail image, the area with horizontal stripes in detail image 1 is the overlapping area of ​​detail image 1 and detail image 2, and the matching feature points of detail image 1 are , the matching feature points of detail image 2 are , matching feature points Matching feature points Constitute matching feature point pairs, the red dotted arrows are matching feature points Located at the position of detail image 1, the red solid line is the distance between two matching feature points.

[0084] Step S43: The matching feature point pairs that can be matched together form an evaluation point set, and the initial reprojection error value of the evaluation point set is calculated. ;

[0085] Specifically, the initial reprojection error value of the evaluation point set is The calculation formula is as follows:

[0086] ,

[0087] n is the number of matching feature point pairs in the evaluation point set, ( ) is the number of pixels in the current detail image after projection. i The coordinates of the matching feature points, ( ) is the neighboring detail image of the current detail image after projection. i The coordinates of the matched feature points.

[0088] It should be noted that if Figure 5 As shown, detail image 1 is used as an estimated value and detail image 2 is used as an actual value to calculate the initial reprojection error value of the evaluation point set. , to evaluate the stitching quality of the overlapping area of ​​detail image 1 and detail image 2.

[0089] Step S44: Use the gradient descent method to adjust the projection coordinates of each corner point in the current detail image, and calculate the current reprojection error value of the evaluation point set during the corner point projection coordinate adjustment process. (The calculation formula is the same as the above initial reprojection error value ), when the current reprojection error value of the evaluation point set Less than the termination iteration value , stop adjusting the projection coordinates of each corner point in the current detail image, and then use the projection coordinates of each corner point in the current detail image at this time as the optimal projection coordinates;

[0090] It should be noted that if Figure 7 As shown, the projection coordinates are shown Move one pixel to the right and arrive at The subsequent image changes, because the projection coordinates of the upper right corner point have changed, the homography matrix will also change accordingly, the projection coordinates of all feature points in the detail image will be updated, the matching feature point pairs in the evaluation point set will also be updated, and the current reprojection error value of the evaluation point set after the matching feature point pairs are updated is calculated. , to evaluate this move.

[0091] Specifically, the step S44 further includes:

[0092] Step S441: Set a termination iteration value , and taking any corner point in the current detail image whose projection coordinates have not been adjusted as the current corner point;

[0093] It should be noted that, when the projection coordinates of the four corner points in the current detail image have not been adjusted, one corner point is selected clockwise from the four corner points in the current detail image as the current corner point.

[0094] Step S442: Calculate the current reprojection error value of the evaluation point set after the initial projection coordinates of the current corner point in the current detail image are moved one pixel in each of the four directions of upward, downward, left and right. ;

[0095] If the current reprojection error values ​​in the four directions are all less than the termination iteration value then stop moving the projection coordinates of the current corner point in the current detail image, and then use the initial projection coordinates of the current corner point in the current detail image as the optimal projection coordinates of the current corner point in the current detail image;

[0096] On the contrary, if there is a current reprojection error value Greater than the termination iteration value direction, select from the four directions: up, down, left, and right The direction with the largest value is moved to the direction where the initial projection coordinates of the current corner point in the current detail image are The moving direction of the maximum value of is moved by one pixel to obtain the moved projection coordinates of the current corner point in the current detail image, and then return to execute the step S442 to calculate the current reprojection error value of the evaluation point set after the moved projection coordinates of the current corner point in the current detail image are moved one pixel in each of the four directions of upward, downward, left, and right. , until the current reprojection error values ​​in the four directions are all less than the termination iteration value .

[0097] Step S45: Calculate the best homography matrix between the current detail image and the panoramic image based on the original coordinates of each corner point in the current detail image before projection and the best projection coordinates of each corner point in the current detail image after projection, and reproject each current detail image into the panoramic image based on the best homography matrix between each current detail image and the panoramic image to obtain the adjusted panoramic image.

[0098] The image stitching and calibration method for a camera array with a long focal length and a large field of view provided in an embodiment of the present invention is applied to a finite camera array, and the result after a rough projection of the image to be matched is continuously iteratively adjusted by using the idea of ​​iterative optimization to obtain a high-precision alignment model.

[0099] It is to be understood that the above embodiments are merely exemplary embodiments used to illustrate the principles of the present invention, but the present invention is not limited thereto. For those of ordinary skill in the art, various modifications and improvements can be made without departing from the spirit and essence of the present invention, and these modifications and improvements are also considered to be within the scope of protection of the present invention.

Claims

1. A method for image stitching and calibration of camera array with long focal length and large field of view, characterized in that: The camera array image stitching calibration method for long focal length and large field of view includes: Step S1: calibrating the panoramic camera to obtain a calibrated panoramic camera; Step S2: using a plurality of detail cameras and the calibrated panoramic camera to respectively photograph the target object to obtain a plurality of detail images and a panoramic image; Step S3: projecting the plurality of detail images into the panoramic image to obtain a projected panoramic image; Step S4: adjusting each detail image in the projected panoramic image to obtain an adjusted panoramic image; The step of adjusting each detail image in the projected panoramic image to obtain an adjusted panoramic image further includes: Step S41: taking any unadjusted detail image in the projected panoramic image as a current detail image; Step S42: searching for all feature point pairs in the overlapping area between the current detail image and its adjacent detail image in the projected panoramic image, and selecting matching feature point pairs that can be matched together from all feature point pairs in the overlapping area; Step S43: The matching feature point pairs that can be matched together form an evaluation point set, and the initial reprojection error value of the evaluation point set is calculated. ; Step S44: Use the gradient descent method to adjust the projection coordinates of each corner point in the current detail image, and calculate the current reprojection error value of the evaluation point set during the corner point projection coordinate adjustment process. , when the current reprojection error value of the evaluation point set Less than the termination iteration value , stop adjusting the projection coordinates of each corner point in the current detail image, and then use the projection coordinates of each corner point in the current detail image at this time as the optimal projection coordinates; Step S45: calculating the best homography matrix between the current detail image and the panoramic image according to the original coordinates of each corner point in the current detail image before projection and the best projection coordinates of each corner point in the current detail image after projection, and re-projecting each current detail image into the panoramic image according to the best homography matrix between each current detail image and the panoramic image to obtain the adjusted panoramic image; Among them, the initial reprojection error value of the evaluation point set is The calculation formula is as follows: ， n is the number of matching feature point pairs in the evaluation point set, is the number of pixels in the current detail image after projection. i The coordinates of the matching feature points, ( ) is the neighboring detail image of the current detail image after projection. i The coordinates of the matched feature points.

2. The method for image stitching and calibration of a camera array with a long focal length and a large field of view according to claim 1, characterized in that: The method of calibrating the panoramic camera to obtain a calibrated panoramic camera further includes: Obtain the internal parameter M and distortion parameter d of the panoramic camera; ， in, represents the focal length, dX It represents the physical length of a pixel in the X direction on the photosensitive plate of the panoramic camera. d Y Indicates the physical length of a pixel in the Y direction on the photosensitive plate of the panoramic camera. Indicates the X coordinate of the center of the panoramic camera's photosensitive plate in the pixel coordinate system. Indicates the Y coordinate of the center of the panoramic camera's photosensitive plate in the pixel coordinate system. Indicates the angle between the horizontal and vertical sides of the panoramic camera's photosensitive plate; The radial distortion of the panoramic camera is corrected using a radial distortion model formula, wherein the radial distortion model formula is as follows: ， The tangential distortion of the panoramic camera is corrected using a tangential distortion model formula, wherein the tangential distortion model formula is as follows: ， in, is the current pixel coordinate of the original image, is the current pixel coordinate of the image after distortion correction, are radial distortion parameters, are tangential distortion parameters, The center pixel coordinates of the original image and the current pixel coordinates The distance between.

3. The method for image stitching and calibration of camera array with long focal length and large field of view according to claim 1, characterized in that: The step of projecting the plurality of detail images into the panoramic image to obtain a projected panoramic image further includes: Using the SIFT feature point extraction and matching algorithm, all feature points in the panoramic image and all feature points in each detail image are extracted respectively, and using the RANSAC algorithm to calculate four pairs of key feature points between each detail image and the panoramic image; According to the four pairs of key feature points between each detail image and the panoramic image, the homography matrix H between each detail image and the panoramic image is calculated. The calculation formula of the homography matrix H is as follows: , in,( ) is the coordinate of the key feature point in the detail image, ( ) is the coordinate of the key feature points in the panoramic image, is an element in the homography matrix H; According to the homography matrix H between each detail image and the panoramic image, each detail image is projected into the panoramic image, and the projected panoramic image is obtained after the multiple detail images are projected; and the original coordinates of the four corner points of the upper left, lower left, upper right, and lower right in each detail image before projection and the projection coordinates of the four corner points projected into the panoramic image are recorded, totaling 4 pairs of corner point coordinates, wherein the calculation formula of the projection coordinates of the four corner points projected into the panoramic image is as follows: ， in, is an element in the homography matrix H, are the original coordinates of the corner points in the detail image before projection, It is the projection coordinate of the corner point in the detail image in the panoramic image after projection.

4. The method for image stitching and calibration of camera array with long focal length and large field of view according to claim 1, characterized in that: The step S44 further includes: Step S441: Set a termination iteration value , and taking any corner point in the current detail image whose projection coordinates have not been adjusted as the current corner point; Step S442: Calculate the current reprojection error value of the evaluation point set after the initial projection coordinates of the current corner point in the current detail image are moved one pixel in each of the four directions of upward, downward, left and right. ; If the current reprojection error values ​​in the four directions are all less than the termination iteration value , then stop moving the projection coordinates of the current corner point in the current detail image, and then use the initial projection coordinates of the current corner point in the current detail image as the optimal projection coordinates of the current corner point in the current detail image; On the contrary, if there is a current reprojection error value Greater than the termination iteration value direction, select from the four directions: up, down, left, and right The direction with the largest value is moved to the direction where the initial projection coordinates of the current corner point in the current detail image are The moving direction of the maximum value of is moved by one pixel to obtain the moved projection coordinates of the current corner point in the current detail image, and then return to execute the step S442 to calculate the current reprojection error value of the evaluation point set after the moved projection coordinates of the current corner point in the current detail image are moved one pixel in each of the four directions of upward, downward, left, and right. , until the current reprojection error values ​​in the four directions are all less than the termination iteration value .

Images