A method for locating and converting panoramic bird's-eye view images of vehicle-mounted surround view cameras based on drones

Abstract

The present invention relates to a method for calibrating and converting a panoramic bird's-eye view image of a vehicle-mounted surround-view camera based on a drone, comprising the following steps: S101: calibrating the vehicle-mounted camera's internal parameters to obtain an internal parameter matrix and a distortion vector; S102: placing and fixing multiple checkerboard calibration plates around the vehicle; S103: collecting images from the vehicle-mounted camera and images from the drone camera; S104: preprocessing the vehicle-mounted camera and drone camera images; S105: iteratively calculating a homography matrix based on the vehicle-mounted camera and drone images, wherein the drone image is a reference bird's-eye view, using a corner detection algorithm and a RANSAC algorithm; S106: converting the vehicle-mounted camera image into a bird's-eye view and performing panoramic stitching based on the homography matrix, camera internal parameters, and distortion vector. The calibration method provided by the present invention can be flexibly calibrated using only a small calibration plate and has no position requirements, thereby significantly improving calibration efficiency and achieving high calibration accuracy. In addition, the conversion and stitching method provided by the present invention can achieve good results and real-time performance, and is convenient for adjusting the results according to needs.

Description

Technical Field

[0001] The present invention relates to the fields of camera calibration and image processing, and in particular to a method for calibrating and converting a panoramic bird's-eye view image of a vehicle-mounted surround-view camera. Background Art

[0002] With the development of autonomous driving technology, the importance of perception systems, as a crucial component, continues to grow. Vision-driven perception technologies provide the necessary foundation and support for all levels of autonomous driving. For example, on-board surround view systems, which use onboard cameras to capture panoramic images and perform further detection and decision-making, are currently widely used in areas such as automated parking and lane keeping.

[0003] To better capture the vehicle's surroundings and minimize blind spots, on-board surround view systems typically consist of multiple fisheye cameras mounted on the front, back, left, and right sides of the vehicle. By transforming the raw images from the on-board cameras, they create a final bird's-eye view image and stitch it together to create a complete panoramic bird's-eye view.

[0004] Because fisheye cameras have a large field of view, the resulting raw images are spherical, so they must first be dedistorted. Because camera sensors inevitably have errors during manufacturing and installation, parameters such as the optical axis and focal length deviate from standard values. Therefore, the camera's intrinsic parameters must be obtained through calibration. The distortion model of a fisheye camera is described by a function composed of polynomials, determined through nonlinear optimization during the calibration process. Therefore, the purpose of fisheye camera calibration is to obtain the camera's intrinsic parameters K and the distortion coefficient D. These two parameters are used to complete image dedistortion.

[0005] To convert the dedistorted camera images into a bird's-eye view, it's also necessary to determine the camera's extrinsic parameters—that is, a description of the camera's position, which can be represented by a rotation matrix, translation matrix, or other form. Based on the intrinsic and extrinsic parameters of each camera, a converted bird's-eye view is obtained. This is then stitched together based on the camera's perspective and position to create a panoramic bird's-eye view. Stitching methods include pre-dividing non-overlapping regions and directly stitching the camera images together, which results in unnatural transitions at the seams. Alternatively, fusion stitching algorithms can be used to achieve smooth transitions at the seams, resulting in a better visual effect.

[0006] For the calibration of camera extrinsic parameters, existing methods usually require accurately determining the specific coordinates of the corner points of the calibration plate in the real-world coordinate system, and calculating the correspondence based on the pixel coordinates of the corner points of the calibration plate in the on-board camera image to obtain the projection transformation matrix H for transforming into a bird's-eye view.

[0007] However, these current calibration methods have many shortcomings. For example, the accuracy requirements for the position of the calibration plate are extremely high. When using the calibration plate for calibration, the position is determined by manually measuring the actual distance of the calibration plate or its corner points, which has a large error; or calibration is performed using a huge calibration cloth to determine the position of the corner points, but the accuracy requirements for the vehicle parking position are high, and the calibration process is cumbersome and requires a lot of manpower costs. At the same time, the accuracy may be low due to uneven calibration cloth, uneven ground, inaccurate vehicle position, etc.; or some calibration methods also require the relative position between cameras to be known and accurately determined, which increases the complexity of calibration and makes it difficult to eliminate installation errors; in addition, for the corner points of the calibration plate in the camera image, manual annotation is often used to find points and correspond, which increases time cost and also brings certain errors.

[0008] For commercial products requiring high precision, companies typically create dedicated calibration rooms, precisely place calibration plates on the floor and walls, and precisely secure the vehicle's position to achieve the required accuracy. This approach, still based on traditional calibration methods, is costly, inflexible, and cumbersome.

[0009] In summary, the existing calibration methods are inefficient, inaccurate, and have many limitations. Moreover, if the accuracy of calibration based on traditional methods is to be improved, the cost will increase sharply. Summary of the Invention

[0010] The present invention is a method for marking and converting a panoramic bird's-eye view image of a vehicle-mounted surround view camera in order to solve the defects of the above-mentioned prior art.

[0011] The purpose of the present invention can be achieved by the following technical solutions:

[0012] A method for locating and converting a panoramic bird's-eye view image of a vehicle-mounted surround view camera based on an unmanned aerial vehicle comprises the following steps:

[0013] S101: Perform intrinsic parameter calibration on the vehicle-mounted camera to obtain the intrinsic parameter matrix K and distortion vector D;

[0014] S102: placing and fixing multiple checkerboard calibration plates around the vehicle;

[0015] S103: Collecting images from the vehicle-mounted camera and the drone camera;

[0016] S104: Preprocessing images from the vehicle-mounted camera and the drone camera;

[0017] S105: Based on the vehicle-mounted camera and the drone image, where the drone image is a reference bird's-eye view, the homography matrix H is iteratively calculated using a corner detection algorithm and a RANSAC algorithm;

[0018] S106: Perform bird's-eye view conversion and panoramic stitching on the vehicle camera image according to the homography matrix H, camera intrinsic parameters and distortion vector.

[0019] Preferably, the vehicle-mounted camera is a fisheye surround view camera, which is installed one at each of the front, rear, left and right sides of the vehicle. The height and angle of the installation can make the field of view cover the range required for the bird's-eye view, and adjacent vehicle-mounted cameras have a certain overlapping field of view.

[0020] Preferably, the intrinsic parameter calibration of the vehicle-mounted camera is performed by using a flat checkerboard calibration plate and adopting a specific camera intrinsic parameter calibration algorithm for calibration.

[0021] Preferably, the camera intrinsic calibration algorithm can complete online and offline camera calibration, including fisheye camera and ordinary camera models, and supports three inputs: camera, video, and image. It generates the camera intrinsic parameter K and distortion vector D, and displays the reprojection error.

[0022] f x and f y is the focal length of the x-axis and y-axis, c x and c y is the optical axis position of the x-axis and y-axis

[0023] D=[k1,k2,k3,k4], k1-k4 are the coefficients of the fisheye camera distortion model polynomial, which is

[0024] θ d =θ(1+k1θ 2 +k2θ 4 +k3θ 6 +k4θ 8 ), θ and θ d is the distance from the center before and after distortion.

[0025] Preferably, the plurality of checkerboard calibration plates are placed and fixed around the vehicle. Unlike the existing method, there is no need to precisely control their positions. They can be placed arbitrarily to ensure that the calibration plates are clear and complete in the camera image.

[0026] Preferably, the calibration plate is composed of black and white checkerboard grids of a fixed size, with unequal numbers of grids in length and width, and no less than 20 inner corner points.

[0027] Preferably, the image is collected from the vehicle-mounted camera, and a corner detection algorithm is used to detect the image to ensure that the corners of the calibration plate can be detected, and the current frame image is captured and saved in a storage device.

[0028] Preferably, the drone camera image needs to be flown directly above the center of the vehicle, and ensure that the drone camera can clearly capture all calibration plates, and the captured frame image is saved in a storage device.

[0029] Preferably, the preprocessing of the vehicle-mounted camera and the drone camera images is to perform dedistortion processing on the vehicle-mounted camera images based on the intrinsic parameter matrix K and the distortion vector D obtained by the intrinsic parameter calibration, and use the new camera intrinsic parameter K new Adjust the image focus and frame to ensure the field of view, rotate, crop, center and zoom the drone camera image to achieve the ideal bird's-eye view size and field of view, while ensuring the uniformity of calibration.

[0030] f is the focal length, α is the zoom factor, and W and H are the width and height of the image.

[0031] Preferably, the homography matrix H is calculated based on the vehicle-mounted camera and drone images by using a corner detection algorithm to obtain the coordinates of the corner points of the calibration plate in the vehicle-mounted camera and drone camera images, selecting the corresponding four point coordinates to solve the linear equation to obtain the homography matrix, and iterating the optimal solution using the RANSAC algorithm.

[0032] The image transformation relationship is

[0033] Preferably, the bird's-eye view conversion of the vehicle-mounted camera image is performed by calculating the dedistortion mapping matrix based on the camera intrinsic parameter K and the distortion vector D, and then calculating the mapping matrix after the homography transformation based on the homography matrix H, and realizing the transformation of the vehicle-mounted camera image to the bird's-eye view through the final mapping matrix.

[0034] Preferably, the panoramic stitching is performed by calculating an image mask MASK based on the vehicle size and the bird's-eye view size, calculating weights on the overlapping parts of adjacent camera images, and performing a weighted sum operation based on the weighted MASK to obtain the final panoramic stitching bird's-eye view.

[0035] Overlap weight dist A and dist B is the distance from the point to the border of the adjacent camera area,

[0036] Bird's-eye view I i is the original vehicle camera image, f i (·) is the mapping relationship between each pixel from the vehicle-mounted camera perspective to the bird's-eye view perspective, which is determined by K, D, and H obtained from the calibration of each camera.

[0037] Compared with the prior art, the present invention has the following advantages:

[0038] (1) The present invention only requires a plurality of small calibration plates to complete the calibration, which is low in cost and high in efficiency;

[0039] (2) During calibration, there is no need to precisely determine the position of the calibration plate; it can be placed arbitrarily. There are no requirements for the parking position and angle of the vehicle, which is highly flexible.

[0040] (3) There is no strict requirement for the camera installation position, and there is no need to measure its position. The calibration method can be applied to any car model and has strong versatility;

[0041] (4) Obtain coordinates through corner detection algorithm, reducing labor costs and errors;

[0042] (5) Using drone calibration is more flexible and free, and the drone camera image is used as a reference for the panoramic bird's-eye view. By adjusting the drone camera image, the bird's-eye view results with different requirements such as field of view, size, and orientation can be achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 The overall process of the calibration and conversion splicing method of the present invention;

[0044] Figure 2 Schematic diagram of a calibration plate in an embodiment of the present invention;

[0045] Figure 3 A top view of a calibration scene in an embodiment of the present invention;

[0046] Figure 4 Collecting calibration images for the drone in an embodiment of the present invention;

[0047] Figure 5 This is the image preprocessing process in an embodiment of the present invention;

[0048] Figure 6 is the fused stitched image MASK in the embodiment of the present invention (taking the front camera as an example);

[0049] Figure 7 This is the stitching result of the original camera image and the final bird's-eye view in an embodiment of the present invention. DETAILED DESCRIPTION

[0050] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments described are only a portion of the embodiments of the present invention, not all of them. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without inventive effort should fall within the scope of protection of the present invention.

[0051] The embodiment of the present invention provides a method for calibrating and converting a panoramic bird's-eye view image of a vehicle-mounted surround view camera based on a drone. The overall process of the calibration method is as follows: Figure 1 As shown, the steps include:

[0052] S101: Perform intrinsic parameter calibration on the vehicle-mounted camera to obtain the intrinsic parameter matrix K and the distortion vector D.

[0053] Among them, the on-board camera is a fisheye surround-view camera, one is installed at the front, back, left and right of the vehicle. The installation height and angle can make the field of view cover the range required for the bird's-eye view, and adjacent on-board cameras have a certain overlapping field of view.

[0054] For example, the vehicle-mounted camera can be installed above the front license plate, above the rear license plate, on the left and right rearview mirrors, and tilted downward at a certain angle to eliminate blind spots in the near field of vision.

[0055] In addition, the vehicle-mounted camera can also be an ordinary camera or other wide-angle camera, and the number can be more than 4, as long as there is basically no blind spot around the vehicle and there is a common viewing area between adjacent cameras.

[0056] Camera intrinsic calibration is implemented using self-written code based on OpenCV. It can complete online and offline camera calibration, including fisheye camera and ordinary camera models. It supports three types of input: camera, video, and image. It generates camera intrinsic parameters K and distortion vector D, and displays reprojection error.

[0057] f x and f y is the focal length of the x-axis and y-axis, c x and c y is the optical axis position of the x-axis and y-axis

[0058] D=[k1,k2,k3,k4], k1-k4 are the coefficients of the fisheye camera distortion model polynomial, which is

[0059] θ d =θ(1+k1θ 2 +k2θ 4 +k3θ 6 +k4θ 8 ), θ and θ d is the distance from the center before and after distortion.

[0060] like Figure 2As shown, the calibration plate uses a black and white checkerboard. The points at the intersection of the four black and white squares are the corner points to be detected. To identify the horizontal and vertical directions of the calibration plate, the number of checkerboard squares in the horizontal and vertical directions needs to be inconsistent. At the same time, the number of internal corner points is increased as much as possible to ensure the robustness of the calibration. The calibration plate should be flat and accurate to improve the accuracy of the calibration results. The appropriate size should be selected, which is convenient for use but not too small, so as not to make the calibration plate too small in the figure and difficult to detect corner points. In this embodiment, the calibration plate has 7*6 internal corner points, and the side length of each small black and white square is 100mm.

[0061] To calibrate camera intrinsic parameters, a handheld calibration plate is moved in front of the camera to obtain images of the plate at different distances and angles. The camera captures images at a certain frame interval and detects the coordinates of the calibration plate's corner points in the images for calculation. The final calibration camera intrinsic parameters K and distortion vector D are obtained through optimization based on the fisheye camera model. During calibration, good lighting conditions must be ensured, and movement of the calibration plate should be minimized to minimize shake and motion blur. To improve calibration accuracy, a sufficient number of high-quality, evenly distributed images must be collected, and judgments must be made based on the dedistorted images and reprojection errors.

[0062] Using online camera intrinsic calibration, image capture and calculations are performed instantly, displaying the continuously optimized dedistortion map and reprojection error in real time. Once a stable and optimal result is achieved, the calibration result is displayed, quickly and conveniently. Alternatively, a series of calibration plate images or videos can be stored for manual processing and filtering before performing calibration calculations, further ensuring calibration accuracy.

[0063] S102: Place and fix multiple checkerboard calibration plates around the vehicle.

[0064] The calibration plate can be the checkerboard calibration plate mentioned in the above internal reference calibration, that is, Figure 2 shown.

[0065] Figure 3 This is a top view of the calibration scene, as shown in the figure. There are no strict positioning requirements for the calibration plates; they can be placed anywhere within the camera's field of view. For example, when using four calibration plates, place one each near the front, rear, left, and right cameras of the vehicle, securing them to the ground to prevent movement. To facilitate subsequent processing, choose an appropriate distance and ensure good lighting. Avoid making the calibration plates too small or unclear in the image, making it difficult to detect corner points.

[0066] S103: Collect vehicle-mounted camera images and drone camera images.

[0067] After securing the calibration plate, capture images from the vehicle's cameras and verify them using a corner detection algorithm. Once the detection is correct, the image is stored. Additionally, to facilitate corner detection, manually adjust image parameters such as exposure and contrast to make the checkerboard calibration plate clearer.

[0068] Figure 4 This is a schematic diagram of collecting calibration images for a drone. As shown in the figure, the drone needs to be equipped with a high-resolution planar camera (if a wide-angle camera is used, internal calibration must be performed as described in S101). Adjust the camera to a downward viewing angle and fly over the vehicle. At this time, try to ensure that the drone is directly above the center of the vehicle, and the flight altitude is at least enough to ensure that the camera can fully capture all calibration plates. Adjust the field of view according to the required range, and collect camera images after the drone is stable. There are no strict requirements for the position and height of the drone in this step, but try to ensure that the drone is stable and the camera is parallel to the ground.

[0069] S104: Preprocess the images from the vehicle-mounted camera and the drone camera.

[0070] Figure 5 This is the image preprocessing process, which processes the images of the vehicle-mounted camera and the drone camera respectively.

[0071] The camera intrinsic parameter K and distortion vector D obtained by the camera intrinsic parameter calibration in S101 are used to dedistort the original vehicle-mounted camera, and the new camera intrinsic parameter K is used. new Adjust the optical axis position of the image to the center and adjust the focal length to avoid excessive image cropping during dedistortion, which may result in a narrow field of view and a lack of common viewing area between adjacent cameras.

[0072] f is the focal length, α is the zoom factor, and W and H are the width and height of the image.

[0073] The drone camera image is rotated to align with the desired orientation of the final bird's-eye view and cropped to the same scale as the final bird's-eye view. The image is then centered by surrounding the vehicle with a rectangular frame. The center of the rectangular frame is translated to the center of the image to achieve centering and eliminate the horizontal position error of the drone. Finally, the entire image is scaled according to the pixel size of the checkerboard calibration plate in the image to eliminate the height error of the drone and achieve the desired physical scale and overall size of the final image. The preprocessed drone image at this point serves as the reference bird's-eye view. Multiple copies are made, each image manually masking the calibration plate and retaining only one. This image is then combined with the preprocessed vehicle camera image to form a corresponding image.

[0074] S105. Calculate the homography matrix H based on the vehicle-mounted camera and the drone image.

[0075] Several sets of corresponding images from the front, rear, left and right cameras are input separately. The coordinates of the corner points of the calibration plate in the images of the vehicle camera and the drone camera are obtained through the corner detection algorithm. The corresponding four point coordinates are selected to solve the linear equation to obtain the homography matrix. When a large number of coordinate points are actually input, the final homography matrix H of the four cameras can be obtained by iterating the RANSAC algorithm multiple times.

[0076] When calculating the homography matrix based on OpenCV, you can use the findhomography function.

[0077] The image transformation relationship is

[0078] S106: Perform bird's-eye view conversion and panoramic stitching on the vehicle-mounted camera images.

[0079] Based on the final bird's-eye view size and the size of the vehicle within the bird's-eye view, the field of view of each camera can be divided and the corresponding image mask MASK is obtained. Based on the calibration parameters obtained in step S101, the original image is dedistorted using the camera intrinsic parameter K, the distortion vector D, and the new camera intrinsic parameter. The transformed bird's-eye view angle is calculated using the homography matrix H and then ANDed with the image mask MASK to obtain the bird's-eye view of the corresponding area. Finally, the bird's-eye view images from the four cameras are stitched together to form a complete panoramic bird's-eye view.

[0080] In order to improve real-time performance, the steps of dedistortion and bird's-eye view transformation can be combined and calculated into a mapping matrix to realize the mapping transformation from the vehicle-mounted camera image directly to the final bird's-eye view.

[0081] In order to further improve the effect and solve the problem of unevenness at the seams of adjacent cameras, a fusion algorithm can be used to stitch the bird's-eye view panoramic image. The four cameras each retain a larger area and calculate the weight in the overlapping part of the adjacent camera images. That is, in the overlapping area, the distance ratio of the point in the area to the boundary is used as the weight to achieve the image gradient effect. For example, the image mask MASK obtained by the previous camera is as follows Figure 6 Finally, the weighted summation operation is performed based on the weighted MASK to obtain the final panoramic stitching bird's-eye view.

[0082] Overlap weight dist A and dist B is the distance from the point to the border of the adjacent camera area,

[0083] Bird's-eye view I i is the original vehicle camera image, f i(·) is the mapping relationship between each pixel from the vehicle-mounted camera perspective to the bird's-eye view perspective, which is determined by K, D, and H obtained from the calibration of each camera.

[0084] Figure 7 The original camera image input and the final generated panoramic stitching bird's-eye view in this embodiment are shown.

[0085] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and such modifications or substitutions are intended to be within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be subject to the scope of protection of the claims.

Claims

1. A method for marking and converting a panoramic bird's-eye view image of a vehicle-mounted surround view camera based on a drone, characterized in that the steps include: S101: Perform intrinsic parameter calibration on the vehicle-mounted camera to obtain the intrinsic parameter matrix K and distortion vector D; S102: placing and fixing multiple checkerboard calibration plates around the vehicle; S103: Collecting images from the vehicle-mounted camera and the drone camera; S104: Preprocessing images from the vehicle-mounted camera and the drone camera; S105: Based on the vehicle-mounted camera and the drone image, where the drone image is a reference bird's-eye view, the homography matrix H is iteratively calculated using a corner detection algorithm and a RANSAC algorithm; S106: Perform bird's-eye view conversion and panoramic stitching on the vehicle camera image according to the homography matrix H, camera intrinsic parameters and distortion vector.

2. The method for locating and converting a panoramic bird's-eye view image of a vehicle-mounted surround view camera based on an unmanned aerial vehicle according to claim 1, characterized in that: The vehicle-mounted cameras are fisheye surround-view cameras, one of which is installed at the front, rear, left, and right of the vehicle. The height and angle at which the vehicle-mounted cameras are installed enable the field of view to cover the range required for a bird's-eye view, and adjacent vehicle-mounted cameras have a certain overlapping field of view.

3. The method for marking and converting a panoramic bird's-eye view image of a vehicle-mounted surround view camera based on an unmanned aerial vehicle according to claim 1, characterized in that: The step S101 performs intrinsic parameter calibration on the vehicle-mounted camera by using a flat checkerboard calibration plate and a specific camera intrinsic parameter calibration algorithm.

4. The method for locating and converting a panoramic bird's-eye view image of a vehicle-mounted surround view camera based on an unmanned aerial vehicle according to claim 3, characterized in that: The camera intrinsic parameter calibration algorithm is used to complete online and offline camera calibration, including fisheye camera and ordinary camera models. It supports three inputs: camera, video, and image. It generates the camera intrinsic parameter K and distortion vector D, and displays the reprojection error. f x and f y is the focal length of the x-axis and y-axis, c x and c y is the optical axis position of the x-axis and y-axis D=[k1,k2,k3,k4], where k1-k4 are the coefficients of the fisheye camera distortion model polynomial, which is: θ d =θ(1+k1θ 2 +k2θ 4 +k3θ 6 +k4θ 8 ), θ and θ d is the distance from the center before and after distortion.

5. The method for marking and converting a panoramic bird's-eye view image of a vehicle-mounted surround view camera based on an unmanned aerial vehicle according to claim 1, characterized in that: The checkerboard calibration plate is composed of black and white checkerboards of a fixed size, with unequal numbers of squares in the length and width directions and no less than 20 inner corner points. The position of the checkerboard calibration plate does not need to be precisely controlled, and it can be placed arbitrarily to ensure that the checkerboard calibration plate is clear and complete in the camera image.

6. The method for locating and converting a panoramic bird's-eye view image of a vehicle-mounted surround view camera based on an unmanned aerial vehicle according to claim 1, characterized in that: The preprocessing of the vehicle-mounted camera and the drone camera image is to perform dedistortion processing on the vehicle-mounted camera image based on the intrinsic parameter matrix K and the distortion vector D obtained by the intrinsic parameter calibration, and use the new camera intrinsic parameter K new Adjust the image focus and frame size to ensure the field of view, rotate, crop, center, and zoom the drone camera image to achieve the ideal bird's-eye view size and field of view, while ensuring the uniformity of the calibration: f is the focal length, α is the zoom factor, and W and H are the width and height of the image.

7. The method for locating and converting a panoramic bird's-eye view image of a vehicle-mounted surround view camera based on an unmanned aerial vehicle according to claim 1, characterized in that: The homography matrix H is calculated based on the vehicle-mounted camera and drone images. The coordinates of the corner points of the calibration plate in the vehicle-mounted camera and drone camera images are obtained by the corner detection algorithm. The corresponding four point coordinates are selected to solve the linear equation to obtain the homography matrix, and the optimal solution is iterated by the RANSAC algorithm: The image transformation relationship is 8. The method for locating and converting a panoramic bird's-eye view image of a vehicle-mounted surround view camera based on an unmanned aerial vehicle according to claim 1, characterized in that: The bird's-eye view conversion of the vehicle camera image is performed by calculating the dedistortion mapping matrix based on the camera intrinsic parameter K and the distortion vector D, and then calculating the mapping matrix after the homography transformation based on the homography matrix H. The final mapping matrix is ​​used to realize the transformation of the vehicle camera image to the bird's-eye view.

9. The method for locating and converting a panoramic bird's-eye view image of a vehicle-mounted surround view camera based on an unmanned aerial vehicle according to claim 1, characterized in that: The conversion and stitching method is to calculate the image mask MASK based on the vehicle size and the bird's-eye view size, calculate the weights in the overlapping parts of adjacent camera images, and perform a weighted sum operation based on the weighted MASK to obtain the final panoramic stitching bird's-eye view: in: Overlap weight dist A and dist B is the distance from the point to the border of the adjacent camera area, Bird's-eye view I i is the original vehicle camera image, f i (·) is the mapping relationship between each pixel from the vehicle-mounted camera perspective to the bird's-eye view perspective, which is determined by K, D, and H obtained from the calibration of each camera.

Images