A panoramic image stitching method applied to a vehicle

Abstract

The application provides a panoramic image splicing method applied to a vehicle, and specifically comprises the following steps: acquiring an image imageL sent by a roof camera, creating a joint-eliminating image; performing cylindrical projection on the imageL image and the joint-eliminating image, and acquiring feature point pairs matched with each other of adjacent spliced cylindrical projection images; respectively performing projection transformation on the cylindrical projection images and corresponding cylindrical projection joint-eliminating images by using a change matrix; performing image fusion on the cylindrical projection images and the cylindrical projection joint-eliminating images after the projection transformation; combining the fused images, and acquiring a panoramic image. In the technical scheme, the joint-eliminating image is created, the sizes of the images are first up-sampled to form an image group, and then the image group after down-sampling is accumulated to obtain the panoramic image. The color difference and the splicing joint are avoided.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image stitching of vehicles, in particular to a panoramic image stitching method applied to vehicles. BACKGROUND

[0002] In recent years, with the continuous progress of science and technology, the automobile technology is advancing towards intelligentization, and people have put forward more requirements for intelligent vehicles. Instead of regarding the vehicle as a simple means of transportation, the vehicle is required to have automatic driving, vehicle entertainment and other functions to improve the comfort of vehicle users. The development of vehicle entertainment function enables people to make online shopping, listen to music and watch movies. At the same time, the vehicle camera is used to shoot the scenery outside the vehicle. However, due to the limited field of view of the camera, if the scenery on both sides of the road needs to be shot during self-driving travel, a panoramic image is formed, and multiple cameras need to be configured to shoot simultaneously in the prior art, which results in high cost due to the installation of multiple cameras on the roof of the vehicle. SUMMARY

[0003] Based on the defects in the prior art, the present application provides a panoramic image stitching method applied to vehicles, comprising:

[0004] An image imageL sent by a roof camera is acquired, and an image with the same size as the imageL is created, and a seam-removing image filled with the same constant is created;

[0005] The imageL image and the seam-removing image are subjected to cylindrical projection to acquire a cylindrical projection image and a cylindrical projection seam-removing image, and feature point pairs matched with each other are acquired from adjacent stitched cylindrical projection images;

[0006] The cylindrical projection image and the corresponding cylindrical projection seam-removing image are subjected to projection transformation by using a change matrix; the cylindrical projection image and the cylindrical projection seam-removing image after the projection transformation are subjected to image fusion; the fused image is combined to acquire a panoramic image.

[0007] A panoramic image stitching method applied to vehicles, and further optionally, edge filling is performed on the cylindrical projection image and the cylindrical projection seam-removing image, so that the filled image is not smaller than the size of the image formed after the adjacent image stitching.

[0008] A panoramic image stitching method applied to vehicles, and further optionally, the roof camera is corrected by using a checkerboard, and the intrinsic and extrinsic parameters of the vehicle camera are calibrated by using Zhang Zhengyou method;

[0009] If the roof camera is a fisheye lens, the fisheye lens needs to be subjected to distortion correction to obtain a distortion correction coefficient;

[0010] Activate the panoramic shooting mode, use the rotatable roof camera to shoot from multiple angles to obtain the original sequence of images, and obtain the imageL for stitching after distortion correction.

[0011] A panoramic image stitching method for vehicles, further optionally, the method for cylindrical images specifically includes:

[0012] Construct cylindrical projection relations and transform the image into a cylindrical projection image using these relations.

[0013] The precise position of each pixel is calculated using bilinear interpolation to obtain the projected image;

[0014] The cylindrical projection image is binarized by adaptive thresholding, contour detection is performed, contour coordinates are obtained, and a new projection image is obtained based on the contour coordinates.

[0015] The edge adjustment strategy sets the corner points within a preset range around the image to 0.

[0016] A panoramic image stitching method for vehicles, further optionally, involves obtaining mutually matched feature point pairs through the SIFT algorithm after cylindrical projection of the image.

[0017] A panoramic image stitching method for vehicles, further optionally, includes the SIFT algorithm specifically comprising:

[0018] The image is Gaussian blurred to remove secondary information, and the image is downsampled into different octaves, with different Gaussian kernels used for convolution within each octave.

[0019] Find the extreme points in the difference pyramid, and perform comparison tests and edge tests on the extreme points;

[0020] The keypoint neighborhood gradient method uses histograms to determine the main direction of keypoints;

[0021] Statistical analysis of the gradient orientation histogram of feature points in the neighborhood yields feature descriptors.

[0022] By comparing the feature descriptors of adjacent images pairwise, several pairs of matching feature points are identified.

[0023] A panoramic image stitching method for vehicles, optionally, preprocesses the matching feature point pairs obtained by the SIFT algorithm to obtain high-quality matching feature point pairs. Specifically, the quality of all mutually matching feature point pairs is evaluated by calculating the Q value through a preset model, and then they are sorted in descending order according to the Q value. Matching feature point pairs with Q less than a preset threshold P1 are filtered out, and the remaining matching feature point pairs form high-quality matching feature point pairs.

[0024] A panoramic image stitching method applied to a vehicle, further optionally, randomly selecting a matching feature point pair with the least model parameter estimation from high-quality matching feature point pairs, and then calculating a perspective transformation matrix Th;

[0025] One of the feature points in all matching feature point pairs of the reference image is perspective transformed using the perspective transformation matrix Th, and the distance Dh between the corresponding feature point in the cylindrical projection image to be stitched and the perspective-transformed feature point is calculated;

[0026] When Dh in the matching feature point pair is less than a preset comparison threshold C2, the point is determined to be an inlier, otherwise it is determined to be an outlier;

[0027] Then, the total number of inliers inlier_num is counted, and when inlier_num is within a preset threshold range, the model is determined to be the optimal model, and the statistical error under the model is recorded;

[0028] The statistical error is defined as the cumulative sum of all Dh distances under the perspective transformation matrix Th;

[0029] The high-quality matching feature point pairs are traversed to obtain the statistical error corresponding to the perspective transformation matrix Th obtained by different matching feature point pairs, and all inliers corresponding to the minimum statistical error are obtained as the optimal inlier set.

[0030] A panoramic image stitching method applied to a vehicle, further optionally, after obtaining the transformation matrix Ma, the image to be stitched and the corresponding seam-removing image are projected into the coordinate system of the adjacent image through Ma to obtain a first fusion image and a first fusion seam-removing image, so that the coordinate reference of the image to be stitched is unified.

[0031] A panoramic image stitching method applied to a vehicle, further optionally, the image fusion method comprises:

[0032] A second fusion image group, a second fusion image adjacent stitching image group, and a second fusion seam-removing image group are obtained, and images of the same size are sequentially taken from the second fusion image group, the second fusion image adjacent stitching image group, and the second fusion seam-removing image group, and then fused;

[0033] The fusion method formula is as follows:

[0034] Imgs[i] = Imgb1[i] * Seam_img[i] + Imgb2[i] * (1 - Seam_img[i])

[0035] Wherein, i indicates the index of corresponding image in the fusion image group, Imgb1[i] indicates the second fusion image with the order of i, Imgb2[i] indicates the adjacent spliced image of the second fusion image with the order of i, Seam_img[i] indicates the second fusion image with the order of i, and Imgs[i] indicates the third fusion image with the order of i.

[0036] The fused image is added into a new image fusion group to form a third fusion image group.

[0037] The third fusion image group is obtained, and the images in the third fusion image group are sorted according to the image size.

[0038] The first image is taken out, the first image is up-sampled to be equal to the size of the second image, and then the second image is added to obtain a result, which is recorded as accumulated_img.

[0039] Then, accumulated_img is up-sampled, so that the up-sampled image is equal to the size of the third image, and then the third image is added.

[0040] In succession, the traversal is performed, and finally the image is obtained as a panoramic image.

[0041] Beneficial effects:

[0042] 1. In the technical scheme provided by the present application, a rotatable roof camera is used to shoot the surroundings of the roof, the obtained images are spliced, and a panoramic image is obtained, which is completed by only one roof camera.

[0043] 2. In the image splicing process, in order to avoid color difference and splicing seam in the overlapping area, the present embodiment creates a seam-eliminating image, and the size of the image is up-sampled to form an image group, and then the image group is down-sampled to form a corresponding cumulative image group, so as to obtain a panoramic image and avoid color difference and splicing seam.

[0044] 3. In the technical scheme provided by the present application, the image formed by the splicing seam is adaptively cut, and the imaging quality of the edge of the panoramic image is maximally reserved. DETAILED DESCRIPTION

[0045] The following drawings only schematically illustrate and explain the present application, and do not limit the scope of the present application.

[0046] Fig. 1 The present application is an embodiment of an image panoramic splicing method flow chart.

[0047] Fig. 2 The present application is an embodiment of an image panoramic splicing method flow chart.

[0048] Fig. 3 The flow chart of the method for adaptive cropping of panoramic images in an embodiment of the present application. DETAILED DESCRIPTION

[0049] In order to make the technical features, objectives and effects of the present application more clearly understood, the specific embodiments of the present application will be described below with reference to the drawings, in which the same reference numerals represent the same parts. In order to make the drawings simple, only the parts related to the present application are schematically shown in the drawings, and the actual structure as a product is not represented. In addition, in order to make the drawings simple and easy to understand, in some drawings, only one of the parts having the same structure or function is schematically shown, or only one of them is marked.

[0050] As is well known to those skilled in the art of control systems, functional modules, applications (APP), they can take any appropriate form, both hardware and software, both discrete multiple functional modules, and multiple functional units integrated into one hardware. As the simplest form, the control system can be a controller, such as a combination logic controller, a microprogrammed controller, etc., as long as it can implement the operations described in the present application. Of course, the control system can also be integrated into a physical device as different modules, which do not deviate from the basic principles and scope of protection of the present application.

[0051] In the present application, "connection" can include direct connection, indirect connection, communication connection, and electrical connection, unless otherwise specified.

[0052] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the present disclosure. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and / or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. As used herein the term "and / or" includes any and all combinations of one or more of the associated listed items

[0053] It should be understood that the term "vehicle" or "vehicular" or other similar terminology used herein generally includes a motor vehicle, such as a passenger automobile, including sport utility vehicles (SUV), buses, trucks, passenger automobiles of various types, including various commercial vehicles, watercraft, including boats and ships, aircraft, and the like, and includes hybrid vehicles, electric vehicles, plug-in hybrid electric vehicles, hydrogen vehicles, and other alternative fuel vehicles (e.g., fuel from a source other than petroleum). As referred to herein, a hybrid vehicle is a vehicle having two or more sources of power, such as both gasoline power and electric power.

[0054] Further, the controller of the present disclosure can be embodied as a non-transitory computer readable medium on a computer readable medium containing executable program instructions executed by a processor, controller, or the like. Examples of the computer readable medium include, but are not limited to, ROM, RAM, compact disc (CD)-ROMs, magnetic tapes, floppy disks, flash memories, smart cards, and optical data storage devices. The computer readable recording medium can also be distributed over network coupled computer systems so that the computer readable medium is stored and executed in a distributed fashion, e.g., by a remote processing server or a controller area network (CAN).

[0055] The present application provides a panoramic image stitching algorithm based on a panoramic image stitching algorithm, as shown in Figs. 1 to 3 Specifically, at least includes:

[0056] Obtain the image imageL sent by the roof camera, create an image with the same size as the imageL, and fill the seam image with the same constant;

[0057] Cylindrical projection is performed on the imageL image and the seam image to obtain a cylindrical projection image and a cylindrical projection seam image;

[0058] Boundary filling is performed on the cylindrical projection image and the cylindrical projection seam image, so that the image after boundary filling is not less than the size of the image formed after adjacent image stitching;

[0059] Obtain the feature point pairs matched with each other of the adjacent stitched cylindrical projection images;

[0060] The change matrix is calculated by constructing a transmission transformation relationship using the feature point pairs matched with each other;

[0061] Obtain the change matrix of the adjacent stitched images;

[0062] The projection transformation is performed on the cylindrical projection image and the corresponding cylindrical projection seam image using the change matrix respectively; image fusion is performed on the cylindrical projection image and the cylindrical projection seam image after projection transformation; the fused image is combined to obtain a panoramic image.

[0063] Specifically, referring to Fig. 2 , the image acquisition of the roof camera specifically includes:

[0064] The roof camera is corrected by a chessboard, and the internal and external parameters of the vehicle-mounted camera are calibrated by Zhang Zhengyou method;

[0065] If the roof camera is a fisheye lens, the fisheye lens needs to be corrected for distortion to obtain a distortion correction coefficient;

[0066] Start panoramic shooting mode, use the rotatable camera installed on the roof to take pictures at multiple angles, obtain the original sequence image, and obtain the image imageL after distortion correction;

[0067] Specifically, in order to make the spliced image more consistent with the observation effect of the human eye, the image is projected on a cylindrical surface before splicing in this embodiment. Compared with the traditional cylindrical projection, the method of cylindrical projection in this embodiment is improved:

[0068] Create a joint-removing image with the same size as the spliced image, and all pixels in the joint-removing image are set to 1 or an integer less than or equal to 255;

[0069] Cylindrical projection is performed on the image imageL and the joint-removing image respectively to obtain a cylindrical projection image and a cylindrical projection joint-removing image;

[0070] The method of cylindrical image specifically includes the following steps:

[0071] A cylindrical projection relationship is constructed, and the image is converted into a cylindrical projection image through the cylindrical projection relationship;

[0072] The accurate position of each pixel point is calculated by bilinear interpolation to obtain the projected image.

[0073] The cylindrical projection relationship is:

[0074]

[0075]

[0076] Wherein, u, v are the pixel coordinates of the projected image, x, y are the pixel coordinates of the image before projection, f is the focal length of the camera, and width and height are the width and height of the image before projection, respectively;

[0077] Specifically, after the cylindrical projection, based on the unmanned aerial vehicle imaging characteristics, the applicant finds that more outlier points will be generated at the edge thereof, and the outlier points will affect the subsequent corner detection. Therefore, the cylindrical projection image is binarized by an adaptive threshold, contour detection is performed, contour coordinates are obtained, and a new cylindrical projection image and a new cylindrical projection seam-elimination image are obtained according to the contour coordinates, so as to eliminate the points in the outlier range;

[0078] The existing corner points in the preset range around the image are set to 0 through an edge adjustment strategy;

[0079] When the cylindrical projection is performed, the mutually matched feature point pairs are obtained through the SIFT algorithm;

[0080] The specific steps of the SIFT algorithm include:

[0081] The Gaussian blur of the image is removed to remove the secondary information, and the image is down-sampled into different octaves, and different Gaussian kernels are convolved in the octaves;

[0082] Extreme points in the difference pyramid are found, and the extreme points are subjected to comparison testing and edge testing;

[0083] The key point neighborhood gradient method is used to determine the main direction of the key point;

[0084] The feature point field gradient direction histogram is counted to obtain a feature description operator;

[0085] The feature description operators of adjacent images are compared in pairs to find a plurality of mutually matched feature point pairs.

[0086] Specifically, the vehicle-mounted camera usually adopts a fisheye camera, has a large visual range, and the feature points in the overlapping area between images are hundreds or thousands of feature points relative to other scenes. Due to the influence of lighting and shooting angle on the image, the obtained corner points may have false corner points. In addition, calculation errors exist in the obtained angle, and these angle points with large errors will participate in matching, which will lead to a long matching time on the one hand, and the wrong angle points will also cause the distortion of the overlapping area image on the other hand. In order to solve this problem, the following solution is adopted in this embodiment:

[0087] The quality evaluation of all mutually matched feature point pairs is calculated through a preset model to obtain a Q value, and then the mutually matched feature point pairs are arranged in descending order according to the Q value, and the matching feature point pairs with a Q value less than a preset threshold P are filtered out to form high-quality matching feature point pairs;

[0088] Specifically, the preset model can be a projection homography matrix and a similar distance, and can be determined by clustering and calculating the similar distance Q value of the feature points;

[0089] Randomly select the least data of model parameter estimation from high-quality matching feature points, and then calculate the perspective change equation;

[0090] Specifically, the model parameters select random 4 matching feature points, and estimate the change matrix transformation Th through perspective transformation;

[0091] One of the feature points in all the matching feature points of the reference image is transformed by the perspective change matrix Th, and the distance Dh between the corresponding feature points in the image to be spliced and the feature points after perspective transformation is calculated;

[0092] When Dh obtained from the feature point data is less than the preset comparison threshold C2, it is determined that the point is an inlier, otherwise it is determined as an outlier;

[0093] Then the total number of inliers inlier_num is counted, and when inlier_num is within the preset threshold range, it is determined that the model is the optimal model, and the statistical error under the model is recorded;

[0094] The statistical error is defined as the cumulative sum of all Dh distances under the perspective transformation matrix Th;

[0095] Then repeat the above process, when the change matrix transformation Th is calculated, the corresponding statistical error of Th is obtained, and the optimal inlier set corresponding to the minimum statistical error is obtained, that is, the final matching mutual matching feature point pair is obtained.

[0096] Specifically, after the spliced image is projected on the cylindrical surface, in order to be able to splice, it is necessary to unify the coordinate reference, so that the coordinate reference with overlapping area can be found for fusion.

[0097] The matching feature point pair of the image to be spliced is calculated to obtain the transformation matrix Ma of the splicing projection transformation.

[0098] Specifically, it includes:

[0099] Get the matching feature points feature_points1, feature_points2 of adjacent images;

[0100] Through the matching feature points feature_points1, feature_points2, the change matrix is constructed by using the projection change relationship;

[0101] feature_points2 = Ma*feature_points1

[0102] Since the coordinates of feature_points2 and feature_points1 are known, the coordinates are substituted to obtain the transformation matrix Ma;

[0103] Specifically, after obtaining the transformation matrix Ma, the image to be spliced is projected to the coordinate system of the adjacent image thereof by Ma, so that the coordinate reference of the adjacent spliced image is unified.

[0104] Specifically, in the embodiment, the image to be spliced and the corresponding joint-elimination image are projected to the coordinate system of the adjacent image by Ma respectively, to obtain a first fusion image and a first fusion joint-elimination image, so that the coordinate reference of the image to be spliced is unified, which is beneficial to the splicing of the adjacent images.

[0105] Specifically, image fusion is more critical. If the fusion is not good, a large splicing joint or color difference will be generated in the overlapping area. In order to solve this problem, the embodiment provides the following solutions, which are as follows:

[0106] Obtain a first fusion image, a first fusion joint-elimination image, and a spliced image adjacent to the first fusion image.

[0107] Set a preset integer threshold C2, and perform 2 n times down-sampling on the first fusion image, the first fusion joint-elimination image, and the spliced image adjacent to the first fusion image, respectively, wherein n is an integer in the range of [0, C2], and C2 can be equal to 4 or 5, for example.

[0108] Store the obtained down-sampled images to form a first fusion image group, a first fusion joint-elimination image group, and a first spliced image group.

[0109] Specifically, since 2 n times down-sampling is performed, after sampling, a corresponding image array is formed. For example, if C2 is equal to n, after sampling the first fusion image, a 1, 2, 4, …, 2 c2 times down-sampled image group is formed as the first fusion image group.

[0110] Respectively, the first fusion image group or the spliced image adjacent to the first fusion image is sequentially subtracted from the bottom layer to the next top layer in order, and the image obtained by 2 times up-sampling of the lower layer image and the top layer image is recorded as a second fusion image group or a second fusion image adjacent spliced image group.

[0111] Obtain the size of the corresponding image in the second fusion image group or the second fusion image adjacent spliced image group, and select the image group of the corresponding size from the first joint-elimination image group to form a second fusion joint-elimination image group.

[0112] Specifically, after the spliced image is processed, the adjacent spliced image is fused.

[0113] Specifically, the image fusion method is as follows:

[0114] Obtaining a second fusion image group, a second fusion image adjacent splicing image group, a second fusion seam-eliminating image group, sequentially taking out images of the same size from the second fusion image group, the second fusion image adjacent splicing image group and the second fusion seam-eliminating image group respectively, and then performing the following fusion:

[0115] Imgs[i] = Imgb1[i] * Seam_img[i] + Imgb2[i] * (1 - Seam_img[i]),

[0116] Wherein, i represents the index of the corresponding image in the fusion image group, Imgb1[i] represents the second fusion image of order i, Imgb2[i] represents the second fusion image adjacent splicing image of order i, Seam_img[i] represents the second fusion seam-eliminating image of order i; Imgs[i] represents the third fusion image of order i;

[0117] Adding the fused image into a new image fusion group to form a third fusion image group;

[0118] Obtaining the third fusion image group, and sorting the images in the third fusion image group according to the image size;

[0119] Taking out the first image, performing up-sampling on the first image, and then adding the second image to obtain a result recorded as accumulated_img;

[0120] Then, performing up-sampling on accumulated_img, so that the up-sampled image is less than or equal to the size of the third image, and then adding the third image;

[0121] Iterating in turn, and finally obtaining the image as a panoramic image.

[0122] Specifically, since there may be many images in panoramic image splicing, image splicing is performed in a dark rotation order, for example, if four images are spliced, the first two images are spliced and fused, and then spliced with the third image, and so on, to obtain the final panoramic image.

[0123] Specifically, the obtained fused image is spliced and fused again until the images used for splicing are traversed, and the final panoramic image is obtained.

[0124] Specifically, due to the use of multiple images for stitching, in the case of the present embodiment using a cylindrical projection and edge adjustment strategy, irregular black edges will be generated at the upper and lower edges of the image along the y-axis direction, in order to remove the black edges and retain the integrity of the stitched image as much as possible, the present embodiment provides an adaptive irregular black edge removal method, as shown in Fig. 3 Specifically, the adaptive irregular black edge removal method comprises the following steps:

[0125] The acquired panoramic image is converted into a gray-scale image; the gray-scale image is binarized, and a preset comparison value threshold C3 is set;

[0126] When the pixel value of the gray-scale image is greater than C3, the pixel value of the gray-scale image is set to 255, and when the pixel value of the gray-scale image is less than C3, the pixel value of the gray-scale image is set to 0, to obtain a binary image;

[0127] Specifically, C3 can be set to a number within 2-10;

[0128] An adaptive comparison reference threshold TC is set to the width of the panoramic image / L, and L is 50-200

[0129] The binary image is traversed from top to bottom along the y-axis, and when the number of coordinate points with a pixel value equal to 0 in all (x, yi) of the binary image under the same yi coordinate is less than TC, the upper boundary top_border=yi of the image is obtained, wherein x takes a value range of [0, the width of the panoramic image],

[0130] The binary image is traversed from bottom to top along the y-axis, and when the number of coordinate points with a pixel value equal to 0 in all (x, yi) of the binary image under the same yi coordinate is less than TC, the lower boundary bottom_border=yi of the image is obtained, wherein x takes a value range of [0, the width of the panoramic image],

[0131] The corresponding image on the panoramic image is obtained by the upper boundary and the lower boundary, to obtain the final panoramic image.

[0132] The above is only a preferred embodiment of the present application, and the present application is not limited to the above embodiment. Those skilled in the art can clearly see that the form in the embodiment is not limited to this, and the adjustment method is also not limited to this. It can be understood that other improvements and changes directly derived or thought of by those skilled in the art without departing from the basic concept of the present application should be considered to be included in the protection scope of the present application.

Claims

1. A panoramic image stitching method applied to a vehicle, characterized by, The method comprises the following steps: An image imageL sent by a roof camera is acquired, and a seam-removing image with the same size as the imageL is created, and pixels of the seam-removing image are filled with the same constant; The imageL image and the seam-removing image are subjected to cylindrical projection to acquire a cylindrical projection image and a cylindrical projection seam-removing image, and feature point pairs matched with each other in the adjacent spliced cylindrical projection images are acquired; The cylindrical projection image and the corresponding cylindrical projection seam-removing image are subjected to projection transformation by using a change matrix; image fusion is performed on the cylindrical projection image and the cylindrical projection seam-removing image after the projection transformation; and the fused image is combined to acquire a panoramic image; After the change matrix is acquired, the to-be-spliced image and the corresponding seam-removing image are projected into the coordinate system of the adjacent image by using the change matrix to acquire a first fused image and a first fused seam-removing image, so that the coordinate reference of the to-be-spliced image is unified; The image fusion method comprises the following steps: A second fused image group, a second fused image adjacent spliced image group and a second fused seam-removing image group are acquired, and images with the same size are sequentially taken out from the second fused image group, the second fused image adjacent spliced image group and the second fused seam-removing image group, and then fusion is performed; The fusion formula is as follows: Imgs[i] = Imgb1[i] * Seam_img[i] + Imgb2[i] * (1 - Seam_img[i]) Wherein, i represents the index of the corresponding image in the fused image group, Imgb1[i] represents the second fused image with the sequence i, Imgb2[i] represents the second fused image adjacent spliced image with the sequence i, Seam_img[i] represents the second fused seam-removing image with the sequence i, and Imgs[i] represents the third fused image with the sequence i; The fused image is added into a new image fusion group to form a third fused image group; The third fused image group is acquired, and the images in the third fused image group are sorted according to the image size; The first image is taken out, and the first image is subjected to up-sampling to be equal to the size of the second image, and then the second image is added to obtain a result, which is recorded as accumulated_img; Then, the accumulated_img is subjected to up-sampling, so that the up-sampled image is equal to the size of the third image, and then the third image is added; The above steps are sequentially repeated to finally obtain the panoramic image.

2. The method of claim 1, wherein the panoramic image stitching method is applied to a vehicle. Edge filling is performed on the cylindrical projection image and the cylindrical projection seam-removing image, so that the filled image is not less than the size of the image formed after the adjacent images are spliced.

3. The method of claim 1, wherein the panoramic image stitching method is applied to a vehicle. The roof camera is corrected by using a checkerboard, and the intrinsic parameters and the extrinsic parameters of the vehicle-mounted camera are calibrated by using the Zhang Zhengyou method; If the roof camera is a fisheye lens, the fisheye lens needs to be subjected to distortion correction to acquire a distortion correction coefficient; The panoramic shooting mode is started, the rotatable roof camera is used for multi-angle rotation shooting, the original sequence image is acquired, and the image imageL used for splicing is acquired after distortion correction.

4. The method of claim 1, wherein the panoramic image stitching method is applied to a vehicle. The method for acquiring the cylindrical image specifically comprises the following steps: A cylindrical projection relationship is constructed, and the image is converted into a cylindrical projection image through the cylindrical projection relationship; The accurate position of each pixel point is calculated through bilinear interpolation to obtain the projected image; The cylindrical projection image is binarized through an adaptive threshold, contour detection is performed, and contour coordinates are obtained, so that a new projection image is obtained according to the contour coordinates; The corner points existing in a preset range around the image are set to 0 through an edge adjustment strategy.

5. The method of claim 1, wherein the panoramic image stitching method is applied to a vehicle. When the image is projected on the cylinder, the feature point pairs matched with each other are obtained through the SIFT algorithm.

6. The method of claim 5, wherein the panoramic image stitching method is applied to a vehicle. The SIFT algorithm specifically includes: The Gaussian blur of the image is removed to remove the secondary information, the image is down-sampled into different octaves, and different Gaussian kernels are convolved in the octaves; Extreme points in the difference pyramid are found, and the extreme points are subjected to comparison testing and edge testing; The key point neighborhood gradient method histogram determines the main direction of the key point; The feature point field gradient direction histogram is counted to obtain a feature description operator. A plurality of pairs of mutually matched feature points are found through pairwise comparison of the feature description operators of adjacent images.

7. The method of claim 5, wherein the panoramic image stitching method is applied to a vehicle. The matching feature point pairs obtained through the SIFT algorithm are preprocessed to obtain high-quality matching feature point pairs. In the preprocessing, all the mutually matched feature point pairs are subjected to quality evaluation through calculation of Q values according to a preset model, and then the matching feature point pairs with Q values less than a preset threshold P1 are filtered out in descending order of Q values, and the remaining matching feature point pairs form the high-quality matching feature point pairs.

8. The method of claim 7, wherein the panoramic image stitching method is applied to a vehicle. The matching feature point pair with the least model parameter estimation is randomly selected from the high-quality matching feature point pairs, and a perspective transformation matrix Th is calculated. One of the feature point pairs in the reference image is subjected to perspective transformation using the perspective transformation matrix Th, and the distance Dh between the corresponding feature point in the cylindrical projection image to be spliced and the feature point subjected to perspective transformation is calculated. When Dh in the matching feature point pair is less than a preset comparison threshold C2, the point is determined to be an inlier, and otherwise, the point is determined to be an outlier. The total number of inliers inlier_num is then counted, and when inlier_num is within a preset threshold range, the model is determined to be an optimal model, and the statistical error under the model is recorded. The statistical error is defined as the cumulative sum of all Dh distances under the perspective transformation matrix Th. The high-quality matching feature point pairs are traversed, the statistical errors corresponding to the perspective transformation matrices Th obtained by different matching feature point pairs are obtained, and all the inliers corresponding to the minimum statistical error are obtained as an optimal inlier set.

Images