Calibration method for fisheye panoramic stitching

Abstract

The invention provides a fisheye panoramic stitching calibration method, which is applied to a shooting device, the shooting device comprises a first fisheye camera and a second fisheye camera, and the shooting directions of the first fisheye camera and the second fisheye camera are opposite. The calibration method comprises the following steps: acquiring a first image obtained by shooting two calibration plates by a first fisheye camera and a second image obtained by shooting two same calibration plates by a second fisheye camera; the two calibration plates are symmetrically arranged on the two sides of the shooting device; and according to the coordinates of the preset identifiers on the two calibration plates in the first image, the coordinates of the preset identifiers on the two calibration plates in the second image, the image parameters of the first image and the image parameters of the second image, calibrating the shooting device to obtain calibration data of the shooting device. Calibration of the shooting device can be completed by using the first image and the second image, and the calibration efficiency of the shooting device can be improved. And the calibration data can be used for panoramic image splicing, so that the image splicing effect can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of camera calibration, and particularly relates to a fisheye panoramic splicing calibration method. BACKGROUND

[0002] In the related art, when a fisheye camera is calibrated, more data needs to be collected, and calibration data of the fisheye camera is determined, so that a situation that a long time is consumed for calibrating the fisheye camera is easily caused, and thus calibration efficiency of the fisheye camera is poor. SUMMARY

[0003] The main purpose of the present application is to provide a fisheye panoramic splicing calibration method, and aims to solve the technical problem of how to improve calibration efficiency of a fisheye camera.

[0004] In a first aspect, the present application provides a fisheye panoramic splicing calibration method, applied to a shooting device, the shooting device comprising a first fisheye camera and a second fisheye camera, a shooting direction of the first fisheye camera being opposite to a shooting direction of the second fisheye camera;

[0005] The calibration method comprises:

[0006] obtaining a first image obtained by the first fisheye camera shooting two calibration boards and a second image obtained by the second fisheye camera shooting the same two calibration boards; the two calibration boards are symmetrically arranged on two sides of the shooting device, and an arrangement direction of the two calibration boards is perpendicular to the shooting direction of the first fisheye camera and the shooting direction of the second fisheye camera;

[0007] calibrating the shooting device according to coordinates of preset marks on the two calibration boards in the first image, coordinates of the preset marks on the two calibration boards in the second image, initial image parameters of the first image and initial image parameters of the second image, to obtain calibration data of the shooting device.

[0008] In a second aspect, the present application provides an electronic device, comprising a shooting device, a memory and a processor; the shooting device comprising a first fisheye camera and a second fisheye camera, a shooting direction of the first fisheye camera being opposite to a shooting direction of the second fisheye camera;

[0009] The memory is configured to store a computer program.

[0010] The processor is configured to execute the computer program and implement steps of the fisheye panoramic splicing calibration method when the computer program is executed.

[0011] In a third aspect, the present application provides a computer readable storage medium, and the computer readable storage medium stores a computer program. When the computer program is executed by a processor, the steps of the fish-eye panoramic image stitching calibration method are implemented.

[0012] The present application provides a fish-eye panoramic image stitching calibration method. The fish-eye panoramic image stitching calibration method comprises the following steps: obtaining a first image captured by a first fish-eye camera and a second image captured by a second fish-eye camera, wherein the two images are captured by the two cameras from two calibration boards; the two calibration boards are symmetrically arranged on two sides of a shooting device, and the arrangement direction of the two calibration boards is perpendicular to the shooting direction of the first fish-eye camera and the shooting direction of the second fish-eye camera; and calibrating the shooting device according to the coordinates of the preset marks on the two calibration boards in the first image, the coordinates of the preset marks on the two calibration boards in the second image, the image parameters of the first image, and the image parameters of the second image, to obtain calibration data of the shooting device.

[0013] The calibration boards are symmetrically arranged on the two sides of the shooting device, and the first fish-eye camera and the second fish-eye camera are used to synchronously capture the images of the calibration boards on the two sides, such as the first image and the second image. The first image and the second image can be used to extract the coordinates of the preset marks on the calibration boards in the corresponding images and extract the initial image parameters of the corresponding images, and then the shooting device is calibrated to obtain the calibration data of the shooting device. Since the calibration can be completed by capturing the double-view images only once, the calibration efficiency of the shooting device is improved. Meanwhile, the calibration environment only needs to symmetrically place the calibration boards, and does not need to rely on complex laboratory conditions, so that the calibration difficulty of the shooting device is reduced. The generated calibration data can be directly used for the stitching of the images of the target objects captured by the first fish-eye camera and the second fish-eye camera synchronously, such as the first target image and the second target image, to obtain the corresponding panoramic image, so that the image stitching convenience and the image stitching effect of the shooting device are improved. BRIEF DESCRIPTION OF DRAWINGS

[0014] Figure 1 A flowchart of a fish-eye panoramic image stitching calibration method provided by an embodiment of the present application;

[0015] Figure 2 A calibration environment diagram of a shooting device related to an embodiment of the present application;

[0016] Figure 3 A diagram of geometric transformation processing of the first image and the second image related to an embodiment of the present application;

[0017] Figure 4 A diagram of the first image or the second image related to an embodiment of the present application;

[0018] Figure 5A schematic diagram of a first image and a second image involved in an embodiment of the present application;

[0019] Figure 6 A schematic diagram of a first image and a second image involved in an embodiment of the present application;

[0020] Figure 7 A correlation curve between a second displacement error and a horizontal offset involved in an embodiment of the present application;

[0021] Figure 8 A schematic diagram of image stitching corresponding to a first target image and a second target image involved in an embodiment of the present application;

[0022] Figure 9 A schematic diagram of first alignment processing on a first target projection image and a second target projection image involved in an embodiment of the present application;

[0023] Figure 10 A schematic diagram of second alignment processing on a first target projection image and a second target projection image involved in an embodiment of the present application;

[0024] Figure 11 A schematic block diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0025] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some of the embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of the present application.

[0026] The flowcharts shown in the drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor do they have to be executed in the order described. For example, some operations / steps can be further decomposed, combined or partially merged, so the actual execution order may be changed according to the actual situation.

[0027] The present application provides a calibration method for fisheye panoramic stitching. The calibration method for fisheye panoramic stitching can be applied to a shooting device. The shooting device can be arranged on an electronic device, which can be a security monitoring device, a vehicle-mounted system, a sports camera, a drone, etc. Some embodiments of the present application will be described in detail below with reference to the drawings. In the case of no conflict, the embodiments described below and the features in the embodiments can be combined with each other.

[0028] Please refer to Figure 1 , Figure 1is a flowchart of a calibration method of fisheye panorama stitching provided by an embodiment of the present application.

[0029] As shown in the figure, the calibration method of fisheye panorama stitching comprises steps S101 to S102. Figure 1

[0030] Illustratively, the photographing device comprises a first fisheye camera and a second fisheye camera, and the photographing direction of the first fisheye camera is opposite to that of the second fisheye camera.

[0031] As shown in the figure, the first fisheye camera and the second fisheye camera comprise two fisheye cameras arranged back to back, the photographing direction of the first fisheye camera is photographing direction 1, and the photographing direction of the second fisheye camera is photographing direction 2. The photographing direction 1 and the photographing direction 2 can be located on the same straight line and point to opposite directions, and thus the photographing direction of the first fisheye camera is opposite to that of the second fisheye camera. Figure 2

[0032] S101, acquire a first image obtained by the first fisheye camera photographing two calibration boards and a second image obtained by the second fisheye camera photographing the same two calibration boards; the two calibration boards are symmetrically arranged on the two sides of the photographing device, and the arrangement direction of the two calibration boards is perpendicular to the photographing direction of the first fisheye camera and the photographing direction of the second fisheye camera.

[0033] Illustratively, the calibration board comprises an ArUco calibration board, a chessboard calibration board, a ChArUco calibration board, etc., which is not limited herein.

[0034] As shown in the figure, Figure 2 ​​As shown, the calibration environment in which the photographing device is located includes two calibration boards, such as calibration board 1 and calibration board 2. The calibration board 1 and the calibration board 2 are symmetrically arranged on the two sides of the photographing device. For example, the calibration board 1 is located on the left side of the photographing device, and the calibration board 2 is located on the right side of the photographing device. The distance between the calibration board 1 and the photographing device is d1, and the distance between the calibration board 2 and the photographing device is d2. When the calibration environment is set, d1 can be made as equal as possible to d2. Correspondingly, in the case that the calibration board 1 and the calibration board 2 are symmetrically arranged on the two sides of the photographing device, the height of the calibration board 1 is as equal as possible to that of the calibration board 2. The arrangement direction of the calibration board 1 and the calibration board 2 can be used to indicate that the arrangement direction is from the left side of the photographing device to the right side of the photographing device, and then the arrangement direction of the calibration board 1 and the calibration board 2 can be perpendicular to the photographing direction of the first fisheye camera and the photographing direction of the second fisheye camera. In the case that d1 is not equal to d2 and / or the height of the calibration board 1 is not equal to the height of the calibration board 2, through the subsequent calibration process of the photographing device, the distance difference between the calibration board 1 and the calibration board 2 and the height difference between the calibration board 1 and the calibration board 2 can be eliminated or reduced as much as possible, and the determination accuracy of the corresponding calibration data of the photographing device is adversely affected, and then the corresponding calibration data is obtained, which is beneficial to reduce the calibration difficulty of the photographing device while improving the calibration accuracy of the photographing device.

[0035] For example, the first fisheye camera can photograph the two calibration boards in the calibration environment to obtain a first image. Correspondingly, the second fisheye camera can photograph the two calibration boards in the calibration environment to obtain a second image. The first image and the second image can include the preset marks on the two calibration boards.

[0036] The preset marks on the two calibration boards included in the first image and the second image can be used to calibrate the photographing device to obtain the calibration data of the photographing device.

[0037] S102, according to the coordinates of the preset marks on the two calibration boards in the first image, the coordinates of the preset marks on the two calibration boards in the second image, the initial image parameters of the first image and the initial image parameters of the second image, calibrating the photographing device to obtain the calibration data of the photographing device.

[0038] For example, based on a preset corner detection algorithm, the first image and the second image are respectively subjected to corner detection to obtain the coordinates of the preset marks on the two calibration boards in the first image and the coordinates of the preset marks on the two calibration boards in the second image.

[0039] The preset marks on the two calibration boards included in the first image and the second image can have corresponding corner points. Based on this, the first image and the second image are respectively subjected to corner point detection by using a preset corner point detection algorithm, to obtain the coordinates of the preset marks on the two calibration boards in the first image and the coordinates of the preset marks on the two calibration boards in the second image.

[0040] The preset marks on the two calibration boards can include one or more corner points. The one or more corner points included in the preset marks on the two calibration boards are respectively converted into a target point, and then the coordinates of the preset marks on the two calibration boards in the first image are determined according to the coordinates of the respective target points of the preset marks on the two calibration boards in the first image, and the coordinates of the preset marks on the two calibration boards in the second image are determined according to the coordinates of the respective target points of the preset marks on the two calibration boards in the second image.

[0041] As shown in Figure 3 , the first image includes image A1, and the second image includes image A2. The preset marks on the two calibration boards can correspond to regions of interest (ROI) in image A1, such as regions L1 and L2. Correspondingly, the preset marks on the two calibration boards can correspond to regions R1 and R2 in image A2. By using a corner point detection algorithm, the respective corner point coordinates of regions L1, L2, R1 and R2 are identified. Correspondingly, the coordinates of the preset marks on the two calibration boards in image A1 are determined according to the respective corner point coordinates of regions L1 and L2, and the coordinates of the preset marks on the two calibration boards in image A2 are determined according to the respective corner point coordinates of regions R1 and R2.

[0042] Taking regions L1, L2, R1 and R2 as rectangles, which can include four corner points, as an example, the corner point coordinates of the respective four corner points of regions L1, L2, R1 and R2 are averaged to obtain the coordinates of the respective region center points of regions L1, L2, R1 and R2. The respective region center points of regions L1, L2, R1 and R2 are determined as target points, and then the coordinates of the respective region center points of regions L1 and L2 are determined as the coordinates of the preset marks on the two calibration boards in the first image, and the coordinates of the respective region center points of regions R1 and R2 are determined as the coordinates of the preset marks on the two calibration boards in the second image.

[0043] Of course, this is not limited to this. For example, the minimum corner coordinates of each region can be determined from the corner coordinates of the four corners of each region L1, L2, R1, and R2. The corner points with the minimum corner coordinates of each region L1, L2, R1, and R2 are determined as target points. Then, the minimum corner coordinates of each region L1 and L2 are determined as the coordinates of the preset marks on the two calibration plates in the first image, and the minimum corner coordinates of each region R1 and R2 are determined as the coordinates of the preset marks on the two calibration plates in the second image. There are no restrictions on this.

[0044] By determining the coordinates of the preset marks on the two calibration plates in the second image based on the coordinates of the target points corresponding to the preset marks on the two calibration plates, and abstracting the preset marks on the two calibration plates into corresponding target points, the adverse effects of the shape of the preset marks on the calibration accuracy of the shooting device can be reduced when calibrating the shooting device and obtaining the calibration data of the shooting device in the subsequent process, thereby improving the calibration accuracy of the shooting device.

[0045] For example, based on a preset circle detection algorithm, circle detection is performed on the first image and the second image respectively to obtain the initial image parameters of the first image and the initial image parameters of the second image.

[0046] Since the first image was captured by a first fisheye camera and the second image was captured by a second fisheye camera, based on the optical characteristics of the fisheye cameras that both the first and second images correspond to, it can be determined that the image regions included in each of the first and second images can be circular or elliptical, such as... Figure 4 As shown in the red circle. Based on this, using a preset circle detection algorithm, circle detection is performed on the first image and the second image respectively to obtain the initial image parameters of the first image and the initial image parameters of the second image. The initial image parameters of the first image may include the initial center coordinates and the radius corresponding to the first image. The initial image parameters of the second image may include the initial center coordinates and the radius corresponding to the second image. The center of the circle corresponding to the first image and the second image is shown in the red circle. Figure 4 The green dots are shown. The preset markings on the two calibration plates included in both the first and second images are as follows... Figure 4 As shown in the blue box. Of course, it is not limited to this, and no restrictions are imposed here.

[0047] like Figure 3 As shown, the initial image parameters of image A1 may include (x l0 ,y l0 ,R l ). (x l0 ,yl0 R is used to indicate the initial center coordinates of the circle corresponding to the first image. l Used to indicate the radius corresponding to the first image. The initial image parameters for image A2 may include (x... r0 ,y r0 ,R r ). (x r0 ,y r0 R is used to indicate the initial center coordinates of the circle corresponding to the second image. r Used to indicate the radius corresponding to the second image. l0 With x r0 For example, they can also be collectively referred to as x0, y l0 With y r0 For example, they can also be collectively referred to as y0, R l With R r For example, they can also be collectively referred to as R.

[0048] The coordinates of the preset markers on the two calibration plates in the first image, the coordinates of the preset markers on the two calibration plates in the second image, the initial image parameters of the first image, and the initial image parameters of the second image can be used to determine the calibration data of the shooting device.

[0049] For example, since the shooting direction of the first fisheye camera in the shooting device is opposite to that of the second fisheye camera, and two calibration plates are symmetrically arranged on both sides of the shooting device, with the arrangement direction of the two calibration plates perpendicular to the shooting directions of both the first and second fisheye cameras, the first image and the second image can include different side regions of the same calibration plate and overlapping regions. Figure 4 As shown, the two calibration plates include calibration plate 1 and calibration plate 2. A first fisheye camera is used for forward-facing shots, and a second fisheye camera is used for rearward-facing shots. The first fisheye camera can capture the areas where calibration plates 1 and 2 are each located in front of the first fisheye camera and behind the second fisheye camera, as well as the areas where calibration plates 1 and 2 are each located to the side of the first fisheye camera and to the side of the second fisheye camera. Correspondingly, the second fisheye camera can capture the areas where calibration plates 1 and 2 are each located behind the first fisheye camera and in front of the second fisheye camera, as well as the areas where calibration plates 1 and 2 are each located to the side of the first fisheye camera and to the side of the second fisheye camera. The areas located in front of the first fisheye camera and behind the second fisheye camera can be different side regions from the areas located to the side of the first fisheye camera and in front of the second fisheye camera. The areas located to the side of the first fisheye camera and to the side of the second fisheye camera can overlap.

[0050] In the case that the camera is calibrated and the calibration data of the camera is obtained, it is necessary to ensure that the same two preset marks on the two calibration boards included in the first image and the second image can be spliced together to form the complete two calibration boards under the action of the calibration data. Since the two calibration boards are symmetrically arranged on both sides of the camera, the same two preset marks on the two calibration boards should also have symmetry on the first image and the second image.

[0051] Therefore, based on the coordinates of the same two preset marks on the two calibration boards in the first image, the coordinates of the same two preset marks on the two calibration boards in the second image, the initial image parameters of the first image, and the initial image parameters of the second image, it can be determined whether the same two preset marks on the two calibration boards have symmetry on the first image and the second image, such as determining whether the same two preset marks on the two calibration boards are vertically aligned on the same image, and determining whether the same two preset marks on the two calibration boards are horizontally aligned on different images.

[0052] As shown in Figure 3 , the first mapping matrix is used to perform geometric transformation on the image A1 to obtain the image B1. The region L3 included in the image B1 can correspond to the region L1 included in the image A1, and the region L4 included in the image B1 can correspond to the region L2 included in the image A1, which are used to indicate the preset marks on the two calibration boards.

[0053] For example, the initial image parameters of the image A1 include (x l0 ,y l0 ,R l ), the coordinates of the region L1 on the image A1 can be represented as (x a1 ,y a1 ), and the coordinates of the region L3 on the image B1 can be represented as (x L3 ,y L3 ).

[0054] The display formula corresponding to the first fisheye camera can be represented as:

[0055]

[0056] wherein atan2 represents signed arctan(y / x), and the calculation result of atan2 can cover four quadrants. Correspondingly, the first mapping matrix corresponding to the first fisheye camera can be represented as:

[0057] x L3 =f x (x a1 ,y a1 ,x l0 ,y l0 ,R l )

[0058] y L3 =f y (x a1 ,y a1 ,x l0 ,y l0 ,R l )

[0059] Among them, f x Used to indicate the mapping transformation of the horizontal coordinates (x-coordinate) using the first mapping matrix, f y This is used to indicate the mapping transformation of the vertical coordinates (y-coordinate) using the first mapping matrix. The first mapping matrix corresponding to the first fisheye camera can also be called the implicit formula corresponding to the first fisheye camera.

[0060] Similarly, when performing geometric transformations on image A2 according to the second mapping matrix to obtain image B2, the initial image parameters (x) of image A2 can also be used. r0 ,y r0 ,R r ) and the coordinates (x, y) of region R1 on image A2. b1 ,y b1 ), for the coordinates (x, y) of region R3 on image B2 R3 ,y R3 To determine.

[0061] By determining whether the preset markings on the two calibration plates are symmetrical in images B1 and B2, it can be determined whether the preset markings on the two calibration plates are symmetrical in images A1 and A2. For example, if region L3 is horizontally aligned with region R3, region L4 is horizontally aligned with region R4, region L3 is vertically aligned with region L4, and region R3 is vertically aligned with region R4, it can be determined that the preset markings on the two calibration plates are symmetrical in images B1 and B2, and thus, it can be determined that the preset markings on the two calibration plates are symmetrical in images A1 and A2.

[0062] If the preset markers on the two identical calibration plates are not symmetrical in the first and second images, it can be determined that the imaging device has not completed calibration. Based on this, at least one of the following can be adjusted: the coordinates of the preset markers on the two calibration plates in the first image, the coordinates of the preset markers on the two calibration plates in the second image, the initial image parameters of the first image, and the initial image parameters of the second image, until the preset markers on the two identical calibration plates are symmetrical in the first and second images.

[0063] The first mapping matrix corresponding to the first fisheye camera and the second mapping matrix corresponding to the second fisheye camera can be used to determine the coordinates of the preset marks on the two calibration boards in the first projection image, which will change with the change of the initial image parameters of the first image. The coordinates of the preset marks on the two calibration boards in the second projection image will change with the change of the initial image parameters of the second image. Therefore, by adjusting one of the initial image parameters of the first image and the initial image parameters of the second image, one of the first mapping matrix and the second mapping matrix can be adjusted, so that the same preset marks on the two calibration boards have symmetry in the first projection image and the second projection image, and the target mapping matrix corresponding to the first fisheye camera and the second fisheye camera is obtained, and the calibration data of the photographing device is determined, that is, the calibration of the photographing device is completed.

[0064] The calibration boards symmetrically arranged on both sides of the photographing device are used to synchronously collect the images of the calibration objects on both sides by the first fisheye camera and the second fisheye camera, such as the first image and the second image. The first image and the second image can be used to extract the coordinates of the preset marks on the calibration boards in the corresponding images and extract the initial image parameters of the corresponding images, and then calibrate the photographing device to obtain the calibration data of the photographing device. Since the calibration can be completed by collecting the double-view images only once, the calibration efficiency of the photographing device is improved. Meanwhile, the calibration environment only needs to symmetrically place the calibration boards, without relying on complex laboratory conditions, so as to reduce the calibration difficulty of the photographing device. The generated calibration data can be directly used for the stitching of the target images, such as the first target image and the second target image, which are synchronously collected by the first fisheye camera and the second fisheye camera, to obtain the corresponding panoramic image, so as to improve the image stitching convenience of the photographing device.

[0065] In some embodiments, one of the first mapping matrix corresponding to the first fisheye camera and the second mapping matrix corresponding to the second fisheye camera is adjusted according to the coordinates of the preset marks on the two calibration boards in the first image, the coordinates of the preset marks on the two calibration boards in the second image, the initial image parameters of the first image and the initial image parameters of the second image, to obtain the target mapping matrix corresponding to the first fisheye camera and the second fisheye camera respectively; and the calibration data of the photographing device is determined according to the target mapping matrix corresponding to the first fisheye camera and the second fisheye camera respectively.

[0066] For example, whether the same preset marks on the two calibration boards have symmetry in the first image and the second image can be evaluated according to the coordinates of the preset marks on the two calibration boards in the first image, the coordinates of the preset marks on the two calibration boards in the second image, the initial image parameters of the first image and the initial image parameters of the second image.

[0067] In a case that the preset marks on the same two calibration boards do not have symmetry on the first image and the second image, one of the first mapping matrix corresponding to the first fisheye camera and the second mapping matrix corresponding to the second fisheye camera can be adjusted until the preset marks on the same two calibration boards have symmetry on the first image and the second image, so as to determine the target mapping matrix corresponding to the first fisheye camera and the second fisheye camera respectively, and obtain the calibration data of the photographing apparatus.

[0068] Taking the adjustment of the first mapping matrix as an example. In the process of adjusting the first mapping matrix, the first projection image corresponding to the first image obtained by performing the geometric transformation on the first image by using the adjusted first mapping matrix will change. The second projection image corresponding to the second image obtained by performing the geometric transformation on the second image by using the unadjusted second mapping matrix will not change. By taking the unchanging second projection image as a reference image, whether the preset marks on the same two calibration boards have symmetry on the changed first projection image and the unchanging second projection image can be evaluated. In a case that the preset marks on the same two calibration boards have symmetry on the changed first projection image and the unchanging second projection image, the adjusted first mapping matrix is determined as the target mapping matrix of the first fisheye camera, and the unadjusted second mapping matrix is determined as the target mapping matrix of the second fisheye camera. Of course, it is not limited to this, and is not limited herein.

[0069] In a case that the target mapping matrix corresponding to the first fisheye camera and the target mapping matrix corresponding to the second fisheye camera are determined, the target mapping matrix corresponding to the first fisheye camera and the target mapping matrix corresponding to the second fisheye camera can be used to transform the image obtained by the photographing apparatus by photographing the same object into a corresponding projection image. The same object has symmetry on the corresponding projection image, which is beneficial to improve the convenience of subsequent splicing of the image photographed by the photographing apparatus. The target mapping matrix corresponding to the first fisheye camera and the target mapping matrix corresponding to the second fisheye camera can be used by the electronic device to splice the image obtained by the photographing apparatus by photographing the same object, and then the electronic device can determine the target mapping matrix corresponding to the first fisheye camera and the target mapping matrix corresponding to the second fisheye camera as the calibration data of the photographing apparatus.

[0070] In some implementations, a geometric transformation is performed on the first image according to a first mapping matrix to obtain a first projected image corresponding to the first image; a geometric transformation is performed on the second image according to a second mapping matrix to obtain a second projected image corresponding to the second image; a first distance between the preset markers on the two calibration plates on the first projected image is determined based on the coordinates of the preset markers on the two calibration plates in the first image and the initial image parameters of the first image; a second distance between the preset markers on the two calibration plates on the second projected image is determined based on the coordinates of the preset markers on the two calibration plates in the second image and the initial image parameters of the second image; and one of the first mapping matrix and the second mapping matrix is ​​adjusted based on the first distance and the second distance to obtain the target mapping matrix corresponding to the first fisheye camera and the second fisheye camera respectively.

[0071] Since the first and second distances are determined based on the first and second projected images (the first and second projected images being determined based on the first image and the second projected image respectively), it can be determined that the first distance is determined based on the coordinates of the preset markers on the two calibration plates in the camera coordinate system corresponding to the first fisheye camera, and the second distance is determined based on the coordinates of the preset markers on the two calibration plates in the camera coordinate system corresponding to the second fisheye camera. The coordinates of the preset markers on the calibration plates in the corresponding camera coordinate system can include coordinates in the horizontal direction (x-coordinate) and coordinates in the vertical direction (y-coordinate). The x-coordinate can be used to determine the horizontal distance between different preset markers, and the y-coordinate can be used to determine the vertical distance between different preset markers.

[0072] like Figure 3 As shown, the first image includes image A1, and the second image includes image A2. Based on the first mapping matrix, image A1 undergoes a geometric transformation to obtain a first projected image corresponding to image A1. The first projected image includes image B1. Based on the second mapping matrix, image A2 undergoes a geometric transformation to obtain a second projected image corresponding to image A2. The second projected image includes image B2.

[0073] Taking two markers, marker 1 and marker 2, as an example, the preset markers on marker 1 can correspond to regions L1 of image A1, R1 of image A2, L3 of image B1, and R3 of image B2, respectively. The preset markers on marker 2 can correspond to regions L2 of image A1, R2 of image A2, L4 of image B1, and R4 of image B2, respectively.

[0074] According to the coordinates of the region L1 in the image A1 and the initial image parameters of the image A1, the coordinates of the region L3 on the image B1 are determined. According to the coordinates of the region L2 in the image A1 and the initial image parameters of the image A1, the coordinates of the region L4 on the image B1 are determined. According to the coordinates of the region L3 and the region L4 respectively on the image B1, the first vertical distance Δ1 and the first horizontal distance Δ3 between the region L3 and the region L4 are determined. Based on this, according to the first vertical distance Δ1 and the first horizontal distance Δ3, the first distance D1 can be determined.

[0075] Similarly, according to the coordinates of the region R1 in the image A2 and the initial image parameters of the image A2, the coordinates of the region R3 on the image B2 are determined. According to the coordinates of the region R2 in the image B2 and the initial image parameters of the image A2, the coordinates of the region R4 on the image B2 are determined, and then according to the coordinates of the region R3 and the region R4 respectively on the image B2, the second vertical distance Δ2 and the second horizontal distance Δ4 between the region R3 and the region R4 are determined, so as to determine the second distance D2.

[0076] In the case of determining the first distance D1 and the second distance D2, whether the preset marks on the same two calibration boards are vertically aligned on the first projection image or the second projection image, and whether the preset marks on the same two calibration boards are horizontally aligned on the first projection image and the second projection image can be evaluated according to the first distance D1 and the second distance D2, and then whether the preset marks on the same two calibration boards have symmetry on the first image and the second image can be determined.

[0077] For example, in the case that the first distance D1 is equal to the second distance D2, such as the first vertical distance Δ1 is equal to the second vertical distance Δ2, and the first horizontal distance Δ3 is equal to the second horizontal distance Δ4, according to the first mapping matrix corresponding to the first fisheye camera and the second mapping matrix corresponding to the second fisheye camera, the target mapping matrix corresponding to each of the first fisheye camera and the second fisheye camera can be determined.

[0078] For another example, in the case that the first distance D1 is not equal to the second distance D2, such as the first vertical distance Δ1 is not equal to the second vertical distance Δ2, and / or the first horizontal distance Δ3 is not equal to the second horizontal distance Δ4, one of the first mapping matrix and the second mapping matrix is adjusted according to the first distance D1 and the second distance D2 until the first distance D1 is equal to the second distance D2. In the case that the first distance D1 is equal to the second distance D2, the target mapping matrix corresponding to each of the first fisheye camera and the second fisheye camera can be determined.

[0079] In some implementations, the vertical offset corresponding to the first vertical distance and the second vertical distance is determined; the horizontal offset corresponding to the first horizontal distance and the second horizontal distance is determined; and one of the first mapping matrix and the second mapping matrix is ​​adjusted according to the vertical offset and the horizontal offset to obtain the target mapping matrix corresponding to the first fisheye camera and the second fisheye camera respectively.

[0080] Since the first distance and the second distance are determined based on the coordinates of the preset markings on the calibration board in the corresponding camera coordinate system, the vertical offset and the horizontal offset are equivalent to the offsets in the corresponding camera coordinate system.

[0081] like Figure 3 As shown, the first image includes image A1, the second image includes image A2, the first projected image includes image B1, and the second projected image includes image B2.

[0082] Taking two markers, marker 1 and marker 2, as an example, the coordinates of region L1 on image A1 can be represented as (x... a1 ,y a1 The coordinates of region L2 on image A1 can be represented as (x... a2 ,y a2 The coordinates of region L3 on image B1 can be represented as (x... L3 ,y L3 The coordinates of region L4 on image B1 can be represented as (x... L4 ,y L4 The coordinates of region R1 on image A2 can be represented as (x... b1 ,y b1 The coordinates of region R2 on image A2 can be represented as (x... b2 ,y b2 The coordinates of region R3 on image B2 can be represented as (x... R3 ,y R3 The coordinates of region R4 on image B2 can be represented as (x... R4 ,y R4 ).

[0083] The first vertical distance Δ1 between the preset marks on calibration plate 1 and calibration plate 2 on image B1 can be expressed as:

[0084]

[0085] The second vertical distance Δ2 between the preset marks on calibration plate 1 and calibration plate 2 on image B2 can be expressed as:

[0086]

[0087] The first horizontal distance Δ3 of the preset marks on the calibration plate 1 and the calibration plate 2 on the image B1 can be represented as:

[0088] The second horizontal distance Δ4 of the preset marks on the calibration plate 1 and the calibration plate 2 on the image B2 can be represented as:

[0089] In the process of adjusting one of the first mapping matrix and the second mapping matrix, the optimization solving target can be set as: making y L3 = y R3 , y L4 = y R4 , x L3 = x L4 , x R3 = x R4 , which is equivalent to Δ1 = Δ2 and Δ3 = Δ4, that is, the calibration plate 1 is horizontally aligned on the image B1 and the image B2, the calibration plate 2 is horizontally aligned on the image B1 and the image B2, the calibration plate 1 and the calibration plate 2 are vertically aligned on the image B1, and the calibration plate 1 and the calibration plate 2 are vertically aligned on the image B2.

[0090] Taking adjusting the second mapping matrix as an example.

[0091] The vertical offset corresponding to the first vertical distance and the second vertical distance can be represented as Δy. The vertical offset Δy can be understood as adjusting the initial image parameter y r0 of the image A2 according to the vertical offset Δy. According to the vertical offset Δy, the adjustment of the second mapping matrix can be represented as:

[0092]

[0093] The horizontal offset corresponding to the first horizontal distance and the second horizontal distance can be represented as Δx. The horizontal offset Δx can be understood as adjusting the initial image parameter x r0 of the image A2 according to the horizontal offset Δx. According to the horizontal offset Δx, the adjustment of the second mapping matrix can be represented as:

[0094]

[0095] Correspondingly, the optimization solving target is transformed into: Δ2 * (Δy) = Δ1, Δ4 * (Δx) = Δ2.

[0096] Based on this, the vertical offset Δy and the horizontal offset Δx can be determined by using numerical calculation in combination with the above formula.

[0097] Having calculated the vertical offset Δy and the horizontal offset Δx, the initial image parameters y of image A2 can be adjusted according to the vertical offset Δy. r0 Adjustments are made, and the initial image parameters x of image A2 are adjusted according to the horizontal offset Δx. r0 Adjustments are made to adjust the second mapping matrix corresponding to the second fisheye camera. Under the influence of the vertical offset Δy and the horizontal offset Δx, the optimization objective can be achieved, that is, to obtain y... L3 =y R3 y L4 =y R4 x L3 =x L4 x R3 =x R4 This is equivalent to making Δ1 = Δ2 and Δ3 = Δ4, which completes the calibration of the shooting device. That is, the second mapping matrix, adjusted according to the vertical offset Δy and the horizontal offset Δx, can be determined as the target mapping matrix corresponding to the second fisheye camera, and the first mapping matrix can be determined as the target mapping matrix corresponding to the first fisheye camera.

[0098] Of course, it is not limited to this. For example, the first mapping matrix corresponding to the first fisheye camera can be adjusted according to the vertical offset Δy and the horizontal offset Δx to obtain the target mapping matrix corresponding to the first fisheye camera, and the second mapping matrix corresponding to the second fisheye camera can be determined as the target mapping matrix corresponding to the second fisheye camera. There are no restrictions here.

[0099] Based on this, according to the coordinates of the preset marks on the two calibration plates in the first image, the coordinates of the preset marks on the two calibration plates in the second image, the initial image parameters of the first image, and the initial image parameters of the second image, the target mapping matrix corresponding to the first fisheye camera and the second fisheye camera can be determined, thereby determining the calibration data of the shooting device without needing to obtain a lot of data, which is conducive to improving the convenience of calibrating the shooting device.

[0100] In some implementations, a first target image is obtained by capturing the target object with a first fisheye camera, and a second target image is obtained by capturing the same target object with a second fisheye camera; the first target image is subjected to geometric transformation processing according to the target mapping matrix corresponding to the first fisheye camera to obtain a first target projection image corresponding to the first target image; the second target image is subjected to geometric transformation processing according to the target mapping matrix corresponding to the second fisheye camera to obtain a second target projection image corresponding to the second target image; the first target projection image and the second target projection image are subjected to image stitching processing to obtain a target panoramic image corresponding to the first target projection image and the second target projection image.

[0101] Once the calibration data of the shooting device is determined, i.e., the calibration of the first and second fisheye cameras is completed, image stitching processing can be performed on the images captured by the shooting device based on the calibration data to determine the corresponding panoramic image. For example, based on the target mapping matrix corresponding to the first fisheye camera and the target mapping matrix corresponding to the second fisheye camera, image stitching processing can be performed on the first target image captured by the first fisheye camera and the second target image captured by the second fisheye camera to obtain the corresponding target panoramic image.

[0102] Taking calibration plate 1 and calibration plate 2 as examples, the first target image may include image A1, and the second target image may include image A2. The first target projection image may include image B1 corresponding to image A1. The second target projection image may include image B2 corresponding to image A2.

[0103] like Figure 5 As shown, before the imaging device is calibrated, it is easy for the vertical distance of region L3 in image B1 to be unequal to the vertical distance of region R3 in image B2, and / or the vertical distance of region L4 in image B1 to be unequal to the vertical distance of region R4 in image B2. After the imaging device is calibrated, under the action of the target mapping matrix corresponding to the first fisheye camera and the target mapping matrix corresponding to the second fisheye camera, the first vertical distance of region L3 and region L4 in image B1 can be made equal to the second vertical distance of region R3 and region R4 in image B2. This allows region L3 and region R3 to have the ability to be horizontally aligned, and region L4 and region R4 can also have the ability to be horizontally aligned. Consequently, the vertical distance of region L3 in image B1 to be equal to the vertical distance of region R3 in image B2, and the vertical distance of region L4 in image B1 to be equal to the vertical distance of region R4 in image B2.

[0104] For example, if the target mapping matrix corresponding to the second fisheye camera is obtained by adjusting the second mapping matrix, and if image B2 corresponding to image A2 is determined based on the target mapping matrix corresponding to the second fisheye camera, then regions R3 and R4 in image B2 have a first displacement error in the vertical direction. Figure 5 As shown. The first displacement error can be represented as error. x =2Δy(y a1 -y a2 ), where Δy can be used to indicate the initial image parameters y of image A2 according to the vertical deviation Δy. r0 Make adjustments, y a1 With y a2The first displacement error can be determined according to the placing heights of the two calibration boards when the second fisheye camera captures the second target image. Based on this, if the heights of the two calibration boards are consistent during the calibration of the photographing device, the first displacement error can be 0. Correspondingly, if there is a height deviation between the two calibration boards during the calibration of the photographing device, the first target projection image and the second target projection image can be aligned, such as vertical alignment, to eliminate the deviation.

[0105] As shown in Figure 6 , before the photographing device is calibrated, the horizontal distance of the region L3 corresponding to the image B1 can be different from the horizontal distance of the region R3 corresponding to the image B2, and / or the horizontal distance of the region L4 corresponding to the image B1 can be different from the horizontal distance of the region R4 corresponding to the image B2. After the photographing device is calibrated, under the action of the target mapping matrix corresponding to the first fisheye camera and the target mapping matrix corresponding to the second fisheye camera, the first vertical distance of the region L3 and the region L4 in the image B1 can be equal to the second vertical distance between the region R3 and the region R4, so that the region L3 and the region L4 can have the ability of vertical alignment, and the region R3 and the region R4 can have the ability of vertical alignment, thereby the horizontal distance of the region L3 corresponding to the image B1 can be equal to the horizontal distance of the region R3 corresponding to the image B2, and the horizontal distance of the region L4 corresponding to the image B1 can be equal to the horizontal distance of the region R4 corresponding to the image B2.

[0106] For example, in the case where the target mapping matrix corresponding to the second fisheye camera is obtained by adjusting the second mapping matrix, if the image B2 corresponding to the image A2 is determined according to the target mapping matrix corresponding to the second fisheye camera, the region R3 and the region R4 in the image B2 have a second displacement error in the horizontal direction, as shown in Figure 6 . The second displacement error can be represented as error y = newy b1 -y b1 -newy b2 +y b2 . Wherein, newy b1 , newy b2 may be used to indicate the vertical coordinates of the region R1 and the region R2 respectively after adjusting the initial image parameter x r0 of the image A2 according to the horizontal deviation amount Δx, y b1 , y b1 is used to indicate the vertical coordinates of the region R1 and the region R2 respectively before adjusting the initial image parameter x r0 of the image A2 according to the horizontal deviation amount Δx.

[0107] Through simulation experiments, the second displacement error and the initial image parameter xr0 The correlation curve between the corresponding horizontal offset Δx is as shown in Figure 7 . Wherein, x in the correlation curve is used to indicate the horizontal offset Δx, and y is used to indicate the second displacement error.

[0108] It can be known from Figure 7 that in the actual splicing image (input: 2 images of 2048*2048, output: one image of 4096*2048), the circle detection radius size is 1000 pixels, and since the center coordinate x r0 of the circle in the actual adjustment will not exceed 20 pixels, the second displacement error generated will not exceed 1 pixel. In the simulation formula corresponding to the correlation curve, 0.1 is the difference ratio between the coordinates of the preset mark on the second image and the initial image parameters of the second image, such as the difference ratio between the coordinates (x b1 ,y b1 ) of the region R1 on the image A2 and the initial image parameters (x r0 ,y r0 ,R r ) of the image A2. In the simulation formula corresponding to the correlation curve, 950 is determined according to the x coordinate difference between the coordinates of the preset mark on the second image and the initial image parameters of the second image, such as the x coordinate of the region R1 on the image A2 is x b1 , and the x coordinate included in the initial image parameters of the image A2 is x r0 , so the x coordinate difference can be expressed as x b1 -x r0 .

[0109] Of course, the circle detection radius size, the difference ratio between the coordinates of the preset mark on the second image and the initial image parameters of the second image, and the x coordinate difference between the coordinates of the preset mark on the second image and the initial image parameters of the second image are usually taken as values, and can also be other values, which are only examples and are not limited herein.

[0110] Therefore, the second displacement error can be considered as not affecting the final calibration result of the photographing device, and further can be considered as not adversely affecting the image splicing accuracy of the image photographed by the photographing device.

[0111] Based on this, after the photographing device completes the calibration, the calibration data of the photographing device can be used for image splicing processing of the image B1 and the image B2 to obtain the target panorama corresponding to the image B1 and the image B2. Of course, the first target projection image is not limited to the image B1, and the second target projection image is not limited to the image B2, which is not limited herein.

[0112] As shown in Figure 8As shown, the process of image stitching processing on the first target projection image and the second target projection image can include: acquiring the first target image and the second target image; determining the corresponding first target projection image according to the first target image; determining the corresponding second target projection image according to the second target image; identifying the first stitching fusion region in the first target projection image and the second stitching fusion region in the second target projection image; performing stitching fusion processing on the first stitching fusion region and the second stitching fusion region to obtain a preset panoramic image corresponding to the first target projection image and the second target projection image; and performing geometric transformation processing on the preset panoramic image to obtain a target panoramic image corresponding to the preset panoramic image.

[0113] The target panoramic image is a target panoramic image corresponding to the first target projection image and the second target projection image. The preset panoramic image includes a stitching fusion region corresponding to the first stitching fusion region and the second stitching fusion region. The target panoramic image includes the processed stitching fusion region.

[0114] Taking the first target projection image including the image B1 and the second target projection image including the image B2 as an example. The first stitching fusion region includes the region L3 and the region L4, and the second stitching fusion region includes the region R3 and the region R4. Under the action of the target mapping matrix corresponding to each of the first fisheye camera and the second fisheye camera, the region L3 can be horizontally aligned with the region R3, the region L3 can be vertically aligned with the region L4, the region L4 can be horizontally aligned with the region R4, and the region R3 can be vertically aligned with the region R4. In this case, the first stitching fusion region and the second stitching fusion region can be stitched and fused according to the symmetry between the region L3 and the region R3 and the symmetry between the region L4 and the region R4, and then the target panoramic image corresponding to the first target projection image and the second target projection image is determined.

[0115] Based on this, in the case of determining the calibration data of the shooting device, such as the target mapping matrix corresponding to each of the first fisheye camera and the second fisheye camera, the target mapping matrix can be used for image stitching processing on the first target image and the second target image, which is beneficial to improve the image stitching convenience and accuracy of the image shot by the shooting device.

[0116] In some embodiments, the first target projection image and the second target projection image are subjected to image alignment processing to obtain a processed first target projection image and a processed second target projection image.

[0117] The processed first target projection image and the processed second target projection image are subjected to image stitching processing to obtain a corresponding target panoramic image.

[0118] For example, after determining the first target projection map and the second target projection map, the first target projection map and the second target projection map can be aligned to obtain the processed first target projection map and the processed second target projection map.

[0119] Alignment processing may include a first alignment process and a second alignment process.

[0120] Taking a target object including calibration plate 1 and calibration plate 2 as an example, the first target image may include image A1, and the second target image may include image A2. The first target projection image may include image B1 corresponding to image A1.

[0121] The second target projection image may include image B2 corresponding to image A2.

[0122] like Figure 9 As shown, when the first vertical distance Δ1 between regions L3 and L4 in image B1 is equal to the second vertical distance Δ2 between regions R3 and R4 in image B2, the y-coordinate of one of images B1 and B2 can be adjusted to perform a first alignment process on images B1 and B2, so that regions L3 in image B1 and R3 in image B2 can be horizontally aligned, and regions L4 in image B1 and R4 in image B2 can be horizontally aligned.

[0123] like Figure 10 As shown, when the first horizontal distance Δ3 between regions L3 and L4 in image B1 is equal to the second horizontal distance Δ4 between regions R3 and R4, the x-coordinate of one of images B1 and B2 can be adjusted to perform a second alignment process on images B1 and B2, so that region L3 in image B1 can be vertically aligned with region L4, region R3 in image B2 can be vertically aligned with region R4, and the horizontal distances of regions L3, L4, R3, and R4 in their respective images are equal.

[0124] Given processed images B1 and B2, image stitching can be performed on them to obtain the corresponding target panoramic image.

[0125] Of course, it is not limited to this, for example, in the process of determining the calibration data of the shooting device, such as the target mapping matrix corresponding to the first fisheye camera and the target mapping matrix corresponding to the second fisheye camera, the first projection image corresponding to the first image and the second projection image can also be aligned to obtain the corresponding first alignment offset and the second alignment offset. Correspondingly, one of the first mapping matrix corresponding to the first fisheye camera and the second mapping matrix corresponding to the second fisheye camera can be adjusted in combination with the first alignment offset, the second alignment offset, the first vertical offset and the horizontal offset, to obtain the target mapping matrix corresponding to the first fisheye camera and the second fisheye camera respectively. The target mapping matrix can be used for subsequent image stitching processing of the target images obtained by the first fisheye camera and the second fisheye camera synchronously capturing the target object, to obtain the corresponding panoramic image without the need for further alignment processing.

[0126] Based on this, in the case that the electronic device performs image alignment processing on the first target projection image and the second target projection image, and performs image stitching processing on the processed first target projection image and the processed second target projection image to obtain the corresponding target panoramic image, the first target projection image and the second target projection image before stitching can be aligned, which is beneficial to improve the image stitching convenience and accuracy of the image captured by the shooting device, and further beneficial to improve the panoramic image quality of the target panoramic image.

[0127] The calibration method for fisheye panoramic stitching provided in the above embodiment includes: obtaining a first image obtained by a first fisheye camera shooting two calibration boards, and a second image obtained by a second fisheye camera shooting the same two calibration boards; the two calibration boards are symmetrically arranged on both sides of the shooting device, and the arrangement direction of the two calibration boards is perpendicular to the shooting direction of the first fisheye camera and the shooting direction of the second fisheye camera; according to the coordinates of the preset marks on the two calibration boards in the first image, the coordinates of the preset marks on the two calibration boards in the second image, the initial image parameters of the first image and the initial image parameters of the second image, the shooting device is calibrated to obtain the calibration data of the shooting device.

[0128] The first fisheye camera and the second fisheye camera are used to synchronously collect images of the two calibration objects, such as a first image and a second image. The first image and the second image can be used to extract coordinates of preset marks on the calibration objects in the corresponding images and extract initial image parameters of the corresponding images, and then calibrate the photographing device to obtain calibration data of the photographing device. Since the calibration can be completed by collecting the double-view images only once, the calibration efficiency of the photographing device is improved. Meanwhile, the calibration environment only needs to symmetrically place the calibration objects, and does not need to rely on complex laboratory conditions, so that the calibration difficulty of the photographing device is reduced. The generated calibration data can be directly used for stitching of a target object image, such as a first target image and a second target image, collected by the first fisheye camera and the second fisheye camera synchronously, to obtain a corresponding panoramic image, so that the image stitching convenience of the photographing device is improved.

[0129] The embodiment of the present application further provides a schematic block diagram of an eye panoramic stitching calibration module. The eye panoramic stitching calibration module can be configured in a server or an electronic device, and is used for executing the eye panoramic stitching calibration method.

[0130] The eye panoramic stitching calibration module comprises an image acquisition module and a calibration module.

[0131] The image acquisition module is used to acquire a first image obtained by the first fisheye camera photographing two calibration objects and a second image obtained by the second fisheye camera photographing the same two calibration objects. The two calibration objects are symmetrically arranged on two sides of the photographing device, and the arrangement direction of the two calibration objects is perpendicular to the photographing direction of the first fisheye camera and the photographing direction of the second fisheye camera.

[0132] The calibration module is used to determine calibration data of the photographing device according to coordinates of preset marks on the two calibration objects in the first image, coordinates of the preset marks on the two calibration objects in the second image, initial image parameters of the first image, and initial image parameters of the second image.

[0133] It should be explained that, for the convenience and brevity of description, the specific working process of the device, the modules and the units described above can refer to the corresponding process in the foregoing method embodiments, and will not be described herein.

[0134] The methodologies of the present application can be employed in a variety of computer system contexts. For example, personal computers, server computers, hand-held or laptop devices, tablet devices, multiprocessor systems, microprocessor-based systems, set top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, and the like. The present application can be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, and the like, that perform particular tasks or implement particular abstract data types. The present application can also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules can be located in both local and remote computer storage media including memory storage devices.

[0135] By way of example, the above-described methods, devices can be implemented as a computer program that can run on an electronic device.

[0136] Referring to Figure 11 , Figure 11 is a structural schematic block diagram of an electronic device provided by an embodiment of the present application. The electronic device can be a server or an electronic device.

[0137] As Figure 11 shown, the electronic device includes a photographing device, a processor, a memory, and a network interface connected through a system bus, wherein the memory can include a storage medium and an internal memory. The photographing device includes a first fisheye camera and a second fisheye camera, and a photographing direction of the first fisheye camera is opposite to a photographing direction of the second fisheye camera.

[0138] The storage medium can store an operating system and a computer program. The computer program includes program instructions, which, when executed, can cause the processor to perform the steps of any one of the calibration methods for fisheye panorama stitching.

[0139] The processor is configured to provide computing and control capabilities to support the operation of the entire electronic device.

[0140] The internal memory provides an environment for the execution of the computer program in the storage medium, and the computer program, when executed by the processor, can cause the processor to perform the steps of any one of the calibration methods for fisheye panorama stitching.

[0141] The network interface is configured to perform network communication, such as sending assigned tasks.

[0142] Those skilled in the art can understand that Figure 11It should be noted that the structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the electronic device to which the scheme of the present application is applied. The specific electronic device can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0143] It should be understood that the processor can be a central processing unit (CPU), and the processor can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0144] In one embodiment, the processor is configured to execute computer programs and implement the following steps when executing the computer programs:

[0145] obtaining a first image obtained by the first fisheye camera capturing two calibration boards and a second image obtained by the second fisheye camera capturing the same two calibration boards; the two calibration boards are symmetrically arranged on two sides of the photographing device, and the arrangement direction of the two calibration boards is perpendicular to the photographing direction of the first fisheye camera and the photographing direction of the second fisheye camera;

[0146] According to the coordinates of the preset marks on the two calibration boards in the first image, the coordinates of the preset marks on the two calibration boards in the second image, the initial image parameters of the first image and the initial image parameters of the second image, the calibration data of the photographing device is determined.

[0147] It should be noted that those skilled in the art can clearly understand that, for the convenience and brevity of description, the above description of the specific working process of the calibration of the photographing device can refer to the corresponding process in the calibration method for fisheye panoramic stitching described above, and will not be repeated here.

[0148] The embodiments of the present application also provide a computer readable storage medium, and the computer readable storage medium stores a computer program. The method implemented by the computer program executed by the processor can refer to each embodiment of the calibration method for fisheye panoramic stitching.

[0149] The computer readable storage medium can be an internal storage unit of the electronic device, such as a hard disk or a memory of the electronic device. The computer readable storage medium can also be an external storage device of the electronic device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, and the like.

[0150] It should be understood that the terms used herein are merely for the purpose of describing particular embodiments and are not intended to limit the present application. As used in the specification and the appended claims, the singular forms "a," "an" and "the" are intended to include plural forms as well, unless the context clearly indicates otherwise.

[0151] It should also be understood that the term "and / or" as used herein refers to any combination of associated terms, as well as all possible combinations, and includes these combinations. It should be noted that the terms "comprise", "comprising", or any other variant thereof are intended to encompass non-exclusive inclusion, so that processes, methods, articles or systems including a series of elements include not only those elements, but also other elements not explicitly listed, or other elements inherent to such processes, methods, articles or systems. Without more limitations, the element defined by the phrase "comprising a" does not exclude the presence of additional identical elements in the process, method, article or system including the element.

[0152] The above-mentioned sequence numbers of the embodiments of the present application are only for description, and do not represent the advantages or disadvantages of the embodiments. The above description is merely a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A calibration method for fish-eye panoramic stitching, characterized in that, The application is applied to a shooting device, the shooting device comprising a first fisheye camera and a second fisheye camera, the shooting direction of the first fisheye camera being opposite to the shooting direction of the second fisheye camera; The calibration method comprises: obtaining a first image obtained by the first fisheye camera shooting two calibration boards and a second image obtained by the second fisheye camera shooting the same two calibration boards; the two calibration boards are symmetrically arranged on two sides of the shooting device, and the arrangement direction of the two calibration boards is perpendicular to the shooting direction of the first fisheye camera and the shooting direction of the second fisheye camera; calibrating the shooting device according to the coordinates of the preset marks on the two calibration boards in the first image, the coordinates of the preset marks on the two calibration boards in the second image, the initial image parameters of the first image and the initial image parameters of the second image, to obtain calibration data of the shooting device.

2. The calibration method of claim 1, wherein, The calibration method comprises: adjusting one of the first mapping matrix corresponding to the first fisheye camera and the second mapping matrix corresponding to the second fisheye camera according to the coordinates of the preset marks on the two calibration boards in the first image, the coordinates of the preset marks on the two calibration boards in the second image, the initial image parameters of the first image and the initial image parameters of the second image, to obtain the target mapping matrix corresponding to the first fisheye camera and the second fisheye camera respectively; determining the calibration data of the shooting device according to the target mapping matrix corresponding to the first fisheye camera and the second fisheye camera respectively.

3. The calibration method of claim 2, wherein, The calibration method comprises: adjusting one of the first mapping matrix corresponding to the first fisheye camera and the second mapping matrix corresponding to the second fisheye camera according to the coordinates of the preset marks on the two calibration boards in the first image, the coordinates of the preset marks on the two calibration boards in the second image, the initial image parameters of the first image and the initial image parameters of the second image, to obtain the target mapping matrix corresponding to the first fisheye camera and the second fisheye camera respectively, comprising: performing geometric transformation processing on the first image according to the first mapping matrix, to obtain a first projection image corresponding to the first image; performing geometric transformation processing on the second image according to the second mapping matrix, to obtain a second projection image corresponding to the second image; determining a first distance of the preset marks on the two calibration boards in the first projection image according to the coordinates of the preset marks on the two calibration boards in the first image and the image parameters of the first image; According to the coordinates of the preset marks on the two calibration boards in the second image and the image parameters of the second image, a second distance of the preset marks on the two calibration boards in the second projection image is determined; According to the first distance and the second distance, one of the first mapping matrix and the second mapping matrix is adjusted to obtain a target mapping matrix corresponding to the first fisheye camera and the second fisheye camera respectively.

4. The calibration method of claim 3, wherein The first distance includes a first vertical distance and a first horizontal distance of the preset marks on the two calibration boards in the first projection image, and the second distance includes a second vertical distance and a second horizontal distance of the preset marks on the two calibration boards in the second projection image; According to the first distance and the second distance, one of the first mapping matrix and the second mapping matrix is adjusted to obtain a target mapping matrix corresponding to the first fisheye camera and the second fisheye camera respectively, including: A vertical offset corresponding to the first vertical distance and the second vertical distance is determined; A horizontal offset corresponding to the first horizontal distance and the second horizontal distance is determined; According to the vertical offset and the horizontal offset, one of the first mapping matrix and the second mapping matrix is adjusted to obtain a target mapping matrix corresponding to the first fisheye camera and the second fisheye camera respectively.

5. The calibration method according to any one of claims 1 to 4, characterized in that, Further comprising: A first target image obtained by the first fisheye camera shooting a target object and a second target image obtained by the second fisheye camera shooting the same target object are acquired; According to the target mapping matrix corresponding to the first fisheye camera, the first target image is geometrically transformed to obtain a first target projection corresponding to the first target image; According to the target mapping matrix corresponding to the second fisheye camera, the second target image is geometrically transformed to obtain a second target projection corresponding to the second target image; The first target projection and the second target projection are image-stitched to obtain a target panorama corresponding to the first target projection and the second target projection.

6. The calibration method of claim 5, wherein, Further comprising: The first target projection and the second target projection are image-aligned to obtain a processed first target projection and a processed second target projection; The first target projection and the second target projection are image-stitched to obtain a target panorama corresponding to the first target projection and the second target projection, including: The processed first target projection and the processed second target projection are image-stitched to obtain a corresponding target panorama.

7. The calibration method according to any one of claims 1 to 4, characterized in that, Further comprising: Based on a preset corner point detection algorithm, corner points of the first image and the second image are detected respectively to obtain the coordinates of the preset marks on the two calibration boards in the first image and the coordinates of the preset marks on the two calibration boards in the second image.

8. The calibration method according to any one of claims 1 to 4, characterized in that, Further comprising: Based on a preset circle detection algorithm, the first image and the second image are subjected to circle detection respectively to obtain initial image parameters of the first image and initial image parameters of the second image.

9. An electronic device, comprising: The electronic device comprises a photographing device, a memory and a processor; the photographing device comprises a first fisheye camera and a second fisheye camera, a photographing direction of the first fisheye camera is opposite to a photographing direction of the second fisheye camera; The memory is configured to store a computer program. The processor is configured to execute the computer program and implement the steps of the calibration method for fisheye panorama splicing according to any one of claims 1 to 8 when executing the computer program.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program, when executed by the processor, implements the steps of the calibration method for fisheye panorama splicing according to any one of claims 1 to 8.

Images