System and method for calculating relative rotation and translation of back-to-back cameras

Abstract

The application discloses a system and method for calculating the relative rotation and translation of back-to-back cameras, including two modes of fixing and adjusting: in the fixing mode, a reversible additional camera is installed beside the back-to-back cameras, the additional camera is turned to take pictures of a plane pattern from different positions and angles, the internal parameters and external parameters of each camera are calculated, and then the inverse operation of the external parameters is performed to obtain the rotation and translation between the back-to-back cameras; when the relative angle between the back-to-back cameras changes, the angle is automatically detected, the external parameters of the additional camera calculated in the fixing mode are modified, the inverse operation of the external parameters is performed again, and the rotation and translation between the back-to-back cameras are adjusted.

Description

TECHNICAL FIELD

[0001] The present application relates to a system and method for calculating the relative rotation and translation of back-to-back cameras. BACKGROUND

[0002] When calibrating two cameras, the field of view of the two cameras needs to overlap each other and the same pattern needs to be captured. Currently, the field of view of commercially available back-to-back dual cameras is greater than 180 degrees. The overlapping image parts are used for comparison, so that the brightness and color of the image have good quality at the splicing position. However, because the overlapping image is at the edge and is greatly deformed, the relative rotation and translation between the back-to-back dual cameras cannot be accurately calculated. When the field of view of the back-to-back dual cameras is less than 180 degrees, because there is no overlapping image part, the related art cannot calculate the relative rotation and translation between the back-to-back dual cameras.

[0003] After calibrating the two cameras according to the related art, if the relative position and angle between the two cameras change, the calibration pattern needs to be captured again, and the internal and external parameters need to be calculated again. This is a difficult task for general users. SUMMARY

[0004] Therefore, the present application provides a system and method for calculating the relative rotation and translation of back-to-back cameras to at least partially solve the above technical problems.

[0005] According to an embodiment of the present application, as one aspect of the present application, the present application provides a system and method for calculating the relative rotation and translation of back-to-back cameras, which includes calculating the relative rotation and translation when the field of view of the back-to-back dual cameras is less than 180 degrees and there is no overlap, so that the image content captured by the back-to-back dual cameras can correspond to each other. The system includes two modes: a fixed mode for calculating the preset rotation and translation between the back-to-back cameras, and an adjustment mode for automatically adjusting the rotation and translation between the back-to-back cameras when the relative angle between the back-to-back cameras changes, without the need to capture the calibration pattern again.

[0006] According to an embodiment of the present application, a system for calculating the relative rotation and translation of back-to-back cameras is provided, which comprises a first camera, a second camera, an extra camera, a planar pattern, an internal parameter calculation unit and an external parameter calculation unit in a fixed mode. The first camera and the second camera form the back-to-back cameras, and the view angles of the first camera, the second camera and the extra camera are all less than 180 degrees. The view angles of the first camera and the second camera do not overlap with each other. The extra camera is fixed beside the first camera and the second camera and is installed on a turning shaft. The turning shaft passes through between the first camera and the second camera, and is used to turn the extra camera and the first camera in the same direction and make the view angles of the extra camera and the first camera overlap with each other, and then turn the extra camera and the second camera in the same direction and make the view angles of the extra camera and the second camera overlap with each other. The translation position (x, y, z) and the turning angle (yaw, roll, pitch) of the extra camera are fixed and known by the mechanism, and are set as a third external parameter. A planar pattern with a known size is used, the planar pattern is moved and rotated when the first camera and the extra camera are in the same direction, and a first set of non-coplanar images are captured by the first camera and the extra camera; the planar pattern is moved and rotated when the second camera and the extra camera are in the same direction, and a second set of non-coplanar images are captured by the second camera and the extra camera. The internal parameter calculation unit calculates the first internal parameter of the first camera, the second internal parameter of the second camera and the third internal parameter of the extra camera respectively using the first set of non-coplanar images and the second set of non-coplanar images. The external parameter calculation unit calculates the first external parameter using the first set of non-coplanar images, the first internal parameter and the third internal parameter. The first external parameter comprises the rotation and translation between the first camera and the extra camera. The external parameter calculation unit calculates the second external parameter using the second set of non-coplanar images, the second internal parameter and the third internal parameter. The second external parameter comprises the rotation and translation between the second camera and the extra camera. When the first camera is taken as a reference, the external parameter calculation unit performs reverse operation on the second external parameter, multiplies the first external parameter and the third external parameter, and then performs reverse operation again to obtain the rotation (yaw, roll, pitch) and translation (x, y, z) between the first camera and the second camera; or when the second camera is taken as a reference, the external parameter calculation unit performs reverse operation on the first external parameter, multiplies the second external parameter and the third external parameter, and then performs reverse operation again to obtain the rotation (yaw, roll, pitch) and translation (x, y, z) between the first camera and the second camera.

[0007] According to an embodiment of the present application, a back-to-back camera with adjustment mode is provided, comprising a first camera, a second camera, an automatic angle detecting unit and an external parameter calculating unit. The first camera and the second camera form a back-to-back camera, and the angle of view of the first camera and the second camera is less than 180 degrees and does not overlap each other. The first camera has a preset first external parameter, including a rotation amount and a translation amount, which is stored in an internal memory or an external memory of the first camera. The second camera has a preset second external parameter, including a rotation amount and a translation amount, which is stored in an internal memory or an external memory of the second camera. The automatic angle detecting unit has a default third external parameter, including a rotation amount and a translation amount, which is stored in an internal memory or an external memory of the automatic angle detecting unit. The preset first external parameter, the preset second external parameter and the default third external parameter are all results of fixed mode calculation. The automatic angle detecting unit is used to detect a relative angle between the first camera and the second camera, and the relative angle can be changed in a single axis. When the relative angle is changed, the automatic angle detecting unit automatically modifies the third external parameter. The external parameter calculating unit is used to multiply the default first external parameter, the preset second external parameter and the modified third external parameter, and then perform reverse operation to obtain the rotation amount and the translation amount of the first camera and the second camera after the relative angle is changed.

[0008] According to an embodiment of the present application, a method for calculating the relative rotation and translation between two back-to-back cameras is provided. The two back-to-back cameras are composed of a first camera and a second camera, and the view angle of each camera is less than 180 degrees and does not overlap with each other. The method for calculating the relative rotation and translation between the two back-to-back cameras includes a fixed mode and an adjustment mode. In the fixed mode, an additional camera is fixed beside the first camera and the second camera, and is installed on a flipping shaft. The flipping shaft passes through the first camera and the second camera, and is used to flip the additional camera to the same direction as the first camera and to make the view angle of the additional camera overlap with the view angle of the first camera, and then to flip the additional camera to the same direction as the second camera and to make the view angle of the additional camera overlap with the view angle of the second camera. The flipping position and the flipping angle of the additional camera are fixed and known, and are set as a third external parameter. When the first camera and the additional camera are in the same direction, a planar pattern of a known size is moved and rotated, and a first set of non-coplanar images is captured by the first camera and the additional camera. When the second camera and the additional camera are in the same direction, the planar pattern is moved and rotated, and a second set of non-coplanar images is captured by the second camera and the additional camera. A first internal parameter of the first camera, a second internal parameter of the second camera, and a third internal parameter of the additional camera are calculated respectively using the first set of non-coplanar images and the second set of non-coplanar images. A first external parameter including the rotation and translation between the first camera and the additional camera is calculated using the first set of non-coplanar images, the first internal parameter, and the third internal parameter. A second external parameter including the rotation and translation between the second camera and the additional camera is calculated using the second set of non-coplanar images, the second internal parameter, and the third internal parameter. The second external parameter is inversely operated with respect to the first camera, and is multiplied by the first external parameter and the third external parameter, and then is inversely operated again to obtain the rotation (yaw, roll, pitch) and translation (x, y, z) between the first camera and the second camera. Or, the first external parameter is inversely operated with respect to the second camera, and is multiplied by the second external parameter and the third external parameter, and then is inversely operated again to obtain the rotation (yaw, roll, pitch) and translation (x, y, z) between the first camera and the second camera. The first external parameter, the second external parameter, and the third external parameter calculated in the fixed mode are stored as default values. In the adjustment mode, an automatic angle detection unit is used to detect the relative angle between the first camera and the second camera, and the relative angle can be changed in a single axis. When the relative angle is changed, the automatic angle detection unit automatically modifies the third external parameter. An external parameter operation unit is used to multiply the default first external parameter, the preset second external parameter, and the modified third external parameter, and then inversely operates to obtain the rotation and translation between the first camera and the second camera after the relative angle is changed.

[0009] For a better understanding of the above-described and other aspects of the present application, a detailed description is given below with reference to the accompanying drawings, in which: BRIEF DESCRIPTION OF DRAWINGS

[0010] Figure 1 A system for calculating relative rotation and translation of back-to-back cameras is illustratively shown according to an embodiment.

[0011] Figure 2 An electronic device employing back-to-back cameras is illustratively shown according to an embodiment.

[0012] Figures 3-4 A first camera, a second camera, and an extrinsic camera are illustratively shown according to an embodiment.

[0013] Figure 5 A planar pattern is illustratively shown according to an embodiment.

[0014] Figure 6 Computation of first intrinsic parameters of a first camera, second intrinsic parameters of a second camera, and third intrinsic parameters of an extrinsic camera are illustratively shown according to an embodiment.

[0015] Figure 7 Computation of first extrinsic parameters of a first camera is illustratively shown.

[0016] Figure 8 Computation of second extrinsic parameters of a second camera is illustratively shown.

[0017] Figure 9A Rotation and translation between a first camera and a second camera are illustratively shown.

[0018] Figure 9B Rotation and translation between a first camera and a second camera are illustratively shown.

[0019] Figures 10-11 A first camera, a second camera, and an extrinsic camera are illustratively shown according to another embodiment.

[0020] Figure 12 A flowchart of a method for calculating relative rotation and translation of back-to-back cameras is illustratively shown according to an embodiment.

[0021] BRIEF DESCRIPTION OF DRAWINGS

[0022] 100: electronic device

[0023] 110: back-to-back cameras

[0024] 111: first camera

[0025] 112: second camera

[0026] 113, 113': extrinsic camera

[0027] 130, 130': planar pattern

[0028] 140: internal parameter calculation unit

[0029] 150: external parameter calculation unit

[0030] 160: automatic angle detection unit

[0031] 900: system

[0032] A1: first internal parameter

[0033] A2: second internal parameter

[0034] A3: third internal parameter

[0035] AX1: flipping axis

[0036] C1, C2: image center

[0037] IM1: first set of non-coplanar images

[0038] IM2: second set of non-coplanar images

[0039] S110, S120, S130, S140, S150, S160: steps

[0040] Rt1: first external parameter

[0041] Rt2: second external parameter

[0042] Rt3, Rt3': third external parameter

[0043] Rt, Rt': rotation and translation DETAILED DESCRIPTION

[0044] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in details below with reference to specific embodiments and the accompanying drawings.

[0045] Reference is made to Figure 1Fig. 9 schematically illustrates a system 900 for calculating the relative rotation and translation of the back-to-back cameras 110 according to an embodiment. The system 900 for calculating the relative rotation and translation of the back-to-back cameras 110 includes the first camera 111, the second camera 112, the extra camera 113, the planar pattern 130, 130', the internal parameter computation unit 140, and the external parameter computation unit 150. The planar pattern 130 and the planar pattern 130' are the same planar pattern at different positions. The internal parameter computation unit 140 and the external parameter computation unit 150 are used to perform parameter computation, such as a chip, a circuit, a circuit board, a computer program, or a computer readable recording medium.

[0046] The extra camera 113 only needs to be set in a fixed mode. Please refer to Figures 3-4 Fig. 10 schematically illustrates the first camera 111, the second camera 112, and the extra camera 113, 113' according to an embodiment. The extra camera 113 and the extra camera 113' are the same camera flipped to different positions. The viewing angle of the first camera 111, the second camera 112, and the extra camera 113 are all less than 180 degrees. The viewing angle of the first camera 111 and the viewing angle of the second camera 112 do not overlap with each other. The extra camera 113 is fixed beside the first camera 111 and the second camera 112. The extra camera 113 is mounted on a flipping axis AX1. The flipping axis AX1 passes through between the first camera 111 and the second camera 112. As shown in the figure, the flipping axis AX1 is used to flip the extra camera 113 in the same direction as the first camera 111 and make the viewing angle of the extra camera 113 and the first camera 111 overlap with each other. Figure 3 Fig. 11 schematically illustrates the first camera 111, the second camera 112, and the extra camera 113, 113' according to an embodiment. The extra camera 113 and the extra camera 113' are the same camera flipped to different positions. The viewing angle of the first camera 111, the second camera 112, and the extra camera 113 are all less than 180 degrees. The viewing angle of the first camera 111 and the viewing angle of the second camera 112 do not overlap with each other. The extra camera 113 is fixed beside the first camera 111 and the second camera 112. The extra camera 113 is mounted on a flipping axis AX1. The flipping axis AX1 passes through between the first camera 111 and the second camera 112. As shown in the figure, the flipping axis AX1 is used to flip the extra camera 113 in the same direction as the first camera 111 and make the viewing angle of the extra camera 113 and the first camera 111 overlap with each other.

[0047] Fig. 12 schematically illustrates the first camera 111, the second camera 112, and the extra camera 113, 113' according to an embodiment. The extra camera 113 and the extra camera 113' are the same camera flipped to different positions. The viewing angle of the first camera 111, the second camera 112, and the extra camera 113 are all less than 180 degrees. The viewing angle of the first camera 111 and the viewing angle of the second camera 112 do not overlap with each other. The extra camera 113 is fixed beside the first camera 111 and the second camera 112. The extra camera 113 is mounted on a flipping axis AX1. The flipping axis AX1 passes through between the first camera 111 and the second camera 112. As shown in the figure, the flipping axis AX1 is used to flip the extra camera 113 in the same direction as the first camera 111 and make the viewing angle of the extra camera 113 and the first camera 111 overlap with each other. Figure 4 Fig. 13 schematically illustrates the first camera 111, the second camera 112, and the extra camera 113, 113' according to an embodiment. The extra camera 113 and the extra camera 113' are the same camera flipped to different positions. The viewing angle of the first camera 111, the second camera 112, and the extra camera 113 are all less than 180 degrees. The viewing angle of the first camera 111 and the viewing angle of the second camera 112 do not overlap with each other. The extra camera 113 is fixed beside the first camera 111 and the second camera 112. The extra camera 113 is mounted on a flipping axis AX1. The flipping axis AX1 passes through between the first camera 111 and the second camera 112. As shown in the figure, the flipping axis AX1 is used to flip the extra camera 113 in the same direction as the first camera 111 and make the viewing angle of the extra camera 113 and the first camera 111 overlap with each other.

[0048] The position (x, y, z) of the extra camera 113 is fixed by the mechanism of the flipping axis AX1, and the flipping angle (yaw, roll, pitch) is also fixed by the mechanism of the flipping axis AX1 at two angles corresponding to the respective angles of the first camera 111 and the second camera 112. When the extra camera 113 is flipped, the first camera 111 and the second camera 112 are fixed, for example, fixed on a pre-made jig. The position and the flipping angle of the extra camera 113 are known, set as the third external parameter Rt3.

[0049] Please refer to Figure 5 Fig. 14 schematically illustrates the planar pattern 130 according to an embodiment. The planar pattern 130 is, for example, a checkerboard pattern, which can be moved horizontally, vertically, front and back, and rotated. The pattern and size of the planar pattern 130 are known.

[0050] As shown in Figure 1 and Figure 3 , when the first camera 111 and the extrinsic camera 113 are co-directional, the planar pattern 130 is moved and rotated, and a first set of non-coplanar images IM1 is captured by the first camera 111 and the extrinsic camera 113. The first set of non-coplanar images IM1 includes multiple pairs of images, each pair of images consisting of two images captured by the first camera 111 and the extrinsic camera 113, respectively, of the planar pattern 130 in the same state. As the planar pattern 130 is moved and rotated, multiple pairs of non-coplanar images IM1 can be obtained.

[0051] As shown in Figure 1 and Figure 4 , when the second camera 112 and the extrinsic camera 113 are co-directional, the planar pattern 130 is moved and rotated, and a second set of non-coplanar images IM2 is captured by the second camera 112 and the extrinsic camera 113. The second set of non-coplanar images IM2 includes multiple pairs of images, each pair of images consisting of two images captured by the second camera 112 and the extrinsic camera 113, respectively, of the planar pattern 130 in the same state. As the planar pattern 130 is moved and rotated, multiple pairs of non-coplanar images IM2 can be obtained. The number of the first set of non-coplanar images IM1 and the number of the second set of non-coplanar images IM2 can be the same or different.

[0052] Please refer to Figure 1 and Figure 6 , Figure 6 schematically shows the operation of the first intrinsic parameter A1 of the first camera 111, the second intrinsic parameter A2 of the second camera 112, and the third intrinsic parameter A3 of the extrinsic camera 113. The intrinsic parameter operation unit 140 uses the first set of non-coplanar images IM1 and the second set of non-coplanar images IM2 to calculate the first intrinsic parameter A1 of the first camera 111, the second intrinsic parameter A2 of the second camera 112, and the third intrinsic parameter A3 of the extrinsic camera 113, respectively. In each pair of images of the first set of non-coplanar images IM1 or each pair of images of the second set of non-coplanar images IM2, there is a corresponding position of the grid points on the planar pattern 130, and through these corresponding relationships, the first intrinsic parameter A1 of the first camera 111, the second intrinsic parameter A2 of the second camera 112, and the third intrinsic parameter A3 of the extrinsic camera 113 can be calculated.

[0053] Please refer to Figure 1 and Figure 7 , Figure 7 schematically shows the operation of the first extrinsic parameter Rt1 of the first camera 111. The extrinsic parameter operation unit 150 uses the first set of non-coplanar images IM1, the first intrinsic parameter A1, and the third intrinsic parameter A3 to calculate the first extrinsic parameter Rt1. The first extrinsic parameter Rt1 includes the rotation and translation between the first camera 111 and the extrinsic camera 130.

[0054] Referring to Figure 1 and Figure 8 , Figure 8 The calculation of the second extrinsic parameter Rt2 of the second camera 112 is shown. The extrinsic parameter calculation unit 150 calculates the second extrinsic parameter Rt2 using the second set of non-coplanar images IM2, the second intrinsic parameter A2, and the third intrinsic parameter A3. The second extrinsic parameter Rt2 includes the rotation and translation between the second camera 112 and the extraneous camera 130.

[0055] Referring to Figure 9A , the rotation (yaw, roll, pitch) and translation (x, y, z) Rt between the first camera 111 and the second camera 112 is shown. The extrinsic parameter calculation unit 150 performs an inverse operation of the translation of the first extrinsic parameter Rt1, multiplies the inverse operation of the first extrinsic parameter Rt1 (indicated by a negative sign), the second extrinsic parameter Rt2, and the third extrinsic parameter Rt3, and then performs an inverse operation (indicated by a negative sign) to obtain the rotation (yaw, roll, pitch) and translation (x, y, z) Rt between the first camera 111 and the second camera 112.

[0056] Referring to Figure 9B , the rotation (yaw, roll, pitch) and translation (x, y, z) Rt between the first camera 111 and the second camera 112 is shown. The extrinsic parameter calculation unit 150 performs an inverse operation of the translation of the first extrinsic parameter Rt1, multiplies the inverse operation of the first extrinsic parameter Rt1 (indicated by a negative sign), the second extrinsic parameter Rt2, and the third extrinsic parameter Rt3, and then performs an inverse operation (indicated by a negative sign) to obtain the rotation (yaw, roll, pitch) and translation (x, y, z) Rt between the first camera 111 and the second camera 112.

[0057] The technology provided in the present case can be applied to other cases in addition to the case of Figures 3-4 . For example, referring to Figures 10-11 , the first camera 111, the second camera 112, and the extraneous camera 113 according to another embodiment are shown. As shown in Figure 10 , the image center C1 of the first camera 111 is aligned with the image center C2 of the second camera 112. As shown in Figure 11 , the image center C1 of the first camera 111 is located to the left of the image center C2 of the second camera 112; and the image center C2 of the second camera 112 is located to the left of the image center C1 of the first camera 111. Figures 10-11 The implementation cases of the above-mentioned embodiments are also applicable to the present technology.

[0058] The present application further provides a method for calculating the relative rotation and translation Rt between the back-to-back cameras 110 according to the above-mentioned technique. Please refer to Figure 12 , which schematically shows a flow chart of the method for calculating the relative rotation and translation Rt between the back-to-back cameras 110 according to an embodiment. As shown in Figure 1 , in step S110, when the first camera 111 and the extrinsic camera 113 are co-directional, the planar pattern 130 is moved and rotated, and a first set of non-coplanar images IM1 is captured by the first camera 111 and the extrinsic camera 113; when the second camera 112 and the extrinsic camera 113 are co-directional, the planar pattern 130 is moved and rotated, and a second set of non-coplanar images IM2 is captured by the second camera 112 and the extrinsic camera 113.

[0059] Then, as shown in Figure 6 , in step S120, the first internal parameter Al of the first camera 111, the second internal parameter A2 of the second camera 112 and the third internal parameter A3 of the extrinsic camera 113 are calculated respectively using the first set of non-coplanar images IM1 and the second set of non-coplanar images IM2.

[0060] Then, as shown in Figure 7 , in step S130, the first external parameter Rt1 is calculated using the first set of non-coplanar images IM1, the first internal parameter Al and the third internal parameter A3.

[0061] Then, as shown in Figure 8 , in step S140, the second external parameter Rt2 is calculated using the second set of non-coplanar images IM2, the second internal parameter A2 and the third internal parameter A3.

[0062] Then, as shown in Figures 9A-9B , in step S150, the second external parameter Rt2 is inversely calculated with respect to the first camera 111, and the inversely calculated second external parameter Rt2 (i.e. -Rt2) is multiplied by the first external parameter Rt1 and the third external parameter Rt3, and then inversely calculated again to obtain the rotation (yaw, roll, pitch) and translation (x, y, z) Rt between the first camera 111 and the second camera 112; or the first external parameter Rt1 is inversely calculated with respect to the second camera 112, and the inversely calculated first external parameter Rt1 (i.e. -Rt1) is multiplied by the second external parameter Rt2 and the third external parameter Rt3, and then inversely calculated again to obtain the rotation (yaw, roll, pitch) and translation (x, y, z) Rt between the first camera 111 and the second camera 112.

[0063] The above is about the fixed mode. The first external parameter Rt1, the second external parameter Rt2 and the third external parameter Rt3 obtained by the fixed mode can be stored in advance for the adjustment mode. One of the first external parameter Rt1 and the second external parameter Rt2 is the parameter after the reverse operation. The adjustment mode can automatically calculate the rotation and translation Rt without the additional camera 113.

[0064] Please refer to Figure 2 , which schematically shows an electronic device 100 with back-to-back cameras 110 with an automatic adjustment mode, including a first camera 111, a second camera 112, an automatic angle detection unit 160 and an external parameter operation unit 150. The automatic angle detection unit 160 is used to sense the angle of rotation of the first camera 111 and the second camera 112, such as an optical sensor, a magnetic sensor, an electromagnetic sensor or a gear module. The external parameter operation unit 150 is used for operation, such as a chip, a circuit, a circuit board, a computer program or a computer readable recording medium.

[0065] The viewing angle of the first camera 111 and the second camera 112 is less than 180 degrees and does not overlap each other. The first camera 111 has a preset first external parameter Rt1, including a preset rotation and translation between the first camera and the additional camera. The first external parameter Rt1 is stored in the internal memory or external memory of the first camera 111. The second camera 112 has a preset second external parameter Rt2, including a preset rotation and translation between the second camera and the additional camera. The second external parameter Rt2 is stored in the internal memory or external memory of the second camera 112.

[0066] The automatic angle detection unit 160 is used to detect a relative angle between the first camera 111 and the second camera 112. The relative angle can vary in a single axis. The automatic angle detection unit 160 has a default third external parameter Rt3, which is stored in the internal memory or external memory of the automatic angle detection unit 160. When the relative angle changes, the automatic angle detection unit 160 automatically modifies the third external parameter Rt3 to obtain a third external parameter Rt3'.

[0067] The external parameter operation unit 150 is used to multiply the first external parameter Rt1, the second external parameter Rt2 and the modified third external parameter Rt3', and then perform a reverse operation to obtain the rotation and translation Rt' of the first camera 111 and the second camera 112 after the relative angle changes. One of the first external parameter Rt1 and the second external parameter Rt2 has been reversed in the fixed mode.

[0068] Even when the visual angles of the first camera 111 and the second camera 112 do not overlap, the relative rotation amount and the translation amount Rt after the first camera 111 and the second camera 112 are rotated can still be accurately calculated, so that the technical bottleneck of the back-to-back camera 110 is greatly broken through.

[0069] The above-described specific embodiments further illustrate the purposes, technical solutions and beneficial effects of the present application. It should be understood that the above-described specific embodiments are merely examples of the present application and are not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A system for calculating the relative rotation and translation of back-to-back cameras, comprising: The first camera has a field of view of less than 180 degrees; The second camera has a field of view of less than 180 degrees, and the field of view of the first camera does not overlap with the field of view of the second camera. An external camera with a field of view of less than 180 degrees is fixed next to the first camera and the second camera. The external camera is mounted on a flip axis that passes between the first camera and the second camera. The flip axis is used to flip the external camera so that it is in the same direction as the first camera and the field of view of the external camera overlaps with that of the first camera. Then, it is flipped so that it is in the same direction as the second camera and the field of view of the external camera overlaps with that of the second camera. The translation position and flip angle of the external camera are fixed by a mechanism and set as a third external parameter. A planar pattern, the size of which is known, is moved and rotated when the first camera and the external camera are in the same direction, and a first set of non-coplanar images are captured by the first camera and the external camera. When the second camera and the external camera are in the same direction, the planar pattern is moved and rotated, and a second set of non-coplanar images are captured by the second camera and the external camera. The internal parameter calculation unit uses the first set of non-coplanar images and the second set of non-coplanar images to calculate the first internal parameters of the first camera, the second internal parameters of the second camera, and the third internal parameters of the external camera, respectively; and The external parameter calculation unit uses the first set of non-coplanar images, the first internal parameters, and the third internal parameters to calculate the first external parameter, which includes rotation and translation. The external parameter calculation unit uses the second set of non-coplanar images, the second internal parameters, and the third internal parameters to calculate the second external parameter, which includes rotation and translation. When the first camera is used as a reference, the external parameter calculation unit performs a reverse calculation on the second external parameter, multiplies it with the first external parameter and the third external parameter, and then performs a reverse calculation to obtain the rotation and translation between the first camera and the second camera. Or When the second camera is used as a reference, the external parameter calculation unit performs a reverse calculation on the first external parameter, multiplies it with the second external parameter and the third external parameter, and then performs a reverse calculation to obtain the rotation and translation between the first camera and the second camera.

2. The system according to claim 1, wherein, The first camera and the second camera have a fixed relative angle, and the flip angle of the additional camera is the same as the relative angle.

3. The system according to claim 1, wherein, The image center of the first camera is aligned with the image center of the second camera.

4. The system according to claim 1, wherein, The image center of the first camera is located to the right of the image center of the second camera; and the image center of the second camera is located to the right of the image center of the first camera.

5. The system according to claim 1, wherein, The image center of the first camera is located to the left of the image center of the second camera; and the image center of the second camera is located to the left of the image center of the first camera.

6. A method for calculating the relative rotation and translation of back-to-back cameras, wherein, An external camera is fixed next to a first camera and a second camera. The viewing angles of the first camera, the second camera, and the external camera are all less than 180 degrees. The external camera is mounted on a flip axis that passes between the first camera and the second camera. The flip axis is used to flip the external camera so that it is aligned with the first camera and their viewing angles overlap. Then, it is flipped again so that it is aligned with the second camera and their viewing angles overlap. The translational position and flip angle of the external camera are fixed by a mechanism and set as a third external parameter. The method includes: When the first camera and the external camera are in the same direction, a planar pattern is moved and rotated, and a first set of non-coplanar images are captured by the first camera and the external camera. When the second camera and the external camera are in the same direction, the planar pattern is moved and rotated, and a second set of non-coplanar images are captured by the second camera and the external camera. Using the first set of non-coplanar images and the second set of non-coplanar images, calculate the first intrinsic parameters of the first camera, the second intrinsic parameters of the second camera, and the third intrinsic parameters of the external camera, respectively. Using the first set of non-coplanar images, the first intrinsic parameter, and the third intrinsic parameter, a first extrinsic parameter is calculated, which includes rotation and translation. Using the second set of non-coplanar images, the second intrinsic parameter, and the third intrinsic parameter, a second extrinsic parameter is calculated, which includes rotation and translation; and When the first camera is used as a reference, the second external parameter is inversely calculated, multiplied by the first external parameter and the third external parameter, and then inversely calculated again to obtain the rotation and translation between the first camera and the second camera; or when the second camera is used as a reference, the first external parameter is inversely calculated, multiplied by the second external parameter and the third external parameter, and then inversely calculated again to obtain the rotation and translation between the first camera and the second camera.

7. The method according to claim 6, wherein, The first camera and the second camera have a fixed relative angle, and the flip angle of the additional camera is the same as the relative angle.

8. The method according to claim 6, wherein, The image center of the first camera is aligned with the image center of the second camera.

9. The method according to claim 6, wherein, The image center of the first camera is located to the right of the image center of the second camera; and the image center of the second camera is located to the right of the image center of the first camera.

10. The method according to claim 6, wherein, The image center of the first camera is located to the left of the image center of the second camera; and the image center of the second camera is located to the left of the image center of the first camera.

Images