Panoramic image stitching method and terminal

Abstract

The present invention discloses a panoramic image stitching method and terminal, which determines the position relationship between adjacent cameras and performs stereo correction on the adjacent cameras so that the coordinate system between the adjacent cameras moves only in a straight line, and obtains the baseline length between the adjacent cameras and the corrected camera focal length based on the stereo correction; at the same time, the user's preset depth information is obtained, so as to achieve panoramic stitching of the images taken by the adjacent cameras under this depth information according to the baseline length and the camera focal length, and obtain a panoramic image. In this way, there is no need to perform feature point matching, avoid dependence on parameters of the actual scene, and the camera shooting scene can be converted at will, thereby improving the use scenario of panoramic camera shooting, and there is no need to perform multiple complex calculations, which reduces the requirements for chip performance and improves the stitching speed.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a panoramic image stitching method and a terminal. Background Art

[0002] At present, there are three methods for realizing panoramic image stitching: the first is panoramic image stitching based on global mapping; the second is panoramic image stitching based on local mapping; the third is panoramic image stitching based on seam (stitching line). All three stitching methods are based on the feature points of the real scene, that is, image stitching is performed after the panoramic calibration is completed based on a specific scene. After the scene is converted, the current calculation parameters will no longer be applicable to other scenes, and the parameter calibration needs to be performed again. At this time, there will be some disadvantages. For example, if there are few feature points in the current scene, the stitching parameters cannot be calculated, and if the panoramic parameter calibration of the current scene needs to be performed once for each frame, this requires high performance of the chip and it is difficult to meet the requirements of real-time video. Summary of the invention

[0003] The technical problem to be solved by the present invention is to provide a panoramic image stitching method and a terminal, which avoids dependence on parameters of the actual scene and realizes panoramic image stitching for scenes of any depth.

[0004] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0005] A panoramic image stitching method, comprising:

[0006] Determine the pose relationship between adjacent cameras;

[0007] Performing stereo correction on the adjacent cameras according to the posture relationship to obtain a baseline length between the adjacent cameras and a corrected camera focal length;

[0008] Get preset depth information;

[0009] The images taken by the adjacent cameras are stitched together according to the baseline length, camera focal length and depth information to obtain a panoramic image.

[0010] In order to solve the above technical problems, another technical solution adopted by the present invention is:

[0011] A panoramic image stitching terminal includes a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor implements the following steps when executing the computer program:

[0012] Determine the pose relationship between adjacent cameras;

[0013] Performing stereo correction on the adjacent cameras according to the posture relationship to obtain a baseline length between the adjacent cameras and a corrected camera focal length;

[0014] Get preset depth information;

[0015] The images taken by the adjacent cameras are stitched together according to the baseline length, camera focal length and depth information to obtain a panoramic image.

[0016] The beneficial effects of the present invention are: by determining the posture relationship between adjacent cameras and performing stereo correction on the adjacent cameras, the coordinate system between the adjacent cameras moves only in a straight line, and the baseline length between the adjacent cameras and the corrected camera focal length are obtained based on the stereo correction; at the same time, the depth information preset by the user is obtained, so that the panoramic stitching of the images taken by the adjacent cameras under this depth information is realized according to the baseline length and the camera focal length to obtain a panoramic image. In this way, there is no need to match feature points, avoid dependence on parameters of the actual scene, and the camera shooting scene can be converted at will, which improves the use scenario of panoramic camera shooting, and there is no need to perform multiple complex calculations, which reduces the requirements for chip performance and improves the stitching speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 A flowchart of a panoramic image stitching method provided by an embodiment of the present invention;

[0018] Figure 2 A program flow chart of a panoramic image stitching method provided by an embodiment of the present invention;

[0019] Figure 3 A flowchart of another panoramic image stitching method provided by an embodiment of the present invention;

[0020] Figure 4 A schematic diagram of the structure of a panoramic image stitching terminal provided by an embodiment of the present invention;

[0021] Description of labels:

[0022] 301. Memory; 302. Processor. DETAILED DESCRIPTION

[0023] In order to explain the technical content, achieved objectives and effects of the present invention in detail, the following is an explanation in conjunction with the implementation modes and the accompanying drawings.

[0024] Please refer to Figure 1 , an embodiment of the present invention provides a panoramic image stitching method, comprising:

[0025] Determine the pose relationship between adjacent cameras;

[0026] Performing stereo correction on the adjacent cameras according to the posture relationship to obtain a baseline length between the adjacent cameras and a corrected camera focal length;

[0027] Get preset depth information;

[0028] The images taken by the adjacent cameras are stitched together according to the baseline length, camera focal length and depth information to obtain a panoramic image.

[0029] From the above description, it can be seen that the beneficial effects of the present invention are: by determining the posture relationship between adjacent cameras and performing stereo correction on adjacent cameras, the coordinate system between adjacent cameras only moves in a straight line, and the baseline length between adjacent cameras and the corrected camera focal length are obtained based on the stereo correction; at the same time, the user's preset depth information is obtained, so that the panoramic stitching of the images taken by the adjacent cameras under this depth information is realized according to the baseline length and the camera focal length to obtain a panoramic image. In this way, there is no need to match feature points, avoid dependence on parameters of the actual scene, and the camera shooting scene can be converted at will, which improves the use scenario of panoramic camera shooting, and there is no need to perform multiple complex calculations, which reduces the requirements for chip performance and improves the stitching speed.

[0030] Furthermore, the step of stitching the images captured by the adjacent cameras according to the baseline length, the camera focal length and the depth information to obtain the panoramic image includes:

[0031] Calculate the disparity between the images captured by the adjacent cameras according to the baseline length, the camera focal length and the depth information;

[0032] The images taken by the adjacent cameras are stitched together according to the parallax to obtain a panoramic image.

[0033] From the above description, it can be seen that based on the baseline length between adjacent cameras and the camera focal length, the parallax between adjacent cameras can be directly calculated in a scene where the user gives depth information. Compared with the current method of panoramic stitching through feature points, the present invention can avoid obtaining parallax through feature point matching, thereby avoiding dependence on parameters of the actual scene. The device can change the shooting scene at will and can perform panoramic stitching on scenes of any depth.

[0034] Furthermore, the step of stitching the images captured by the adjacent cameras according to the baseline length, the camera focal length and the depth information to obtain the panoramic image includes:

[0035] Select one of the adjacent cameras as a reference camera and the other camera as a target camera;

[0036] Acquire sampling points on a reference image taken by the reference camera;

[0037] Determine a mapping point of the sampling point on the target image captured by the target camera according to the baseline length, the camera focal length and the depth information;

[0038] The sampling points and the mapping points are used as feature points after the reference image and the target image are matched, and the reference image and the target image are spliced ​​according to the feature points to obtain a panoramic image.

[0039] From the above description, it can be seen that the feature points under the depth information can also be generated by the baseline length and the camera focal length. Since the feature points are directly obtained by mapping, there is no need to perform feature point matching between images. Compared with the matching method through actual feature points, the present invention constructs feature points in a way that does not need to consider the parameter calibration of the actual scene, but only needs to consider the preset depth. It can better adapt to the changeable and even extreme scenes that appear in the actual application process, and does not need to perform feature point matching, which has the advantage of faster splicing speed.

[0040] Furthermore, determining the position relationship between adjacent cameras includes:

[0041] Perform single-target positioning on adjacent cameras to obtain the first camera intrinsic parameter and the second camera intrinsic parameter;

[0042] The adjacent cameras are stereo-calibrated according to the first camera intrinsic parameter and the second camera intrinsic parameter to obtain relative position parameters between the adjacent cameras as the posture relationship between the adjacent cameras.

[0043] From the above description, it can be seen that the first camera intrinsic parameters and the second camera intrinsic parameters of adjacent cameras are respectively obtained based on single-target positioning, and then the relative position parameters between adjacent cameras are determined through stereo calibration, so as to determine the posture relationship between adjacent cameras, so as to obtain accurate information for subsequent stereo correction and ensure stitching accuracy. At the same time, the posture relationship obtained by single-target positioning and stereo calibration is simple in calculation, which effectively reduces the requirements on chip performance and ensures the needs of real-time shooting.

[0044] Furthermore, the obtaining of preset depth information includes:

[0045] A target area in the image captured by the adjacent camera is selected, and actual depth information of the target area is obtained as preset depth information.

[0046] From the above description, it can be seen that the actual depth information can be obtained by analyzing the target area in the image, thereby avoiding the problem of feature point matching in the panoramic image stitching process, thereby avoiding dependence on the parameters of the actual scene. The device can change the shooting scene at will and can perform panoramic stitching of scenes of any depth.

[0047] Please refer to Figure 4 Another embodiment of the present invention provides a panoramic image stitching terminal, including a memory, a processor, and a computer program stored in the memory and running on the processor, and the processor implements the following steps when executing the computer program:

[0048] Determine the pose relationship between adjacent cameras;

[0049] Performing stereo correction on the adjacent cameras according to the posture relationship to obtain a baseline length between the adjacent cameras and a corrected camera focal length;

[0050] Get preset depth information;

[0051] The images taken by the adjacent cameras are stitched together according to the baseline length, camera focal length and depth information to obtain a panoramic image.

[0052] From the above description, it can be seen that the beneficial effects of the present invention are: by determining the posture relationship between adjacent cameras and performing stereo correction on adjacent cameras, the coordinate system between adjacent cameras only moves in a straight line, and the baseline length between adjacent cameras and the corrected camera focal length are obtained based on the stereo correction; at the same time, the user's preset depth information is obtained, so that the panoramic stitching of the images taken by the adjacent cameras under this depth information is realized according to the baseline length and the camera focal length to obtain a panoramic image. In this way, there is no need to match feature points, avoid dependence on parameters of the actual scene, and the camera shooting scene can be converted at will, which improves the use scenario of panoramic camera shooting, and there is no need to perform multiple complex calculations, which reduces the requirements for chip performance and improves the stitching speed.

[0053] Furthermore, the step of stitching the images captured by the adjacent cameras according to the baseline length, the camera focal length and the depth information to obtain the panoramic image includes:

[0054] Calculate the disparity between the images captured by the adjacent cameras according to the baseline length, the camera focal length and the depth information;

[0055] The images taken by the adjacent cameras are stitched together according to the parallax to obtain a panoramic image.

[0056] From the above description, it can be seen that based on the baseline length between adjacent cameras and the camera focal length, the parallax between adjacent cameras can be directly calculated in a scene where the user gives depth information. Compared with the current method of panoramic stitching through feature points, the present invention can avoid obtaining parallax through feature point matching, thereby avoiding dependence on parameters of the actual scene. The device can change the shooting scene at will and can perform panoramic stitching on scenes of any depth.

[0057] Furthermore, the step of stitching the images captured by the adjacent cameras according to the baseline length, the camera focal length and the depth information to obtain the panoramic image includes:

[0058] Select one of the adjacent cameras as a reference camera and the other camera as a target camera;

[0059] Acquire sampling points on a reference image taken by the reference camera;

[0060] Determine a mapping point of the sampling point on the target image captured by the target camera according to the baseline length, the camera focal length and the depth information;

[0061] The sampling points and the mapping points are used as feature points after the reference image and the target image are matched, and the reference image and the target image are spliced ​​according to the feature points to obtain a panoramic image.

[0062] From the above description, it can be seen that the feature points under the depth information can also be generated by the baseline length and the camera focal length. Since the feature points are directly obtained by mapping, there is no need to perform feature point matching between images. Compared with the matching method through actual feature points, the present invention constructs feature points in a way that does not need to consider the parameter calibration of the actual scene, but only needs to consider the preset depth. It can better adapt to the changeable and even extreme scenes that appear in the actual application process, and does not need to perform feature point matching, which has the advantage of faster splicing speed.

[0063] Furthermore, determining the position relationship between adjacent cameras includes:

[0064] Perform single-target positioning on adjacent cameras to obtain the first camera intrinsic parameter and the second camera intrinsic parameter;

[0065] The adjacent cameras are stereo-calibrated according to the first camera intrinsic parameter and the second camera intrinsic parameter to obtain relative position parameters between the adjacent cameras as the posture relationship between the adjacent cameras.

[0066] From the above description, it can be seen that the first camera intrinsic parameters and the second camera intrinsic parameters of adjacent cameras are respectively obtained based on single-target positioning, and then the relative position parameters between adjacent cameras are determined through stereo calibration, so as to determine the posture relationship between adjacent cameras, so as to obtain accurate information for subsequent stereo correction and ensure stitching accuracy. At the same time, the posture relationship obtained by single-target positioning and stereo calibration is simple in calculation, which effectively reduces the requirements on chip performance and ensures the needs of real-time shooting.

[0067] Furthermore, the obtaining of preset depth information includes:

[0068] A target area in the image captured by the adjacent camera is selected, and actual depth information of the target area is obtained as preset depth information.

[0069] From the above description, it can be seen that the actual depth information can be obtained by analyzing the target area in the image, thereby avoiding the problem of feature point matching in the panoramic image stitching process, thereby avoiding dependence on the parameters of the actual scene. The device can change the shooting scene at will and can perform panoramic stitching of scenes of any depth.

[0070] The embodiments of the present invention provide a panoramic image stitching method and a terminal, which can be applied to a device for shooting panoramic images, avoid the device's dependence on parameters of the actual scene, and realize panoramic image stitching for scenes of any depth. After the user sets the corresponding depth information, the device can switch the shooting scene at will. The following is an explanation through a specific embodiment:

[0071] Please refer to Figure 1 to Figure 2 , Embodiment 1 of the present invention is:

[0072] A panoramic image stitching method, comprising:

[0073] S1. Determine the pose relationship between adjacent cameras.

[0074] In some embodiments, the adjacent cameras are two adjacent cameras in a horizontally placed panoramic camera group.

[0075] Specifically, the S1 includes:

[0076] S11, performing single-target positioning on adjacent cameras respectively to obtain a first camera intrinsic parameter and a second camera intrinsic parameter;

[0077] In some embodiments, single-target positioning is performed on adjacent cameras respectively, and a first camera intrinsic parameter of the first camera P1 is obtained as K1, and a second camera intrinsic parameter of the second camera P2 is obtained as K2.

[0078] S12. Perform stereo calibration on the adjacent cameras according to the first camera intrinsic parameter and the second camera intrinsic parameter to obtain relative position parameters between the adjacent cameras as the posture relationship between the adjacent cameras.

[0079] It should be noted that the relative position parameters include a rotation matrix R and a translation matrix T.

[0080] It should be noted that according to P = RP w +T, can convert the coordinate point in the world coordinate system to the coordinate point in the camera coordinate system. x = KP, which can convert the coordinate point group of the camera coordinate system into the coordinate point of the pixel coordinate system. w represents the coordinate point in the world coordinate system, and P represents P in the camera coordinate systemw The mapping point, P x represents the mapping point of P in the pixel coordinate system, K represents the camera intrinsic parameter, R represents the rotation matrix, and T represents the translation matrix.

[0081] In some embodiments, the step S12 is specifically: based on P x = KP, and the coordinate point P of the pixel coordinate system is converted according to the first camera intrinsic parameter K1 and the second camera intrinsic parameter K2 x Convert to the coordinate point P of the camera coordinate system to obtain P1 and P2. At the same time, based on P = RP w +T, and taking the first camera P1 as the reference system, obtain the rotation matrix R2 and translation matrix T2 of the second camera P2, and use them as the position relationship between the adjacent cameras. That is, R2 represents the rotation matrix from the second camera P2 to the first camera P1.

[0082] In some embodiments, the position relationship between the first camera P1 and the second camera P2 can be represented by a rotation matrix and a translation matrix as (R, T), then the position relationship of the first camera P1 is (I, 0), where I indicates that the rotation matrix is ​​a unit matrix, and 0 indicates that the translation matrix is ​​0, that is, the first camera P1 is the coordinate origin of the reference system, then the position relationship of the second camera P2 is (R2, T2). On this basis, the adjacent cameras are stereo rectified according to the position relationship between the first camera P1 and the second camera P2 to obtain new cameras P1' and P2'.

[0083] S2. Perform stereo correction on the adjacent cameras according to the posture relationship to obtain a baseline length between the adjacent cameras and a corrected camera focal length.

[0084] In some embodiments, after stereo calibration based on the above-mentioned posture relationship (I, 0) of the first camera P1 and the posture relationship (R2, T2) of the second camera P2, R1' and R2' are obtained, where R1' and R2' represent the rotation matrix from the original cameras P1 and P2 to the new cameras P1' and P2', that is, the new camera P1'=R1'×P1, the new camera P2'=R2'×P2. At this time, the camera intrinsic parameters of the new cameras P1' and P2' obtained after rotation are the same, and the focal length is also the same, that is, the focal length of the calibrated cameras is f.

[0085] In some embodiments, after stereoscopic correction is performed on the adjacent cameras, a reprojection matrix Q for mutual conversion between the pixel coordinate system and the world coordinate system is obtained;

[0086] in,

[0087] Among them, (u1, v1) is the principal point coordinate of the first camera P1, (u2, v2) is the principal point coordinate of the second camera P2, f is the corrected camera focal length, and b is the baseline length between adjacent cameras. It should be noted that the principal point coordinate represents the mapping point of the origin of the camera coordinate system on the pixel coordinate system, and the baseline length represents the distance between the optical centers of the first camera P1 and the second camera P2. That is, the baseline length b between adjacent cameras can be obtained according to the reprojection matrix Q, the principal point coordinate of the first camera P1, the principal point coordinate of the second camera P2, and the corrected camera focal length f.

[0088] In an optional implementation, the posture relationship between the adjacent cameras is determined by a checkerboard pattern and stereo correction is performed.

[0089] S3. Obtain preset depth information.

[0090] Specifically, S3 includes:

[0091] S31 . Select a target area in the image captured by the adjacent camera, and obtain actual depth information of the target area as preset depth information.

[0092] In an optional implementation, S3 specifically includes: presetting different empirical depth values ​​according to different types of actual scenes as preset depth information, that is, configuring multiple scene modes on the interface of the device for users to select and set, each scene mode corresponds to a different empirical depth value, wherein the scene modes include indoor scenes and outdoor scenes, etc.

[0093] In an optional implementation, S3 specifically includes: directly acquiring preset depth information, that is, the user directly inputs a corresponding depth value.

[0094] In some embodiments, the preset depth information is z.

[0095] S4. Stitching the images taken by the adjacent cameras according to the baseline length, camera focal length and depth information to obtain a panoramic image.

[0096] Specifically, the S4 includes:

[0097] S41, calculating the disparity between the images taken by the adjacent cameras according to the baseline length, the camera focal length and the depth information;

[0098] In some embodiments, the first camera P1 and the second camera P2 are set horizontally, and there is a horizontal parallax between the new cameras P1' and P2'. Based on the formula z = bf / (x1-x2), where x1 is the coordinate point of point A in the pixel coordinate system of the new camera P1', and x2 is the coordinate point of point A in the pixel coordinate system of the new camera P2'. According to the depth information z, the baseline length b, and the corrected focal length f, it can be obtained that x1-x2 = bf / z, where the horizontal parallax d = x1-x2.

[0099] S42: stitching the images taken by the adjacent cameras according to the parallax to obtain a panoramic image.

[0100] In some embodiments, the parallax is the distance between the imaging points of the same object on two different cameras, so the parallax can be eliminated by moving the image by the same number of pixels as the parallax, thereby achieving the effect of panoramic image stitching. For example, taking the center of the image taken by the new camera P1' as the origin, the center of the image taken by the new camera P2' is moved by d pixels and then stitched to obtain a panoramic image.

[0101] Please refer to Figure 3 , Embodiment 2 of the present invention is:

[0102] A panoramic image stitching method is different from the first embodiment in that the implementation method of step S4 is different.

[0103] In this embodiment, S4 includes:

[0104] S41, selecting one of the adjacent cameras as a reference camera and the other camera as a target camera;

[0105] In some embodiments, the step S41 specifically includes: selecting the first camera P1 among the adjacent cameras as a reference camera and the second camera P2 as a target camera.

[0106] S42, obtaining sampling points on the reference image taken by the reference camera;

[0107] It should be noted that there are multiple sampling points.

[0108] S43, determining a mapping point of the sampling point on the target image captured by the target camera according to the baseline length, the camera focal length and the depth information;

[0109] In some embodiments, the step S43 is specifically as follows: first, the sampling point C1 is mapped to the image corresponding to the new camera P1' through the rotation matrix R1' from the first camera P1 to the new camera P1', so as to obtain the coordinate x1 of the correction mapping point C1' of the image corresponding to the new camera P1'; then, according to the formula z=bf / (x1-x2), x2=x1-bf / z is transformed to obtain the coordinate x2 of the correction mapping point C2' of the image corresponding to the new camera P2'; finally, the correction mapping point C2' is mapped to the target image of the second camera P2 through the rotation matrix R2' from the second camera P2 to the new camera P2', so as to obtain the mapping point C2 on the target image captured by the second camera P2.

[0110] That is, the sampling point C1 on the reference image of the first camera P1 and the mapping point C2 on the target image of the second camera P2 are pixels with the same name in one-to-one correspondence. It should be noted that the pixels with the same name are imaging points of the same object in multiple cameras.

[0111] S44: Using the sampling points and the mapping points as feature points after the reference image and the target image are matched, and splicing the reference image and the target image according to the feature points to obtain a panoramic image.

[0112] In some embodiments, the reference image and the target image may be stitched together using a commonly used feature point stitching method to obtain a panoramic image, which is not limited herein. In this way, the panoramic image stitching method described in the present invention may also achieve panoramic image stitching based on feature points.

[0113] Please refer to Figure 4 , Embodiment 3 of the present invention is:

[0114] A panoramic image stitching terminal includes a memory 301, a processor 302, and a computer program stored in the memory 301 and running on the processor 302. When the processor 302 executes the computer program, each step in a panoramic image stitching method described in Embodiment 1 or Embodiment 2 is implemented.

[0115] In summary, the panoramic image stitching method and terminal provided by the present invention completely determine the position relationship between adjacent cameras and perform stereoscopic correction on adjacent cameras, so that the coordinate system between adjacent cameras only moves in a straight line, so that the three-dimensional stereo relationship between panoramic images is converted into a two-dimensional plane relationship. Based on the stereoscopic correction, the baseline length between adjacent cameras and the corrected camera focal length are obtained, thereby determining the relative position relationship between the images taken by the two cameras. At the same time, based on the depth information preset by the user, the parallax of the camera under this depth information is calculated according to the baseline length and the camera focal length. Based on the calculated parallax, the corresponding image is moved by a certain number of pixels and then panoramic stitching is performed. In this way, there is no need to perform feature point matching to remove scene depth and stitching parameters, avoiding dependence on the actual scene. In another optional implementation, based on the depth information preset by the user, the already matched pixels with the same name are actively searched as feature points for panoramic stitching by selecting sampling points and determining mapping points, effectively avoiding the defect of unstable stitching effect caused by the feature point matching method. Moreover, the camera shooting scene can be changed at will by stitching panoramic images with the user's preset depth information, which expands the use scenarios of panoramic camera shooting. In addition, no feature point matching is required during the stitching process, which avoids multiple complex calculations and reduces the requirements for chip performance, thereby improving the stitching speed. In the prior art, the depth of the object photographed by the device is unknown, so its depth information needs to be calculated when stitching panoramic images. In the application scenario of the present invention, the user has obtained the corresponding depth information according to the actual use scenario of the device, so the device can be separated from the current actual scene, avoiding the problem of failure of depth information detection due to extreme scenes in actual use.

[0116] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent transformations made using the contents of the present invention's specification and drawings, or directly or indirectly applied in related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A panoramic image stitching method, characterized in that: include: Determine the pose relationship between adjacent cameras; Performing stereo correction on the adjacent cameras according to the posture relationship to obtain a baseline length between the adjacent cameras and a corrected camera focal length; Get preset depth information; splicing the images taken by the adjacent cameras according to the baseline length, the camera focal length and the depth information to obtain a panoramic image; The adjacent cameras are stereo-corrected according to the position and posture relationship to obtain the baseline length between the adjacent cameras and the corrected camera focal length, which includes: After stereo correction is performed according to the position relationship between the first camera and the second camera in the adjacent cameras, a rotation matrix R1' of the first camera, a rotation matrix R2' of the second camera, a pixel coordinate system, and a reprojection matrix Q for mutual conversion with the world coordinate system are obtained; The new camera P1'=R1'×P1 obtained after the stereo calibration of the first camera, wherein P1 represents a coordinate point of the first camera coordinate system; The new camera P2'=R2'×P2 obtained after the second camera is stereo rectified, where P2 represents a coordinate point of the second camera coordinate system; The new camera P1' and the new camera P2' have the same camera intrinsic parameters, and the calibrated camera focal lengths are also the same; Obtaining a baseline length between the adjacent cameras according to the reprojection matrix Q, the principal point coordinates of the first camera, the principal point coordinates of the second camera, and the corrected camera focal length; The principal point coordinates represent the mapping point of the origin of the camera coordinate system on the pixel coordinate system; The baseline length represents the distance between the optical centers of the first camera and the second camera; The step of stitching the images captured by the adjacent cameras according to the baseline length, the camera focal length and the depth information to obtain a panoramic image comprises: Calculate the disparity between the images captured by the adjacent cameras according to the baseline length, the camera focal length and the depth information; The images taken by the adjacent cameras are stitched together according to the parallax to obtain a panoramic image.

2. A panoramic image stitching method according to claim 1, characterized in that: The images taken by the adjacent cameras are stitched together according to the baseline length, camera focal length and depth information to obtain a panoramic image replaced by: Select one of the adjacent cameras as a reference camera and the other camera as a target camera; Acquire sampling points on a reference image taken by the reference camera; Determine a mapping point of the sampling point on the target image captured by the target camera according to the baseline length, the camera focal length and the depth information; The sampling points and the mapping points are used as feature points after the reference image and the target image are matched, and the reference image and the target image are spliced ​​according to the feature points to obtain a panoramic image.

3. The panoramic image stitching method according to claim 1, characterized in that: Determining the position relationship between adjacent cameras includes: Perform single-target positioning on adjacent cameras to obtain the first camera intrinsic parameter and the second camera intrinsic parameter; The adjacent cameras are stereo-calibrated according to the first camera intrinsic parameter and the second camera intrinsic parameter to obtain relative position parameters between the adjacent cameras as the posture relationship between the adjacent cameras.

4. The panoramic image stitching method according to claim 1, characterized in that: The obtaining of preset depth information includes: A target area in the image captured by the adjacent camera is selected, and actual depth information of the target area is obtained as preset depth information.

5. A panoramic image stitching terminal, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor implements the following steps when executing the computer program: Determine the pose relationship between adjacent cameras; Performing stereo correction on the adjacent cameras according to the posture relationship to obtain a baseline length between the adjacent cameras and a corrected camera focal length; Get preset depth information; splicing the images taken by the adjacent cameras according to the baseline length, the camera focal length and the depth information to obtain a panoramic image; The adjacent cameras are stereo-corrected according to the position and posture relationship to obtain the baseline length between the adjacent cameras and the corrected camera focal length, which includes: After stereo correction is performed according to the position relationship between the first camera and the second camera in the adjacent cameras, a rotation matrix R1' of the first camera, a rotation matrix R2' of the second camera, a pixel coordinate system, and a reprojection matrix Q for mutual conversion with the world coordinate system are obtained; The new camera P1'=R1'×P1 obtained after the stereo calibration of the first camera, wherein P1 represents a coordinate point of the first camera coordinate system; The new camera P2'=R2'×P2 obtained after the second camera is stereo rectified, where P2 represents a coordinate point of the second camera coordinate system; The new camera P1' and the new camera P2' have the same camera intrinsic parameters, and the calibrated camera focal lengths are also the same; Obtaining a baseline length between the adjacent cameras according to the reprojection matrix Q, the principal point coordinates of the first camera, the principal point coordinates of the second camera, and the corrected camera focal length; The principal point coordinates represent the mapping point of the origin of the camera coordinate system on the pixel coordinate system; The baseline length represents the distance between the optical centers of the first camera and the second camera; The step of stitching the images captured by the adjacent cameras according to the baseline length, the camera focal length and the depth information to obtain a panoramic image comprises: Calculate the disparity between the images captured by the adjacent cameras according to the baseline length, the camera focal length and the depth information; The images taken by the adjacent cameras are stitched together according to the parallax to obtain a panoramic image.

6. A panoramic image stitching terminal according to claim 5, characterized in that: The images taken by the adjacent cameras are stitched together according to the baseline length, camera focal length and depth information to obtain a panoramic image replaced by: Select one of the adjacent cameras as a reference camera and the other camera as a target camera; Acquire sampling points on a reference image taken by the reference camera; Determine a mapping point of the sampling point on the target image captured by the target camera according to the baseline length, the camera focal length and the depth information; The sampling points and the mapping points are used as feature points after the reference image and the target image are matched, and the reference image and the target image are spliced ​​according to the feature points to obtain a panoramic image.

7. The panoramic image stitching terminal according to claim 5, characterized in that: Determining the position relationship between adjacent cameras includes: Perform single-target positioning on adjacent cameras to obtain the first camera intrinsic parameter and the second camera intrinsic parameter; The adjacent cameras are stereo-calibrated according to the first camera intrinsic parameter and the second camera intrinsic parameter to obtain relative position parameters between the adjacent cameras as the posture relationship between the adjacent cameras.

8. The panoramic image stitching terminal according to claim 5, characterized in that: The obtaining of preset depth information includes: A target area in the image captured by the adjacent camera is selected, and actual depth information of the target area is obtained as preset depth information.

Images