Image stitching method and device

[0034] Example one:

[0035] figure 2 A flowchart of an image stitching method provided by the first embodiment of the present invention is shown, and the details are as follows:

[0036] Step S21, selecting one field of view from the multiple fields of view as the reference field of view.

[0037] In this step, multiple cameras shoot multiple fields of view from different angles, and any two adjacent fields of view have a certain overlap area, so that the fields of view shot by any two cameras can transition through the overlap area. Choose a field of view from the multiple fields of view as the reference field of view.

[0038] Step S22: Determine a projection transformation matrix from the other fields of view except the reference field of view to the reference field of view among the multiple fields of view.

[0039] In this step, when a reference field of view is selected, the other fields of view are projectively transformed into the reference coordinate system of the reference field of view according to the mapping relationship. The field of view adjacent to the reference field of view can directly calculate the field of view to the reference field of view. The projection transformation matrix of the field, and then the projection transformation to the reference coordinate system; and the field of view that is not adjacent to the reference field of view needs to calculate the projection transformation matrix of the field of view to the reference field of view through the intermediate transition field of view. Such as image 3 As shown, the digital node represents the field of view shot by a group of 3*3 camera arrays. Now, the images of these nine fields of view are to be panoramic stitched. It is assumed that the fields of view adjacent in space have a certain overlap area. In the panoramic stitching process, assuming that the field of view 1 is used as the reference field of view, the projection transformation matrix of the field of view 2 projected to the field of view 1 can be directly calculated, and the field of view 6 that is not adjacent to the field of view 1 must pass the intermediate transition view The projection transformation matrix is ​​calculated by the field to be mapped to the reference coordinate system, and the intermediate transition field of view between field 1 and field 6 is not unique. image 3 Two lines are used to mark two of the paths from field of view 1 to field of view 6, and two calculation methods of the projection transformation matrix from field of view 6 to field of view 1 can be obtained through these two paths.

[0040] Wherein, the step of determining the projection transformation matrix of the field of view other than the reference field of view among the multiple fields of view to the reference field of view specifically includes:

[0041] A1. Determine the shortest path for projecting other fields of view in the plurality of fields of view except the reference field of view to the reference field of view. In this step, the shortest path of the field of view projected to the reference field of view refers to the path of the field of view projected to the reference field of view through the least intermediate transition field of view.

[0042] Due to the projection transformation of the field of view, there is bound to be a certain error. The more the number of projection transformations, the greater the accumulated error. When the error accumulates to a certain level, mismatching will occur, which directly affects To the effect of a panoramic image. In order to reduce the cumulative error, it is necessary to determine the shortest path from the field of view to the reference field of view to reduce the number of projection transformations.

[0043] Among them, the shortest path from the field of view to the reference field of view is determined through the following steps:

[0044] A11. Generate projection transformation maps of the multiple fields of view;

[0045] A12. The projection transformation graph uses whether two nodes are connected to indicate whether the two fields of view are adjacent;

[0046] A13. Generate a reachable path table according to the projection transformation graph, where the reachable path table stores information about whether two fields of view corresponding to two nodes are adjacent;

[0047] A14. Generate a transformation tree of the multiple fields of view according to the generated reachable path table, the transformation tree storing information about the shortest path from the field of view corresponding to each node to the reference field of view;

[0048] A15. Determine, according to the generated transformation tree, a shortest path for projecting other fields of view except the reference field of view into the reference field of view among the multiple fields of view.

[0049] In the above steps, the concept of "graph" in the data structure is introduced, and different nodes are used to identify each field of view, thereby generating projection transformation maps of multiple fields of view. In the projection transformation graph, two connected nodes represent two adjacent nodes. To image 3 The 9 fields of view shown as an example, the generated projection transformation map is as follows Figure 4 Shown in Figure 4 Middle, node V i (I=1,2,…,9) represents the field of view i, from Figure 4 It can be seen that the field of view 6 can be from the field of view 1 through the field of view 2, and then from the field of view 2 to the field of view 6, or from the field of view 1 through the field of view 5, and then from the field of view 5 to the field of view 6, and so on. After the projection transformation map is generated, the reachable path table described in Table 1 can be obtained according to the projection transformation map:

[0050] Table 1:

[0051] Node

[0052] In Table 1, (V i ,V j )=1 means that node i and node j are directly connected. In Table 1, "1" and blank are used to indicate whether two nodes are directly connected. In actual situations, other information can also be used to identify whether two nodes are directly connected, that is, to identify whether the two fields of view corresponding to the two nodes are adjacent. There is no limitation here. From the information recorded in Table 1, we can know the node directly connected to any node by looking up Table 1. According to the reachable path table in Table 1, it can be generated as Figure 5 The transformation tree shown. in Figure 5 Among them, the path from any node to node 1 is the shortest path, that is, the number of projection transformations from any field of view to the reference field of view is the smallest.

[0053] A2, according to the determined shortest path, calculate a projection transformation matrix of the multiple fields of view other than the reference field of view projected to the reference field of view. In step A2, the value in each projection transformation matrix is ​​the pixel value.

[0054] Wherein, the projection transformation matrix of other fields of view in the plurality of fields of view except the reference field of view is calculated to the reference field of view by the following steps:

[0055] A21. Determine whether any one of the multiple fields of view is adjacent to the reference field of view;

[0056] A22. When any one of the multiple fields of view is adjacent to the reference field of view, according to the translation matrix and zoom matrix of the camera coordinate system relative to the world coordinate system of any one of the multiple fields of view And a rotation matrix, which determines a projection transformation matrix for projecting any one of the multiple fields of view onto the reference field of view;

[0057] A23. When any one of the multiple fields of view is not adjacent to the reference field of view, according to the translation matrix and zoom of the camera coordinate system relative to the world coordinate system of any one of the multiple fields of view The matrix and the rotation matrix determine the projection transformation matrix of the first field of view to the second field of view on the shortest path from any field of view to the reference field of view; the first field of view and the second field of view The two fields of view are adjacent, and the path from the second field of view to the reference field of view is shorter than the path from the first field of view to the reference field of view; and calculate the shortest distance between any one of the multiple fields of view and the reference field of view The product of the projection transformation matrices of any two adjacent fields of view on the path is used as the projection transformation matrix of any one of the multiple fields of view to the reference field of view.

[0058] In the above steps, the "first" of the "first field of view" and the "second" of the "second field of view" are only used to distinguish different fields of view, and have no other meaning. When the field of view is adjacent to the reference field of view, the projection transformation matrix of the field of view to the reference field of view can be determined according to the translation matrix, zoom matrix and rotation matrix of the camera coordinate system relative to the world coordinate system of the field of view. Such as Image 6 As shown, suppose that for a point P(X,Y,Z) in space, the corresponding points in the images taken by two cameras at different positions are respectively P 1 And P 2 , That is, the two fields of view captured by the two cameras have overlapping areas. P 1 And P 2 The coordinates of are recorded as X1=(x1,y1,w) T , X 2 =(x2,y2,w) T , We can see:

[0059] X 1 =V 1 R 1 T 1 p, X 2 =V 2 R 2 T 2 p

[0060] Where T i ,V i ,R i (i=1,2) are the translation matrix, zoom matrix and rotation matrix of the camera coordinate system relative to the world coordinate system. From the above formula, p=X 1 V 1 -1 R 1 -1 T 1 -1 , Then X 2 =V 2 R 2 T 2 X 1 V 1 -1 R 1 -1 T 1 -1 When the field of view is not adjacent to the reference field of view, the projection transformation matrix of any two adjacent fields of view on any shortest path in the transformation tree is determined. Of course, since this embodiment only needs to obtain the field of view to the reference field of view Therefore, when calculating the projection transformation matrix of two adjacent fields of view, it is to calculate the projection transformation matrix of the node in the transformation tree to its parent node, refer to Figure 5 , When calculating the projection transformation matrix of node 3 to node 1, it is the calculation of the projection transformation matrix of node 3 to node 2, instead of the calculation of the projection transformation matrix of node 2 to node 3, it is the calculation of node 2 to node 1 Instead of calculating the projection transformation matrix of node 1 to node 2. Wherein, the calculation method of the transformation matrices of two adjacent fields of view is the same as that of step A22, and will not be repeated here. After determining the projection transformation matrix of any two adjacent fields of view on any shortest path, the product of the field of view and the projection transformation matrix of any two adjacent fields of view on the shortest path of the reference field of view is taken as the view The projection transformation matrix of the field projection to the reference field of view. Reference Figure 5 , Assuming that the projection transformation matrix M23 of node 3 projected to node 2 is determined, and the projection transformation matrix M12 of node 2 projected to node 1, then the projection transformation matrix of the field of view corresponding to node 3 is projected to the reference field of view M13=M12*M23, Among them, "*" means matrix multiplication.

[0061] Step S23: Determine, according to the projection transformation matrix, the projection transformation fields of view of the multiple fields of view except the reference field of view.

[0062] In this step, the projective transformation field of view is equal to the product of the field of view and the projection transformation matrix of the field of view. For example, assuming that the projection transformation matrix of the field of view D is T(), and the pixel point set of the field of view D is E, then the projected transformation field of view of the field of view D=T(E).

[0063] Step S24: Determine the sum of the deformation variables of the projection transformation field of view of the field of view other than the reference field of view among the multiple fields of view.

[0064] In this step, when there are multiple projection transformation fields of view, the deformation amount of each projection transformation field of view is calculated, and the obtained deformation amount of each projection transformation field of view is accumulated.

[0065] Among them, the sum of the deformation variables of the projection transformation field of view is determined through the following steps:

[0066] B1. Determining the deformation amount of the projection transformation field of view of any field of view other than the reference field of view among the multiple fields of view.

[0067] B2. Accumulate the deformation of the projection transformation field of view of all the fields of view except the reference field of view among the multiple fields of view.

[0068] Wherein, in step B1, the step of determining the deformation amount of the projection transformation field of view of any field of view other than the reference field of view of the plurality of fields of view specifically includes:

[0069] B11. Before any field of view in the plurality of fields of view except the reference field of view is projected onto the reference field of view, determine any point of view in any field of view to the coordinates established before the projection transformation Euclidean distance of the origin of the system;

[0070] B12. After the any field of view is projected onto the reference field of view, determine the Euclidean distance from any point of view in the field of view to the origin of the coordinate system established after the projection transformation;

[0071] B13. Calculate the difference between the Euclidean distance from any viewpoint in the field of view to the origin of the coordinate system established before the projection transformation and the Euclidean distance from any viewpoint in the field of view to the origin of the coordinate system established after the projection transformation ;

[0072] B14. Accumulate the Euclidean distance difference of all viewpoints in the field of view, as the sum of the deformation variables of the projected transformation field of view corresponding to the field of view.

[0073] In the above steps, the Euclidean distance formula is d=sqrt(∑(xi1-xi2)^2), i=1,2..n, n refers to the dimension of the calculated Euclidean distance, for example, when calculating the two-dimensional view When the Euclidean distance of the viewpoint in the field, n=2. In order to more clearly illustrate the calculation of the deformation of the projectively transformed field of view corresponding to a field of view, a specific example is used as follows:

[0074] See Figure 7 with Figure 8 , Figure 7 Shows a schematic diagram of the field of view before the projection transformation, Figure 8 show Figure 7 After the projection transformation of the field of view shown, the corresponding projection transformation field of view. Assuming that before the projection transformation, the set of pixels on the boundary of the field of view is E, the set of pixels corresponding to the field of view after the projection transformation is E′, and the projection transformation matrix is ​​T(), then E′=T(E). For any point X ∈ E, the coordinates in the new coordinate system after the projection transformation are X′, X′ ∈ E′, and the center point O corresponds to O′ in the new coordinate system after the projection transformation, then in the original field of view, any The Euclidean distance d between the point X ∈ E and the center point O is:

[0075] d=||X-O||

[0076] The Euclidean distance d′ between the corresponding point X′∈E′ and the center point O′ after the projection transformation of the point X∈E is:

[0077] d′=||X′-O′||

[0078] Then the difference Δd of the Euclidean distance before and after the projection transformation is:

[0079] Δd=|d-d′|

[0080] Suppose there are N pixels in the set E, that is, N viewpoints, calculate the Δd corresponding to each pixel, and then add them all together as a measure Figure 7 The deformation of the field of view shown after the projection transformation, the deformation σ is:

[0081] σ = X i = 1 N Δdi

[0082] According to the above calculation formula, determine the deformation of the projective transformation field of view of any field of view, and accumulate the deformation of the determined projective transformation field of view of any field of view to obtain the multiple fields of view other than the reference field of view. The projection transformation of all fields of view is the sum of the deformation of the field of view.

[0083] Repeating steps S21 to S24, and obtaining all the fields of view in the plurality of fields of view as the reference field of view in turn, the sum of the deformation variables of the projection transformation fields of view of the fields of view except the reference field of view;

[0084] Step S25: Determine the minimum value of the sum of the deformation variables of the projection-transformed fields of view of the fields of view except the reference field of view when all fields of view of the plurality of fields of view are sequentially used as the reference field of view.

[0085] Assuming that there are m fields of view that need to be panoramic stitched, and the spatially adjacent fields of view have a certain overlap area, the m fields of view are respectively the field of view F 1 ,F 2 ,...,F m , Select F 1 As the reference field of view, calculate F according to step S24 1 As the reference field of view, the field of view F is excluded from the m fields of view 1 Then repeat steps S21 to S24 to select each field of view as the reference field of view, and use the shortest path projection matrix algorithm to calculate the projection transformation matrix from each field of view to the reference field of view. , And calculate the deformation of the field of view of each projection transformation, and then calculate the cumulative deformation of the final panoramic image, and finally calculate all m cumulative deformations. Compare the m cumulative deformation variables, find the smallest cumulative deformation variable among the m cumulative deformation variables, and then determine which field of view the smallest cumulative deformation variable is calculated from, and determine the view field corresponding to the reference field of view. field.

[0086] In step S26, the field of view corresponding to the minimum value of the sum of the deformation variables of the projection transformation field of view is used as the reference field of view to splice images.

[0087] In this step, after the reference field of view is determined, all fields of view except the reference field of view among the multiple fields of view are projected onto the reference field of view to splice images in the multiple fields of view.

[0088] In this embodiment, any field of view is selected from a plurality of fields of view as the reference field of view, and the shortest path from other fields of view to the reference field of view is determined, and then calculated according to the determined shortest path except for the reference field. The projection transformation matrix from other fields of view to the reference field of view is used to determine the projection transformation field of view for other fields of view except the reference field according to the calculated projection transformation matrix. Finally, the deformation of each projection transformation field of view is calculated, and then Calculate the cumulative deformation of the final panoramic image. Select another field of view from multiple fields of view as the reference field of view, determine the cumulative deformation of the panoramic image generated when other fields of view are used as the reference field of view, and compare each field of view as the reference field of view The cumulative deformation of the panoramic image generated during the field, to determine which field of view the smallest cumulative deformation is calculated from as the reference field of view, and finally use the reference field of view corresponding to the smallest cumulative deformation as the reference field of view The field of view performs image stitching. Since the reference field of view is determined by comparing all the fields of view as the cumulative deformation of the reference field of view to generate the panoramic image, and the cumulative deformation is calculated based on the shortest path projected from the field of view to the reference field of view, this embodiment is adopted. The provided method can select the optimal reference field of view, so that when the reference field of view selected in this embodiment is used as the reference field of view of the stitched image, it can ensure that the cumulative deformation of the obtained panoramic image is minimized, and the stitched image is improved quality.

[0089] Embodiment two:

[0090] The following specifically describes how to determine the shortest path projected from the field of view to the reference field of view. In this embodiment, each field of view is abstracted as a node, which is described in detail as follows:

[0091] First, create three tables, namely HIGHER table, TEMP table and LOWER table. The HIGHER table stores the determined nodes and is initially empty; the LOWER table stores the undetermined nodes and initially contains all m nodes; the TEMP table is a temporary table. Initially empty.

[0092] The algorithm flow is:

[0093] Step 1: Assuming that node Vs is selected as the source node, remove it from the LOWER table and put it into the HIGHER table;

[0094] Step 2: Traverse each node in the HIGHER table, for any node V i ∈HIGHER table, find the node V in the LOWER table i All adjacent nodes, first calculate these nodes and V i The projection transformation matrix between each node, and then calculate the projection transformation matrix from each node to the source node, where any node V j To source node V s The projection transformation matrix M sj =M si ·M ij , Remove the nodes for which the projection transformation matrix has been obtained from the LOWER table and put them into the TEMP table;

[0095] Step 3: Clear the HIGHER table, put all the nodes in the TEMP table into the HIGHER table, and then clear the TEMP table;

[0096] Step 4: Repeat steps 2 to 3 until the LOWER table is empty.

[0097] From the above algorithm, we can get the projection transformation matrix from all fields of view to the reference field of view in the transformation tree. To image 3 As an example, assuming the field of view 1 is used as the reference field of view, the process of applying the above algorithm is as follows:

[0098] Create three tables for these nodes, namely HIGHER=[],LOWER=[V 1 ,V 2 ,...,V 9 ],TEMP=[].

[0099] Let node V 1 Is the source node, HIGHER=[V 1 ],LOWER=[V 2 ,V 3 ,...,V 9 ];

[0100] Find node V in HIGHER table in LOWER table 1 Adjacent node V 2 , V 4 , V 5 , The projection transformation matrix to be calculated is the projection transformation matrix itself of their adjacent field of view, respectively M 12 , M 14 , M 15 , Delete these nodes from the LOWER table, put them into the TEMP table, empty the HIGHER table, put all the nodes in the TEMP table into the HIGHER table, and then empty the TEMP table, at this time HIGHER=[V 2 ,V 4 ,V 5 ],LOWER=[V 3 ,V 6 ,V 7 ,V 8 ,V 9 ],TEMP=[];

[0101] Find the node V in the HIGHER table in turn in the LOWER table 2 , V 4 , V 5 Adjacent nodes, by V 2 Find adjacent node V 3 , V 6 , First calculate V 2 And V 3 The projection transformation matrix between M 23 , V 2 And V 6 The projection transformation matrix between M 26 , Then the projection transformation matrix M can be calculated on this basis 13 =M 12 ·M 23 , M 16 =M 12 ·M 26 , Will V 3 And V 6 Deleted from the LOWER table and included in the TEMP table, at this time LOWER=[V 7 ,V 8 ,V 9 ],TEMP=[V 3 ,V 6 ]; Similarly, continue to look for node V in the LOWER table 4 Adjacent nodes, then V 7 And V 8 , Directly calculate M 47 And M 48 , Followed by M 17 =M 14 ·M 47 , M 18 =M 14 ·M 48 , Will V 7 And V 8 Deleted from the LOWER table and included in the TEMP table, at this time LOWER=[V 9 ],TEMP=[V 3 ,V 6 ,V 7 ,V 8 ]; Continue to find node V in the LOWER table 5 Adjacent nodes, then V 9 , Directly calculate M 59 , Followed by M 19 =M 15 ·M 59 , Will V 9 Deleted from the LOWER table and included in the TEMP table, at this time LOWER=[],TEMP=[V 3 ,V 6 ,V 7 ,V 8 ,V 9 ]; At this time, the LOWER table is empty and the program ends.

[0102] Can get Figure 5 Transform tree. The path from any node to the source node obtained by the above calculation process is the shortest path, that is, the number of projection transformations from any field of view to the reference field of view is the smallest.

[0103] Embodiment three:

[0104] Picture 9 A structural diagram of an image splicing device provided by the third embodiment of the present invention is shown. For ease of description, only parts related to the embodiment of the present invention are shown. The image splicing device corresponds to the image splicing method of the foregoing embodiment.

[0105] In this embodiment, the image splicing device includes: a field of view selection unit 91, a projection transformation matrix determination unit 92, a projection transformation view field determination unit 93, a first deformation amount determination unit 94, a second deformation amount determination unit 95, image splicing Unit 96. among them:

[0106] The field of view selection unit 91 is used to select a field of view from multiple fields of view as the reference field of view.

[0107] The projection transformation matrix determining unit 92 is configured to determine a projection transformation matrix from the other fields of view except the reference field of view to the reference field of view among the multiple fields of view.

[0108] When the field of view selection unit 91 selects a reference field of view, the other fields of view are projectively transformed into the reference coordinate system of the reference field of view according to the mapping relationship. The field of view adjacent to the reference field of view can directly calculate the field of view to the reference. The projection transformation matrix of the field of view, and then the projection transformation to the reference coordinate system; and the field of view not adjacent to the reference field of view needs to calculate the projection transformation matrix of the field of view to the reference field of view through the intermediate transition field of view.

[0109] As a preferred embodiment of the present invention, the projection transformation matrix determination unit 92 includes: a shortest path determination module and a projection transformation matrix calculation module. among them:

[0110] The shortest path determination module is used to determine the shortest path of the field of view other than the reference field of view among the plurality of fields of view projected to the reference field of view. Further, the shortest path determination module includes: a projection transformation map generating module, configured to generate projection transformation maps of the multiple fields of view. The projection transformation graph uses whether two nodes are connected to indicate whether the two fields of view are adjacent. The reachable path table generating module is configured to generate a reachable path table according to the projection transformation graph, and the reachable path table stores information about whether two fields of view corresponding to two nodes are adjacent. The transformation tree generation module is configured to generate transformation trees of the multiple fields of view according to the generated reachable path table, and the transformation tree stores the information of the shortest path of the field of view corresponding to each node to the reference field of view. The projection path determination module is configured to determine, according to the generated transformation tree, the shortest path for the other fields of view except the reference field of view to be projected to the reference field of view.

[0111] The projection transformation matrix calculation module is configured to calculate the projection transformation matrix of the multiple fields of view except the reference field of view to the reference field of view according to the determined shortest path. Further, the projection transformation matrix calculation module includes: an adjacent judgment module for judging whether any one of the multiple fields of view is adjacent to the reference field of view. A matrix generation module for when any one of the multiple fields of view is adjacent to the reference field of view, according to the translation of the camera coordinate system relative to the world coordinate system in any one of the multiple fields of view The matrix, the scaling matrix, and the rotation matrix determine the projection transformation matrix of any one of the multiple fields of view to the reference field of view. When any one of the multiple fields of view is not adjacent to the reference field of view, according to the translation matrix, zoom matrix, and zoom matrix of the camera coordinate system in which any one of the multiple fields of view is located relative to the world coordinate system The rotation matrix determines the projection transformation matrix of the second field of view to the first field of view on the shortest path from any field of view of the plurality of fields of view to the reference field of view. The second field of view is adjacent to the first field of view, and the path from the first field of view to the reference field of view is shorter than the path from the second field of view to the reference field of view. Calculate the product of the projection transformation matrix of any one of the multiple fields of view and any two adjacent fields of view on the shortest path of the reference field of view, as the projection of any of the multiple fields of view The projection transformation matrix to the reference field of view.

[0112] Since the projection transformation matrix of the field of view is calculated through the shortest path from the field of view to the reference field of view, the resulting projection transformation matrix has the least error.

[0113] The projection transformation field of view determination unit 93 is configured to determine, according to the projection transformation matrix, the projection transformation fields of view of the multiple fields of view other than the reference field of view.

[0114] The projection transformation field of view is equal to the product of the field of view and the projection transformation matrix of the field of view.

[0115] The first deformation amount determining unit 94 is configured to determine the sum of the deformation amounts of the projectively transformed fields of view of the fields of view other than the reference field of view among the plurality of fields of view.

[0116] The first deformation determining unit 94 first determines the deformation of the projection transformation field of view of any one of the multiple fields of view except the reference field of view, and then accumulates the deformation of the projection transformation field of view of all the fields of view. Among them, the deformation of the projected transformation field of view of a field of view is the sum of the deformations of all the viewpoints (or pixels) in the projected transformation field of view.

[0117] Specifically, the first deformation amount determining unit 94 includes: a deformation amount calculation module and a deformation amount accumulation module.

[0118] among them:

[0119] The deformation calculation module is used to determine the deformation of the projection transformation field of view of any field of view except the reference field of view among the multiple fields of view. Wherein, the specific step of determining the deformation of the projection transformation field of view of any field of view includes: before any field of view of the plurality of fields of view except the reference field of view is projected onto the reference field of view, determining The Euclidean distance from any viewpoint in the any field of view to the origin of the coordinate system established before the projection transformation. After the any field of view is projected onto the reference field of view, the Euclidean distance from any point of view in the any field of view to the origin of the coordinate system established after the projection transformation is determined. Calculate the difference between the Euclidean distance from any viewpoint in the field of view to the origin of the coordinate system established before the projection transformation and the Euclidean distance from any viewpoint in the field of view to the origin of the coordinate system established after the projection transformation. The Euclidean distance difference of all viewpoints in the field of view is accumulated as the sum of the deformation variables of the projected transformation field of view corresponding to the field of view.

[0120] The deformation accumulation module is used to accumulate the deformation of the projection transformation field of view of all the fields of view except the reference field of view.

[0121] Among them, the Euclidean distance formula is d=sqrt(∑(xi1-xi2)^2), i=1,2..n, and n refers to the dimension of the calculated Euclidean distance.

[0122] The second deformation determining unit 95 is configured to determine the deformation of the projection transformation field of view of the multiple fields of view except the reference field of view when all the fields of view in the plurality of fields of view are sequentially used as the reference field of view The minimum value of the sum.

[0123] By cyclically operating the field of view selection unit 91, the projection transformation matrix determination unit 92, the projection transformation view field determination unit 93, and the first deformation determination unit 94, it is obtained that all of the multiple fields of view are sequentially used as the reference field of view. The sum of the deformation variables of the projection transformation field of view of the field of view other than the reference field of view in each field of view, and then compare the sum of the obtained multiple deformation variables to determine the minimum value of the sum of the deformation variables.

[0124] The image splicing unit 96 is used for splicing images by using the field of view corresponding to the minimum value of the sum of the deformation variables of the projection transformation field of view as the reference field of view.

[0125] In this embodiment, the reference field of view used by the stitched image is determined by comparing all the fields of view as the reference field of view to generate the cumulative deformation of the panoramic image, and the cumulative deformation is based on the shortest field of view projected to the reference field of view. The path is calculated. Therefore, using the reference field of view determined in this embodiment as the reference field of view of the stitched image can ensure that the cumulative deformation of the obtained panoramic image is minimized, thereby improving the quality of the stitched image.

[0126] A person of ordinary skill in the art can understand that all or part of the steps in the method of the foregoing embodiments can be implemented by a program instructing relevant hardware. The program can be stored in a computer readable storage medium. The storage medium, such as ROM/RAM, magnetic disk, optical disk, etc.