Panoramic image interpolation method and device

Abstract

The embodiment of the invention provides a panoramic image interpolation method and device. Comprising the following steps: determining a point I0 on a panoramic longitude and latitude graph I; mapping to a fisheye image according to a conversion model, and taking four adjacent integer coordinate adjacent points; a unit sphere S is determined according to the panoramic longitude and latitude graph I, the panoramic longitude and latitude graph I is pasted to the unit sphere, plane mapping is carried out by taking the panoramic longitude and latitude graph I as the center, a projection plane P is obtained, calculation is carried out by using a least square method, and a homography matrix is obtained; and completing interpolation according to the homography matrix. Through the steps of mapping, projection, matrix solving and interpolation, the edge sawtooth problem is optimized, meanwhile, the accuracy and universality of the interpolation process are guaranteed, and the method is suitable for various panoramic unfolding scenes based on fisheye images.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method and apparatus for interpolating panoramic images. Background Technology

[0002] Image processing is the technique of using computers to analyze images to achieve desired results. It is also known as image processing. Image processing generally refers to digital image processing. A digital image is a large two-dimensional array obtained by capturing images using equipment such as industrial cameras, video cameras, and scanners. The elements of this array are called pixels, and their values ​​are called grayscale values.

[0003] Typical panoramic images are unfolded using an interpolation model based on a fisheye image and the intrinsic and extrinsic parameters between calibrated lenses. Bilinear interpolation is commonly used. However, because the actual fisheye unfolding model is non-linear, it easily leads to jagged edges. Therefore, this invention proposes a panoramic image interpolation method and apparatus to at least partially solve the problems that may exist in the prior art. Summary of the Invention

[0004] In view of the above problems, a panoramic image interpolation method and apparatus are proposed to overcome or at least partially solve the above problems.

[0005] A panoramic image interpolation method, the method comprising:

[0006] Obtain a panoramic latitude and longitude map I, and determine a point I0 on the panoramic latitude and longitude map I;

[0007] Based on the preset latitude and longitude map to fisheye map conversion model, Mapping to fisheye image Above, recorded as and take The four nearest integer coordinates of the points are denoted as . ;

[0008] The unit sphere S is determined based on the panoramic latitude and longitude map I, and the panoramic latitude and longitude map I is then affixed to the unit sphere. Up, obtain exist Corresponding spatial points on as well as exist The point on is ;by Perform a planar mapping centered on the target plane to obtain the projection plane P, and determine... Four projection points projected onto projection plane P ;

[0009] Based on the four integer coordinates of neighboring points and 4 projection points The homography matrix is ​​obtained by using the least squares method. ;

[0010] Based on the homography matrix And the point F1(x, y) on the fisheye diagram F, which is the center point P0(0, 0) of the projection plane P, then F1 is the point where I0 is corrected on F, that is... , complete the interpolation.

[0011] Optionally, acquiring the panoramic latitude and longitude map I and determining a point I0 on the panoramic latitude and longitude map I includes:

[0012] Obtain the fisheye image F, and unfold it with latitude and longitude map according to the preset unfolding model to obtain panoramic latitude and longitude map I. Take a point on panoramic latitude and longitude map I and denote it as I0.

[0013] Optionally, the step of converting a latitude and longitude map to a fisheye image according to a preset conversion model is performed. Mapping to fisheye image Above, recorded as ,include:

[0014] Take a point I0 on the panoramic latitude and longitude map I, and set the longitude of point I0 as φ and the latitude as θ;

[0015] The radius r after fisheye distortion is obtained by calculating using the distortion function f(θ) and latitude. d =f(θ); where f(θ) is a polynomial function with θ as the variable;

[0016] Based on radius r d and x-axis and the ordinate We perform a transformation calculation to obtain the mapping of I0 on the fisheye image F, denoted as F0.

[0017] Optionally, the taking The four nearest integer coordinates of the points are denoted as . This includes: taking the four integer coordinate neighbors F0. 00 F 01 F 10 F 11 Let these 4 points be F. g4 , specifically:

[0018] Take coordinates xF 00 =floor(xF0), yF 00 =floor(yF0) is F 00 Where floor represents rounding down to the nearest integer;

[0019] Take coordinates xF 01 =xF 00 +1, yF 01 =yF 00 For F 01 ;

[0020] Take coordinates xF 10 =xF 00 yF 10 =yF 00 +1 is F 10 ;

[0021] Take coordinates xF 11 =xF 00 +1, yF 11 =yF 00 +1 is F 11 .

[0022] Optionally, the step involves determining the unit sphere S based on the panoramic latitude and longitude map I, and then attaching the panoramic latitude and longitude map I to the unit sphere. Up, obtain exist Corresponding spatial points on as well as exist The point on is ;include:

[0023] Based on the surface containing the longitude and latitude of the panoramic latitude and longitude map I, a unit sphere S is determined by extending this surface, and the panoramic latitude and longitude map I is then attached to the unit sphere. Above, obtain and The corresponding spatial point located on the unit sphere S ;

[0024] In terms of spatial points Centered on the panoramic latitude and longitude map I, a planar mapping is performed to obtain the projection plane P, and... The corresponding location is in The point on is .

[0025] Optionally, the so-called Perform a planar mapping centered on the target plane to obtain the projection plane P, and determine... Four projection points projected onto projection plane P ,include:

[0026] by Centered on, determine with The plane of the cut surface or the plane parallel to the cut surface is mapped to obtain the projection plane P. Projected onto the projection plane P, we get P00 P 01 P 10 P 11 Four projection points, denoted as .

[0027] Optionally, the so-called Centered on, determine with The plane of the cut surface or the plane parallel to the cut surface is mapped to obtain the projection plane P. Projected onto the projection plane P, we get P 00 P 01 P 10 P 11 Four projection points, denoted as Specifically, it includes:

[0028] Setting S p Let P be any point on the sphere S. p Let its mapping point on the projection plane P be, then we have Where View is the view matrix and Project is the perspective matrix;

[0029] Using the view matrix, the spatial point S0 is rotated to the point S1=(0, 0, -1) on the sphere S. The view matrix is ​​obtained as follows: S0 is rotated 90 degrees counterclockwise along the same longitude to obtain u;

[0030] Where t = S0, r = cross(t, u), v = cross(r, t), and cross represents the cross product of two vectors.

[0031] but ,

[0032] Then in On On a plane, according to By projecting, a projection plane is obtained. ,in,

[0033] , , The focal length represents the projection imaging plane;

[0034] Seeking Coordinates are ), Coordinates are , coordinate( , Coordinates are .

[0035] Optionally, the step of calculating the homography matrix using the least squares method based on four integer coordinate neighboring points and four projection points includes:

[0036] According to P g4 and F g4 The coordinates are used to calculate the homography matrix H using the least squares method, such that... , specifically,

[0037] set up ,

[0038] Then there is , ;

[0039] Rearranging terms yields a linear form.

[0040]

[0041] Further,

[0042]

[0043] make

[0044]

[0045] ,

[0046] Then there is Then, we can solve the equation: That is, to obtain .

[0047] Optionally, based on the homography matrix H and the point F1(x, y) of the center point P0(0, 0) of the projection plane P on the fisheye image F, F1 is the corrected value of I0 on F, i.e. To complete the interpolation, including:

[0048] According to the homography matrix H, the point F1(x, y) on the fisheye diagram F of the center point P0(0, 0) of the projection plane P is the point where I0 is corrected on F. Complete the interpolation;

[0049] Using the homography matrix H, we obtain After obtaining the coordinates of F1, the color value is obtained by bilinear interpolation, which is the color value of point I0.

[0050] An interpolation device for panoramic images, the device comprising:

[0051] The image acquisition module is used to acquire a panoramic latitude and longitude map I and determine a point I0 on the panoramic latitude and longitude map I;

[0052] The image mapping module is used to convert latitude and longitude maps into fisheye images according to a preset conversion model. Mapping to fisheye image Above, recorded as and take The four nearest integer coordinates of the points are denoted as . ;

[0053] The coordinate projection module is used to determine the unit sphere S based on the panoramic latitude and longitude map I, and to attach the panoramic latitude and longitude map I to the unit sphere. Up, obtain exist Corresponding spatial points on as well as exist The point on is ;by Perform a planar mapping centered on the target plane to obtain the projection plane P, and determine... Four projection points projected onto projection plane P ;

[0054] The coordinate calculation module is used to calculate the coordinates of four neighboring integer coordinates. and 4 projection points The homography matrix is ​​obtained by using the least squares method. ;

[0055] The interpolation calculation module is used to calculate the interpolation value based on the homography matrix. and projection plane center point fisheye image Points on ,but That is exist The value point after the upper correction, i.e. , complete the interpolation.

[0056] A computer-readable storage medium storing a computer program that, when executed by a processor, implements a method for interpolating panoramic images.

[0057] A computer program product includes a computer program that, when executed by a processor, implements an interpolation method for panoramic images.

[0058] The embodiments of the present invention have the following advantages:

[0059] By acquiring a panoramic latitude and longitude map I and determining a point I0 on the panoramic latitude and longitude map I; and according to a preset conversion model from latitude and longitude map to fisheye image, the conversion is performed... Mapping to fisheye image Above, recorded as and take The four nearest integer coordinates of the points are denoted as . Determine the unit sphere S based on the panoramic latitude and longitude map I, and then attach the panoramic latitude and longitude map I to the unit sphere. Up, obtain exist Corresponding spatial points on as well as exist The point on is ;by Perform a planar mapping centered on the target plane to obtain the projection plane P, and determine... Four projection points projected onto projection plane P Based on 4 integer coordinate neighboring points and 4 projection points The homography matrix is ​​obtained by using the least squares method. According to the homography matrix And the point F1(x, y) on the fisheye diagram F, which is the center point P0(0, 0) of the projection plane P, then F1 is the point where I0 is corrected on F, that is... The interpolation process is completed through mapping, projection, matrix solving, and interpolation steps. This process determines the core workflow of panoramic image interpolation, breaking through the nonlinear adaptation limitations of traditional bilinear interpolation, solving the edge jaggedness problem at its root, and ensuring the accuracy and versatility of the interpolation process. It is applicable to various panoramic unfolding scenarios based on fisheye images. Attached Figure Description

[0060] To more clearly illustrate the technical solution of the present invention, the accompanying drawings used in the description of the present invention will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0061] Figure 1 This is a flowchart of the steps of a panoramic image interpolation method provided in some embodiments of the present invention;

[0062] Figure 2 This is a structural module diagram of a panoramic image interpolation device provided in some embodiments of the present invention. Detailed Implementation

[0063] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0064] Reference Figure 1 The diagram illustrates a panoramic image interpolation method provided by some embodiments of the present invention, which may specifically include the following steps:

[0065] Step S1: Obtain panoramic latitude and longitude map I, and determine a point I0 on the panoramic latitude and longitude map I;

[0066] Step S2: Based on the preset latitude and longitude map to fisheye map conversion model, convert... Mapping to fisheye image Above, recorded as and take The four nearest integer coordinates of the points are denoted as . ;

[0067] Step S3: Determine the unit sphere S based on the panoramic latitude and longitude map I, and attach the panoramic latitude and longitude map I to the unit sphere. Up, obtain exist Corresponding spatial points on as well as exist The point on is ;by Perform a planar mapping centered on the target plane to obtain the projection plane P, and determine... Four projection points projected onto projection plane P ;

[0068] Step S4: Based on the four integer coordinate neighboring points and 4 projection points The homography matrix is ​​obtained by using the least squares method. ;

[0069] Step S5: Based on the homography matrix And the point F1(x, y) on the fisheye diagram F, which is the center point P0(0, 0) of the projection plane P, then F1 is the corrected value of I0 on F, that is... , complete the interpolation.

[0070] By employing steps of mapping, projection, matrix solving, and interpolation, the core process of panoramic image interpolation is determined, overcoming the nonlinear adaptation limitations of traditional bilinear interpolation, optimizing the edge jaggedness problem, and ensuring the accuracy and versatility of the interpolation process. This approach is applicable to various panoramic unfolding scenarios based on fisheye images.

[0071] It should be noted that since panoramic cameras are generally fisheye lens cameras, the images output by such cameras will produce large distortions. The images processed by traditional methods will have distorted edges and a high rate of jagged edges. Especially when applied outdoors, the smoothness of details such as road markings and building corners is insufficient. The method of this application can improve the smoothness of image edges.

[0072] In some embodiments of this application, the above step S1, obtaining a panoramic latitude and longitude map I and determining a point I0 on the panoramic latitude and longitude map I, includes: obtaining a fisheye image F, and unfolding the latitude and longitude map according to a preset unfolding model to obtain a panoramic latitude and longitude map I, and taking a point on the panoramic latitude and longitude map I, denoted as I0.

[0073] In some embodiments of this application, in step S2 above, according to a preset conversion model from latitude and longitude map to fisheye image, the... Mapping to fisheye image Above, recorded as This includes: taking a point I0 on the panoramic latitude and longitude map I, and setting the longitude of point I0 as φ and the latitude as θ;

[0074] The radius r after fisheye distortion is obtained by calculating using the distortion function f(θ) and latitude. d =f(θ); where f(θ) is a polynomial function with θ as the variable;

[0075] Based on radius r d and x-axis and the ordinate We perform a transformation calculation to obtain the mapping of I0 on the fisheye image F, denoted as F0.

[0076] set up The longitude of the point is latitude is The radius after fisheye distortion is , It is a distortion function. Generally speaking A polynomial function with variable . Then x-coordinate y-axis Refine the mapping logic from I0 to F0 and F g4The selection rules ensure the accuracy of the mapping points and the representativeness of the neighboring points, providing reliable coordinate data support for subsequent projection transformation and matrix solving.

[0077] Panoramic images are generally unfolded using latitude and longitude maps, denoted as panoramic latitude and longitude maps. . Represented as One point above. Based on the conversion model from latitude and longitude maps to fisheye maps, [the following is done / implied]. Mapping to fisheye image Above, recorded as .Pick The four nearest integer coordinates , , , Let these 4 points be... .

[0078] In step S2 above, take The four nearest integer coordinates of the points are denoted as . , including: taking The four nearest integer coordinates , , , Let these 4 points be... , specifically:

[0079] Take coordinates , for ,in, Indicates rounding down to the nearest integer;

[0080] Take coordinates , for

[0081] Take coordinates , +1 is ;

[0082] Take coordinates xF 11 =xF 00 +1, yF 11 =yF 00 +1 is .

[0083] Through standardization The coordinate calculation rules, through rounding down and neighbor point offset logic, ensure... The coordinates are integers, which conforms to the discrete characteristics of image pixels, avoids calculation errors caused by non-integer coordinates, and ensures the feasibility of subsequent calculations.

[0084] In some embodiments of this application, step S3 above, determining the unit sphere S based on the panoramic latitude and longitude map I, and attaching the panoramic latitude and longitude map I to the unit sphere... Up, obtain exist Corresponding spatial points on as well as exist The point on is ,include:

[0085] Based on the surface containing the longitude and latitude of the panoramic latitude and longitude map I, a unit sphere S is determined by extending this surface, and the panoramic latitude and longitude map I is then attached to the unit sphere. Above, obtain and The corresponding spatial point located on the unit sphere S ;

[0086] In terms of spatial points Centered on the panoramic latitude and longitude map I, a planar mapping is performed to obtain the projection plane P, and... The corresponding location is in The point on is .

[0087] A connection is established between the panoramic latitude and longitude map I, the unit sphere S, and the projection plane P. Through spherical mapping and projection transformation, the nonlinear fisheye unfolding model is transformed into a quantifiable planar projection relationship, providing a reasonable spatial transformation logic for solving the homography matrix. A projection plane P is established centered at S0, and F... g4 The corresponding S g4 Projecting onto P yields P g4 The projection process fully considers spatial geometric relationships to ensure P g4 With F g4 The correspondence is consistent.

[0088] Furthermore, in step S3 above, with Perform a planar mapping centered on the target plane to obtain the projection plane P, and determine... Four projection points projected onto projection plane P ,include:

[0089] by Centered on, determine with The plane of the cut surface or the plane parallel to the cut surface is mapped to obtain the projection plane P. Projected onto the projection plane P, we get P 00 P 01 P 10 P 11 Four projection points, denoted as .

[0090] Furthermore, the aforementioned Centered on, determine with The plane of the cut surface or the plane parallel to the cut surface is mapped to obtain the projection plane P. Projected onto the projection plane P, we get P 00 P 01 P 10 P 11 Four projection points, denoted as Specifically, it includes:

[0091] Setting S p Let P be any point on the sphere S. p Let its mapping point on the projection plane P be, then we have Where View is the view matrix and Project is the perspective matrix;

[0092] Using the view matrix, the spatial point S0 is rotated to the point S1=(0, 0, -1) on the sphere S. The view matrix is ​​obtained as follows: S0 is rotated 90 degrees counterclockwise along the same longitude to obtain u;

[0093] Where t = S0, r = cross(t, u), v = cross(r, t), and cross represents the cross product of two vectors.

[0094] but ,

[0095] Then in On On a plane, according to By projecting, a projection plane is obtained. ,in,

[0096] , , The focal length represents the projection imaging plane;

[0097] Seeking Coordinates are ), Coordinates are , coordinate( , Coordinates are .

[0098] By refining the solution process for the View matrix and Project matrix, and clarifying the rules for vector operations and matrix construction, the establishment of the projection plane P and the projection point P are made possible. g4 The calculation is operable and avoids the loss of accuracy caused by the ambiguity of the projection rules.

[0099] Furthermore, in step S4 above, based on the four integer coordinate neighboring points and the four projection points, the homography matrix is ​​calculated using the least squares method, including:

[0100] According to P g4 and F g4 The coordinates are used to calculate the homography matrix H using the least squares method, such that... , specifically,

[0101] set up ,

[0102] Then there is , ;

[0103] Rearranging terms yields a linear form.

[0104]

[0105] Further,

[0106]

[0107] make

[0108]

[0109] ,

[0110] Then there is Then, we can solve the equation: That is, to obtain .

[0111] It should be noted that the above Multiplying matrix A and matrix h yields matrix b.

[0112] Based on the above-described solution logic for the homography matrix H, specifically by establishing a system of linear equations using the least squares method, we can ensure that the H matrix accurately represents P. g4 With F g4 The mapping relationship provides a reliable matrix basis for the subsequent calculation of correction points.

[0113] Furthermore, in step S5 above, based on the homography matrix and the point F1(x, y) of the center point P0(0, 0) of the projection plane P on the fisheye diagram F, F1 is the corrected value of I0 on F, i.e. To complete the interpolation, including:

[0114] According to the homography matrix H, the point F1(x, y) on the fisheye diagram F of the center point P0(0, 0) of the projection plane P is the point where I0 is corrected on F. Complete the interpolation;

[0115] Using the homography matrix H, we obtain After obtaining the coordinates of F1, the color value is obtained by bilinear interpolation, which is the color value of point I0.

[0116] The final steps of interpolation are clearly defined. The correction point F1 is determined using the H matrix and P0, and color values ​​are obtained using bilinear interpolation. This ensures the continuity and accuracy of the color at point I0, further enhancing the visual effect of the panoramic image. This results in a natural transition between image color values ​​and surrounding pixels, without any jagged edges.

[0117] like Figure 2 As shown, some embodiments of the present invention also provide an apparatus for interpolating panoramic images, used to implement the above-mentioned panoramic image interpolation method, which may specifically include the following modules:

[0118] Image acquisition module 100 is used to acquire a panoramic latitude and longitude map I and determine a point I0 on the panoramic latitude and longitude map I;

[0119] Image mapping module 200 is used to convert latitude and longitude maps into fisheye images according to a preset conversion model. Mapping to fisheye image Above, recorded as and take The four nearest integer coordinates of the points are denoted as . ;

[0120] The coordinate projection module 300 is used to determine the unit sphere S based on the panoramic latitude and longitude map I, and to attach the panoramic latitude and longitude map I to the unit sphere. Up, obtain exist Corresponding spatial points on as well as exist The point on is ;by Perform a planar mapping centered on the target plane to obtain the projection plane P, and determine... Four projection points projected onto projection plane P ;

[0121] The coordinate calculation module 400 is used to calculate the coordinates of four integer neighboring points. and 4 projection points The homography matrix is ​​obtained by using the least squares method. ;

[0122] Interpolation calculation module 500 is used to calculate based on the homography matrix. And the point F1(x, y) on the fisheye diagram F, which is the center point P0(0, 0) of the projection plane P, then F1 is the corrected value of I0 on F, that is... , complete the interpolation.

[0123] Some embodiments of the present invention also provide an electronic device, including a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program, when executed by the processor, implements the method described above.

[0124] Some embodiments of the present invention also provide a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, it implements the method described above.

[0125] Some embodiments of the present invention also provide a computer program product, including a computer program that, when executed by a processor, implements the method described above.

[0126] As the device embodiment is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.

[0127] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0128] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, apparatus, or computer program products. Therefore, embodiments of the present invention can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of the present invention can take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0129] Embodiments of the present invention are described with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0130] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0131] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0132] Although preferred embodiments of the present invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present invention.

[0133] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes the aforementioned element.

[0134] The above provides a detailed description of the panoramic image interpolation method and apparatus. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. An interpolation method of a panoramic image, characterized by, The method includes: acquiring a panoramic latitude-longitude map and determining the panoramic latitude-longitude map a point on the above ; Based on the preset latitude and longitude map to fisheye map conversion model, Mapping to fisheye image Above, recorded as and take The four nearest integer coordinates of the points are denoted as . ; According to the panoramic latitude and longitude map Determine the unit ball and the panoramic latitude and longitude map Stick to unit ball Up, obtain exist Corresponding spatial points on as well as exist The point on is ;by By performing planar mapping around the center, the projection plane is obtained. and determine Projected onto the projection plane The four projection points on ; Based on the four integer coordinates of neighboring points and 4 projection points The homography matrix is ​​obtained by using the least squares method. ; Based on the homography matrix and projection plane center point fisheye image Points on ,but That is exist The value point after the upper correction, i.e. , complete the interpolation.

2. The method according to claim 1, characterized in that, The acquisition of panoramic latitude and longitude maps And determine the panoramic latitude and longitude map. A little bit ,include: Obtain fisheye image It unfolds the map using latitude and longitude based on a preset unfolding model to obtain a panoramic latitude and longitude map. And take a panoramic latitude and longitude map Let's take a point above and denote it as... .

3. The method according to claim 1, characterized in that, The conversion model from a preset latitude and longitude map to a fisheye image will... Mapping to fisheye image ,include: Take panoramic latitude and longitude map A little bit ,set up The longitude of the point is φ, and the latitude is θ. Through distortion function Calculated using latitude, the radius after fisheye distortion is obtained as follows: ;in, Let θ be a polynomial function; According to radius and x-axis and the ordinate Perform conversion calculations to obtain fisheye image The mapping on is denoted as .

4. The method according to claim 1, characterized in that, The taking The four nearest integer coordinates of the points are denoted as . , including: taking The four nearest integer coordinates , , , Let these 4 points be... , specifically: Take coordinates , for ,in, Indicates rounding down to the nearest integer; Take coordinates , for Take coordinates , +1 is ; Take coordinates xF 11 =xF 00 +1, yF 11 =yF 00 +1 is .

5. The method according to claim 1, characterized in that, The unit sphere S is determined based on the panoramic latitude and longitude map I, and the panoramic latitude and longitude map I is then attached to the unit sphere. Up, obtain exist Corresponding spatial points on as well as exist The point on is ;include: Based on the surface containing the longitude and latitude of the panoramic latitude and longitude map I, a unit sphere S is determined by extending this surface, and the panoramic latitude and longitude map I is then attached to the unit sphere. Above, obtain and The corresponding spatial point located on the unit sphere S ; With spatial points Centered on the panoramic latitude and longitude map I, a planar mapping is performed to obtain the projection plane P, and... The corresponding location is in The point on is .

6. The method according to claim 1, characterized in that, The Perform a planar mapping centered on the target plane to obtain the projection plane P, and determine... Four projection points projected onto projection plane P ,include: by Centered on, determine with The plane of the cut surface or the plane parallel to the cut surface is mapped to obtain the projection plane P. Projected onto the projection plane P, we get P 00 P 01 P 10 P 11 Four projection points, denoted as .

7. The method according to claim 6, characterized in that, The Centered on, determine with The plane of the cut surface or the plane parallel to the cut surface is mapped to obtain the projection plane P. Projected onto the projection plane P, we get P 00 P 01 P 10 P 11 Four projection points, denoted as Specifically, it includes: Setting S p Let P be any point on the sphere S. p Let its mapping point on the projection plane P be, then we have Where View is the view matrix and Project is the perspective matrix; Using the view matrix, the spatial point S0 is rotated to the point S1=(0, 0, -1) on the sphere S. The view matrix is ​​obtained as follows: S0 is rotated 90 degrees counterclockwise along the same longitude to obtain u; Where t = S0, r = cross(t, u), v = cross(r, t), and cross represents the cross product of two vectors. but , Then in On On a plane, according to By projecting, a projection plane is obtained. ,in, , , The focal length represents the projection imaging plane; Seek Coordinates are ), Coordinates are , coordinate( , Coordinates are .

8. The method according to claim 1, characterized in that, The homography matrix is ​​calculated using the least squares method based on four integer coordinate neighboring points and four projection points, including: According to P g4 and F g4 The coordinates are used to calculate the homography matrix H using the least squares method, such that... , specifically, set up , Then there is , ; Rearranging terms yields a linear form. Further, make 、 Then there is Then, we can solve the equation: That is, to obtain .

9. The method according to claim 1, characterized in that, Based on the homography matrix H and the point F1(x, y) of the center point P0(0, 0) of the projection plane P on the fisheye image F, F1 is the corrected value of I0 on F, i.e. To complete the interpolation, including: According to the homography matrix H, the point F1(x, y) on the fisheye diagram F of the center point P0(0, 0) of the projection plane P is the point where I0 is corrected on F. Complete the interpolation; Using the homography matrix H, we obtain After obtaining the coordinates of F1, the color value is obtained by bilinear interpolation, which is the color value of point I0.

10. A panoramic image interpolation device, characterized in that, The device includes: The image acquisition module is used to acquire a panoramic latitude and longitude map I and determine a point I0 on the panoramic latitude and longitude map I; The image mapping module is used to convert latitude and longitude maps into fisheye images according to a preset conversion model. Mapping to fisheye image Above, recorded as and take The four nearest integer coordinates of the points are denoted as . ; The coordinate projection module is used to determine the unit sphere S based on the panoramic latitude and longitude map I, and to attach the panoramic latitude and longitude map I to the unit sphere. Up, obtain exist Corresponding spatial points on as well as exist The point on is ;by Perform a planar mapping centered on the target plane to obtain the projection plane P, and determine... Four projection points projected onto projection plane P ; The coordinate calculation module is used to calculate the coordinates of four neighboring integer coordinates. and 4 projection points The homography matrix is ​​obtained by using the least squares method. ; The interpolation calculation module is used to calculate the interpolation value based on the homography matrix. And the point F1(x, y) on the fisheye diagram F, which is the center point P0(0, 0) of the projection plane P, then F1 is the point where I0 is corrected on F, that is... , complete the interpolation.

Images