Method for mapping on panoramic image

Abstract

The invention discloses a method for mapping on a panoramic image. The method comprises the following steps: acquiring the panoramic image; a corresponding three-dimensional coordinate system is established for the panoramic image, and the panoramic image covers a 360-degree view field in the horizontal direction and covers a 180-degree view field in the vertical direction with the center of sphere as the original point; a user selects four points on the panoramic image to form a rectangle as a mapping area of the to-be-pasted image in the panoramic image, and the four points are transformed; obtaining a to-be-mapped image, establishing a plane-coordinate system for the to-be-mapped image, and performing transformation processing on four vertexes of the to-be-mapped image; performing target equation calculation on the transformed selected points on the panoramic image and the transformed selected points of the to-be-pasted image; traversing each pixel point of the panoramic image to obtain a final image; according to the method, a reversible transformation model can be accurately described, and forward reversible transformation of the chartlet can be realized.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method for mapping on a panoramic image. Background Art

[0002] In graphic design, people will apply stickers to specified locations in images to meet their needs. In video design, people will apply stickers to specified locations in videos to meet their needs. Naturally, when people obtain panoramic images, they will have the need to apply stickers to specified locations in the panoramic images.

[0003] The current panoramic editors on the market, such as JianE Panorama, Jiushang VR, Xuanyun Panorama, and Kujiale, all treat the image to be pasted as a two-dimensional plane when pasting it into a panoramic picture. By manually adjusting the position, size, and flipping angle of the plane, they gradually move closer to a plane embedded in the panoramic picture. This operation process not only requires the technical level of the staff, but is also time-consuming and labor-intensive. In addition, manual adjustment cannot guarantee that the size and angle of the flat image are consistent with a plane in the panoramic picture. Summary of the Invention

[0004] The purpose of this section is to summarize some aspects of the embodiments of the present invention and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section and the abstract and title of this application to avoid obscuring the purpose of this section, the abstract and the title of the invention, and such simplifications or omissions should not be used to limit the scope of the present invention.

[0005] In view of the problems existing in the panoramic image processing technology in the above-mentioned prior art, the present invention is proposed. The present invention can accurately describe the reversible transformation model and map the forward reversible transformation.

[0006] To solve the above technical problems, the present invention provides the following technical solution: a method for mapping a panoramic image, comprising the following steps:

[0007] Acquire panoramic images;

[0008] A corresponding three-dimensional coordinate system is established for the panoramic image, with the center of the sphere as the origin. The panoramic image covers a 360-degree field of view horizontally and a 180-degree field of view vertically;

[0009] The user selects four points on the panoramic image to form a rectangle as the mapping area of the image to be pasted in the panoramic image, and transforms the four points;

[0010] Obtain the image to be mapped, establish a plane coordinate system for the image to be mapped, and perform transformation processing on the four vertices of the image to be mapped;

[0011] Calculate the target equation for the selected points after the transformation of the panoramic image and the selected points after the transformation of the image to be pasted;

[0012] Traverse each pixel of the panoramic image to obtain the final image.

[0013] As a preferred solution of the method of mapping on a panoramic image of the present invention, the step of transforming the four selected points on the panoramic image is specifically as follows:

[0014] Four points p i (x i ,y i , z i )’s coordinates are p0(x0, y0, z0), p1(x1, y1, z1), p2(x2, y2, z2), p3(x3, y3, z3);

[0015] The average longitude and latitude of the four selected points are converted into u and v, specifically,

[0016] According to the coordinates of the center point of the mapping area Generate the mapping point A'' of point A on the XZ-axis plane. u is the angle between point A'' and the positive axis of the Z-axis, and v is ∠AOA''. Convert it to u and v, where u∈[-180°, 180°] and v∈[-90°, 90°].

[0017] Set 7 parameters rotatez, rotatex, rotatey, HorizonRotate, VorticalRotate, scalex, scaley; set the initial values of the 7 parameters, rotatez, rotatex, rotatey are all set to 0, HorizonRotate = -u, VorticalRotate = v, scalex = 1.0,

[0018] The angle VAngle is obtained by the law of cosines using p2 and p1, and the angle HAngle is obtained by the law of cosines using p0 and p1;

[0019] will p i Normalized to p i ”(x i ”,y i ”, z i ”).

[0020] As a preferred solution of the method for mapping on a panoramic image of the present invention, the step of transforming the four vertices on the image to be mapped is specifically as follows:

[0021] The plane coordinate system takes the upper left corner of the image to be pasted as the origin, the width direction as the X axis, and the height direction as the Y axis. The coordinates of the four vertices of the image to be pasted are q0(0,0), q1(w,0), q2(0,h), q3(w,h), where w and h are the width and height of the image to be pasted respectively.

[0022] q i (i=0, 1, 2 or 3) to perform scaling transformation, as shown in formula (1),

[0023] Will Convert from the plane coordinate system to be mapped to the three-dimensional coordinate system, as shown in formula (2), in express The first element of express The second element of

[0024] right Perform a rotation transformation around its own center, and the rotation matrix calculation is as shown in formula (3) and formula (4).

[0025] Formula (3) is,

[0026]

[0027] Where, roll = -rotatez, pitch = -rotatex, yaw = -rotatey;

[0028] Formula (4) is,

[0029] Yes Perform a rotation transformation around its own center to generate the point;

[0030] right Perform depth migration, as shown in formula (5), Depth is a constant value;

[0031] right Perform a rotation transformation around the origin of the three-dimensional coordinate system. The rotation matrix is calculated as follows:

[0032]

[0033] (6), where pitch2 = HorizonRotate, yaw2 = VorticalRotate;

[0034]

[0035] Will Normalized to

[0036] As a preferred solution of the method of mapping on a panoramic image of the present invention, the target equation calculation formula is:

[0037]

[0038] Use the optimization algorithm to solve the 7 parameters in the objective equation.

[0039] As a preferred solution of the method for mapping on a panoramic image of the present invention, when traversing each pixel point of the panoramic image, the following steps are included:

[0040] For any pixel point in the panoramic image, convert it into three-dimensional coordinates B(a, b, c);

[0041] The origin and point B in the three-dimensional coordinate system form a ray;

[0042] If the ray intersects the X plane, the intersection point is Then the point is covered by the map, and if there is no intersection, then the point does not need to be covered by the map;

[0043] Will After inverse transformation using the above formulas (1) to (7), the point is transformed into a point on the coordinate system of the plane to be mapped, and the point information is taken to cover point B in the panoramic image.

[0044] As a preferred solution of the method of mapping on a panoramic image of the present invention, the specific steps of taking the point information to cover point B in the panoramic image after inverse transformation are as follows:

[0045] The parameters HorizonRotate and VorticalRotate in the objective equation are solved by the optimization algorithm and brought into formula (6), and then obtained according to formula (7):

[0046] Will Substitute into formula (5) to obtain

[0047] Solve the parameters rotatez, rotatex, and rotatey in the target equation through the optimization algorithm and bring them into formula (3), and then get them according to formula (4):

[0048] Will Substitute into formula (2) to obtain

[0049] Solve the parameters scalex and scaley in the target equation through the optimization algorithm and bring them into formula (1). According to formula (1), Find r i Point coordinates;

[0050] Take r i The pixel information of the point is transferred to cover point B in the panoramic image.

[0051] The beneficial effects of the present invention are as follows: the present invention accurately describes a reversible transformation model, replaces the pixels of each point in the area to be mapped with the pixels of the corresponding points in the mapping image, and performs a forward reversible transformation of the mapping; then, an optimization algorithm is used to ensure that the mapping area is geometrically aligned with the target area to reduce distortion; finally, ray-plane detection is used to ensure seamless fusion of the mapping, avoiding blur or jagged edges caused by interpolation in traditional methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. Those skilled in the art can also derive other drawings based on these drawings without inventive effort. Among them:

[0053] Figure 1 It is a schematic diagram of the process of the present invention.

[0054] Figure 2 The coordinate A of the center point of the mapping area and the mapping point A generated on the XZ axis plane.

[0055] Figure 3 The diagram shows the angle between point A” and the positive Z axis.

[0056] Figure 4 Schematic diagram of v being ∠AOA”.

[0057] Figure 5 A schematic diagram of the traversal process.

[0058] Figure 6 is the panoramic image to be mapped.

[0059] Figure 7 The image to be mapped.

[0060] Figure 8 This is a panoramic image after mapping using the present invention. DETAILED DESCRIPTION

[0061] To make the above-mentioned objects, features, and advantages of the present invention more clearly understood, the following detailed description of the specific embodiments of the present invention is given in conjunction with the accompanying drawings. It is obvious that the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in this field without creative work should fall within the scope of protection of the present invention.

[0062] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0063] Secondly, the term "one embodiment" or "embodiment" herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in various places throughout this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive of other embodiments.

[0064] The present invention is described in detail with reference to schematic diagrams. For ease of illustration, cross-sectional views of device structures may be partially enlarged and not to scale when describing embodiments of the present invention. Furthermore, the schematic diagrams are merely illustrative and should not limit the scope of the present invention. Furthermore, in actual production, the three-dimensional dimensions of length, width, and depth should be included.

[0065] In the description of the present invention, it should be noted that the terms "upper, lower, inner, and outer" and other references to orientations or positional relationships are based on the orientations or positional relationships shown in the accompanying drawings and are intended solely to facilitate and simplify the description of the present invention. They are not intended to indicate or imply that the devices or components referred to must have, be constructed, or operate in a specific orientation, and therefore should not be construed as limitations on the present invention. Furthermore, the terms "first, second, or third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0066] In this disclosure, unless otherwise specified or limited, the terms "mounted," "connected," and "connected" should be interpreted broadly. For example, they may refer to fixed, removable, or integral connections. They may also refer to mechanical, electrical, or direct connections, indirect connections through an intermediary, or internal communication between two components. Those skilled in the art will understand the specific meanings of these terms in this disclosure.

[0067] Example 1

[0068] Reference Figure 1, which is the first embodiment of the present invention, provides a method for mapping on a panoramic image, and uses this method to perform forward reversible transformation of the mapping.

[0069] A method for mapping a panoramic image comprises the following steps:

[0070] S1: Acquire panoramic image;

[0071] S2: Establish a corresponding three-dimensional coordinate system for the panoramic image, with the center of the sphere as the origin. The panoramic image covers a 360-degree field of view in the horizontal direction and a 180-degree field of view in the vertical direction;

[0072] S3: The user selects four points on the panoramic image to form a rectangle as the mapping area of the image to be pasted in the panoramic image, and transforms the four points. The specific steps are as follows:

[0073] Four points p i (x i ,y i , z i )’s coordinates are p0(x0, y0, z0), p1(x1, y1, z1), p2(x2, y2, z2), p3(x3, y3, z3);

[0074] The average longitude and latitude of the four selected points are converted into u and v, specifically,

[0075] According to the coordinates of the center point of the mapping area Generate the mapping point A' in the XZ axis plane. u is the angle between point A' and the positive axis of the Z axis, and v is ∠AOA'. Convert it to u and v, u∈[-180°, 180°], v∈[-90°, 90°], as shown in the following example: Figures 2 to 4 ;

[0076] Set 7 parameters rotatez, rotatex, rotatey, HorizonRotate, VorticalRotate, scalex, and scaley. Set the initial values of the 7 parameters: rotatez, rotatex, and rotatey are all set to 0, HorizonRotate = -u, VorticalRotate = v, and scalex = 1.0.

[0077] The angle VAngle is obtained by the law of cosines using p2 and p1, and the angle HAngle is obtained by the law of cosines using p0 and p1;

[0078] will p i Normalized to p i ”(x i”,y i ”, z i ”).

[0079] S4: Get the image to be mapped, establish its own plane coordinate system for the image to be mapped, and take the four vertices of the image to be mapped for transformation processing. The specific steps are:

[0080] The plane coordinate system takes the upper left corner of the image to be pasted as the origin, the width direction as the X axis, and the height direction as the Y axis. The coordinates of the four vertices of the image to be pasted are q0(0,0), q1(w,0), q2(0,h), q3(w,h), where w and h are the width and height of the image to be pasted respectively; q i With p i One-to-one correspondence, if the order of q0, q1, q2, q3 is upper left corner, upper right corner, lower left corner, upper right corner, then the order of p0, p1, p2, p3 is upper left corner, upper right corner, lower left corner, upper right corner;

[0081] q i (i=0, 1, 2 or 3) to perform scaling transformation, as shown in formula (1),

[0082] Will Convert from the plane coordinate system to be mapped to the three-dimensional coordinate system, as shown in formula (2), in express The first element of express The second element of

[0083] right Perform a rotation transformation around its own center, and the rotation matrix calculation is as shown in formula (3) and formula (4).

[0084] Formula (3) is,

[0085]

[0086] Where, roll = -rotatez, pitch = -rotatex, yaw = -rotatey;

[0087] Formula (4) is,

[0088] Yes Perform a rotation transformation around its own center to generate the point;

[0089] right Perform depth migration, as shown in formula (5), Depth is a constant value;

[0090] right Perform a rotation transformation around the origin of the three-dimensional coordinate system. The rotation matrix is calculated as follows:

[0091]

[0092] (6), where pitch2 = HorizonRotate, yaw2 = VorticalRotate;

[0093]

[0094] Will Normalized to

[0095] S5: Calculate the target equation for the selected points after the transformation of the panoramic image and the selected points after the transformation of the image to be pasted. The calculation formula is:

[0096]

[0097] Use an optimization algorithm (the optimization algorithm is preferably the Levenberg-Marquardt algorithm, which is a prior art. CvLevMarq in Opencv is a class for implementing the Levenberg-Marquardt (LM) algorithm, which can be directly called to solve the optimal solution of the 7 parameters. It is not described in detail in this application) to solve the 7 parameters (rotatez, rotatex, rotatey, HorizonRotate, VorticalRotate, scalex, scaley) in the target equation.

[0098] S6: Traverse each pixel of the panoramic image to obtain the final image;

[0099] Among them, such as Figure 5 The specific steps of traversal are:

[0100] For any pixel point in the panoramic image, convert it into three-dimensional coordinates B(a, b, c);

[0101] The origin and point B in the three-dimensional coordinate system form a ray;

[0102] If the ray is parallel to the X-plane ( Four points form an X plane) have an intersection point, the intersection point is Then the point is covered by the map, and if there is no intersection, then the point does not need to be covered by the map;

[0103] Will After inverse transformation using the above formulas (1) to (7), the point is transformed into the coordinate system of the plane to be mapped, and the information of the point is taken to cover point B in the panoramic image;

[0104] After inverse transformation, the specific steps of taking the point information to cover point B in the panoramic image are as follows:

[0105] The parameters HorizonRotate and VorticalRotate in the objective equation are solved by the optimization algorithm and brought into formula (6), and then obtained according to formula (7):

[0106] Will Substitute into formula (5) to obtain

[0107] Solve the parameters rotatez, rotatex, and rotatey in the target equation through the optimization algorithm and bring them into formula (3), and then get them according to formula (4):

[0108] Will Substitute into formula (2) to obtain

[0109] Solve the parameters scalex and scaley in the target equation through the optimization algorithm and bring them into formula (1). According to formula (1), Find r i Point coordinates;

[0110] Take r i The pixel information of the point is transferred to cover point B in the panoramic image.

[0111] The mapping technology of the present invention is used for forward reversible transformation, accurately describing the reversible transformation model, replacing the pixels of each point in the area to be mapped with the pixels of the corresponding points in the mapping image; an optimization algorithm is used to ensure geometric alignment between the mapping area and the target area to reduce distortion; and ray-plane detection is used to ensure seamless fusion of the mapping, avoiding blur or jagged edges caused by interpolation in traditional methods.

[0112] Example 2

[0113] This embodiment provides a method for attaching an image to a panoramic image. The difference from Embodiment 1 is that this embodiment uses scientific demonstration to verify the real effect of this method.

[0114] like Figures 6 to 8 As shown, Figure 6 is the selected panoramic image, Figure 7 is the image to be posted, Figure 8 is the panoramic image after mapping, from Figure 8 It can be seen that in the panoramic image after mapping, the mapping image maintains the aspect ratio of the mapping, the image to be pasted completely fits the surface of the area to be mapped, the system dynamically adjusts to compensate for the longitude and latitude deformation, no perspective distortion will be generated, and the text on the image to be pasted is not stretched or deformed.

[0115] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, and all of these should be included in the scope of the technical solutions of the present invention.

Claims

1. A method for mapping a panoramic image, characterized by: The following steps are included: Acquire panoramic images; A corresponding three-dimensional coordinate system is established for the panoramic image, with the center of the sphere as the origin. The panoramic image covers a 360-degree field of view horizontally and a 180-degree field of view vertically; The user selects four points on the panoramic image to form a rectangle as the mapping area of the image to be pasted in the panoramic image, and transforms the four points; Obtain the image to be mapped, establish a plane coordinate system for the image to be mapped, and perform transformation processing on the four vertices of the image to be mapped; Calculate the target equation for the selected points after the transformation of the panoramic image and the selected points after the transformation of the image to be pasted; Traverse each pixel of the panoramic image to obtain the final image.

2. The method for attaching a map to a panoramic image according to claim 1, wherein: The specific steps of transforming the four selected points on the panoramic image are: Four points p i (x i ,y i , z i )’s coordinates are p0(x0, y0, z0), p1(x1, y1, z1), p2(x2, y2, z2), p3(x3, y3, z3); The average latitude and longitude are set by the selected four points and converted into u and v, specifically, According to the coordinate A of the center point of the mapping area Generate the mapping point A'' of point A on the XZ-axis plane. u is the angle between point A'' and the positive axis of the Z-axis, and v is ∠AOA''. Convert it to u and v, where u∈[-180°, 180°] and v∈[-90°, 90°]. Set 7 parameters rotatez, rotatex, rotatey, HorizonRotate, VorticalRotate, scalex, scaley; set the initial values of the 7 parameters, rotatez, rotatex, rotatey are all set to 0, HorizonRotate = -u, VorticalRotate = v, scalex = 1.0, The angle VAngle is obtained by the law of cosines using p2 and p1, and the angle HAngle is obtained by the law of cosines using p0 and p1; will p i Normalized to p" i (x” i ,y” i ,z” i ).

3. The method for attaching a map to a panoramic image according to claim 2, wherein: The specific steps for transforming the four vertices on the texture image are: The plane coordinate system takes the upper left corner of the image to be pasted as the origin, the width direction as the X axis, and the height direction as the Y axis. The coordinates of the four vertices of the image to be pasted are q0(0,0), q1(w,0), q2(0,h), q3(w,h), where w and h are the width and height of the image to be pasted respectively. q i (i=0, 1, 2 or 3) to perform scaling transformation, as shown in formula (1), Will Convert from the plane coordinate system to be mapped to the three-dimensional coordinate system, as shown in formula (2), in express The first element of express The second element of right Perform a rotation transformation around its own center, and the rotation matrix calculation is as shown in formula (3) and formula (4). Formula (3) is, Where, roll = -rotatez, pitch = -rotatex, yaw = -rotatey; Formula (4) is, Yes Perform a rotation transformation around its own center to generate the point; right Perform depth migration, as shown in formula (5), Depth is a constant value; right Perform a rotation transformation around the origin of the three-dimensional coordinate system. The rotation matrix is calculated as follows: (6), where pitch2 = HorizonRotate, yaw2 = VorticalRotate; Will Normalized to 4. The method for attaching a map to a panoramic image according to claim 3, wherein: The target equation is calculated as follows: Use the optimization algorithm to solve the 7 parameters in the objective equation.

5. The method for attaching a map to a panoramic image according to claim 4, wherein: When traversing each pixel of the panoramic image, the following steps are included: For any pixel point in the panoramic image, convert it into three-dimensional coordinates B(a, b, c); The origin and point B in the three-dimensional coordinate system form a ray; If the ray intersects the X plane, the intersection point is Then the point is covered by the map, and if there is no intersection, then the point does not need to be covered by the map; Will After inverse transformation using the above formulas (1) to (7), the point is transformed into a point on the coordinate system of the plane to be mapped, and the point information is taken to cover point B in the panoramic image.

6. The method for attaching a map to a panoramic image according to claim 5, wherein: After inverse transformation, the specific steps of taking the point information to cover point B in the panoramic image are as follows: The parameters HorizonRotate and VorticalRotate in the objective equation are solved by the optimization algorithm and brought into formula (6), and then obtained according to formula (7): Will Substitute into formula (5) to obtain Solve the parameters rotatez, rotatex, and rotatey in the target equation through the optimization algorithm and bring them into formula (3), and then get them according to formula (4): Will Substitute into formula (2) to obtain Solve the parameters scalex and scaley in the target equation through the optimization algorithm and bring them into formula (1). According to formula (1), Find r i Point coordinates; Take r i The pixel information of the point is transferred to cover point B in the panoramic image.

Images