A method for generating panoramic video from panoramic pictures

Abstract

The present invention relates to a method for generating a panoramic video from a panoramic picture. The method is based on the panoramic picture, superimposes scene annotation materials such as audio, video, and pictures inside the panoramic picture, adopts a universal video format as the panoramic video, and enriches the display elements of the panoramic picture. The panoramic video generated by the present invention can be directly played and used on a variety of short video platforms such as Douyin and Xiaohongshu, thereby enhancing the use effect and dissemination range of the panoramic picture.

Description

Technical Field

[0001] The invention relates to a method for generating a panoramic video from a panoramic picture, and belongs to the technical field of generation and playback of panoramic videos. Background Art

[0002] As a new video field, panoramic video is usually a spherical video with a 360° field of view. Standing inside the sphere and watching it, it feels like being in real space. By rotating the spherical video, you can see whatever you want in the 360° space and be completely immersed in the environment shown in the video.

[0003] Panoramic video is an interactive multimedia, but currently panoramic video is mainly generated by 360-degree shooting with dedicated panoramic cameras and other equipment, or generated based on panoramic pictures. The visual elements of this type of panoramic video are mainly images that actually exist in space and continuous panoramic images. It lacks scene depth annotation and interactive information. Therefore, the existing technology supplements the details of the panoramic video by adding hotspots to the panoramic video to associate various pictures, videos, sounds, etc., such as panoramic house viewing, panoramic digital scenic spots, panoramic exhibition hall visits, etc., which have more typical applications.

[0004] Currently, panoramic house viewing, panoramic digital scenic spots and other contents generated based on panoramic pictures, although some interactive materials such as audio, video or pictures have been added, they often exist in the form of HTML5 programs, or can only be opened with a dedicated player. The playback and interaction of hot data are achieved through program settings. However, they cannot be directly played and used on the current mainstream short video platforms (such as: Douyin, Xiaohongshu, etc.). Summary of the invention

[0005] The technical problem to be solved by the present invention is: to provide a method for generating a panoramic video by using panoramic pictures, which can add hot spots to the panoramic video. The panoramic video generated by this method can be played and used on mainstream short video platforms.

[0006] In order to solve the above technical problems, the technical solution proposed by the present invention is: a method for generating a panoramic video from a panoramic picture, comprising the following steps:

[0007] S1. Select a panoramic picture for generating a panoramic video;

[0008] S2, importing the picture material into the panoramic picture to merge the picture material with the panoramic picture;

[0009] The method of importing any image material into a panoramic image includes the following steps:

[0010] S21, convert the panoramic image into a spherical latitude and longitude coordinate format, assuming that the radius of the sphere is R and the center of the sphere is O, and select a position on the sphere As the center point of the picture material, and make the picture material perpendicular to OE;

[0011] S22, let the pixel width of the panoramic image be W P , high is H P , the pixel width of the image material is W, the height is H, and the spherical radius R is normalized to 1;

[0012] Take the center point of the image material as the origin, establish a rectangular coordinate system, traverse all pixels in the image material, and for any pixel A(x,y), its normalized coordinates are marked as (u,v), then:

[0013] u=2x / W-1,v=2y / H-1, where u∈[-1,1],v∈[-1,1];

[0014] S23, calculating the longitude and latitude coordinates of each pixel in the image material projected onto the spherical surface

[0015] where θ∈[-π / 2,π / 2],

[0016] in

[0017] S24, convert the longitude and latitude coordinates of each pixel in the image material into the coordinates (X p ,Y p ):

[0018]

[0019] S25, based on the coordinate position (X p ,Y p ), replacing pixels at corresponding positions in the panoramic image with pixels in the picture material, so that the picture material and the panoramic image are merged to obtain a fused panoramic image;

[0020] S3. All panoramic pictures obtained after processing in step S2 are synthesized into a panoramic video according to a set time slice sequence.

[0021] The present invention performs secondary editing and synthesis on the panoramic picture and the picture material used as the scene annotation material to obtain a fused panoramic picture, enriching the display elements of the panoramic picture, and serving as a frame of a panoramic video, thereby generating a panoramic video according to the time slice sequence. In other words, the present invention can directly display hotspot data in a panoramic video. Compared with the prior art, the scene annotation material no longer needs to be displayed through a program, but is directly played in a universal panoramic video format, so that it can be directly used on a variety of short video platforms such as Douyin and Xiaohongshu, enhancing the use effect and dissemination range of panoramic pictures. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 It is a schematic diagram of converting a panoramic image into a spherical latitude and longitude coordinate format.

[0023] Figure 2 It is a schematic diagram of the location of image materials.

[0024] Figure 3 It is a schematic diagram of importing image materials into a panoramic image.

[0025] Figure 4 This is a schematic diagram of the time slice setting. DETAILED DESCRIPTION

[0026] The existing technology adds various hot data such as pictures, videos, and sounds to panoramic videos, and requires the use of programs to interact with the panoramic videos. Therefore, it cannot be directly played and used on short video platforms, and a solution is urgently needed.

[0027] Example 1

[0028] This embodiment shows a method for generating a panoramic video from a panoramic picture, which fuses various picture hot spots into the panoramic picture and then generates a panoramic video. The method includes the following steps:

[0029] S1. Select a panoramic picture from which a panoramic video needs to be generated, and then generate a panoramic video based on the panoramic picture.

[0030] S2. Set scene annotations, select required hotspot data of picture materials, import the picture materials into the panoramic picture, and merge the picture materials with the panoramic picture; preferably, when importing the picture materials, the length and width of the picture can be appropriately adjusted as needed.

[0031] The method of importing any image material into a panoramic image includes the following steps:

[0032] S21, such as Figure 1 and Figure 2 As shown, the panoramic image is converted into a spherical latitude and longitude coordinate format. Assuming that the radius of the sphere is R and the center of the sphere is O, a position is selected on the sphere. As the center point of the image material, and make the image material perpendicular to OE, where θ 0 is the latitude, is the longitude;

[0033] S22, after setting the position and size of the picture material, overlay the picture material into the corresponding area of ​​the panoramic picture, and then generate a new panoramic picture, that is, Figure 3 As shown, let the pixel width of the panoramic image be W P , high is H P , the pixel width of the image material is W, the height is H, and the spherical radius R is normalized to 1;

[0034] Taking the center point of the image material as the origin, a rectangular coordinate system is established on the plane of the image material, and all pixels in the image material are traversed. For any pixel A(x,y), its normalized coordinates are marked as (u,v), then:

[0035] u=2x / W-1,v=2y / H-1, where u∈[-1,1],v∈[-1,1];

[0036] S23, based on the stereoscopic projection method, calculating the longitude and latitude coordinates of each pixel in the image material projected onto the spherical surface

[0037] where θ∈[-π / 2,π / 2],

[0038] in

[0039] Where θ is the latitude, is the longitude;

[0040] S24, convert the longitude and latitude coordinates of each pixel in the image material into the coordinates (X p ,Y p ):

[0041]

[0042] S25, overlaying the image material onto the panoramic image, that is, according to the coordinate position (X) of each pixel in the image material in the panoramic image, p ,Y p ), replace the pixels at the corresponding positions in the panoramic image with the pixels in the picture material, so that the picture material and the panoramic image are merged to obtain a fused panoramic image.

[0043] S3, all the picture materials are processed according to step S2 to obtain a series of fused panoramic pictures, and all the panoramic pictures obtained after the processing of step S2 are synthesized into a panoramic video according to the set time slice sequence. This is the prior art. The panoramic video can adopt the universal format MPEG4 format, support mobile phones, computers and a variety of playback software to play, so that it can be directly uploaded to the short video platform for playback and use.

[0044] When using this embodiment, the center point of the picture material may not be located at the center of the panoramic picture, so the user needs to rotate the panoramic video to see the content of the picture material. Therefore, this embodiment can also be improved as follows:

[0045] Before executing step S3, the fused panoramic image is translated to translate the center point of the image material to the center position of the panoramic image.

[0046] In this embodiment, each pixel in the fused panoramic image is translated along the X-axis, and the Y-axis coordinate remains unchanged. In this way, the X-axis coordinate origin of the translated panoramic image coincides with point E, so that the plane image material is located at the center of the panoramic image. The translation amount is

[0047] In order to ensure that the position after translation is still within the width of the panoramic image, assume that the coordinates of a pixel before translation are (x, y), and the coordinates of the pixel after translation are (x ′ ,y ′ ),but:

[0048] x ′ =(x+d x )mod W p ,y ′ =y, where mod is the modulus operator;

[0049] Through the above processing, the coordinates of each pixel in the panoramic image are translated, and the picture material will be located at the center of the panoramic image. In the final panoramic video, the picture material will also be located in the center area of ​​the first perspective of the panoramic video playback, so that users can easily view the core content elements without manually rotating the panoramic video.

[0050] Example 2

[0051] The difference between this embodiment and the first embodiment is that this embodiment can display hot spots such as picture materials and video materials in the panoramic video, that is, in addition to importing picture materials into the panoramic picture, video materials are also imported into the panoramic picture.

[0052] During implementation, in step S2, the video material is converted into multiple picture materials by frame, and each picture material is fused into a panoramic picture according to the method of step S2 to generate a fused panoramic picture of each frame. Finally, all panoramic pictures are combined into a panoramic video according to the set time slice sequence.

[0053] Preferably, when the video material is imported into the panoramic picture, each frame of the panoramic picture generated corresponding to the video material is reassembled to generate a new panoramic video clip, the duration of the panoramic video is consistent with the playback duration of the video material, and the panoramic video clip is finally merged into the final panoramic video. Moreover, by panning the fused panoramic picture, the video material can be located in the center area of ​​the panoramic video clip.

[0054] Example 3

[0055] The difference between this embodiment and embodiment one is that this embodiment adds a function of displaying hot spots such as audio materials in the panoramic video, that is, before executing step S3, the audio material is set, and during the entire playback time of the audio material, the panoramic picture is cyclically translated along the X-axis direction at a certain time interval, and the translation amount can be set as needed, so that the center point of the panoramic picture changes continuously, thereby generating a series of panoramic pictures with different center point positions, and the generated panoramic pictures are mixed with the audio material to generate a panoramic video.

[0056] In this embodiment, in order to ensure that the position after translation is still within the width of the panoramic image, it is assumed that the translation amount per second is d x , the coordinates of a pixel before translation are (x, y), and the coordinates of the pixel after translation are (x ′ ,y ′ ),but:

[0057] x ′ =(x+d x )mod W p ,y ′ =y, where mod is the modulus operator;

[0058] Each time the video is panned, a new panoramic image is generated, and the panned panoramic images and audio are combined to form a panoramic video. The playback effect of the panoramic video is that the panoramic image continues to pan and rotate as the audio plays, so that users can experience the 360-degree panoramic image from all angles without manually rotating the panoramic video.

[0059] like Figure 4 As shown, in this embodiment, the time slices of scene materials (including picture materials, video materials, audio materials) and background music in the panoramic video can be set by the following method, and the playback order of the time slices can be flexibly adjusted as needed:

[0060] 1) Panoramic video PanoZ1 with audio material fusion, duration t1;

[0061] 2) PanoZ2, a panoramic video fused with picture materials, with a duration of t2;

[0062] 3) PanoZ3, a panoramic video fused with video materials, with a duration of t3;

[0063] 4) Background music: You can set a background music that covers the entire panoramic video.

[0064] In Embodiment 1 and Embodiment 2, the setting of the time slices may also be adjusted according to the above method, and irrelevant time slices may be deleted.

[0065] Example 4

[0066] The difference between this embodiment and embodiments 1-3 is that: there is only one panoramic picture in embodiments 1-3, while this embodiment uses multiple panoramic pictures, and steps S2 and S3 are executed for each panoramic picture, each panoramic picture corresponds to a panoramic video clip, and then the multiple panoramic video clips are integrated into one panoramic video to generate a panoramic video with richer scene content.

Claims

1. A method for generating a panoramic video from a panoramic picture, comprising the following steps: S1. Select a panoramic picture for generating a panoramic video; S2, importing the picture material into the panoramic picture to merge the picture material with the panoramic picture; The method of importing any image material into a panoramic image includes the following steps: S21, convert the panoramic image into a spherical latitude and longitude coordinate format, assuming that the radius of the sphere is R and the center of the sphere is O, and select a position on the sphere As the center point of the picture material, and make the picture material perpendicular to OE; S22, let the pixel width of the panoramic image be W P , high is H P , the pixel width of the image material is W, the height is H, and the spherical radius R is normalized to 1; Take the center point of the image material as the origin, establish a rectangular coordinate system, traverse all pixels in the image material, and for any pixel A(x,y), its normalized coordinates are marked as (u,v), then: u=2x / W-1,v=2y / H-1, where u∈[-1,1],v∈[-1,1]; S23, calculating the longitude and latitude coordinates of each pixel in the image material projected onto the spherical surface Where θ∈[-π / 2,π / 2], in S24, convert the longitude and latitude coordinates of each pixel in the image material into the coordinates (X p ,Y p ): Y p =(θ / π+0.5)×H p ; S25, based on the coordinate position (X p ,Y p ), replacing pixels at corresponding positions in the panoramic image with pixels in the picture material, so that the picture material and the panoramic image are merged to obtain a fused panoramic image; S3. All panoramic pictures obtained after processing in step S2 are synthesized into a panoramic video according to a set time slice sequence.

2. The method for generating a panoramic video from a panoramic picture according to claim 1, characterized in that: Before executing step S3, the material video is converted into multiple picture materials frame by frame, and each picture material is fused into the panoramic picture according to the method of step S2.

3. The method for generating a panoramic video from a panoramic picture according to claim 1 or 2, characterized in that: When executing step S2, the length and width of the image material are adjusted as needed.

4. The method for generating a panoramic video from a panoramic picture according to claim 1 or 2, characterized in that: Before executing step S3, the fused panoramic image is translated to translate the center point of the image material to the center position of the panoramic image. Assuming that the coordinates of a certain pixel before translation are (x, y), the coordinates of the pixel after translation are (x ′ ,y ′ ),but: Translation x ′ =(x+d x )mod W p ,y ′ =y, where mod is the modulo operator.

5. The method for generating a panoramic video from a panoramic picture according to claim 1 or 2, characterized in that: Before executing step S3, set the audio material, and during the entire playback time of the audio material, circularly translate the panoramic image along the X-axis direction at certain time intervals so that the center point of the panoramic image changes continuously, thereby generating a series of panoramic images with different center point positions.

6. The method for generating a panoramic video from a panoramic picture according to claim 1 or 2, characterized in that: There are multiple panoramic pictures, and step S2 and step S3 are performed on each panoramic picture. Each panoramic picture corresponds to a panoramic video segment, and then the multiple panoramic video segments are integrated into a panoramic video.

Images