Method and device for generating stereoscopic panoramic video, and storage medium

Abstract

The invention provides a method and equipment for generating a three-dimensional panoramic video, and a storage medium. The method comprises the following steps: S1, setting a binocular virtual camera at the position of an observer in an animation scene; s2, establishing a virtual ball surrounding the binocular virtual camera, and dividing a sphere skin into a plurality of view finding blocks; s3, running the animation frame by frame, and enabling the binocular virtual camera to poll and collect the image content in each view finding block; and S4, combining the puzzle blocks into a left monocular panoramic video and a right monocular panoramic video. Therefore, a panoramic video with a three-dimensional effect is produced.

Description

Technical Field

[0001] The present invention relates to video processing technology, and in particular to a method and device for generating stereoscopic panoramic video, and a storage medium. Background Art

[0002] Panoramic video is a type of video that records 360-degree scenes through special shooting equipment. According to the production method, it can be divided into virtual panoramic video (generated by 3D game engines such as Unity and Unreal) and real panoramic video (using panoramic cameras to shoot real scenes). Currently, panoramic video is widely used in the experience of VR products, allowing viewers to use digital helmets to freely change the direction of their heads and watch pre-made video content from various angles.

[0003] The current production process of virtual panoramic video mainly includes: First, edit the animation content in the game engine. This animation content refers to the pre-edited configuration of the position, angle, and movement of each object in the 3D model scene at each point in time. Playback is based on these configurations during preview or generation.

[0004] Next, a virtual camera is set up at the preset user's viewing angle. This camera is configured to capture a panoramic image, for example, by stitching together a series of square photos from top, bottom, left, right, front, and back. For each frame of the animation, the camera is instructed to capture a single shot. These panoramic photos are then stored based on the current playback time and then concatenated chronologically to form a single global video file.

[0005] However, this type of virtual panoramic video production method can only obtain ordinary panoramic videos, but cannot achieve a stereoscopic effect. Therefore, how to produce a panoramic video with a stereoscopic effect still poses certain challenges to those skilled in the art. Summary of the Invention

[0006] Therefore, the main purpose of the present invention is to provide a method and device for generating a stereoscopic panoramic video, and a storage medium, so as to produce a panoramic video with a stereoscopic effect.

[0007] To achieve the above object, according to one aspect of the present invention, a method for generating a stereoscopic panoramic video is provided, comprising the steps of: Step S1 sets a binocular virtual camera at the observer's position in the animation scene; Step S2: creating a virtual sphere surrounding the binocular virtual camera and dividing the sphere skin into a number of viewing blocks; Step S3 runs the animation frame by frame, and makes the binocular virtual camera poll and collect the image content in each viewing block; Step S4 combines the puzzle pieces into left and right monocular panoramic videos.

[0008] Optionally, the step of establishing the virtual ball in step S2 includes: In step S21 , a virtual sphere is established with the center of the binocular virtual camera's eye distance as the sphere center and according to a preset length and radius.

[0009] Optionally, the step of dividing the spherical skin into a plurality of viewing blocks in step S2 includes: Step S22 rasterizes the sphere skin to divide it into a number of framing blocks with the same area or with an area difference within ±10%.

[0010] Optionally, the spherical skin is rasterized and divided into at least 3600 viewing blocks.

[0011] Optionally, the step of setting a binocular virtual camera in step S1 includes: In step S11, two monocular virtual cameras are arranged in parallel at a distance equal to the human eye distance, simulating the freedom of movement of the human head and neck, and establishing the vertical and horizontal rotation freedom coordinates for the binocular virtual cameras.

[0012] Optionally, the step of establishing vertical and horizontal rotational degrees of freedom coordinates for the binocular virtual camera in step S11 includes: Step S111: Using the horizontal line connecting the two monocular virtual cameras as the binocular horizontal axis, establish the vertical rotation freedom coordinates, and limit the rotation angle range to -90° to 90°; Step S112 sets the binocular longitudinal axis vertically at the center of the binocular transverse axis to establish the left and right rotational freedom coordinates.

[0013] Optionally, the step of causing the binocular virtual camera to poll and collect image content in each viewing block in step S3 includes: Step S31: At the observer's position, adjust the sight direction of the binocular virtual camera to align the center of the eye distance with the midpoint of the target framing block; In step S32, all the view blocks are polled frame by frame in the same manner as in step S31, the corresponding image content is collected, and the left and right single targets are recorded as puzzle blocks.

[0014] Optionally, the step of combining the puzzle pieces into left and right monocular panoramic videos in step S4 includes: In step S41, the puzzle pieces are combined into a left-eye panoramic frame and a right-eye panoramic frame according to the frame sequence, the left and right monocular marks and the corresponding viewfinder block positions, and the left and right monocular panoramic videos are synthesized respectively.

[0015] In order to achieve the above-mentioned purpose, according to another aspect of the present invention, a system for generating a stereoscopic panoramic video is provided, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the system implements the steps of any of the above methods for generating a stereoscopic panoramic video.

[0016] In order to achieve the above-mentioned object, according to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, wherein when the computer program is executed, the steps of the method for generating a stereoscopic panoramic video as described above are implemented.

[0017] The method, device, and storage medium for generating stereoscopic panoramic videos provided by the present invention cleverly utilize 3D game engine technology by setting up binocular virtual cameras and establishing a virtual ball in an animation scene to simulate the freedom of human head and neck movement to collect and synthesize left and right monocular panoramic videos, so as to provide viewers with a more vivid stereoscopic virtual reality experience when played by existing VR devices, thereby solving the problem that traditional panoramic video production technology cannot generate stereoscopic panoramic videos. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings, which constitute part of this application, are intended to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are intended to explain the present invention and do not constitute an undue limitation of the present invention. In the accompanying drawings: Figure 1 A schematic diagram of the steps of a method for generating a stereoscopic panoramic video according to the present invention; Figure 2 A conceptual diagram of establishing vertical and horizontal rotational degrees of freedom coordinates for a binocular virtual camera in the method for generating a stereoscopic panoramic video of the present invention; Figure 3 A conceptual diagram of the spatial relationship between a binocular virtual camera and a virtual sphere in the method for generating a stereoscopic panoramic video of the present invention, wherein the virtual sphere is shown in a semi-enclosed form to facilitate the display of the spatial relationship; Figure 4 This is a conceptual diagram of a binocular virtual camera capturing image content within a view block in the method for generating a stereoscopic panoramic video of the present invention. DETAILED DESCRIPTION

[0019] In order to enable those skilled in the art to better understand the technical solutions of the present invention, the specific technical solutions of the present invention will be clearly and completely described below in conjunction with the embodiments to help those skilled in the art further understand the present invention. Obviously, the embodiments described in this case are only embodiments of a part of the present invention, rather than all embodiments. It should be pointed out that, for those of ordinary skill in the art, the embodiments in this application and the features in the embodiments can be combined with each other without departing from the concept of the present invention and without conflicting with each other. Based on the embodiments in the present invention, all other embodiments obtained by those of ordinary skill in the art without making creative work should fall within the disclosure and protection scope of the present invention.

[0020] In addition, the terms "first," "second," "S1," "S2," and the like in the specification, claims, and drawings of the present invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential sequence. It should be understood that the features used in this manner are interchangeable where appropriate, such that the embodiments of the present invention described herein can be implemented in an order other than that described herein. Furthermore, the stages described in each step are not necessarily implemented in the same step. It should be understood that the order of implementation of the contents of each step can be adjusted and interchanged without violating the inventive concept, such that the step embodiments of the present invention described herein can be implemented in an order other than that described herein. Furthermore, the terms "including," "having," and any variations thereof, as used herein, are intended to cover non-exclusive inclusions. Unless otherwise expressly specified or limited, the terms "disposed," "arranged," "installed," "connected," and "connected" should be interpreted broadly, for example, to mean fixed, removable, or integral; mechanical or electrical; direct or indirect through an intermediary; or internal communication between two components. Those skilled in the art will understand the specific meanings of these terms in this case based on specific circumstances and in conjunction with prior art.

[0021] Considering the existing stereoscopic video (3D video) production method, it is generally recorded by two cameras placed side by side, and the video content obtained by the left and right cameras is transmitted to the left and right eyes of the viewer respectively through methods such as polarizing filters, thereby forming an angle difference between the two eyes and producing a stereoscopic perspective effect.

[0022] However, since this method cannot know the viewer's head angle in advance, it is generally impossible to obtain a stereoscopic panoramic video that "has a stereoscopic effect and allows the viewer to change the angle at will" through pre-shooting.

[0023] It can be seen that neither the existing virtual panoramic video production method nor the stereoscopic video production method can directly produce a stereoscopic panoramic video. Therefore, in order to solve this problem, Figures 1 to 4 As shown, the present invention provides a method for generating a stereoscopic panoramic video, the exemplary steps of which include: Step S1 sets a binocular virtual camera at the observer's position in the animation scene.

[0024] For example, before making a video, you need to edit the animation content in the game engine. Then, in the game scene, set the position representing the observer and place two virtual cameras to simulate human binocular vision, for example. Figure 2 As shown, in the example of step S1, the example steps of setting the binocular virtual camera include: In step S11, two monocular virtual cameras are set up in parallel with the distance between human eyes, and are placed in parallel for level viewing. Then, the vertical and horizontal rotational freedom coordinates are established for the binocular virtual cameras by simulating the freedom of movement of the human head and neck.

[0025] Among them, reference Figure 2 As shown in FIG, an example of steps for establishing the vertical and horizontal rotation degrees of freedom coordinates for a binocular virtual camera includes: In step S111 , the horizontal line connecting the two monocular virtual cameras is used as the binocular horizontal axis to establish the vertical rotation degree of freedom coordinates, and the rotation angle range is limited to -90° to 90°.

[0026] Step S112 sets the binocular longitudinal axis vertically at the center of the binocular transverse axis to establish the left and right rotation freedom coordinates, and the rotation angle range is not limited.

[0027] Step S2 establishes a virtual sphere surrounding the binocular virtual camera and divides the sphere skin into a number of viewing blocks.

[0028] For example, refer to Figure 3 As shown, the steps of establishing the virtual ball include: Step S21 creates a virtual sphere with the binocular virtual camera's eye distance as the sphere center and a preset length and radius. The length and radius can be freely adjusted according to the scene viewing distance, various film and television effects, and content needs, and are not limited in this example.

[0029] Examples of steps for dividing the sphere skin into several framing blocks include: Step S22 rasterizes the sphere skin to divide it into a number of framing blocks with the same area or with an area difference within ±10%.

[0030] For example Figure 3As shown, the spherical skin can be divided into a number (e.g., at least 3600 blocks) of substantially equal area and close to a square. To facilitate the final synthesis of the video, the blocks can be numbered according to their positions for subsequent splicing.

[0031] Step S3 runs the animation frame by frame, and causes the binocular virtual camera to poll and collect the image content in each viewing block.

[0032] For example, refer to Figure 4 As shown, the example steps for collecting the image content in each viewing block are: Step S31: At the observer's position, adjust the sight lines of the binocular virtual camera to align the center of the eye distance with the midpoint of the target viewfinder block. In this way, the two monocular virtual cameras can respectively capture the image content in the viewfinder block.

[0033] Step S32 runs the animation content frame by frame, pauses after each frame is played, and in the same manner as step S31, the binocular virtual camera is made to poll all the viewfinders to capture the corresponding image content, and records the left and right single targets as puzzle blocks.

[0034] Step S4 combines the puzzle pieces into left and right monocular panoramic videos.

[0035] For example, the steps of the puzzle piece assembly process include: Step S41 finally combines all puzzle pieces into a left-eye panoramic frame and a right-eye panoramic frame, respectively, according to the frame sequence, the left and right monocular marks, and the corresponding framing block positions. These two frames are labeled with frame numbers. This cycle continues until all animation frames have been played. This completes the left and right panoramic frames of the animation. At this point, the panoramic frames saved by all single-sided cameras are combined into a video according to the frame sequence, thereby synthesizing the left and right monocular panoramic videos VideoL and VideoR, respectively. This completes the entire animation's stereoscopic panoramic video production steps.

[0036] When playing the stereoscopic panoramic video, output VideoL and VideoR to the left and right eye screens of the VR device respectively to obtain a stereoscopic panoramic virtual experience.

[0037] On the other hand, corresponding to the above method, the present invention also provides a system for generating stereoscopic panoramic video, an example of which includes: a memory, a processor, and a computer program stored in the memory and runnable on the processor, wherein when the processor executes the computer program, the system implements the steps of the method for generating stereoscopic panoramic video as described in any of the above examples.

[0038] On the other hand, corresponding to the above method, the present invention also provides a computer-readable storage medium, which stores a computer program, wherein when the computer program is executed, the steps of the method for generating stereoscopic panoramic video as described in any of the above examples are implemented.

[0039] The preferred embodiments of the present invention disclosed above are only used to help illustrate the present invention. The preferred embodiments do not describe all the details in detail, nor do they limit the invention to the specific implementation methods described. Obviously, many modifications and changes can be made based on the contents of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present invention, so that those skilled in the art can better understand and utilize the present invention. The present invention is only limited by the claims and their full scope and equivalents. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

[0040] Those skilled in the art will understand that, in addition to implementing the systems, devices, units, and their respective modules provided by the present invention in purely computer-readable program code, it is entirely possible to implement the same programs in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, embedded microcontrollers, etc. by logically programming the method steps. Therefore, the systems, devices, and their respective modules provided by the present invention can be considered as a hardware component, and the modules included therein for implementing various programs can also be considered as structures within the hardware component; the modules for implementing various functions can also be considered as both software programs for implementing the method and structures within the hardware component.

[0041] In addition, all or part of the steps in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a program. The program is stored in a storage medium and includes a number of instructions for causing a single-chip microcomputer, chip, or processor to execute all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc., various media that can store program code.

[0042] In addition, various implementations of the embodiments of the present invention may be arbitrarily combined, and as long as they do not violate the concept of the embodiments of the present invention, they should also be regarded as the contents disclosed in the embodiments of the present invention.

Claims

1. A method for generating a stereoscopic panoramic video, comprising the steps of: Step S1 sets a binocular virtual camera at the observer's position in the animation scene; Step S2: creating a virtual sphere surrounding the binocular virtual camera and dividing the sphere skin into a number of viewing blocks; Step S3 runs the animation frame by frame, and makes the binocular virtual camera poll and collect the image content in each viewing block; Step S4 combines the puzzle pieces into left and right monocular panoramic videos.

2. The method for generating a stereoscopic panoramic video according to claim 1, wherein the step of creating the virtual sphere in step S2 comprises: In step S21 , a virtual sphere is established with the center of the binocular virtual camera's eye distance as the sphere center and according to a preset length and radius.

3. The method for generating a stereoscopic panoramic video according to claim 1, wherein the step of dividing the spherical skin into a plurality of viewing blocks in step S2 comprises: Step S22 rasterizes and divides the sphere skin into a plurality of framing blocks with the same area or an area difference within ±10% or an area difference within ±10%. 4 . The method for generating a stereoscopic panoramic video according to claim 3 , wherein the spherical skin is rasterized and divided into at least 3,600 viewing blocks.

5. The method for generating a stereoscopic panoramic video according to claim 1, wherein the step of setting a binocular virtual camera in step S1 comprises: In step S11, two monocular virtual cameras are arranged in parallel at a distance equal to the human eye distance, simulating the freedom of movement of the human head and neck, and establishing the vertical and horizontal rotation freedom coordinates for the binocular virtual cameras.

6. The method for generating a stereoscopic panoramic video according to claim 5, wherein the step of establishing vertical and horizontal rotational degrees of freedom coordinates for the binocular virtual camera in step S11 comprises: Step S111: Using the horizontal line connecting the two monocular virtual cameras as the binocular horizontal axis, establish the vertical rotation freedom coordinates, and limit the rotation angle range to -90° to 90°; Step S112 sets the binocular longitudinal axis vertically at the center of the binocular transverse axis to establish the left and right rotational freedom coordinates.

7. The method for generating a stereoscopic panoramic video according to claim 1, wherein the step of causing the binocular virtual camera to poll and capture the image content in each viewing block in step S3 comprises: Step S31: At the observer's position, adjust the sight direction of the binocular virtual camera to align the center of the eye distance with the midpoint of the target framing block; In step S32, all the view blocks are polled frame by frame in the same manner as in step S31, the corresponding image content is collected, and the left and right single targets are recorded as puzzle blocks.

8. The method for generating a stereoscopic panoramic video according to claim 1, wherein the step of combining the puzzle pieces into left and right monocular panoramic videos in step S4 comprises: In step S41, the puzzle pieces are combined into a left-eye panoramic frame and a right-eye panoramic frame according to the frame sequence, the left and right monocular marks and the corresponding viewfinder block positions, and the left and right monocular panoramic videos are synthesized respectively.

9. A system for generating a stereoscopic panoramic video comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the system implements the steps of the method for generating a stereoscopic panoramic video according to any one of claims 1 to 8.

10. A computer-readable storage medium storing a computer program, wherein when the computer program is executed, the steps of the method for generating a stereoscopic panoramic video according to any one of claims 1 to 8 are implemented.

Images