Holographic projection system construction method based on three-dimensional game engine
By laying out virtual cameras and optimizing the capture range in the three-dimensional game engine, combining splicing and mapping technology, the complexity and sound performance of traditional multi-folding holographic projection technology is solved, and efficient and low-cost audio-visual display is achieved, enhancing the audience The immersion.
Patent Information
- Application Number
- CN202411166623.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-23
- Publication Date
- 2024-11-29
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The traditional multi-fold holographic projection technology has complex content production and is expensive, and ignores the positioning and expression of sound in three-dimensional space, affecting the overall sense of reality and immersion.
The holographic projection system construction method based on the three-dimensional game engine is adopted, and by accurately layout of virtual cameras, optimizing the capture range and distance, combined with innovative splicing and mapping technology, efficient visual rendering and all-round stereo sound generation on multi-screen devices are achieved.
Overcoming the limitations of content production complexity, cost and sound performance of traditional multi-fold projection technology, it significantly enhances the audience's immersive experience.
Smart Images

Figure CN119052445A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of optics and imaging technology, and in particular to a method for constructing a holographic projection system based on a three-dimensional game engine. Background Art
[0002] Pseudo-holographic projection technology utilizes optical illusions and special media to create three-dimensional stereoscopic imaging, creating a holographic effect from specific angles. However, a single viewing angle cannot fully display the details and three-dimensionality of a three-dimensional object. This is where multi-fold holographic technology comes in. By combining multiple screens to create a wider display area, multi-fold holographic technology supports 360-degree panoramic viewing from multiple angles, significantly enhancing the sense of immersion. However, traditional multi-fold holographic projection content is complex and costly to produce, and often overlooks the positioning and representation of sound in three-dimensional space, compromising the overall sense of realism and immersion.
[0003] 3D game engines are a revolutionary technology in digital content creation. They are not only the core driving force behind game development but are also widely used in a variety of fields, including film and television special effects, virtual reality (VR), augmented reality (AR), and simulation training. These engines integrate high-performance graphics rendering technology, capable of handling complex lighting, shading, material, and animation effects to create stunning visual experiences. They also feature advanced physics simulation engines that emulate real-world physical laws such as gravity, collisions, and fluid dynamics, adding a greater sense of realism and interactivity to virtual worlds. Of particular note, modern 3D game engines generally include integrated audio processing modules. These modules generate stereo sound effects in real time, closely synchronized with the visual content, based on scene changes within the game or application. By simulating the propagation characteristics of sound in 3D space, such as echo, reverberation, and directionality, the engines create a realistic audio experience that makes players and viewers feel as if they are immersed in a real virtual environment. This dual sense of visual and auditory immersion significantly enhances the level and depth of the user experience.
[0004] In order to simplify the content production process, reduce costs, and enhance the dual visual and auditory experience, the present invention proposes a method for constructing a 360-degree holographic projection system based on a three-dimensional game engine. Summary of the Invention
[0005] The purpose of this invention is to propose a method for constructing a holographic projection system based on a 3D game engine to address the problems of the aforementioned existing technologies. By integrating the advanced technologies of a 3D game engine, efficient visual rendering and omnidirectional stereo sound generation can be achieved on a multi-fold screen device. This system achieves efficient, high-quality audiovisual presentation on a multi-fold screen device through precise virtual camera layout, optimized capture range and distance, and innovative splicing and mapping techniques. This invention not only overcomes the limitations of traditional multi-fold screen projection technology in content production complexity, cost, and sound performance, but also greatly enhances the audience's immersive experience.
[0006] To achieve the above object, the present invention provides the following solutions:
[0007] A method for constructing a holographic projection system based on a three-dimensional game engine, comprising:
[0008] Multiple virtual cameras and audio monitors placed in preset positions capture the target's images and sounds;
[0009] Stitching the images captured by multiple virtual cameras;
[0010] Shooting the stitched image using a main camera; wherein the main camera is a camera other than the virtual camera of the multi-camera;
[0011] Add the target sound and the main camera's shooting image to the 360 holographic device for output.
[0012] Optionally, placing multiple virtual cameras based on a preset method includes:
[0013] Calculate the preset parameters of the virtual camera placed in multiple positions based on the output device and display object;
[0014] Based on the preset parameters, multiple virtual cameras are placed.
[0015] Optionally, the preset parameters include: the number of cameras n, the camera capture range f, and the distance x between the camera and the target.
[0016] Optionally, the virtual camera for placing multiple cameras also includes:
[0017] The distance between the virtual camera of each camera position and the target is always x, and the virtual cameras are evenly distributed and have consistent intervals.
[0018] Optionally, the target image includes:
[0019] Adjust the shooting angles of the multi-camera virtual cameras so that the image of the captured target is centered on a common intersection.
[0020] Optionally, stitching images captured by multiple virtual cameras includes:
[0021] The images captured by the virtual camera are projected onto a quadrilateral of a preset size in the form of a texture, and combined into a splicing plane of a preset size;
[0022] Arrange the centers of the quadrilaterals of the stitching plane in a preset counterclockwise order and rotate them according to the preset Euler angles.
[0023] Optionally, the main camera captures the stitched image including:
[0024] The main optical axis of the main camera passes through the stitching center and is perpendicular to the stitching plane, and orthogonally shoots the stitching plane content in a preset posture.
[0025] Optionally, adding the target sound and the main camera's captured image to the 360 holographic device for output includes:
[0026] Bind the images and sounds captured on the same side, and play them on the screen and speaker on the same side of the 360 holographic device respectively.
[0027] The beneficial effects of the present invention are:
[0028] This invention achieves efficient, high-quality audiovisual presentation on multi-fold screens through precise virtual camera placement, optimized capture range and distance, and innovative splicing and mapping techniques. This invention not only overcomes the limitations of traditional multi-fold screen projection technology in content production complexity, cost, and sound quality, but also significantly enhances the audience's immersive experience. BRIEF DESCRIPTION OF THE DRAWINGS
[0029] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0030] Figure 1 This is a flow chart of a method for constructing a holographic projection system based on a three-dimensional game engine according to an embodiment of the present invention;
[0031] Figure 2 This is a schematic diagram of multi-camera placement according to an embodiment of the present invention;
[0032] Figure 3 Schematic diagram of capturing a multi-camera stitched image according to an embodiment of the present invention;
[0033] Figure 4 This is a schematic diagram of displaying the final output image and sound on a 360 holographic device according to an embodiment of the present invention. DETAILED DESCRIPTION
[0034] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.
[0035] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.
[0036] This embodiment provides a method for constructing a holographic projection system based on a three-dimensional game engine, including:
[0037] Multiple virtual cameras and audio monitors placed in preset positions capture the target's images and sounds;
[0038] Stitching the images captured by multiple virtual cameras;
[0039] Use a main camera to shoot the stitched image; wherein the main camera is a camera other than the virtual camera of the multi-camera;
[0040] Add the target sound and the main camera's shooting image to the 360 holographic device for output.
[0041] Furthermore, placing multiple virtual cameras based on a preset method includes:
[0042] Calculate the preset parameters of the virtual camera placed in multiple positions based on the output device and display object;
[0043] Place multiple virtual cameras based on preset parameters;
[0044] Moreover, the distance between the virtual camera of each camera position and the target is always x, and the virtual cameras are evenly distributed and have consistent intervals.
[0045] Specifically, in this embodiment, the preset parameters include: the number of cameras n, the camera capture range f, and the camera-to-target distance x. The number of cameras is determined based on the number of folding screens in the final 360-degree holographic device in step 6). The camera capture range refers to the lens size when the camera is shooting in orthogonal projection; the camera-to-target distance refers to the Euclidean distance between the camera and the target. The values of these two values need to be calculated based on the maximum size s of the target's directed bounding box.
[0046] The capture range of each virtual camera is set to f; ensure that the distance between the placed virtual cameras and the target is always x, and the virtual cameras are evenly distributed and have consistent intervals.
[0047] Furthermore, the target images include:
[0048] Adjust the shooting angles of the multi-camera virtual cameras so that the image of the captured target is centered on a common intersection.
[0049] Specifically, in this embodiment, the shooting angle needs to be adjusted to adapt to the spliced images, so that the spliced images are rotated and arranged in a certain order with a common intersection as the center.
[0050] Furthermore, stitching the images captured by the multi-position virtual cameras includes:
[0051] The images captured by the virtual camera are projected onto a quadrilateral of a preset size in the form of a texture, and combined into a splicing plane of a preset size;
[0052] Arrange the centers of the quadrilaterals of the stitching plane in a preset counterclockwise order and rotate them according to the preset Euler angles.
[0053] Furthermore, the main camera captures the stitched image including:
[0054] The main optical axis of the main camera passes through the stitching center and is perpendicular to the stitching plane, and shoots the stitching plane content orthogonally in a preset posture.
[0055] Specifically, in this embodiment, the main camera is a camera other than the virtual multi-camera camera, which captures the picture as the final rendering result. The main optical axis of the camera passes through the stitching center and is perpendicular to the stitching plane, and orthogonally shoots the stitching plane content in a certain posture.
[0056] Furthermore, adding the target sound and the main camera's captured image to the 360 holographic device for output includes:
[0057] Bind the images and sounds captured on the same side, and play them on the screen and speaker on the same side of the 360 holographic device respectively.
[0058] Specifically, in this embodiment, the stereo sound comes from the audio monitors of multiple cameras, which bind the images and sounds captured on the same side, and play the bound images and sounds on the screen and speakers on the same side of the 360 holographic device respectively.
[0059] This embodiment can be simply summarized as follows: using the virtual camera in the three-dimensional game engine to evenly surround the target to perform multi-angle image capture and sound capture of the three-dimensional digital product, and output the corresponding images and sounds on the screen and speakers on each side of the 360-degree holographic folding screen to achieve a visual and auditory 360-degree holographic display.
[0060] The following embodiment takes the Unity 3D game engine as an example and selects a 4-fold 360 holographic projection device for output. Figure 1 As shown, the process of the present invention includes the following steps:
[0061] Step 1: Determine the preset parameters: number of cameras n, camera capture range f, and distance between camera and target x;
[0062] Step 2: Set the virtual camera parameters and place the virtual camera at a certain position;
[0063] Step 3: Adjust the virtual camera posture and bind the captured image and sound;
[0064] Step 4: Splice the captured images in a certain format;
[0065] Step 5: Set the main camera to shoot the stitched image;
[0066] Step 6: Play the bound images and sounds on the folding screen and speakers on the same side of the 360 holographic device respectively.
[0067] The spatial coordinates described in this embodiment are all left-handed systems used in mainstream three-dimensional game engines.
[0068] Step 1: Determine the number of cameras n = 4 based on the output device. Based on the maximum size s of the directional bounding box of the 3D target, preset the camera capture range f and the distance x between the camera and the target. Among them, f should be as large as possible and close to x should be much larger than s / 2;
[0069] Step 2: Place the 3D target at the origin of the global coordinate system, as shown in the following example: Figure 2 As shown, create a "Camera Group" object at the origin and group the four cameras into subsets of the object. Each camera is set to "Orthogonal" projection, with the capture range extending from a rectangle with a side length of f. For ease of description, the cameras in the camera group are named and placed according to the following relative coordinates (relative to the camera group object): Front Camera Left camera Rear camera Right camera
[0070] Step 3: Rotate each camera using the following Euler angles: front camera (0,0,45), left camera (0,90,45), back camera (0,180,45), right camera (0,270,45). Obtain the image and sound captured by each camera and bind the sound to the image. The image is output as a texture.
[0071] Step 4: Project the captured content of the four cameras in the camera group onto a quadrilateral of size 1 in the form of texture, and combine them into a 2*2 size "stitching plane". Figure 3 As shown, the centers of the quadrilaterals projecting the content are arranged in the following coordinates (-0.5, 0, -0.5), (0.5, 0, -0.5), (0.5, 0, 0.5), (-0.5, 0, 0.5) in a counterclockwise order from left to front to right to back, and then rotated by the following Euler angles: (90, 0, 0), (90, 0, 90), (90, 0, 180), (90, 0, 270). The order of each projection plane is allowed to change, but the adjacent relationship must remain unchanged.
[0072] Step 5: Set the main camera, place it at (0,1,0), and rotate it with the Euler angle (90,-45,0). Shoot the stitching plane with a shooting size of √2, and capture the content to exclude objects other than the "stitching plane".
[0073] Step 6: The content of the stitching plane captured by the main camera is mapped to the UI canvas in the form of texture, and expanded in a filling manner to adapt to the change of device resolution. Figure 4 As shown, the main camera captures the image and finally projects the four image contents onto a 360-degree holographic projection device with a 4-fold screen through optical refraction through a large screen. The sound corresponding to the image is output through the speakers on each side.
[0074] This embodiment provides a universal 360-degree holographic projection system. By integrating advanced 3D game engine technology, it achieves efficient visual rendering and omnidirectional stereo sound generation on multi-fold screen devices. Through precise virtual camera placement, optimized capture range and distance, and innovative splicing and mapping techniques, this system achieves efficient, high-quality audiovisual presentation on multi-fold screen devices. This invention not only overcomes the limitations of traditional multi-fold screen projection technology in content production complexity, cost, and sound quality, but also greatly enhances the audience's immersive experience.
[0075] The above embodiments are merely descriptions of preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by persons skilled in the art should fall within the scope of protection defined by the claims of the present invention.
Claims
1. A method for constructing a holographic projection system based on a three-dimensional game engine, characterized in that: include: Multiple virtual cameras and audio monitors placed in preset ways to capture the target's images and sounds; Stitch the images captured by virtual cameras from multiple locations; Using a main camera to shoot the stitched image; wherein the main camera is a camera other than the multi-camera virtual camera; Add the target sound and the footage from the main camera to the 360 holographic device for output.
2. The method for constructing a holographic projection system based on a three-dimensional game engine according to claim 1, characterized in that: Virtual cameras that place multiple cameras based on presets include: Based on the output device and display object, calculate the preset parameters of the virtual camera placed in multiple positions; Based on the preset parameters, multiple virtual cameras are placed.
3. The method for constructing a holographic projection system based on a three-dimensional game engine according to claim 2, characterized in that: The preset parameters include: the number of cameras n, the camera capture range f, and the distance between the camera and the target x.
4. The method for constructing a holographic projection system based on a three-dimensional game engine according to claim 3, characterized in that: The virtual cameras for placing multiple cameras also include: The distance between the virtual camera of each camera position and the target is always x, and the virtual cameras are evenly distributed and have consistent intervals.
5. The method for constructing a holographic projection system based on a three-dimensional game engine according to claim 1, characterized in that: The target screens include: Adjust the shooting angles of the multi-camera virtual cameras so that the image of the target is centered on a common intersection.
6. The method for constructing a holographic projection system based on a three-dimensional game engine according to claim 1, characterized in that: The stitching of images captured by virtual cameras from multiple positions includes: Project the images captured by the virtual camera onto a quadrilateral of a preset size in the form of a texture, and combine them into a splicing plane of a preset size; Arrange the centers of the quadrilaterals of the stitching plane in a preset counterclockwise order and rotate them according to the preset Euler angles.
7. The method for constructing a holographic projection system based on a three-dimensional game engine according to claim 6, characterized in that: The main camera captures the stitched image including: The main optical axis of the main camera passes through the stitching center and is perpendicular to the stitching plane, and orthogonally shoots the stitching plane content in a preset posture.
8. The method for constructing a holographic projection system based on a three-dimensional game engine according to claim 1, characterized in that: Adding the target sound and the main camera's footage to the 360 holographic device for output includes: Bind the images and sounds captured on the same side, and play them on the screen and speaker on the same side of the 360 holographic device respectively.
Citation Information
Patent Citations
Real-time holographic projection device and method based on pure color background
CN106454295A
Shared viewing system capable of carrying out virtual photographing
CN107105168A
A holographic communication system with an intelligent speaker
CN109040875A
Display panel, display device and driving method of display device
CN109873021A
Electronic equipment and sound production control method thereof
CN110572502A
