Immersive projection system and correction method therefor

Abstract

A correction method for an immersive projection system, comprising: setting a virtual projection position and a virtual picture deformation of a projection device; setting a virtual photographing position of a photographing device; simulating a projection area of the projection device; defining a plurality of first feature points in the projection area; simulating a virtual static image of the photographing device, the virtual static image comprising the projection area and the first feature points; adjusting the virtual projection position and the virtual picture deformation of the projection device on the basis of the first feature points; and outputting simulation profile information, the simulation profile information comprising the adjusted virtual projection position and the adjusted virtual picture deformation. The present disclosure also provides an immersive projection system.

Description

Immersive projection system and correction method thereof Technical Field

[0001] The present disclosure relates to a projection system, and more particularly to an immersive projection system and a calibration method thereof. Background Art

[0002] Because immersive projection systems offer a truly immersive user experience, they are widely used in applications including flight simulation, cultural and creative exhibitions, star-level simulation planetariums, and theme park entertainment. Immersive projection systems provide users with an ultra-wide viewing angle, giving them the feeling of being immersed in a highly realistic audiovisual environment.

[0003] A typical immersive projection system includes an immersive projection screen, audience seats or a flight cabin, multiple projectors and a set of media playback servers, and may also include any on-site decorative objects.

[0004] To provide an immersive user experience, immersive projection scenes are typically set up on circular, hemispherical, or multi-wall projection screens. To project immersive content onto these screens, multiple projectors must be used for splicing and fusion. Furthermore, content production must be tailored to the projection theme, and the content must be segmented and distributed across the appropriate projectors for playback.

[0005] To attract users, each immersive projection system has different and diverse themes, content, and deployment methods. Because it involves many different professional fields and details, the overall planning and setup of an immersive projection system is very difficult. It usually requires an experienced system integrator to plan and set up each project, and it also requires the integration of multiple different hardware and software tools.

[0006] The overall difficulty of immersive projection systems makes them difficult to quickly promote and set up. Therefore, how to provide an easy-to-setup immersive projection system and its calibration method is one of the current problems that need to be solved.

[0007] Summary of the Invention

[0008] To address the aforementioned difficulties in planning and setting up an immersive projection system, the present disclosure provides an immersive projection system and calibration method thereof. These systems can assist users with everything from immersive projection scene planning, projection simulation, and the installation and setup of multiple projection devices to on-site projection setup using camera-assisted automatic calibration. They can even assist with automatic screen segmentation settings for content players, thereby significantly reducing the difficulty of planning and setting up an immersive projection system.

[0009] The present disclosure provides a calibration method for an immersive projection system, comprising: setting a virtual projection position and virtual image distortion of a projection device; setting a virtual image position of a camera; simulating a projection area of ​​the projection device; defining a plurality of first feature points in the projection area; simulating a virtual static image of the camera, the virtual static image including the projection area and the first feature points; adjusting the virtual projection position and virtual image distortion of the projection device based on the first feature points; and outputting simulation profile information. The simulation profile information includes the adjusted virtual projection position and the adjusted virtual image distortion.

[0010] In certain embodiments, the calibration method disclosed herein further includes: setting a projection position and image distortion of a projection device and a photographic position of a photographic device based on simulation profile information; obtaining a static image via the photographic device; obtaining a plurality of second feature points in a virtual static image based on the first feature points; comparing the first feature points with the second feature points to obtain virtual image difference parameters; and adjusting the projection position and image distortion of the projection device based on the virtual image difference parameters and the static image.

[0011] In some embodiments, obtaining the static image via the photographic device further includes: obtaining a plurality of third feature points in the static image based on the first feature points; and comparing the virtual image difference parameter with the third feature points to obtain the image difference parameter.

[0012] In some embodiments, adjusting the projection position and image deformation of the projection device according to the virtual image difference parameter and the static image further includes adjusting the projection position and image deformation of the projection device according to the image difference parameter.

[0013] In some embodiments, setting the virtual projection position and virtual image deformation of the projection device further includes: setting a virtual color and virtual brightness of the projection device; and adjusting the virtual color and virtual brightness of the projection device based on the first feature points. The simulation profile information also includes adjusted virtual color and virtual brightness information.

[0014] In some embodiments, setting the virtual projection position and virtual image deformation of the projection device further includes setting multiple virtual projection positions and multiple virtual image deformations for multiple projection devices; and adjusting the virtual projection positions and virtual image deformations of the projection devices based on the first feature points further includes adjusting fusion correction information of the projection devices based on the first feature points. The simulation profile information further includes the adjusted fusion correction information.

[0015] In some embodiments, adjusting the virtual projection position and virtual screen deformation of the projection device according to the virtual image difference parameters further includes: obtaining a plurality of second feature points in the virtual static image based on the first feature points; comparing the first feature points with the second feature points to obtain the virtual image difference parameters; and adjusting the virtual photography position of the photography device according to the virtual image difference parameters.

[0016] In some embodiments, adjusting the virtual photographing position of the photographing device according to the virtual image difference parameter further includes adjusting a simulated photographing parameter of the photographing device according to the virtual image difference parameter.

[0017] In certain embodiments, the calibration method disclosed herein further includes: setting a projection position and image distortion of a projection device and a photographic position of a photographic device based on the simulation profile information; setting photographic parameters of the photographic device based on the simulated photographic parameters; obtaining a static image via the photographic device; and adjusting the projection position and image distortion of the projection device based on the virtual image difference parameter and the static image.

[0018] The present disclosure further provides a calibration method for an immersive projection system, comprising: setting a virtual projection position and virtual image distortion of a projection device; setting a virtual photography position of a photography device; obtaining a projection area and a virtual static image through a projection simulation program; defining a plurality of first feature points in the projection area; adjusting the virtual projection position and virtual image distortion of the projection device based on the first feature points; and outputting simulation profile information. The simulation profile information includes the adjusted virtual projection position and the adjusted virtual image distortion.

[0019] In some embodiments, the projection simulation program includes a projection geometry simulation subroutine, a camera device simulation subroutine, a stitching simulation subroutine, a color and brightness simulation subroutine, and a fusion simulation subroutine.

[0020] The present disclosure further provides an immersive projection system, which is coupled to a projection screen. The immersive projection system includes: a projection device having a projection position and image deformation, and projecting a projection area onto the projection screen; and a camera device having a camera position and capturing a static image, the static image including the projection area. The projection position and image deformation of the projection device are set based on the static image and the virtual static image.

[0021] In some embodiments, the virtual static image is obtained through a projection simulation process.

[0022] In summary, the immersive projection system and its calibration method disclosed herein can assist users in scene planning, projection device and camera lens selection, and scene projection simulation during the system planning stage, and can output simulation profile information (profile) based on the projection simulation results. The calibration method of the immersive projection system disclosed herein can, on the one hand, provide users with a reference for the physical installation location, projection orientation, and projection lens selection of the on-site projection device. On the other hand, it can also provide users with setting information such as the projection size, lens seat shift, and keystone correction required for the multi-projection installation setting and the automatic calibration program. In addition, the calibration method of the immersive projection system disclosed herein can also provide the automatic calibration program with recommended information such as the type of camera lens, setting location, and field of view (FOV) to be used, so that the automatic calibration program can smoothly perform automatic splicing, fusion, and color correction assisted by the camera. Furthermore, after the automatic calibration program is completed, relevant information such as the segmentation and fusion band of the corresponding multi-projection screen can be output to set the display card output screen of the media playback server.

[0023] Therefore, as described above, the immersive projection system and calibration method disclosed herein provide a comprehensive solution covering everything from system planning, projection device and lens selection, projection simulation, to automatic projection calibration installation and content player configuration. This can significantly reduce the difficulty of system planning and implementation, facilitating the rapid implementation and promotion of immersive projection systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] FIG1 is a flowchart illustrating the steps of a first embodiment of a calibration method for an immersive projection system disclosed herein;

[0025] FIG2 is a schematic diagram of an immersive projection system disclosed herein;

[0026] 3A to 3G are schematic diagrams showing different immersive projection screens;

[0027] FIG4 is a schematic diagram of a projection cone of a projection device;

[0028] FIG5A is a schematic diagram of a combined projection area of ​​a projection device;

[0029] FIG5B is a schematic diagram of a virtual static image of a photographic device;

[0030] 6A and 6B are schematic diagrams of a fusion process of a flat projection screen;

[0031] 7A and 7B are schematic diagrams of a fusion process for a curved projection screen;

[0032] FIG8 is a flowchart illustrating the steps of a second embodiment of a calibration method for an immersive projection system according to the present disclosure;

[0033] FIG9 is a flowchart illustrating the steps of a fourth embodiment of a calibration method for an immersive projection system according to the present disclosure;

[0034] FIG10 is a flowchart illustrating the steps of a fifth embodiment of a calibration method for an immersive projection system according to the present disclosure;

[0035] FIG. 11 is a schematic diagram of a projection device according to the present disclosure.

[0036] DESCRIPTION OF THE REFERENCE NUMERALS: 1: Immersive projection system 10, 10A-10G: Projection screen 11: Viewer seat 12: Projection device 121: Combined projection area 122: First characteristic point 13: Photographic device 131: Virtual static image 132: Second characteristic point 14: Media playback server 20: Motorized lens mount 21: Projection lens 22: Horizontal shift stepper motor 23: Vertical shift stepper motor 24: Motor drive circuit 25: Electronic pattern deformation adjustment circuit S01-S12, S20-S30: Steps DETAILED DESCRIPTION

[0037] FIG1 is a step diagram of a first embodiment of a calibration method for an immersive projection system disclosed herein. As shown in FIG1 , the calibration method for an immersive projection system disclosed herein includes steps S01 to S07. Step S01 is to set a virtual projection position and a virtual image deformation of a projection device. Step S02 is to set a virtual photographic position of a photographic device. Step S03 is to simulate a projection area of ​​a projection device. Step S04 is to define a plurality of first feature points in the projection area. Step S05 is to simulate a virtual static image of a photographic device. Step S06 is to adjust the virtual projection position and the virtual image deformation of the projection device according to the number of first feature points. Step S07 is to output simulation profile information including the adjusted virtual projection position and the adjusted virtual image deformation.

[0038] FIG2 is a schematic diagram of an immersive projection system 1 disclosed herein. It should be noted that, since the simulated results correspond to the actual installation site, the immersive projection system 1 shown in FIG2 is intended to illustrate both the immersive projection system in a virtual environment and the immersive projection system at the installation site, but is not intended to limit the present invention.

[0039] As shown in Figures 1 and 2, in step S01, the virtual projection position and virtual image deformation (or virtual projection image geometric deformation) of the projection device 12 are set. In step S02, the virtual photography position of the photographic device 13 is set. In some embodiments, the immersive projection system 1 is coordinated with the immersive projection screen 10. The immersive projection system 1, for example, includes a viewer seat 11 (or a flight cabin), a projection device 12, a photographic device 13, and a media playback server 14, but this is not restrictive. In this embodiment, multiple projection devices 12 are used as an example for illustration, but this is not restrictive. In some applications, only a single projection device 12 may be used (for example, Figures 3A to 3B below). The immersive projection screen 10 typically covers a wide viewing angle of at least 120° horizontally to allow users to have an immersive visual experience. Figures 3A to 3G are schematic diagrams showing different immersive projection screens 10A to 10G. The immersive projection screen may include, for example, a one-dimensional curved immersive projection screen 10A (Figure 3A), a two-dimensional curved immersive projection screen 10B (Figure 3B), a cave-shaped immersive projection screen 10C (Figure 3C) composed of multiple walls, a cylindrical immersive projection screen 10D (Figure 3D), a hemispherical screen immersive projection screen 10E (Figure 3E), a full-spherical screen immersive projection screen 10F (Figure 3F), and a horseshoe-shaped immersive projection screen 10G (Figure 3G), etc., but it is not restrictive.

[0040] Referring again to Figures 1 and 2 , in the multi-projection simulation tool, the specifications of the projection devices 12 and camera devices 13 can be configured according to different requirements. Users can set multiple virtual projection positions and virtual image deformations for the projection devices 12, as well as the virtual image position of the camera device 13. It is worth noting that the virtual projection position can represent the location of the projection device 12 or the location of the image projected by the projection device 12's rear lens. Similarly, the virtual image position can represent the location of the camera device 13 or the location of the image captured by the rear lens of the camera device 13. Furthermore, the location of the media playback server 14 is not restrictive; it is used to provide a wide, wide-angle, and appropriately segmented projection content to the projection devices 12. It should be noted that the location of the media playback server 14 does not need to be set in the multi-projection simulation tool. The projection devices 12 can project the display onto the immersive projection screen 10 using a warp and blending method. The camera device 13 is used for automatic projection calibration. In addition, in the multi-projection simulation tool, the field where the immersive projection system 1 is located including the decorative objects 15 can also be simulated.

[0041] It is worth mentioning that the multi-projection simulation tool can provide a layout assist function. The multi-projection simulation tool can provide assistance in selecting the projection device 12 and camera device 13 suitable for the three-dimensional projection scene. The layout assist will automatically calculate the layout information required to meet the requirements of this scene based on the size and shape of the projection screen in the scene, the projection distance of the projection device, the image size, brightness, and other conditions. The layout information includes the number of projection devices, configuration positions, postures, and lens-related settings. It can provide a reference for on-site installation scenes and is used during the auto calibration stage. Furthermore, the layout assist can automatically calculate the virtual projection position and virtual image deformation of the projection device 12 based on the specifications and projection distance of the projection device 12 selected by the user, and the user can make fine adjustments based on this. Furthermore, the choice of projection device 12 is related to the size of the projection scene space, image resolution, splicing and fusion band size, brightness requirements, and of course, the optional projection lens.

[0042] In some embodiments, layout assistance can provide two usage modes: based on the number of projection devices and based on projection distance. When planning the scene installation, it is necessary to meet the overall projection size to cover the entire screen and meet the brightness requirements. Under different projector models and lens combinations, the number of projectors required is different. Through the "based on the number of projectors consideration mode" of layout assistance, the optimal number of projectors configuration can be automatically planned. However, under the spatial conditions of some venues, the distance for projector installation may be limited. Through the "based on the projection distance consideration mode", layout assistance can plan the best layout plan under the known projection distance conditions to meet the projection size and brightness requirements.

[0043] Specifically, if considering the number of projectors, the user must first set the upper limit for the projection size of each projector and the minimum brightness requirement for the projection on the screen. Next, the user must select the projector model and lens configuration. Furthermore, the user must specify the percentage of the blend zone. Based on these conditions, the layout assistant calculates the required number of projectors and related layout information, ultimately presenting the resulting layout in a 3D graphical display.

[0044] On the other hand, if projection distance is a consideration, the user must first specify the projection distance and the minimum brightness required for the projection screen. Next, the user must select the projector model and lens, as well as the percentage of the blend band. Based on these conditions, the layout assistant calculates the required number of projectors and related layout information, ultimately presenting the resulting layout in a 3D graphical representation. The above description is for illustrative purposes only and is not intended to limit the present invention.

[0045] FIG4 is a schematic diagram of the projection frustum of the projection device 12. As shown in FIG4 , the lens of the projection device 12 has a throw ratio (TR), which is defined as the projection distance D divided by the projection screen width W (i.e., TR = D / W). A lens with a smaller throw ratio can project the same size image at a shorter distance. When the size of the on-site space is limited, selecting a projection lens with a smaller throw ratio is generally more suitable for meeting the on-site projection requirements. On the other hand, a projection lens with a smaller throw ratio is also more likely to cause trapezoidal distortion in the projected image. Therefore, the lens selection for the projection device 12 is generally based on the on-site space limitations. The most appropriate projection lens is selected, and the overall projected image needs to be slightly larger than the projection screen size. If projection geometric distortion (such as trapezoidal distortion) occurs due to installation conditions, it can be corrected by adjusting the position of the lens mount of the projection device 12 and using the electronic distortion adjustment function.

[0046] In other words, due to the space limitations of the installation site of the projection device 12 and to avoid interference with the projection light path by on-site viewers, the projection device 12 is generally installed at a higher position. Therefore, the projection optical axis may not be able to maintain a perpendicular condition to the projection surface. In this way, a trapezoidal projection image will be formed on the projection screen. Usually, the projection device 12 has a manual or electric lens mount, and the horizontal and vertical position adjustment of the lens mount can be used to reduce the projection trapezoidal deformation (image deformation) caused by the vertical projection installation condition. Furthermore, the electronic trapezoidal correction circuit configured in the projection device 12 can also perform projection trapezoidal correction. After correction, the projection device 12 can form a rectangular projection image on the projection screen. In the multi-projection simulation tool, it can provide simulated adjustment functions for the horizontal and vertical position adjustment of the lens mount and the electronic trapezoidal (virtual image deformation) correction. Therefore, the multi-projection simulation tool can be used to simulate the projection deformation adjustment correction value required for the on-site installation of the scene projection device 12.

[0047] Specifically, in a 3D simulation engine of a multi-projection simulation tool, a scene is typically described as objects in a 3D space, each object consisting of a plurality of 3D vertices, and these objects are rendered and displayed on a flat screen.

[0048] The rendered scene is relative to the camera, so the scene's vertices must also be defined relative to the camera's viewpoint. When a mesh is drawn in a 3D simulation engine (such as, but not limited to, the OpenGL rendering pipeline or the Direct3D graphics pipeline), the 3D simulation engine's vertex shader processes each vertex so that its position is defined in clip space.

[0049] In a 3D simulation engine, model view projection (MVP) is a series of matrix transformations that can be applied to vertices defined in model space, transforming them from local space to world space via the model matrix. This is then transformed to view space via the view matrix. This is then transformed to clip space via the projection matrix. Finally, the object can be rasterized, which is the process of converting an image represented in a vector graphics format into a bitmap for display on a monitor or projector.

[0050] For example, as shown in the following formula, v'=P×V×M×v

[0051] That is, the vertex position (v) in local space is first transformed by the model matrix (M), then by the view matrix (V), and finally by the projection matrix (P) to the vertex coordinates in clip space. It is worth mentioning that the projection matrix can be an orthographic projection matrix or a prospective projection matrix.

[0052] Specifically, the multi-projection simulation tool can provide users with immersive projection space scene configuration simulation and projection simulation, mainly using the model perspective projection technology mentioned above to convert each three-dimensional object in the projection scene into a two-dimensional dot matrix, and calculate the light, shadow, brightness and color of the three-dimensional objects in the three-dimensional scene through shaders and render them.

[0053] Figure 5A is a schematic diagram of the combined projection area of ​​the projection device, and Figure 5B is a schematic diagram of a virtual static image of the camera device. As shown in Figures 1, 5A, and 5B, in step S03, a projection area 121 of the projection device is simulated. In step S04, a plurality of first feature points 122 within projection area 121 are defined. In step S05, a virtual static image 131 of the camera device is simulated. Virtual static image 131 includes projection area 121 and first feature points 122 (represented by second feature points 132, described later, in Figure 5B).

[0054] In the multi-projection simulation tool, a projection simulation program can be used to simulate the combined projection area 121 of the projection devices and the virtual static image 131. In some embodiments, the projection simulation program may include, for example, a projection geometry simulation subroutine, a camera simulation subroutine, a warping simulation subroutine, a color and brightness simulation subroutine, and a blending simulation subroutine, but this is not limiting. For example, if only a single projection device is used, the warping simulation subroutine and the blending simulation subroutine may not be used.

[0055] The projection geometry simulation subroutine is described below. The projection geometry of the projection image is based on the exit pupil position of the projection device's lens as the origin. It is a viewing frustum that is expanded by the projected horizontal / vertical field of view formed according to the aspect ratio (AR) of the projection image of the projection device and the projection ratio of the projection lens. The surface of objects within the viewing frustum will be rendered into the projection image. The horizontal field of view (HFOV) and the vertical field of view (VFOV) can be calculated by the aspect ratio (AR) of the image and the projection ratio (TR) of the lens using the following formulas: HFOV = 2×atan (0.5 / TR) VFOV = 2×atan (2×TR×AR)

[0056] In practice, the projected image can be adjusted by shifting the lens horizontally or vertically. This creates a lens shift between the central optical axis of the projection device and the center of the actual projected image. By considering the lens shift, the vertical and horizontal fields of view can be calculated. Furthermore, the projection simulation tool's projection simulation on the projection screen is divided into two parts: the projection frame formed on the projection screen, and the projected content image on the projection screen.

[0057] The information about the projector's placement within the scene includes its position, its installation orientation (e.g., elevation, azimuth, and roll), the field of view and exit pupil distance of the projector lens, and the offset between the projection optical axis and the lens. This information can be used within the multi-projection simulation tool to generate a model perspective projection (MVP) matrix, extending from the projection optical origin of the projector.

[0058] In a simulated physical scene, points on the projection screen or wall are converted into homogeneous coordinates on the viewing cone extending from the projector's optical origin using the projection device's model perspective projection matrix. After de-homogeneity and linearization between the near and far planes of the viewing cone, the distances of these points relative to the projection device's optical origin are determined. A depth map table for the viewing cone is created using the depth test function of a 3D simulation engine (such as, but not limited to, OpenGL). The depth map table records the distance information from the projection light path of the projector to the object surface. The depth map table is two-dimensional. If the size setting matches the resolution of the projector, each point index in the depth map table corresponds to a pixel position in the image plane within the projector. Using the depth map table and the field of view of the projector's viewing cone, the horizontal and vertical projection fields of view of the physical scene corresponding to each index position in the depth map can be calculated relative to the projection optical axis.

[0059] By using the horizontal / vertical field of view of the projection device and the index position in the depth information table, the world coordinate corresponding to the index position in the physical world scene can be calculated and used to draw the projection boundary line. To obtain the world coordinate in the physical world scene, first calculate the horizontal field of view (θ) of the index position in the depth information table relative to the projection optical axis of the projector. H ) and vertical field of view (θ V The world coordinates (x, y, z) of the corresponding position in the physical scene can be obtained from the world coordinates (Prj.x, Prj.y, Prj.z) of the projector and the depth d1 using the following formula: x = Prj.x + d1 × tan (θ H ) y=Prj.y+d1×tan(θ V ) z=Prj.z–d1

[0060] Usually, all index positions on the depth information table are converted into world coordinates and connected, so that a projection border can be drawn on any shape.

[0061] The color and brightness simulation subroutine is described below. The content (color) projected on the projection screen is determined by the pixel shader in the graphics processing unit (GPU) of the computer executing the multi-projection simulation tool. The software within the pixel shader must calculate whether the current rendering point on the projection screen is within the projection field of view of the projection device, which serves as a reference for the rendering color mechanism. In certain embodiments, setting the virtual projection position and virtual image deformation of the projection device further includes: setting the virtual color and virtual brightness of the projection device; and adjusting the virtual color and virtual brightness of the projection device based on the first feature points.

[0062] First, the real-time information of the projection device, including the position of the projection device (i.e., the position of the projection optical center), the projection optical axis vector, the horizontal / vertical reference vector of the scene, etc., is passed to the pixel shader after the posture of the projection device is adjusted. The graphics processing unit will send the coordinates of each point (i.e., the rendering point) on the surface of the projection screen in the physical world scene to the pixel shader software for calculation. The software will first calculate the vector formed from the projection optical center to the rendering point, and project this vector onto the plane formed by the projection device optical axis vector and the horizontal / vertical reference vector of the scene, thereby calculating the horizontal / vertical field of view of the rendering point within the visual cone extended by the projection device. If the rendering point is within the projection field of view of the projection device, that is, it belongs to the projection screen, then the rendering point can be rendered into the screen color. If the rendering point is outside the range, it is rendered into the original color of the projection screen.

[0063] The projected image color comes from two sources: a user-defined texture image or the projection screen's own color, simulated through brightness. The pixel color value within the texture image to which a rendering point is mapped is determined by the ratio of the rendering point's horizontal / vertical field of view to the projector's field of view, which determines the texture coordinate. The texture coordinate can be used to find the corresponding pixel's color value, which the pixel shader uses to determine the rendering point's color. The texture coordinate (txelx, txely) can be calculated using the following formula. The horizontal projection field of view of the projection device is (leftFOV, rightFOV), and the vertical projection field of view is (lowFOV, upFOV).

[0064] txelx=(tan(θ H –tan(leftFOV)) / (tan(rightFOV)–tan(leftFOV))

[0065] txely=(tan(θ V –tan(lowFOV)) / (tan(upFOV)–tan(lowFOV))

[0066] When using the projection screen's inherent color to simulate the difference in incident brightness to simulate a projected image, the specified projection screen color is first converted from RGB (Red, Green, Blue) color coordinates to HSV (Hue, Saturation, Value / Brightness) color coordinates. The depth information table is then used to calculate the distance between the rendering point and the projector. The brightness ratio of the HSV color coordinates is then derived, taking into account the projector's own luminance (lumens). Finally, the value is converted back to RGB color coordinates, which represents the color rendered by the pixel shader at that point. To calculate the ratio of the HSV color coordinates, the illuminance of the projection screen under the projection of the projector is first calculated and then divided by the current ambient light intensity to obtain the SLR (Screen Lighting Ratio). For example, if two projectors, model DU5671, each output 6200 lumens (lm), are located at projection distances of 6.5m and 8.5m, respectively, and the room illuminance is 300 lx, the calculated SLRs are 1.23 and 1.86, respectively. Therefore, the HSV color coordinate Value of the projected area of ​​the projection screen is proportionally adjusted, and the simulation result can be consistent with the feeling of real experience.

[0067] Because SLR is very sensitive at close projection distances, the Multi-Projection Simulation Tool linearizes the SLR. This linearized SLR prevents rapid color saturation in close-range projection simulations and avoids color saturation in overlapping areas when using multiple projectors.

[0068] The fusion simulation subroutine is described as follows. The overlapping area generated by the projections of multiple projection devices will make the image in this area brighter than the non-overlapping area due to the sum of the brightness. Therefore, the rendering of the overlapping area must be fused (blend). The fusion simulation is processed in the pixel shader of the graphics processing unit. In some embodiments, setting the virtual projection position and virtual screen deformation of the projection device further includes: setting multiple virtual projection positions and multiple virtual screen deformations of multiple projection devices; adjusting the virtual projection position and virtual screen deformation of the projection device according to the first feature points further includes: adjusting the fusion correction information of the projection devices according to the first feature points.

[0069] The fusion process is, for example, edge blending of multiple projection devices on a flat projection screen. Figures 6A and 6B are schematic diagrams of the fusion process for a flat projection screen. First, as shown in Figure 6A, the horizontal and vertical fields of view corresponding to the overlapping areas of the projections are calculated. For example, the horizontal fields of view θ corresponding to the fusion reference lines in the overlapping areas are calculated. T (such as the fusion reference line corresponding to the upper overlapping area), vertical field of view θ L(For example, the fusion reference line corresponding to the overlapping area on the left.) As shown in Figure 6B, when projecting the overlapping area, the pixel shader proportionally dims the projected image based on the horizontal / vertical field of view of the target rendering point and the horizontal / vertical field of view of the projected overlapping area, so that the total brightness is the same as that of the non-overlapping area. Finally, the color and brightness simulation method described above is used to calculate the simulation result.

[0070] 7A and 7B are schematic diagrams of the fusion processing of the curved projection screen. On the other hand, as shown in FIG7A , the overlapping area of ​​the non-planar projection screen 10 is generally nonlinear. The fusion area is the polygon enclosed by P1 to P6 in the figure. Through the depth information table of each projection device obtained by the projection simulation and the relative position between the projection devices, it can be known that the target rendering point on the projection screen of the projection screen corresponds to the field of view of each projection device. Using the field of view of the projection device for this point, if it falls within the projection field of view of two (or more) projection devices, it can be determined that the point belongs to the fusion area. Based on this, the software algorithm can be used to calculate the point set of the polygon boundary of the fusion area. For example, the algorithm for constructing the polygon of the fusion area projected by two projection devices. Each point in the polygon boundary point set is in the following format:

[0071] P=[(Prj1,HFOV1,VFOV1),(Prj2,HFOV2,VFOV2),…], where

[0072] Prj1, Prj2, ... are projection devices whose projections include this point;

[0073] HFOV1 and VFOV1 are the horizontal / vertical fields of view of the point seen by the projection device Prj1;

[0074] HFOV2 and VFOV2 are the horizontal / vertical fields of view of the point seen by the projection device Prj2;

[0075] And so on.

[0076] Color brightness simulation is performed within the blending area. Based on the field of view of the target rendering point and its distance from the field of view of each projector, the color brightness is distributed to the projectors projecting toward that point using a gradient dimming method. Finally, the color brightness is accumulated within the GPU to produce the screen rendering point. The relationship between the color brightness gradient change percentage and the field of view is as follows.

[0077] Prj1.Intensity(P)%=(FOVH–FOVHm) / (FOVH–FOVH1)×100%

[0078] Prj2.Intensity(P)%=100-Prj1.Intensity(P)%

[0079] Among them, FOVH1 and FOVH are the horizontal field of view angle ranges passing through point P when Prj1 gives a given vertical field of view in the fusion area;

[0080] FOVHm is the horizontal field of view of point P at Prj1.

[0081] In other words, as shown in Figure 7B, segment B is calculated from segment A, and segment D is calculated from segment C, thereby determining the blending area. Because projection depth on a non-planar projection screen varies depending on the curvature, the color brightness distribution must not only consider the field of view but also the depth distance of each projector at that point. For example, the field of view can be used to first calculate the gradient change, dimming the color brightness ratio, and then adjusting the ratio based on the square of the projection depth at that point between projectors.

[0082] The following describes the stitching simulation subroutine. When the projection optical axis is not perpendicular to the projection screen surface, the projected image may exhibit trapezoidal distortion. The multi-projection simulation tool allows the user to adjust the vertical and horizontal keystone correction angles and calculates the keystone correction homography matrix on the image plane of the projection device.

[0083] Based on the horizontal / vertical field of view (FOV) of the target rendering point on the projection screen, the image resolution of the projector, and the projection field of view of the projector, the pixel shader first converts the FOV to the projector's image plane coordinates and performs a co-reflective matrix transformation to obtain new plane coordinates. This represents the position of the rendering point on the projector's image plane. Finally, these new plane coordinates are converted back to the FOV, and the depth information table is used to obtain the world coordinate position of the point after trapezoidal correction, thus drawing the trapezoidally corrected projection bounding box.

[0084] When keystone correction is enabled, the current rendering point in the pixel shader is at its post-keystone correction position. This point must be restored to its pre-correction position to determine whether it is within the projection area and to determine the texture coordinates of the image. For example, an inverse homography matrix transformation can be used to restore the pre-correction position, resulting in a keystone correction simulation with a textured image.

[0085] Furthermore, when projecting onto a curved screen, the curvature of the screen can cause distortion. The Multi-Projection Simulation Tool uses virtual longitude and latitude lines on the curved screen to find a new projection frame that adheres to these lines, while maintaining the maximum possible range within the original projection frame. This frame is the distortion-corrected projection frame.

[0086] Next, the depth information table is scanned in an indexed manner. Combined with the position and field of view of the projection device, the index value of the depth information table corresponding to the position of the projection point on the projection screen can be calculated. By detecting whether the position of the projection point on the projection screen is within the new projection frame, an index value array containing the frame can be formed. The original projection frame index value array is then mapped to the corrected projection frame index value array, and the co-injection matrix between the two is calculated. Similar to the trapezoidal correction method, this matrix also provides pixel shader calculations to transform the position of the projection screen rendering point to achieve the result of projection content image correction. When the curvature of the projection screen is large, it can be divided into sub-areas, and the co-injection matrix can be calculated in each area separately to achieve a more refined effect.

[0087] Referring again to Figures 5A and 5B , after the above simulation process, a projection area 121 of the projection devices can be obtained. Next, a plurality of first feature points 122 are defined within the projection area 121. Next, a virtual static image 131 of the photographic device is simulated using a camera simulation subroutine. In certain embodiments, the virtual static image 131 includes the projection area 121 and the first feature points 122 (represented by second feature points 132 described later in Figure 5B ).

[0088] The camera rig simulation subroutine is described below. The multi-projection simulation tool allows users to create a camera rig within a scene and set parameters such as the camera rig's position within the scene, the camera rig's lens posture (e.g., elevation, azimuth, and roll), and the camera rig's lens field of view.

[0089] When the camera simulation subroutine (camera perspective simulation) is entered, a 3D perspective matrix is ​​created using the camera's parameters, displaying the scene as seen from the camera's perspective, similar to the rendering effect of a photograph taken by the camera. An export function is also provided to save an image file for use in the automatic correction process. This generates a virtual still image 131 captured by the simulated camera. This virtual still image 131 includes the projection area 121 and the first feature points 122.

[0090] Specifically, the difference between the simulated 3D image and the projected image (2D image) captured by the camera is used to calculate a certain ratio to eliminate the distortion caused by the camera. For example, if two projectors are spliced ​​together, in order to ensure that the images are displayed in equal proportions on the projection screen, the two projectors will be spliced ​​and merged using Figure 5A (the actual 3D coordinate system of the scene) to ensure that the left side of the image is larger than the right side.

[0091] If calibration is performed based on the camera's viewing angle, the projected image closer to the camera will appear smaller, while the projected image farther from the camera will appear larger, resulting in the stitching problem of the left side being larger and the right side being smaller. When calibrating with a camera on-site, no information like that shown in Figure 5A is available. This information can only be obtained by using a rangefinder or tape measure to measure the projection screen and determine a 3D coordinate origin to correct for camera distortion.

[0092] Therefore, in order to overcome the complex and time-consuming problems, the present disclosure first creates the information after splicing as shown in FIG. 5A and FIG. 5B in a simulation manner.

[0093] 1 and 2 , in step S06 , the virtual projection position and the virtual image deformation of the projection device 12 are adjusted according to the first feature point.

[0094] In the multi-projection simulation tool, after completing the automatic layout, the layout assistant provides manual fine-tuning of each row of projection units. This fine-tuning includes desktop or ceiling mounting, vertical position, and bracket length. Within this fine-tuning range, the multi-projection simulation tool automatically adjusts the vertical offset of the projector lens to maintain consistent brightness on the projection screen. Furthermore, the color of each row of projection units can be specified here for improved recognition on the layout screen.

[0095] To adjust the position of a single projector, lock the projection in the Projection Reference section of the projector's settings to adjust the left / right, front / back, and top / bottom positions. The 3D simulation will also indicate the range of projector adjustments within the layout space. The Multi-Projection Simulator automatically adjusts the horizontal and vertical offsets and zoom ratios of the projector lenses, and displays the projection range.

[0096] In some embodiments, adjusting the virtual projection position and virtual image deformation of the projection device 12 based on the first feature points further includes: defining a plurality of second feature points in the virtual static image based on the first feature points, and comparing the first feature points with the second feature points to obtain virtual image difference parameters. Adjusting the virtual photography position of the photographic device 13 based on the virtual image difference parameters. In some embodiments, adjusting the virtual photography position of the photographic device 13 based on the virtual image difference parameters further includes: adjusting the simulated photography parameters of the photographic device 13 based on the virtual image difference parameters. In other words, the virtual projection position and virtual image deformation of the projection device 12 can be pre-adjusted based on the virtual image difference parameters obtained as shown in Figures 5A and 5B, while also adjusting the virtual photography position of the photographic device 13. Furthermore, the simulated photography parameters of the photographic device 13, such as color temperature, sensitivity, focal length, and exposure settings, can also be adjusted based on the virtual static image.

[0097] Referring again to Figures 1 and 2, in step S07, simulation overview information is output. The simulation overview information includes the adjusted virtual projection position and the adjusted virtual image deformation. In certain embodiments, the simulation overview information may also include adjusted virtual color and virtual brightness information and adjusted fusion correction information. Finally, based on the above-mentioned projection simulation program, the multi-projection simulation tool can output information including projection device and projection lens selection information, three-view images of the on-site projection device installation position, on-site projection device position information, projection device lens information, offset setting information for each projection device lens, simulated lens position, field of view information, etc.

[0098] In summary, the planning and configuration of existing immersive projection systems is relatively difficult. The calibration method for the immersive projection system disclosed herein can connect the actual locations of the projection device and the photographic device through the simulated world generated by three-dimensional modeling. The projection position of the projection device in the three-dimensional modeling is first optimized by simulating the images obtained by the photographic device, and then the optimized simulation profile information is output.

[0099] Figure 8 is a step diagram of the second embodiment of the calibration method for the immersive projection system disclosed herein. As shown in Figure 8, the calibration method for the immersive projection system of the second embodiment of the present disclosure may further include steps S8 to S12. As shown in Figures 2, 5A, 5B and 8, step S08 is to set the projection position and screen deformation (or geometric deformation of the projection screen) of the projection device 12 and set the photographic position of the photographic device 13 based on the simulation profile information. Step S09 is to obtain a static image through the photographic device 13. Step S10 is to obtain a plurality of second feature points 132 based on the first feature point 122 in the virtual static image 131. Step S11 is to compare the first feature point 122 with the second feature point 132 to obtain a virtual image difference parameter. Step S12 is to adjust the projection position and screen deformation of the projection device 12 based on the virtual image difference parameter and the static image. In some embodiments, acquiring a static image via the camera device 13 further includes: acquiring a plurality of third feature points in the static image based on the first feature points; and comparing the first feature points with the third feature points to obtain an image difference parameter. In some embodiments, adjusting the projection position of the projection device 12 based on the virtual image difference parameter and the static image further includes: adjusting the projection position and image distortion of the projection device 12 based on the virtual image difference parameter and the image difference parameter.

[0100] In some embodiments, a plurality of second feature points 132 are defined in the virtual static image 131 based on the first feature points 122, and the first feature points 122 are compared with the second feature points 132 to obtain virtual image difference parameters. In other words, the information after the splicing of Figures 5A and 5B is first created in a simulation, and then the virtual image difference parameters of the first feature points 122 and the second feature points 132 in Figures 5A and 5B are calculated.

[0101] Specifically, the calibration method of the immersive projection system disclosed herein can use, for example, the first feature point 122 obtained in FIG. 5A to bring in the feature points that need to be calibrated at the actual scene, obtain multiple third feature points by using a static image obtained by the photographic device 12 at the actual scene, and compare the first feature point with the third feature point (similar to the second feature point 132 in FIG. 5B ) to obtain image difference parameters to correspond to the feature points during the optimization simulation (such as the second feature point 132 in FIG. 5B ), and complete the stitching to obtain a two-dimensional stitching result. In other words, the virtual static image of the simulated photographic device can obtain virtual image difference parameters and the static image of the on-site photographic device can obtain image difference parameters. By comparing the difference between the two, the result can be optimized to produce a final nearly proportional picture.

[0102] The calculation method of the actual correction compensation method using analog photography equipment and on-site photography equipment is briefly described as follows.

[0103] First, after establishing the desired camera angle and projection device position during simulation, the simulated camera angle (A) and world coordinates (B) are combined, and the difference between the two is used to generate the variance (X) of each feature point. The simplified formula is as follows.

[0104] X=f(A)–f(B) (1)

[0105] Next, substitute the feature point information obtained in Figure 5B into the coordinates to be corrected in the actual scene. The simplified formula is as follows: Feature Real = Feature Simulation (2)

[0106] The error values ​​of Feature Real's simulation data were optimized using information obtained from on-site photography, and the calculated and spliced ​​results are as follows.

[0107] Feature Real Point(X n ,Y n )=Feature Real Point(n,n)×Optimization Function (3)

[0108] Y=f(Feature Real Point(Xn ,Y n ))

[0109] Finally, the value calculated by formula (1) is used to optimize the result of formula (3).

[0110] Z=f(X×Y) (4)

[0111] Specifically, the image of the projection device is acquired from the simulated photography device, and the feature points of the photography device and the projection device are mapped. These corresponding feature points are then applied to the actual device, and fine-tuning of the image is performed to optimize the correspondence between the feature points.

[0112] In other words, before performing geometric optimization simulations for multiple projectors, the simulated feature information is first established. This feature information is then optimized by connecting the projectors to the on-site camera system. Once the connections are complete, the projector offsets are used to move the projections to the target locations based on the simulation results. The on-site camera system is then positioned to the target locations using the simulated camera positions.

[0113] It is worth noting that the photographic device can be a webcam, SLR camera, mobile camera, or industrial camera. In some embodiments, when using a mobile camera as a platform to capture images, edge processing can be performed directly in the mobile camera to reduce the actual image transmission load.

[0114] Furthermore, in some embodiments, the photographic parameters of the photographic device can be set based on the simulated photographic parameters. In other words, by compensating for image differences between the simulated photographic device and the live photographic device, the photographic position of the live photographic device can be adjusted. Furthermore, as described above, the photographic parameters of the live photographic device, such as shutter speed, color temperature, sensitivity, focal length, and exposure, can also be set based on the simulated photographic parameters.

[0115] In other words, during the parameter verification phase (actual installation), the camera's shutter speed is automatically set to the same frequency as the projector's, using the parameters simulated during the simulation phase to eliminate image retention. Next, the sensitivity is set based on the simulated values ​​to adapt to the scene's ambient light and noise. Finally, the threshold is fine-tuned to correct for optical noise caused by stray light.

[0116] On the other hand, if the test pattern to be detected is blocked by an obstruction and it is impossible to obtain complete feature points, various algorithms can be used to remove unstable feature points to avoid incomplete detection and large detection position errors.

[0117] Before verification, binary thresholding is used to remove noise from horizontal and vertical image stripes, and a contour scan is performed across the entire projection area to establish the projection range. Next, when verifying the first test pattern, the image is segmented into several blocks using a function library, and the accuracy of each segmented image block is calculated. Contouring is then used to create each edge, and the minimum circle algorithm is used to calculate the position and radius of each circle center to obtain the characteristics of the complete projected image. Furthermore, color feature differences are used to calculate the projection angle offset and identify the HDMI IP address of the projection device.

[0118] During the optimization process, more feature points can be collected for different test patterns to increase image registration accuracy. After all test patterns are inspected, a complete feature point fusion is performed on all test patterns to establish a more dense feature location information. To address the problem of vanishing feature points caused by the image exceeding the actual projection area, the corresponding vectors can be calculated from the initially inspected horizontal and vertical stripes and all test patterns. Vector and co-injection matrix methods are then used to optimize the location of feature points during simulation.

[0119] After the optimization is completed, the simulated feature points and the actual image can be used for splicing and fusion between projection devices, and the desired strip fusion can be calculated by this technology.

[0120] Furthermore, after calculating the desired color values ​​(red, green, blue, cyan, magenta, yellow, and white balance) and appropriate brightness values ​​for each projector through color simulation, the camera device then compensates and adjusts for these seven colors, optimizing color projection consistency across multiple projectors.

[0121] Before performing color compensation on multiple projectors, the simulated color results are first assigned to the projectors. To ensure color consistency, the projectors are then optimized for color consistency by capturing the actual scene footage through a live camera connection. The projection positions of the projectors are then moved to the target locations during the simulation.

[0122] When detecting image color, image noise is first removed through binarization and the entire projection area is scanned using contours to establish the projection range point of interest (ROI). Next, the RGB color values ​​of each projector, such as a pseudo code, are calculated from the projected image area. Color compensation is performed by adjusting the seven color values ​​of the projector using seven different test patterns.

[0123] The three main colors are adjusted using seven colors (red, green, blue, cyan, magenta, yellow, and white balance) respectively. The colors do not interfere with each other during the adjustment process. The adjustment method is as follows.

[0124] In the color compensation stage, the number of color compensations is calculated through an algorithm, and the specified number of color compensations is performed in an open-loop manner.

[0125] In summary, the calibration method of the immersive projection system disclosed herein can assist users in scene planning, projection device and camera lens selection, and scene projection simulation during the system planning stage, and can output simulation overview information based on the projection simulation results. The calibration method of the immersive projection system disclosed herein can, on the one hand, provide users with a reference for the physical installation location, projection orientation, and projection lens selection of the on-site projection device; on the other hand, it can also provide users with setting information such as the projection size, lens seat shift, and trapezoidal correction required for the multi-projection installation setting and the automatic calibration program. In addition, the calibration method of the immersive projection system disclosed herein can also provide the automatic calibration program with information such as the type of camera lens to be used, the setting location, and the field of view, so that the automatic calibration program can smoothly perform automatic splicing, fusion, and color correction assisted by the camera. Furthermore, after the automatic calibration program is completed, relevant information such as the segmentation and fusion band of the corresponding multi-projection screen can be output to set the display card output screen of the media playback server.

[0126] Therefore, as described above, the calibration method of the immersive projection system disclosed herein provides a comprehensive solution covering everything from system planning, projection device and lens selection, projection simulation, to automatic projection calibration installation and content player configuration. This can significantly reduce the difficulty of system planning and implementation, facilitating the rapid implementation and promotion of immersive projection systems.

[0127] Figure 9 is a step diagram of the third embodiment of the calibration method for the immersive projection system disclosed herein. As shown in Figure 9, the calibration method for the immersive projection system of the third embodiment of the present disclosure may further include steps S20 to S25. Step S20 is to set the virtual projection position and virtual screen deformation of the projection device. Step S21 is to set the virtual photographic position of the photographic device. Step S22 is to obtain the projection area and the virtual static image through the projection simulation program. Step S23 is to define multiple first feature points in the projection area. Step S24 is to adjust the virtual projection position and virtual screen deformation of the projection device according to the first feature points. Step S25 is to output a simulation profile information including the adjusted virtual projection position and the adjusted virtual screen deformation.

[0128] The calibration method of the immersive projection system of the third embodiment of the present disclosure obtains the projection area and the virtual static image through a projection simulation program (step S22). The specific implementation has been described in detail in the first embodiment above and will not be repeated here.

[0129] FIG10 is a step diagram of the fourth embodiment of the calibration method for the immersive projection system disclosed herein. As shown in FIG10 , the calibration method for the immersive projection system of the fourth embodiment of the present disclosure may further include steps S26 to S30 compared to the third embodiment. Step S26 is to set the projection position and image deformation of the projection device and set the photographic position of the photographic device based on the simulation profile information. Step S27 is to obtain a static image through the photographic device. Step S28 is to obtain a plurality of second feature points based on the first feature point in the virtual static image. Step S29 is to compare the first feature point with the second feature point to obtain a virtual image difference parameter. Step S30 is to adjust the projection position and image deformation of the projection device based on the virtual image difference parameter and the static image.

[0130] The specific implementation of the calibration method of the immersive projection system of the fourth embodiment of the present disclosure has been described in detail in the second embodiment. Furthermore, the photographic parameters of the photographic device can also be set according to the simulated photographic parameters, which will not be repeated here.

[0131] FIG2 is a schematic diagram of an immersive projection system 1 disclosed herein. FIG11 is a schematic diagram of a projection device 12 disclosed herein. The immersive projection system 1 cooperates with a projection screen 10, and the immersive projection system 1 includes a projection device 12 and a photographic device 13. The projection device 12 has a projection position and image deformation (or projection image geometric deformation), and projects a projection area onto the projection screen 10. The photographic device is at a photographic position and obtains a static image. The static image includes the projection area. The projection position and image deformation of the projection device 12 are set based on the static image (for example, through the image difference parameters obtained therefrom) and the virtual static image (for example, through the virtual image difference parameters obtained therefrom). It should be noted that since the results after simulation will correspond to the installation site, the immersive projection system 1 shown in FIG2 is used to illustratively represent both the immersive projection system in the virtual environment and the immersive projection system at the installation site, but it is not used to limit the present invention. It is worth mentioning that, in this embodiment, multiple projection devices 12 are used as an example for illustration, but this is not limiting. In certain applications, only a single projection device 12 may be used (eg, FIG. 3A to FIG. 3B ).

[0132] In certain embodiments, the projection device 12 may include, for example, a motorized lens mount 20, an optional projection lens 21, a horizontal shift stepper motor 22 (connected to a horizontal slide), and a vertical shift stepper motor 23 (connected to a vertical slide), although this is not limiting. Furthermore, the control unit (e.g., an MCU) of the projection device 12 may control the motorized lens mount 20 to move horizontally and vertically via a motor drive circuit 24, causing the projected image to move horizontally and vertically accordingly, although this is not limiting. Furthermore, the projection device 12 may further include an electronic image distortion adjustment circuit 25 to perform geometric distortion correction and blending of the projected image, although this is not limiting.

[0133] The specific implementation of the immersive projection system 1 has been described in detail in the above embodiments and will not be repeated here.

[0134] As mentioned above, the immersive projection system and its calibration method disclosed herein can assist users in scene planning, projection device and camera lens selection, and scene projection simulation during the system planning stage, and can output simulation profile information (profile) based on the projection simulation results. The calibration method of the immersive projection system disclosed herein can, on the one hand, provide users with a reference for the physical installation location, projection orientation, and projection lens selection of the on-site projection device. On the other hand, it can also provide users with setting information such as the projection size, lens seat shift, and keystone correction required for the multi-projection installation setting and the automatic calibration program. In addition, the calibration method of the immersive projection system disclosed herein can also provide the automatic calibration program with recommended information such as the type of camera lens, setting location, and field of view (FOV) to be used, so that the automatic calibration program can smoothly perform automatic splicing, fusion, and color correction assisted by the camera. Furthermore, after the automatic calibration program is completed, relevant information such as the segmentation and fusion band of the corresponding multi-projection screen can be output to set the display card output screen of the media playback server.

[0135] Therefore, as described above, the immersive projection system and calibration method disclosed herein provide a comprehensive solution covering everything from system planning, projection device and lens selection, projection simulation, to automatic projection calibration installation and content player configuration. This can significantly reduce the difficulty of system planning and implementation, facilitating the rapid implementation and promotion of immersive projection systems.

[0136] The components of several embodiments are summarized above so that those skilled in the art to which the present invention belongs can better understand the concepts of the embodiments of the present invention. Those skilled in the art to which the present invention belongs should understand that the embodiments of the present invention can be used as a basis to design or modify other processes and structures to achieve the same purposes and / or the same benefits as the embodiments introduced herein. Those skilled in the art to which the present invention belongs should also understand that these equivalent structures do not depart from the spirit and scope of the present invention, and various changes, substitutions and other options can be made herein without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention shall be as defined by the appended claims.

Claims

1. A calibration method for an immersive projection system, comprising: Setting the virtual projection position and virtual image deformation of the projection device; Setting a virtual photographing position of a photographing device; Simulating a projection area of ​​the projection device; Defining a plurality of first feature points in the projection area; Simulating a virtual static image of the photographic device, the virtual static image including the projection area and the plurality of first feature points; Adjusting the virtual projection position of the projection device and the virtual screen deformation according to the plurality of first feature points; and Output simulation overview information, in, The simulation overview information includes the adjusted virtual projection position and the adjusted virtual picture deformation.

2. The calibration method according to claim 1, further comprising: According to the simulation profile information, setting the projection position and image deformation of the projection device and setting the photographing position of the photographing device; Acquiring a static image via the photographic device; In the virtual static image, obtaining a plurality of second feature points based on the plurality of first feature points; comparing the plurality of first feature points with the plurality of second feature points to obtain a virtual image difference parameter; and The projection position and the screen deformation of the projection device are adjusted according to the virtual image difference parameter and the static image.

3. The calibration method according to claim 2, wherein obtaining the static image via the photographing device further comprises: In the static image, obtaining a plurality of third feature points based on the plurality of first feature points; and The virtual image difference parameter is compared with the plurality of third feature points to obtain an image difference parameter.

4. The calibration method according to claim 3, wherein adjusting the projection position and the image deformation of the projection device according to the virtual image difference parameter and the static image further comprises: The projection position and the picture deformation of the projection device are adjusted according to the image difference parameter.

5. The calibration method according to claim 1, wherein setting the virtual projection position and the virtual image deformation of the projection device further comprises: Setting the virtual color and virtual brightness of the projection device; Adjusting the virtual color and the virtual brightness of the projection device according to the plurality of first feature points; in, The simulation profile information also includes the adjusted virtual color and virtual brightness information.

6. The calibration method according to claim 1, wherein setting the virtual projection position and the virtual image deformation of the projection device further comprises: Setting multiple virtual projection positions and multiple virtual image deformations of multiple projection devices; in, Adjusting the virtual projection position and the virtual image deformation of the projection device according to the plurality of first feature points further comprises: Adjusting fusion correction information of the plurality of projection devices according to the plurality of first feature points; Wherein, the simulation profile information further includes the adjusted fusion correction information.

7. The calibration method according to claim 1, wherein adjusting the virtual projection position and the virtual image deformation of the projection device according to the plurality of first feature points further comprises: In the virtual static image, obtaining a plurality of second feature points based on the plurality of first feature points; comparing the plurality of first feature points with the plurality of second feature points to obtain a virtual image difference parameter; and The virtual photographing position of the photographing device is adjusted according to the virtual image difference parameter.

8. The calibration method according to claim 7, wherein adjusting the virtual photographing position of the photographing device according to the virtual image difference parameter further comprises: The simulated photography parameters of the photography device are adjusted according to the virtual image difference parameters.

9. The calibration method according to claim 8, further comprising: According to the simulation profile information, setting the projection position and image deformation of the projection device and setting the photographing position of the photographing device; According to the simulated photographic parameters, setting photographic parameters of the photographic device; Acquiring a static image via the photographic device; and The projection position and the screen deformation of the projection device are adjusted according to the virtual image difference parameter and the static image.

10. A calibration method for an immersive projection system, comprising: Setting the virtual projection position and virtual image deformation of the projection device; Setting a virtual photographing position of a photographing device; Obtain projection area and virtual static image through projection simulation program; Defining a plurality of first feature points in the projection area; Adjusting the virtual projection position of the projection device and the virtual screen deformation according to the plurality of first feature points; and Output simulation overview information, in, The simulation overview information includes the adjusted virtual projection position and the adjusted virtual picture deformation.

11. The calibration method according to claim 10, further comprising: According to the simulation profile information, setting the projection position and image deformation of the projection device and setting the photographing position of the photographing device; Acquiring a static image via the photographic device; In the virtual static image, obtaining a plurality of second feature points based on the plurality of first feature points; comparing the plurality of first feature points with the plurality of second feature points to obtain a virtual image difference parameter; and The projection position and the screen deformation of the projection device are adjusted according to the virtual image difference parameter and the static image.

12. The calibration method according to claim 11, wherein obtaining the static image via the photographing device further comprises: In the static image, obtaining a plurality of third feature points based on the plurality of first feature points; and The virtual image difference parameter is compared with the plurality of third feature points to obtain an image difference parameter.

13. The calibration method according to claim 12, wherein adjusting the projection position and the image deformation of the projection device according to the virtual image difference parameter and the static image further comprises: The projection position and the picture deformation of the projection device are adjusted according to the virtual image difference parameter and the image difference parameter.

14. The calibration method according to claim 10, wherein the projection simulation program comprises a projection geometry simulation subroutine, a photographic device simulation subroutine, a splicing simulation subroutine, a color and brightness simulation subroutine, and a fusion simulation subroutine.

15. The calibration method according to claim 10, wherein adjusting the virtual projection position and the virtual image deformation of the projection device according to the plurality of first feature points further comprises: The virtual photographing position of the photographing device is adjusted according to the virtual static image.

16. The calibration method according to claim 15, wherein adjusting the virtual photographing position of the photographing device according to the virtual static image further comprises: The simulated photography parameters of the photography device are adjusted according to the virtual static image.

17. The calibration method according to claim 16, further comprising: According to the simulation profile information, setting the projection position and image deformation of the projection device and setting the photographing position of the photographing device; According to the simulated photographic parameters, setting photographic parameters of the photographic device; Acquiring a static image via the photographic device; and The projection position and the screen deformation of the projection device are adjusted according to the virtual image difference parameter and the static image.

18. An immersive projection system, in conjunction with a projection screen, the immersive projection system comprising: A projection device having a projection position and a picture deformation, and projects a projection area onto the projection screen; and A photographing device having a photographing position and acquiring a static image, wherein the static image includes the projection area, in, The projection position and the screen deformation of the projection device are set according to the static image and the virtual static image.

19. The immersive projection system of claim 18, wherein the virtual static image is obtained via a projection simulation process.

20. The immersive projection system according to claim 19, wherein the projection simulation program comprises a projection geometry simulation subroutine, a photographic device simulation subroutine, a splicing simulation subroutine, a color and brightness simulation subroutine, and a fusion simulation subroutine.

Images