Immersive projection system and correction method thereof

Abstract

The invention discloses a correction method of an immersive projection system. The correction method comprises the following steps: setting a virtual projection position and virtual picture deformation of a projection device; setting a virtual photographing position of the photographing device; simulating a projection area of the projection device; defining a plurality of first feature points in the projection area; the virtual static image of the simulation photographic device comprises a projection area and a first feature point; adjusting a virtual projection position and virtual picture deformation of the projection device according to the first feature point; and outputting simulated profile information including the adjusted virtual projection position and the adjusted virtual picture deformation. The invention also provides an immersive projection system.

Description

Technical Field

[0001] The present disclosure relates to a projection system, and more particularly to an immersive projection system and a calibration method thereof. Background Art

[0002] Since the immersive projection system can provide an immersive user experience, it has been widely used in applications including flight simulation, cultural and creative performances, starry sky simulation planetariums, theme park entertainment, etc. The immersive projection system can provide users with an ultra-wide viewing angle visual experience, making users feel like they are in a very realistic audio-visual environment.

[0003] A typical immersive projection system includes an immersive projection screen, viewer seats or a flight cabin, multiple projectors and a set of media playback servers, and may also include any on-site decorative objects.

[0004] In order to provide an immersive user experience, immersive projection scenes are generally set up on projection screens such as rings, hemispherical screens, and multi-walls. To project immersive content in the above projection scenes, multiple projectors must be used for splicing and fusion. In addition, the production of content must also be designed according to the projection theme, and the projection content must be cut and distributed to the corresponding projectors for projection playback.

[0005] In order to attract users, the themes, content and deployment methods of various immersive projections are different and diverse. Because they involve many different professional fields and details, the overall planning and setting of the immersive projection system is very difficult. It usually requires experienced system integrators to plan and set up each project, and it is also necessary to integrate a variety of different hardware and software tools.

[0006] The overall difficulty of the immersive projection system makes it difficult to be quickly promoted and set up. Therefore, how to provide an immersive projection system that is easy to set up and a calibration method thereof is one of the problems that need to be solved urgently. Summary of the invention

[0007] In order to solve the difficulties in planning and setting up the above immersive projection system, the present disclosure provides an immersive projection system and a calibration method thereof, which can assist users in scene planning, projection simulation, installation and setting of multiple projection devices, and camera-assisted automatic calibration to perform on-site installation and setting of projection, and can even assist in the automatic screen segmentation setting of the content player, thereby greatly reducing the difficulty of planning and setting up the immersive projection system.

[0008] The present disclosure provides a calibration method for an immersive projection system, comprising: setting a virtual projection position and a virtual image deformation of a projection device; setting a virtual photography position of a photography device; simulating a projection area of ​​the projection device; defining a plurality of first feature points in the projection area; simulating a virtual static image of the photography device, the virtual static image including the projection area and the first feature points; adjusting the virtual projection position and the virtual image deformation of the projection device according to the first feature points; and outputting simulation profile information. The simulation profile information includes the adjusted virtual projection position and the adjusted virtual image deformation.

[0009] In certain embodiments, the correction method disclosed herein further includes: setting the projection position and screen deformation of the projection device and setting the photographic position of the photographic device according to the simulation profile information; obtaining a static image through the photographic device; obtaining a plurality of second feature points in the virtual static image according to the first feature points; comparing the first feature points with the second feature points to obtain virtual image difference parameters; and adjusting the projection position and screen deformation of the projection device according to the virtual image difference parameters and the static image.

[0010] In some embodiments, obtaining the static image via the photographic device further includes: obtaining a plurality of third feature points in the static image based on the first feature points; and comparing the virtual image difference parameter with the third feature points to obtain the image difference parameter.

[0011] In some embodiments, adjusting the projection position and the image deformation of the projection device according to the virtual image difference parameter and the static image further includes: adjusting the projection position and the image deformation of the projection device according to the image difference parameter.

[0012] In some embodiments, setting the virtual projection position and virtual image deformation of the projection device further includes: setting the virtual color and virtual brightness of the projection device; adjusting the virtual color and virtual brightness of the projection device according to the first feature points. The simulation profile information also includes the adjusted virtual color and virtual brightness information.

[0013] In some embodiments, setting the virtual projection position and virtual screen deformation of the projection device further includes: setting multiple virtual projection positions and multiple virtual screen deformations of multiple projection devices; adjusting the virtual projection positions and virtual screen deformation of the projection device according to the first feature points further includes: adjusting the fusion correction information of the projection devices according to the first feature points. The simulation profile information also includes the adjusted fusion correction information.

[0014] In some embodiments, adjusting the virtual projection position and virtual screen deformation of the projection device according to the virtual image difference parameters further includes: obtaining a plurality of second feature points in the virtual static image according to the first feature points; comparing the first feature points with the second feature points to obtain the virtual image difference parameters; and adjusting the virtual photography position of the photography device according to the virtual image difference parameters.

[0015] In some embodiments, adjusting the virtual photographing position of the photographing device according to the virtual image difference parameter further includes: adjusting the simulated photographing parameters of the photographing device according to the virtual image difference parameter.

[0016] In certain embodiments, the calibration method disclosed herein further includes: setting the projection position and image deformation of the projection device and setting the photographic position of the photographic device according to the simulation profile information; setting the photographic parameters of the photographic device according to the simulation photographic parameters; obtaining a static image through the photographic device; and adjusting the projection position and image deformation of the projection device according to the virtual image difference parameters and the static image.

[0017] The present disclosure further provides a calibration method for an immersive projection system, comprising: setting a virtual projection position and a virtual image deformation of a projection device; setting a virtual photography position of a photography device; obtaining a projection area and a virtual static image through a projection simulation program; defining a plurality of first feature points in the projection area; adjusting the virtual projection position and the virtual image deformation of the projection device according to the first feature points; and outputting simulation profile information. The simulation profile information includes the adjusted virtual projection position and the adjusted virtual image deformation.

[0018] In some embodiments, the projection simulation program includes a projection geometry simulation subroutine, a camera device simulation subroutine, a stitching simulation subroutine, a color and brightness simulation subroutine, and a fusion simulation subroutine.

[0019] The present disclosure further provides an immersive projection system, which cooperates with a projection screen, and includes: a projection device, which has a projection position and a picture deformation, and projects a projection area on the projection screen; and a photographing device, which has a photographing position and obtains a static image, and the static image includes the projection area. The projection position and the picture deformation of the projection device are set according to the static image and the virtual static image.

[0020] In some embodiments, the virtual static image is obtained through a projection simulation process.

[0021] In summary, the immersive projection system and its calibration method disclosed herein can assist users in scene planning, projection device and camera lens selection and scene projection simulation in the system planning stage, and can output simulation profile information (profile) based on the projection simulation results. The calibration method of the immersive projection system disclosed herein can provide users with a reference for the location of the physical installation of the on-site projection device, the projection orientation, and the selection of the projection lens, and on the other hand, it can also provide users with the projection size, lens seat displacement, keystone correction and other setting information required for the multi-projection installation setting and the automatic calibration program. In addition, the calibration method of the immersive projection system disclosed herein can also provide the automatic calibration program with information such as the type of camera lens, setting position, field of view (FOV) and so on, so that the automatic calibration program can smoothly perform automatic splicing, fusion and color correction assisted by the camera. Furthermore, after the automatic calibration program is completed, the relevant information corresponding to the segmentation and fusion band of the multi-projection screen can be output to set the display card output screen of the media playback server.

[0022] Therefore, as mentioned above, the immersive projection system and its correction method disclosed in the present invention provide an overall solution covering everything from system planning, projection device and lens selection, projection simulation to projection automatic correction installation and content player settings, thereby greatly reducing the difficulty of system planning and construction, and facilitating the rapid construction and promotion of the immersive projection system. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 is a step diagram of a first embodiment of a calibration method for an immersive projection system disclosed herein;

[0024] Figure 2 A schematic diagram of an immersive projection system disclosed herein;

[0025] Figures 3A to 3G A schematic diagram showing different immersive projection screens;

[0026] Figure 4 is a schematic diagram of a projection cone of a projection device;

[0027] Figure 5A is a schematic diagram of a combined projection area of ​​a projection device;

[0028] Figure 5B is a schematic diagram of a virtual static image of a photographic device;

[0029] Fig. 6A and Figure 6B It is a schematic diagram of the fusion processing of the flat projection screen;

[0030] Fig. 7A and Figure 7BIt is a schematic diagram of the fusion processing of the curved projection screen;

[0031] Figure 8 is a step diagram of a second embodiment of the calibration method of the immersive projection system disclosed herein;

[0032] Fig. 9 is a step diagram of a fourth embodiment of the calibration method of the immersive projection system disclosed herein;

[0033] Fig.10 is a step diagram of a fifth embodiment of the calibration method of the immersive projection system disclosed herein;

[0034] Fig.11 is a schematic diagram of a projection device disclosed herein.

[0035] Description of Figure Numbers:

[0036] 1: Immersive projection system

[0037] 10, 10A~10G: Projection screen

[0038] 11: Spectator Seats

[0039] 12: Projection device

[0040] 121: Combined projection area

[0041] 122: First feature point

[0042] 13: Photographic Installation

[0043] 131:Virtual static image

[0044] 132: Second characteristic point

[0045] 14: Media playback server

[0046] 20: Electric lens mount

[0047] 21: Projection lens

[0048] 22: Horizontal shift stepper motor

[0049] 23: Vertical shift stepper motor

[0050] 24: Motor drive circuit

[0051] 25: Electronic graphics deformation adjustment circuit

[0052] S01~S12, S20~S30: Steps DETAILED DESCRIPTION

[0053] Figure 1FIG. 1 is a step diagram of a first embodiment of a calibration method for an immersive projection system disclosed herein. Figure 1 As shown, the calibration method of the immersive projection system disclosed in the present invention includes steps S01 to S07. Step S01 is to set the virtual projection position and virtual screen deformation of the projection device. Step S02 is to set the virtual photographic position of the photographic device. Step S03 is to simulate the projection area of ​​the projection device. Step S04 is to define a plurality of first feature points in the projection area. Step S05 is to simulate a virtual static image of the photographic device. Step S06 is to adjust the virtual projection position and virtual screen deformation of the projection device according to the number of first feature points. Step S07 is to output simulation overview information including the adjusted virtual projection position and the adjusted virtual screen deformation.

[0054] Figure 2 is a schematic diagram of an immersive projection system 1 disclosed herein. It should be noted that since the simulation results will correspond to the installation site, Figure 2 The immersive projection system 1 shown is used to illustratively represent both an immersive projection system in a virtual environment and an immersive projection system at an installation site, but is not used to limit the present invention.

[0055] like Figure 1 and Figure 2 As shown, in step S01, the virtual projection position and virtual image deformation (or virtual projection image geometric deformation) of the projection device 12 are set. In step S02, the virtual photography position of the photographic device 13 is set. In some embodiments, the immersive projection system 1 is coordinated with the immersive projection screen 10. The immersive projection system 1, for example, includes a viewer seat 11 (or a flight cabin), a projection device 12, a photographic device 13, and a media playback server 14, but this is not restrictive. In this embodiment, multiple projection devices 12 are used as an example for illustration, but this is not restrictive. In some applications, only a single projection device 12 (for example, as shown below) may be used. FIG. 3A to FIG. 3B ). The immersive projection screen 10 generally covers a wide viewing angle of at least 120° horizontally to allow the user to have an immersive visual experience. Figures 3A to 3G Schematic diagrams showing different immersive projection screens 10A-10G. The immersive projection screen may include, for example, a one-dimensional curved immersive projection screen 10A ( Figure 3A ), two-dimensional curved immersive projection screen 10B ( Figure 3B ), multiple walls form a cave-like immersive projection screen 10C ( Figure 3C ), cylindrical immersive projection screen 10D ( Figure 3D ), hemispherical immersive projection screen 10E ( Figure 3E ), full-screen immersive projection screen 10F ( Figure 3F ), and horseshoe-shaped immersive projection screen 10G ( Figure 3G), etc., but are not restrictive.

[0056] Please refer to Figure 1 and Figure 2 In the multi-projection simulation tool, the specifications of the projection device 12 and the photographic device 13 can be set according to different requirements, and the user can set multiple virtual projection positions and multiple virtual screen deformations of multiple projection devices 12 according to requirements, and set the virtual photographic position of the photographic device 13. It is worth mentioning that the virtual projection position can represent the configuration position of the projection device 12, or the position of the screen projected by the lens after the projection device 12 is configured. Similarly, the virtual photographic position can represent the configuration position of the photographic device 13, or the position of the screen acquired by the lens after the photographic device 13 is configured. In addition, the setting position of the media playback server 14 is non-restrictive, and it is used to provide a set of wide-width, wide-angle and appropriately divided projection content to the projection devices 12. It should be noted that in the multi-projection simulation tool, the position of the media playback server 14 may not be set first. The projection devices 12 can be projected and displayed on the immersive projection screen 10 using a warp and blend method. The photographic device 13 is used for automatic projection correction. In addition, in the multi-projection simulation tool, the field where the immersive projection system 1 is located including the decorative objects 15 can also be simulated.

[0057] It is worth mentioning that the multi-projection simulation tool can provide a layout assist function. The multi-projection simulation tool can provide assistance in selecting the projection device 12 and the photographic device 13 suitable for the scene based on the three-dimensional projection scene. The layout assist will automatically calculate the layout information that needs to meet the needs of this scene based on the size and shape of the projection screen in the scene, the projection distance of the projection device, the screen size, the brightness and other conditions. The layout information includes the number of projection devices, the configuration position, the posture and the lens-related settings, which can provide a reference for the on-site installation scene and be used in the autocalibration stage. Furthermore, the layout assist can automatically calculate the virtual projection position and virtual screen deformation of the projection device 12 based on the specifications and projection distance of the projection device 12 selected by the user, and the user can make fine adjustments based on this. Furthermore, the selection of the projection device 12 is related to the size of the projection scene space, the screen resolution, the size of the splicing and fusion bands, and the brightness requirements, and of course it is also related to the optional projection lens.

[0058] In some embodiments, layout assistance can provide two usage modes: based on the number of projection devices and based on projection distance. When planning the scene installation, it is necessary to meet the overall projection size to cover the entire screen and meet the brightness requirements. Under different projector models and lens combinations, the number of projectors required is different. Through the "based on the number of projectors consideration mode" of layout assistance, the optimal number of projectors configuration can be automatically planned. However, under the spatial conditions of some venues, the distance for projector installation may be limited. Through the "based on the projection distance consideration mode", layout assistance can plan the best layout plan under the known projection distance conditions to meet the projection size and brightness conditions.

[0059] Specifically, if the number of projectors is considered, the user must first set the upper limit of the projection size of a single projector and the minimum brightness requirement of the projection on the screen. Then, the user needs to select the projector model and the lens configured. At the same time, the user needs to give the percentage of the fusion zone. Layout Assistant will calculate the required number of projectors and related layout information based on the above conditions, and finally present the layout results in a three-dimensional diagram.

[0060] On the other hand, if the projection distance is considered, the user must first specify the projection distance and the minimum brightness requirement of the projection on the projection screen. Then, the user needs to select the projector model and the lens configured, and specify the percentage of the fusion band. The layout assistant will calculate the required number of projectors and related layout information based on the above conditions, and finally present the layout results in a three-dimensional diagram. The above is only for illustrative purposes and is not intended to limit the present invention.

[0061] Figure 4 Schematic diagram of the projection frustum of the projection device 12. Figure 4 As shown, the lens of the projection device 12 has a throw ratio (TR), and the definition of the throw ratio (TR) is the projection distance D divided by the projection screen width W (i.e., TR=D / W). A lens with a smaller throw ratio can project a screen of the same size at a shorter distance. When the size of the on-site space is relatively limited, selecting a projection lens with a smaller throw ratio is usually more able to meet the on-site projection requirements. On the other hand, a projection lens with a smaller throw ratio is also more likely to cause trapezoidal deformation of the projection screen. Therefore, the selection of the lens of the projection device 12 is usually based on the on-site space limitations, and the most appropriate projection lens is selected for installation. The overall projection screen needs to be slightly larger than the size of the projection screen. If the projection geometric deformation (such as trapezoidal deformation) occurs due to the installation conditions, it can be corrected by adjusting the position of the lens mount of the projection device 12 and the electronic deformation adjustment function.

[0062] In other words, in response to the space limitations of the installation site of the projection device 12 and to avoid interference of on-site viewers with the projection light path, the projection device 12 is generally installed at a higher position. Therefore, the projection optical axis may not be able to maintain a perpendicular condition to the projection surface. In this way, a trapezoidal projection image will be formed on the projection screen. Usually, the projection device 12 has a manual or electric lens mount, and the projection trapezoidal deformation (image deformation) caused by the vertical projection installation condition can be reduced by adjusting the horizontal and vertical positions of the lens mount. Furthermore, the electronic trapezoidal correction circuit configured by the projection device 12 can also perform projection trapezoidal correction. After correction, the projection device 12 can form a rectangular projection image on the projection screen. In the multi-projection simulation tool, it can provide a simulated adjustment function of the horizontal and vertical position adjustment of the lens mount and the electronic trapezoid (virtual image deformation) correction. Therefore, the multi-projection simulation tool can be used to simulate the projection deformation adjustment correction value required for the on-site installation of the scene projection device 12.

[0063] Specifically, in a 3D simulation engine of a multi-projection simulation tool, a scene is usually described as objects in a 3D space, each object is composed of a number of 3D vertices, and these objects are rendered and displayed on a flat screen.

[0064] The rendered scene is relative to the camera device, so the scene's vertices must also be defined relative to the camera's viewpoint. When drawing a mesh in a 3D simulation engine (such as, but not limited to, OpenGL's rendering pipeline, or Direct3D's graphics pipeline, etc.), the 3D simulation engine's vertex shader processes each vertex so that the vertex's position is defined in clip space.

[0065] In a 3D simulation engine, model view projection (MVP) is a series of matrix transformations that can be applied to vertices defined in model space, transforming them from local space to world space through the model matrix. Then, they are transformed to view space through the view matrix. Next, they are transformed to clip space through the projection matrix. Finally, objects can be rasterized. That is, the process of converting an image represented in a vector graphics format into a bitmap for display or projection.

[0066] For example, as expressed in the following formula,

[0067] v'=P×V×M×v

[0068] That is, the vertex position (v) in the local space is first transformed by the model matrix (M), then by the view matrix (V), and finally by the projection matrix (P) to the vertex coordinates in the clip space. It is worth mentioning that the projection matrix can be an orthographic projection matrix or a prospective projection matrix.

[0069] Specifically, the multi-projection simulation tool can provide users with immersive projection space scene configuration simulation and projection simulation, mainly using the model perspective projection technology mentioned above to convert each three-dimensional object in the projection scene into a two-dimensional dot matrix, and calculate the light, shadow, brightness and color of the three-dimensional objects in the three-dimensional scene through shaders and render them.

[0070] Figure 5A is a schematic diagram of the combined projection area of ​​the projection device, Figure 5B is a schematic diagram of a virtual static image of a photographic device. Figure 1 , Figure 5A and Figure 5B As shown, in step S03, a projection area 121 of a projection device is simulated. In step S04, a plurality of first feature points 122 in the projection area 121 are defined. In step S05, a virtual static image 131 of a photographic device is simulated, and the virtual static image 131 includes the projection area 121 and the first feature points 122 ( Figure 5B (indicated by the second characteristic point 132 described later).

[0071] In the multi-projection simulation tool, the combined projection area 121 of the projection devices and the virtual static image 131 can be simulated through a projection simulation program. In some embodiments, the projection simulation program can include, for example, a projection geometry simulation subroutine, a camera simulation subroutine, a warp simulation subroutine, a color and brightness simulation subroutine, and a blend simulation subroutine, but it is not limiting. For example, if only a single projection device is used, the warp simulation subroutine and the blend simulation subroutine may not be used.

[0072] The projection geometry simulation subroutine is described as follows. The projection geometry of the projection image is based on the exit pupil position of the projection device lens as the origin. According to the aspect ratio (AR) of the projection image of the projection device and the projection ratio of the projection lens, the viewing frustum is formed by the projection horizontal / vertical field of view. The surface of the object within the viewing frustum will be rendered into the projection image. The horizontal field of view (HFOV) and the vertical field of view (VFOV) can be calculated by the aspect ratio (AR) of the image and the projection ratio (TR) of the lens through the following formulas.

[0073] HFOV = 2 × atan (0.5 / TR)

[0074] VFOV = 2 × atan (2 × TR × AR)

[0075] In actual applications, the projection screen can be adjusted by moving the horizontal / vertical position of the lens. At this time, there will be an offset (lens shift) between the central optical axis of the projection device and the center of the actual projection screen. Considering the lens position offset, the vertical field of view and the horizontal field of view can be calculated. Furthermore, the projection simulation tool on the projection screen is divided into two parts: the projection frame formed on the projection screen, and the projection content screen on the projection screen.

[0076] The information of the projection device in the scene includes the position of the projection device, the installation posture of the projection device (such as elevation angle, azimuth angle, roll angle), the field of view and exit pupil distance of the projection device lens, and the offset between the projection optical axis and the lens. Therefore, in the multi-projection simulation tool, a model perspective projection (MVP) matrix can be formed that extends from the projection optical origin of the projection device.

[0077] In the simulated physical world scene, the points on the projection screen or wall are converted into homogeneous coordinates on the visual cone extending from the optical origin of the projector through the model perspective projection matrix of the projection device. After de-homogeneous and linearization between the near plane and the far plane of the visual cone, the distance of the points relative to the optical origin of the projection device can be obtained. The depth map of the visual cone is established through the depth test function of the three-dimensional simulation engine (such as but not limited to OpenGL). The depth map records the distance information of the projection light trace of the projection device to the surface of the object. The depth map is two-dimensional. If the size setting is the same as the resolution of the projection device, the index of each point in the depth map is exactly the pixel position of the image plane inside the projection device. Through the depth map and the visual cone field of view of the projection device, the horizontal / vertical projection field of view formed with the projection optical axis in the physical world scene corresponding to each index position on the depth map can be calculated.

[0078] Through the horizontal / vertical field of view projection of the position of the projection device and the index position of the depth information table, the world coordinates corresponding to the index position in the physical world scene can be calculated to draw the projection boundary line. To obtain the world coordinates in the physical world scene, first calculate the horizontal field of view (θ H ) and vertical field of view (θ V ). The world coordinates (x, y, z) of the corresponding position of the physical scene can be obtained by the world coordinates (Prj.x, Prj.y, Prj.z) of the projector and the depth d1 using the following formula.

[0079] x=Prj.x+d1×tan(θ H )

[0080] y=Prj.y+d1×tan(θ V )

[0081] z=Prj.z–d1

[0082] Usually, all index positions on the depth information table are converted into world coordinates and connected, so that a projection border can be drawn on any shape.

[0083] The color and brightness simulation subroutine is described as follows. The content (color) projected on the projection screen is determined by the pixel shader in the graphics processing unit (GPU) of the computer executing the multi-projection simulation tool. The software in the pixel shader must calculate whether the rendering point on the current projection screen is within the projection field of view of the projection device as a reference for the rendering color mechanism. In some embodiments, setting the virtual projection position and virtual screen deformation of the projection device further includes: setting the virtual color and virtual brightness of the projection device; adjusting the virtual color and virtual brightness of the projection device according to the first feature points.

[0084] First, the real-time information of the projection device, including the position of the projection device (i.e., the position of the projection optical center), the projection optical axis vector, the horizontal / vertical reference vector of the scene, etc., is passed to the pixel shader after the posture of the projection device is adjusted. The graphics processing unit will send the coordinates of each point (i.e., the rendering point) on the surface of the projection screen in the physical world scene to the pixel shader software for calculation. The software will first calculate the vector formed from the projection optical center to the rendering point, and project this vector onto the plane formed by the optical axis vector of the projection device and the horizontal / vertical reference vector of the scene, thereby calculating the horizontal / vertical field of view of the rendering point within the extended cone of the projection device. If the rendering point is within the projection field of view of the projection device, that is, it belongs to the projection screen, the rendering point can be rendered into the screen color at this time. If the rendering point is outside the range, it is rendered into the original color of the projection screen.

[0085] There are two sources for the color of the projected image: a texture image given by the user, or the color of the projection screen itself formed by brightness simulation. The color value of the pixel in the texture image to which the rendering point is mapped is determined by the ratio of the horizontal / vertical field of view of the rendering point to the projection field of view of the projector to locate the texture coordinates. The color value of the corresponding pixel can be found from the texture coordinates, so that the pixel shader can determine the color of the rendering point. The texture coordinates (txelx, txely) can be calculated using the following formula. Among them, the horizontal projection field of view of the projection device is (leftFOV, rightFOV), and the vertical projection field of view is (lowFOV, upFOV).

[0086] txelx=(tan(θ H –tan(leftFOV)) / (tan(rightFOV)–tan(leftFOV))

[0087] txely=(tan(θ V –tan(lowFOV)) / (tan(upFOV)–tan(lowFOV))

[0088] When using the projection screen's own color to simulate the difference in incident brightness to simulate the projection image, first convert the specified projection screen color from RGB (Red, Green, Blue) color coordinates to HSV (Hue, Saturation, Value / Brightness) color coordinates, and calculate the distance between the rendering point and the projector from the depth information table. Referring to the brightness (lumen) output of the projection device itself, derive the brightness ratio of the HSV color coordinate Value value, and finally convert it back to RGB color coordinates, which is the rendering color of the pixel shader at this point. To calculate the ratio of the HSV color coordinate Value value, first calculate the illuminance of the projection screen under the projection of the projection device, and then divide it by the illuminance of the current ambient light source to obtain the SLR (Screen Lighting Ratio). For example, two projectors model DU5671 output 6200 lumens (lm), the projection distances are 6.5m and 8.5m respectively, and the indoor illuminance is 300lx. The calculated SLRs are 1.23 and 1.86 respectively. Therefore, the HSV color coordinate Value of the color of the projected area of ​​the projection screen is proportionally adjusted, and the simulation result can be consistent with the feeling of real experience.

[0089] Since SLR is very sensitive to changes at close projection distances, the multi-projection simulation tool will linearize SLR. The linearized SLR can prevent the color of the projected image from saturating too quickly during close-range projection simulation, and can also avoid color saturation problems in overlapping areas when multiple projectors are projecting.

[0090] The fusion simulation subroutine is described as follows. The overlapping area generated by the projection of multiple projection devices will make the image in this area brighter than the non-overlapping area due to the sum of the brightness. Therefore, the rendering of the overlapping area must be fused. The fusion simulation is processed in the pixel shader of the graphics processing unit. In some embodiments, setting the virtual projection position and virtual screen deformation of the projection device further includes: setting multiple virtual projection positions and multiple virtual screen deformations of multiple projection devices; adjusting the virtual projection position and virtual screen deformation of the projection device according to the first feature points further includes: adjusting the fusion correction information of the projection devices according to the first feature points.

[0091] The blending process includes, for example, edge blending formed by multiple projection devices on a flat projection screen. Fig. 6A and Figure 6B is a schematic diagram of the fusion processing of the flat projection screen. First, Fig. 6A As shown, the horizontal / vertical field of view corresponding to the overlapping area of ​​the projections is calculated, for example, the horizontal field of view θ corresponding to the fusion reference lines of the overlapping area T (such as the fusion reference line corresponding to the upper overlapping area), vertical field of view θ L(such as the fusion reference line corresponding to the overlapping area on the left). Figure 6B As shown, when the pixel shader projects the overlapping area, it dims the projected image in proportion to the horizontal / vertical field of view of the target rendering point and the horizontal / vertical field of view of the overlapping area, so that the total brightness is the same as that of the non-overlapping area. Finally, the simulation result is calculated using the above color and brightness simulation method.

[0092] Fig. 7A and Figure 7B is a schematic diagram of the fusion processing of the curved projection screen. Fig. 7A As shown, the overlapping area of ​​the non-planar projection screen 10 is generally nonlinear. The fusion area is the polygon enclosed by P1 to P6 in the figure. Through the depth information table of each projection device obtained by the projection simulation and the relative position between the projection devices, it can be known that the target rendering point on the projection screen of the projection screen corresponds to the field of view of each projection device. Using the field of view of the projection device to this point, if it falls within the projection field of view of two (or more) projection devices, it can be determined that the point belongs to the fusion area. Based on this, the software algorithm can be used to calculate the point set of the polygon boundary of the fusion area. For example, the algorithm for constructing the polygon of the fusion area projected by two projection devices. Each point in the polygon boundary point set is in the following format:

[0093] P=[(Prj1,HFOV1,VFOV1),(Prj2,HFOV2,VFOV2),…], where

[0094] Prj1, Prj2, ... are projection devices whose projections include the point;

[0095] HFOV1, VFOV1 are the horizontal / vertical fields of view of the point seen by the projection device Prj1;

[0096] HFOV2, VFOV2 are the horizontal / vertical fields of view of the point seen by the projection device Prj2;

[0097] And so on.

[0098] The simulation of color brightness is to distribute the target rendering point to the projection device in the fusion area in a gradient dimming manner according to the field of view of each projection device and the distance from the field of view of each projection device. Finally, the color brightness is accumulated in the graphics processing unit to generate the screen rendering point. The relationship between the gradient change percentage of color brightness and the field of view is as follows.

[0099] Prj1.Intensity(P)%=(FOVH–FOVHm) / (FOVH–FOVH1)×100%

[0100] Prj2.Intensity(P)%=100-Prj1.Intensity(P)%

[0101] Among them, FOVH1 and FOVH are the horizontal field of view angle ranges passing through point P when Prj1 gives a vertical field of view in the fusion area;

[0102] FOVHm is the horizontal field of view of point P at Prj1.

[0103] In other words, if Figure 7B As shown, segment B is calculated from segment A, and segment D is calculated from segment C, and the fusion area is calculated accordingly. Since the projection depth of a non-planar projection screen surface will vary according to the curvature, the distribution of color brightness must not only consider the field of view, but also the depth distance of each projection device at that point. For example, the field of view is used to calculate the gradient change method, dim the color brightness ratio, and then adjust it according to the square ratio of the projection depth of the projection device at that point.

[0104] The following is a description of the splicing simulation subroutine. When the projection optical axis is not perpendicular to the projection screen surface, the projection image will have a trapezoidal deformation problem. The multi-projection simulation tool allows users to adjust the vertical or horizontal trapezoidal correction angle and calculate the trapezoidal correction homography matrix on the image plane of the projection device.

[0105] According to the horizontal / vertical field of view of the target rendering point of the projection screen, the image resolution of the projection device, and the projection field of view of the projection device, the pixel shader first converts the horizontal / vertical field of view to the projection device image plane coordinates for co-projection matrix conversion to obtain new plane coordinates. This is the position of the rendering point corresponding to the projection device image plane. Finally, this new plane coordinate is converted back to the horizontal / vertical field of view, and the depth information table is used to obtain the world coordinate position of the point after trapezoidal correction, so as to depict the projection frame after trapezoidal correction.

[0106] When keystone correction is enabled, the current rendering point in the pixel shader is at the position after keystone correction. The point must be restored to the position before correction to determine whether it is in the projection area and determine the texture coordinates of the image. For example, an inverse homography matrix transformation can be introduced to restore the position before correction to obtain a keystone correction simulation result with a texture image screen.

[0107] In addition, when projecting onto a curved projection screen, the projection will be deformed as the curvature of the projection screen changes. The multi-projection simulation tool can use the virtual longitude / latitude lines on the curved projection screen to find a new projection frame that is close to the longitude / latitude lines and try to maintain the maximum range within the original projection frame. This frame is the projection frame after deformation correction.

[0108] Next, the depth information table is scanned in an indexed manner, and the position and field of view of the projection device are combined to calculate the index value of the depth information table corresponding to the position of the projection point on the projection screen. By detecting whether the position of the projection point on the projection screen is within the new projection border, an index value array containing the border can be formed. Then the original projection border index value array is mapped to the corrected projection border index value array, and the co-injection matrix between the two is calculated. Just like the trapezoidal correction method, this matrix also provides pixel shader calculations to convert the position of the projection screen rendering point to achieve the result of projection content picture correction. When the curvature of the projection screen is large, it can be divided into sub-areas, and the co-injection matrix is ​​calculated in each area to achieve a more refined effect.

[0109] Please refer to Figure 5A and Figure 5B As shown, after the above simulation procedure, the projection area 121 of the projection devices can be obtained. Then, a plurality of first feature points 122 are defined in the projection area 121. Then, the virtual static image 131 of the photography device is simulated using the photography device simulation subroutine. In some embodiments, the virtual static image 131 includes the projection area 121 and the first feature points 122 ( Figure 5B (indicated by the second characteristic point 132 described later).

[0110] The camera device simulation subroutine is described below. The multi-projection simulation tool allows users to create a camera device in a scene and set the camera device's position in the scene, the camera device's lens posture (such as elevation, azimuth, roll angle) and the camera device's lens field of view and other parameters.

[0111] When entering the camera simulation subroutine (camera perspective simulation), the three-dimensional perspective matrix will be established with the camera parameters to display the scene seen from the camera perspective, similar to the rendering effect of the camera. In addition, an export function is provided to save the image file for use by the automatic correction program. Thus, a virtual static image 131 taken by the simulated camera can be obtained, and the virtual static image 131 will include the projection area 121 and the first feature points 122.

[0112] Specifically, the difference between the simulated 3D actual image and the projected image (2D image) obtained by the camera can be used to calculate a certain ratio to eliminate the distortion produced by the camera. For example, if two projection devices are connected, in order to make the images be played on the projection screen in the same proportion, Figure 5A (The actual three-dimensional coordinate system of the scene) is used to stitch and merge the two projection devices so that the picture will not be larger on the left and smaller on the right.

[0113] If the calibration is done based on the camera's viewing angle, the projection image closer to the camera will be relatively small, while the projection image farther from the camera will be relatively large, which will cause the problem of larger left and smaller right after stitching. Figure 5A Unless the user uses a rangefinder or tape measure to measure the projection screen and sets a three-dimensional coordinate origin, this information can be obtained to correct the distortion of the photographic device.

[0114] Therefore, in order to overcome the complex and time-consuming problem, the present invention first establishes a simulation method such as Figure 5A and Figure 5B The spliced ​​information.

[0115] Please refer to Figure 1 and Figure 2 As shown, in step S06, the virtual projection position and virtual image deformation of the projection device 12 are adjusted according to the first feature point.

[0116] In the multi-projection simulation tool, after the automatic layout is completed, the layout assistant can provide manual fine-tuning function for each row of projection devices. The items that can be fine-tuned include whether the installation method is desktop or ceiling, the movement of the upper and lower positions, and the length of the bracket. Within the fine-tunable range, the multi-projection simulation tool will automatically adjust the vertical offset of the projector lens to keep the brightness of the image projected to the projection screen unchanged. In addition, the appearance color of each row of projection devices can also be specified here to improve recognition on the layout screen.

[0117] If you want to adjust the position of a single projector, lock the projection in the projection reference of the projector's settings to adjust the left / right, front / back, top / bottom positions of the projector. At the same time, the 3D simulation screen will indicate the range of fine-tuning of the projector in the layout space. The multi-projection simulation tool will automatically adjust the horizontal offset, vertical offset, and zoom ratio of the projector lens, and display the projection screen range.

[0118] In some embodiments, adjusting the virtual projection position and virtual image deformation of the projection device 12 according to the first feature points further includes: defining a plurality of second feature points in the virtual static image according to the first feature points, and comparing the first feature points with the second feature points to obtain virtual image difference parameters. Adjusting the virtual photography position of the photography device 13 according to the virtual image difference parameters. In some embodiments, adjusting the virtual photography position of the photography device 13 according to the virtual image difference parameters further includes: adjusting the simulation photography parameters of the photography device 13 according to the virtual image difference parameters. In other words, according to Figure 5A and Figure 5BThe obtained virtual image difference parameters can be used to pre-adjust the virtual projection position and virtual image deformation of the projection device 12, and adjust the virtual photography position of the photography device 13. Furthermore, the simulated photography parameters of the photography device 13, such as color temperature, sensitivity, focal length, and exposure, can also be adjusted according to the virtual static image.

[0119] Please refer to Figure 1 and Figure 2 As shown, in step S07, the simulation overview information is output. The simulation overview information includes the adjusted virtual projection position and the adjusted virtual screen deformation. In some embodiments, the simulation overview information may also include the adjusted virtual color and virtual brightness information and the adjusted fusion correction information. Finally, according to the above-mentioned projection simulation program, the multi-projection simulation tool can output information including the selection information of the projection device and the projection lens, the three-view plane of the installation position of the on-site projection device, the on-site projection device position information, the projection device lens information, the offset setting information of the lens of each projection device, and the simulated lens position, field of view information, etc.

[0120] In summary, the planning and setting of existing immersive projection systems are relatively difficult. The calibration method of the immersive projection system disclosed herein can connect the actual positions of the projection device and the photographic device through the simulated world generated by three-dimensional modeling, and first optimize the projection position of the projection device in the three-dimensional modeling by simulating the picture obtained by the photographic device, and then output the optimized simulation overview information.

[0121] Figure 8 FIG. 2 is a step diagram of a second embodiment of the calibration method of the immersive projection system disclosed herein. Figure 8 As shown, the calibration method of the immersive projection system of the second embodiment of the present disclosure may further include steps S8 to S12. Figure 2 , Figure 5A , Figure 5B and Figure 8As shown, step S08 is to set the projection position and screen deformation (or projection screen geometric deformation) of the projection device 12 and the photographing position of the photographing device 13 according to the simulation profile information. Step S09 is to obtain a static image through the photographing device 13. Step S10 is to obtain a plurality of second feature points 132 according to the first feature point 122 in the virtual static image 131. Step S11 is to compare the first feature point 122 with the second feature point 132 to obtain a virtual image difference parameter. Step S12 is to adjust the projection position and screen deformation of the projection device 12 according to the virtual image difference parameter and the static image. In some embodiments, obtaining the static image through the photographing device 13 further includes: obtaining a plurality of third feature points according to the first feature points in the static image; and comparing the first feature points with the third feature points to obtain the image difference parameter. In some embodiments, adjusting the projection position of the projection device 12 according to the virtual image difference parameter and the static image further includes: adjusting the projection position and screen deformation of the projection device 12 according to the virtual image difference parameter and the image difference parameter.

[0122] In some embodiments, in the virtual static image 131, a plurality of second feature points 132 are defined based on the first feature points 122, and the first feature points 122 are compared with the second feature points 132 to obtain virtual image difference parameters. Figure 5A and Figure 5B After splicing, calculate Figure 5A and Figure 5B The virtual image difference parameters of the first feature point 122 and the second feature point 132 in FIG.

[0123] Specifically, the calibration method of the immersive projection system disclosed herein can be used, for example, Figure 5A The first feature point 122 obtained is brought into the feature point that needs to be corrected in the actual scene, and a plurality of third feature points are obtained by using the static image obtained by the photographing device 12 in the actual scene, and the first feature point is compared with the third feature point (similar to Figure 5B The image difference parameters are obtained by using the second feature point 132 to correspond to the feature point during the optimization simulation (e.g. Figure 5B In other words, the virtual static image of the simulated photographic device can obtain virtual image difference parameters and the static image of the on-site photographic device can obtain image difference parameters. By comparing the difference between the two, the result can be optimized to produce a nearly proportional picture.

[0124] The calculation method of the actual correction compensation method using analog photography equipment and on-site photography equipment is briefly described as follows.

[0125] First, after establishing the desired camera angle and projection device position during simulation, the camera angle (A) and the world coordinate (B) during simulation are stitched together, and the difference between the two is used to generate the difference (X) of each feature point. The simplified formula is as follows.

[0126] X = f(A) – f(B) (1)

[0127] Next, Figure 5B Substitute the feature point information obtained in into the coordinates to be corrected at the actual site. The simplified formula is as follows.

[0128] Feature Real = Feature Simulation (2)

[0129] The error value of Feature Real's simulation data is optimized through the information obtained from the on-site photography device, and the result after calculation and splicing is as follows.

[0130] Feature Real Point (X n , Y n ) = Feature Real Point (n, n)×OptimizationFunction (3)

[0131] Y=f(Feature Real Point(X n ,Y n ))

[0132] Finally, the value calculated by equation (1) is used to optimize the result of equation (3).

[0133] Z = f(X×Y) (4)

[0134] Specifically, the image of the projection device is obtained from the simulated photography device established in the simulation, and the correspondence between the feature points of the photography device and the projection device is established. The corresponding feature points are established in the actual device, and the image is fine-tuned to optimize the correspondence between the feature points.

[0135] In other words, before performing the geometric simulation optimization of multiple projection devices, the simulated feature information is first established, and then the feature information of the previous simulation is optimized by connecting with the on-site camera device. After the connection is completed, the offset of the projection device is used to move the projection screen to the target position according to the simulation result, and the on-site camera device is arranged to the target position through the simulated camera device position.

[0136] It is worth mentioning that the photographing device may be a webcam, a single-lens reflex camera, a mobile camera or an industrial camera. In some embodiments, when a mobile camera is used as a platform to acquire images, edge processing may be performed directly in the mobile camera to reduce the actual image transmission load.

[0137] In addition, in some embodiments, the photographic parameters of the photographic device can be set according to the simulated photographic parameters. In other words, the photographic position of the live photographic device can be adjusted by correcting and compensating the image difference between the simulated photographic device and the live photographic device. At the same time, as mentioned above, the photographic parameters of the live photographic device, such as shutter time, color temperature, sensitivity, focal length, and exposure, can also be set according to the simulated photographic parameters.

[0138] In other words, during the parameter verification phase (actual installation phase), the shutter time of the camera device will be automatically set to the same frequency as the projection device based on the parameters simulated during the simulation phase to remove the afterimage effect. Then, the sensitivity is set based on the values ​​during the simulation to adapt to the ambient light brightness and noise of the entire scene. Finally, the optical noise problem caused by stray light is corrected by fine-tuning the threshold.

[0139] On the other hand, if the test pattern to be detected is blocked by an obstruction and it is impossible to obtain complete feature points, in order to avoid excessive detection position errors caused by incomplete detection, various algorithms can be used to remove unstable feature points.

[0140] Before verification, the horizontal and vertical image noise can be removed by binary thresholding, and the entire projection area can be scanned using contours to establish the projection range. Next, when verifying the first test pattern, the image is divided into several blocks through the function library, and each segmented image block is calculated. Then, each edge line is created using the contour, and the position and radius of each circle center are calculated using the minimum circle algorithm to obtain the characteristics of the complete projection image. In addition, the calculation of the projection angle offset and the identification of the HDMI-IP of the projection device are established through the difference in color characteristics.

[0141] During the optimization process, more feature points can be collected for different test patterns to increase the accuracy of image alignment. After all test patterns are detected, all test patterns can be completely fused to establish more dense feature position information. When the image exceeds the actual projection area, the problem of missing feature points (vanishing point) can be solved by calculating the corresponding vectors through the horizontal and vertical images and all test patterns detected at the beginning, and optimizing the position of feature points during simulation through the method of vector and co-reflection matrix.

[0142] After the optimization is completed, the simulated feature points and the actual image can be used for splicing and fusion between projection devices, and the desired band fusion can be calculated by this technology.

[0143] In addition, after calculating the expected color (red, green, blue, cyan, magenta, yellow, white balance) values ​​and appropriate brightness values ​​of each projector through color simulation, the camera device compensates and adjusts the above seven colors to optimize the color projection consistency of multiple projectors.

[0144] Before color compensation of multiple projection devices, the simulated color results are set to the projector device. In order to compensate for the consistency of the simulated colors, the actual scene images are captured through the on-site photography device to optimize the color consistency of the projection device, and the projection position of the projection device is moved to the target position during simulation.

[0145] When detecting the image color, the image noise is first removed by binarization and the entire projection area is scanned using contours to establish the projection range point of interest (ROI). Next, the RGB color values ​​of each projection device are calculated through the projected image area, such as the pseudo code. When color compensation is performed, seven different test patterns are used to adjust the seven color values ​​of the projection device.

[0146] The three main colors are adjusted with seven colors (red, green, blue, cyan, magenta, yellow, and white balance) respectively. The colors in the adjustment process will not interfere with each other. The adjustment method is as follows.

[0147]

[0148] In the color compensation stage, the number of color compensations is calculated through an algorithm, and the specified number of color compensations is performed in an open-loop manner.

[0149] In summary, the calibration method of the immersive projection system disclosed herein can assist users in scene planning, projection device and camera lens selection, and scene projection simulation during the system planning stage, and can output simulation overview information based on the projection simulation results. The calibration method of the immersive projection system disclosed herein can, on the one hand, provide users with a reference for the location of the physical installation of the on-site projection device, the projection orientation, and the selection of the projection lens. On the other hand, it can also provide users with the projection size, lens seat displacement, trapezoidal correction, and other setting information required for the multi-projection installation setting and the automatic calibration program. In addition, the calibration method of the immersive projection system disclosed herein can also provide the automatic calibration program with information such as the type of camera lens, setting position, and field of view that are recommended to be used, so that the automatic calibration program can smoothly perform automatic splicing, fusion, and color correction assisted by the camera. Furthermore, after the automatic calibration program is completed, relevant information corresponding to the segmentation and fusion bands of the multi-projection screen can be output to set the display card output screen of the media playback server.

[0150] Therefore, as mentioned above, the calibration method of the immersive projection system disclosed in the present invention provides an overall solution covering everything from system planning, projection device and lens selection, projection simulation to projection automatic calibration installation and content player settings, thereby greatly reducing the difficulty of system planning and construction, and facilitating the rapid construction and promotion of the immersive projection system.

[0151] Fig. 9 FIG. 1 is a step diagram of a third embodiment of the calibration method of the immersive projection system disclosed herein. Fig. 9 As shown, the calibration method of the immersive projection system of the third embodiment of the present disclosure may further include steps S20 to S25. Step S20 is to set the virtual projection position and virtual screen deformation of the projection device. Step S21 is to set the virtual photography position of the photography device. Step S22 is to obtain the projection area and the virtual static image through the projection simulation program. Step S23 is to define a plurality of first feature points in the projection area. Step S24 is to adjust the virtual projection position and virtual screen deformation of the projection device according to the first feature points. Step S25 is to output a simulation overview information including the adjusted virtual projection position and the adjusted virtual screen deformation.

[0152] The calibration method of the immersive projection system of the third embodiment of the present disclosure is to obtain the projection area and the virtual static image through a projection simulation program (step S22). The specific implementation has been described in detail in the first embodiment above, and will not be repeated here.

[0153] Fig.10 FIG. 4 is a step diagram of a fourth embodiment of the calibration method of the immersive projection system disclosed herein. Fig.10As shown, the calibration method of the immersive projection system of the fourth embodiment of the present disclosure may further include steps S26 to S30 compared to the third embodiment. Step S26 is to set the projection position and screen deformation of the projection device and set the photographic position of the photographic device based on the simulation profile information. Step S27 is to obtain a static image through the photographic device. Step S28 is to obtain a plurality of second feature points in the virtual static image based on the first feature point. Step S29 is to compare the first feature point with the second feature point to obtain a virtual image difference parameter. Step S30 is to adjust the projection position and screen deformation of the projection device based on the virtual image difference parameter and the static image.

[0154] The specific implementation of the calibration method of the immersive projection system of the fourth embodiment of the present disclosure has been described in detail in the second embodiment. Furthermore, the photographic parameters of the photographic device can also be set according to the simulated photographic parameters, which will not be repeated here.

[0155] Figure 2 is a schematic diagram of an immersive projection system 1 disclosed herein. Fig.11 Schematic diagram of the projection device 12 disclosed in the present invention. The immersive projection system 1 cooperates with the projection screen 10, and the immersive projection system 1 includes the projection device 12 and the photographing device 13. The projection device 12 has a projection position and a picture deformation (or a projection picture geometric deformation), and projects a projection area onto the projection screen 10. The photographing device is at a photographing position and obtains a static image. The static image includes the projection area. The projection position and picture deformation of the projection device 12 are set based on the static image (for example, through the image difference parameters obtained therefrom) and the virtual static image (for example, through the virtual image difference parameters obtained therefrom). It should be noted that since the simulated results will correspond to the installation site, Figure 2 The immersive projection system 1 shown is used to illustrate both the immersive projection system in a virtual environment and the immersive projection system at the installation site, but it is not used to limit the present invention. It is worth mentioning that in this embodiment, multiple projection devices 12 are used as an example for explanation, but it is not restrictive. In some applications, only a single projection device 12 (such as the above) can be used. FIG. 3A to FIG. 3B ).

[0156] In some embodiments, the projection device 12 may include, for example, an electric lens holder 20, an optional projection lens 21, a horizontal shift stepper motor 22 (connected to a horizontal moving slide rail), and a vertical shift stepper motor 23 (connected to a vertical moving slide rail), etc., but this is not limiting. Furthermore, the control unit (e.g., MCU) of the projection device 12 may control the electric lens holder 20 to move horizontally and vertically through the motor drive circuit 24, and the projection image will move horizontally and vertically accordingly, but this is not limiting. In addition, the projection device 12 may further include an electronic graphic deformation adjustment circuit 25 to perform geometric deformation correction adjustment and fusion band fusion processing of the projection image, but this is not limiting.

[0157] The specific implementation of the immersive projection system 1 has been described in detail in the above embodiments and will not be repeated here.

[0158] As mentioned above, the immersive projection system and its calibration method disclosed in the present invention can assist users in scene planning, projection device and camera lens selection and scene projection simulation in the system planning stage, and can output simulation profile information (profile) based on the projection simulation results. The calibration method of the immersive projection system disclosed in the present invention can provide users with references to the location of the physical installation of the on-site projection device, the projection orientation, and the selection of the projection lens, and on the other hand, it can also provide users with the projection size, lens seat displacement, keystone correction and other setting information required for the multi-projection installation setting and the automatic calibration program. In addition, the calibration method of the immersive projection system disclosed in the present invention can also provide the automatic calibration program with information such as the type of camera lens, setting position, field of view (FOV) and so on, so that the automatic calibration program can smoothly perform automatic splicing, fusion and color correction assisted by the camera. Furthermore, after the automatic calibration program is completed, the relevant information corresponding to the segmentation and fusion band of the multi-projection screen can be output to set the display card output screen of the media playback server.

[0159] Therefore, as mentioned above, the immersive projection system and its correction method disclosed in the present invention provide an overall solution covering everything from system planning, projection device and lens selection, projection simulation to projection automatic correction installation and content player settings, thereby greatly reducing the difficulty of system planning and construction, and facilitating the rapid construction and promotion of the immersive projection system.

[0160] The components of several embodiments are summarized above so that the technical personnel in the technical field to which the present invention belongs can better understand the concept of the embodiments of the present invention. The technical personnel in the technical field to which the present invention belongs should understand that the embodiments of the present invention can be used as a basis to design or modify other processes and structures to achieve the same purpose and / or achieve the same benefits as the embodiments introduced herein. The technical personnel in the technical field to which the present invention belongs should also understand that these equivalent structures do not deviate from the spirit and scope of the present invention, and various changes, substitutions and other options can be made here without departing from the spirit and scope of the present invention. Therefore, the protection scope of the present invention shall be defined as the appended application claims.

Claims

1. A calibration method for an immersive projection system, comprising: Setting the virtual projection position and virtual image deformation of the projection device; Setting a virtual photographing position of a photographing device; Simulating a projection area of ​​the projection device; Defining a plurality of first feature points in the projection area; Simulating a virtual static image of the photographic device, the virtual static image including the projection area and the plurality of first feature points; Adjusting the virtual projection position of the projection device and the virtual screen deformation according to the plurality of first feature points; and Output simulation overview information, in, The simulation overview information includes the adjusted virtual projection position and the adjusted virtual picture deformation.

2. The calibration method according to claim 1, further comprising: According to the simulation profile information, setting the projection position and image deformation of the projection device and setting the photographing position of the photographing device; Acquiring a static image via the photographic device; In the virtual static image, obtaining a plurality of second feature points based on the plurality of first feature points; comparing the plurality of first feature points with the plurality of second feature points to obtain a virtual image difference parameter; and The projection position and the screen deformation of the projection device are adjusted according to the virtual image difference parameter and the static image.

3. The calibration method according to claim 2, wherein obtaining the static image via the photographing device further comprises: In the static image, obtaining a plurality of third feature points based on the plurality of first feature points; and The virtual image difference parameter is compared with the plurality of third feature points to obtain an image difference parameter.

4. The calibration method according to claim 3, wherein adjusting the projection position and the image deformation of the projection device according to the virtual image difference parameter and the static image further comprises: The projection position and the picture deformation of the projection device are adjusted according to the image difference parameter.

5. The calibration method according to claim 1, wherein setting the virtual projection position and the virtual image deformation of the projection device further comprises: Setting the virtual color and virtual brightness of the projection device; Adjusting the virtual color and the virtual brightness of the projection device according to the plurality of first feature points; in, The simulation profile information also includes the adjusted virtual color and virtual brightness information.

6. The calibration method according to claim 1, wherein setting the virtual projection position and the virtual image deformation of the projection device further comprises: Setting multiple virtual projection positions and multiple virtual image deformations of multiple projection devices; in, Adjusting the virtual projection position and the virtual image deformation of the projection device according to the plurality of first feature points further comprises: Adjusting fusion correction information of the plurality of projection devices according to the plurality of first feature points; Wherein, the simulation profile information further includes the adjusted fusion correction information.

7. The calibration method according to claim 1, wherein adjusting the virtual projection position and the virtual image deformation of the projection device according to the plurality of first feature points further comprises: In the virtual static image, obtaining a plurality of second feature points based on the plurality of first feature points; comparing the plurality of first feature points with the plurality of second feature points to obtain a virtual image difference parameter; and The virtual photographing position of the photographing device is adjusted according to the virtual image difference parameter.

8. The calibration method according to claim 7, wherein adjusting the virtual photographing position of the photographing device according to the virtual image difference parameter further comprises: The simulated photography parameters of the photography device are adjusted according to the virtual image difference parameters.

9. The calibration method according to claim 8, further comprising: According to the simulation profile information, setting the projection position and image deformation of the projection device and setting the photographing position of the photographing device; According to the simulated photographic parameters, setting photographic parameters of the photographic device; Acquiring a static image via the photographic device; and The projection position and the screen deformation of the projection device are adjusted according to the virtual image difference parameter and the static image.

10. A calibration method for an immersive projection system, comprising: Setting the virtual projection position and virtual image deformation of the projection device; Setting a virtual photographing position of a photographing device; Obtain projection area and virtual static image through projection simulation program; Defining a plurality of first feature points in the projection area; Adjusting the virtual projection position of the projection device and the virtual screen deformation according to the plurality of first feature points; and Output simulation overview information, in, The simulation overview information includes the adjusted virtual projection position and the adjusted virtual picture deformation.

11. The calibration method according to claim 10, further comprising: According to the simulation profile information, setting the projection position and image deformation of the projection device and setting the photographing position of the photographing device; Acquiring a static image via the photographic device; In the virtual static image, obtaining a plurality of second feature points based on the plurality of first feature points; comparing the plurality of first feature points with the plurality of second feature points to obtain a virtual image difference parameter; and The projection position and the screen deformation of the projection device are adjusted according to the virtual image difference parameter and the static image.

12. The calibration method according to claim 11, wherein obtaining the static image via the photographing device further comprises: In the static image, obtaining a plurality of third feature points based on the plurality of first feature points; and The virtual image difference parameter is compared with the plurality of third feature points to obtain an image difference parameter.

13. The calibration method according to claim 12, wherein adjusting the projection position and the image deformation of the projection device according to the virtual image difference parameter and the static image further comprises: The projection position and the picture deformation of the projection device are adjusted according to the virtual image difference parameter and the image difference parameter.

14. The calibration method according to claim 10, wherein the projection simulation program comprises a projection geometry simulation subroutine, a photographic device simulation subroutine, a splicing simulation subroutine, a color and brightness simulation subroutine, and a fusion simulation subroutine.

15. The calibration method according to claim 10, wherein adjusting the virtual projection position and the virtual image deformation of the projection device according to the plurality of first feature points further comprises: The virtual photographing position of the photographing device is adjusted according to the virtual static image.

16. The calibration method according to claim 15, wherein adjusting the virtual photographing position of the photographing device according to the virtual static image further comprises: The simulated photography parameters of the photography device are adjusted according to the virtual static image.

17. The calibration method according to claim 16, further comprising: According to the simulation profile information, setting the projection position and image deformation of the projection device and setting the photographing position of the photographing device; According to the simulated photographic parameters, setting photographic parameters of the photographic device; Acquiring a static image via the photographic device; and The projection position and the screen deformation of the projection device are adjusted according to the virtual image difference parameter and the static image.

18. An immersive projection system, in conjunction with a projection screen, the immersive projection system comprising: A projection device having a projection position and a picture deformation, and projects a projection area onto the projection screen; and A photographing device having a photographing position and acquiring a static image, wherein the static image includes the projection area, in, The projection position and the screen deformation of the projection device are set according to the static image and the virtual static image.

19. The immersive projection system of claim 18, wherein the virtual static image is obtained via a projection simulation process.

20. The immersive projection system according to claim 19, wherein the projection simulation program comprises a projection geometry simulation subroutine, a photographic device simulation subroutine, a splicing simulation subroutine, a color and brightness simulation subroutine, and a fusion simulation subroutine.

Images