Unreal engine-based multi-projector planar fusion projection correction method and system

Abstract

The application discloses a kind of based on Unreal Engine's multi-projector plane fusion projection correction method and system, comprising: using Unreal Engine division projection plane, the position P1 of each projector's sub-projection area in projection plane is obtained;Using projector, the calibration interface generated by Unreal Engine is projected to sub-projection area, the grid proportion of calibration interface is determined, generates to be calibrated point, after calibration, the position P2 of calibration point in calibration interface in sub-projection area is stored;According to P1 and P2, the position P3 of calibration point in projection plane of sub-projection area is obtained;P3 and P1 are input PFM format generator, and the PFM file of projector is generated according to affine transformation mode;P3 and P1 are input Alpha generator, and according to the coincidence area and fusion strategy calculated by P1 and P3, transparency cover Alpha file is generated;PFM file and Alpha file are input in Unreal Engine and are projected project content in configuration.This application can meet the projection needs in different light environment, and reduce hardware dependence.

Description

Technical Field

[0001] This invention relates to the field of multi-projection seamless fusion technology, and in particular to a multi-projector planar fusion projection correction method and system based on Unreal Engine. Background Technology

[0002] Multi-projector splicing refers to using two or more projectors to project simultaneously to form a large projection. Compared to a single projector, multi-projector splicing can achieve higher resolution, and compared to a display screen of the same size, the cost of multi-projector splicing is much lower.

[0003] Multi-projector splicing can be categorized into two types based on whether there is overlap between the projected images: non-overlapping splicing and overlapping splicing. Overlapping splicing, where there are no gaps between the projected content, is also known as seamless splicing. Non-overlapping splicing typically reduces gaps by physically adjusting the projector's position, while overlapping splicing projects the same content into the overlapping areas and adjusts the brightness and color of these areas to make them indistinguishable to the naked eye, thus achieving the desired projection effect. Most existing multi-projector splicing solutions rely on hardware calibration plugins for the projectors, which are costly and have significant limitations. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention proposes a multi-projector planar fusion projection correction method and system based on Unreal Engine.

[0005] The specific technical solution is as follows:

[0006] A multi-projector planar fusion projection correction method based on Unreal Engine includes the following steps:

[0007] S1: Use Unreal Engine to divide the entire projection plane and obtain several sub-mesh regions. Arrange the projectors and sub-mesh regions one by one to obtain the position P1 and content of the sub-projection region corresponding to each projector in the entire projection plane, and store it.

[0008] S2: Open the calibration project using Unreal Engine, use a projector to project the calibration interface onto the corresponding sub-projection area, determine the grid ratio used for each calibration interface, generate the points to be calibrated and calibrate the sub-projection areas one by one, and after calibration, store the position P2 of the calibration point in each sub-projection area in the calibration interface.

[0009] S3: Based on P1 and P2, obtain the absolute position P3 of the calibration points of all sub-projection regions on the entire projection plane;

[0010] S4: Input P3 and P1 into the PFM format generator to generate the PFM file required for each projector according to the affine transformation method; input P3 and P1 into the Alpha generator to generate the transparency mask Alpha file required for projection according to the overlapping area and fusion strategy calculated from P1 and P3.

[0011] S5: Input the PFM file and the Alpha file into the nDisplay editor of the Unreal Engine for configuration.

[0012] S6: Use the Unreal Engine to project the project content.

[0013] Further, the step S1 is specifically implemented through the following sub-steps:

[0014] (1.1) Use the Unreal Engine to create a partitioning project, divide the projection plane with length L and width W into an m*n grid, obtaining (m + 1)*(n + 1) grid points, Q

[0018] ,

[0017] , , =(x i , y j ), i = 0, 1, 2, …, m, j = 0, 1, 2, …, n; the abscissa x i of the grid points ranges from 0, L / m, 2L / m, …, iL / m, …, L, and the ordinate y j ranges from 0, W / n, 2W / n, …, jW / n, …, W.

[0015] (1.2) Specify that the kth sub-grid area occupies an m k *n k grid, 0 < m k < m, 0 < n k < n, move the position of each sub-grid area by dragging, so that the sub-grid areas cover the entire projection plane and there is an overlapping area between every two adjacent sub-grid areas, that is, reach the specified position; the overlapping area between sub-grid areas is the area that needs to be fused and projected; store the coordinates of all grid points inside and on the edge of all sub-grid areas. <00…072>(1.3) Arrange multiple projectors so that the projection areas of the projectors correspond to the sub-grid areas one by one. The projection area corresponding to each projector is a sub-projection area. The sub-grid area is a part of the entire projection plane in terms of content, and the sub-projection area is a part of the entire projection plane in terms of physical area. Record the position of each sub-projection area in the entire projection plane as P1 and store it.

[0017] Further, the step S2 is specifically implemented through the following sub-steps:

[0018] (2.1) Use Unreal Engine to create a calibration project and open it. Use a projector to project the calibration interface onto the corresponding sub-projection area. Each projector projects its own calibration interface.

[0019] (2.2) Initialize all calibration interfaces, determine the mesh ratio used by each calibration interface, that is, the mesh ratio inside each sub-mesh region, generate a mesh covering the entire sub-projection region with the corresponding ratio for each calibration interface, and generate movable calibration points at the vertices of each mesh.

[0020] (2.3) Manually click and drag the point to be calibrated with the left mouse button to move the position of the point to be calibrated significantly. Then select the point to be calibrated with the right mouse button and use the arrow keys on the keyboard to fine-tune the position of the point to be calibrated. The points to be calibrated in the overlapping areas between the sub-projection areas need to be completely overlapping. After the overlapping areas are calibrated, adjust the grid blocks in the non-overlapping areas to distribute them evenly.

[0021] (2.4) After all the calibration points have been adjusted, the calibration points of each sub-grid area are obtained. The position P2 of the calibration point of each projector's sub-grid area in the calibration interface is output and stored.

[0022] Further, step S3 specifically involves: P2 using the calibration interface as a reference system and P1 using the entire projection plane as a reference system. When calculating the transformation, the offset vector of each sub-grid region reference system relative to the entire projection plane reference system is calculated based on P1. This offset vector is added to the P2 position to obtain the absolute position P3 of the calibration point.

[0023] Furthermore, in step S4, the generation of the PFM file is specifically achieved through the following sub-steps:

[0024] (4.1) The PFM format generator reads the file storing P1 and P3 information, divides each grid into two triangles along the diagonal, one at the upper right corner and one at the lower left corner, and uses these two triangles as the basic unit T of the affine transformation. Based on the P3 at the corresponding position of T1 generated by P1, T3 is generated, resulting in an affine matrix composed of T1 and T3. The affine matrix is ​​calculated for each pair of triangles. For an m1*n1 subgrid region, 2*m1*n1 affine transformation matrices A will be obtained.

[0025] (4.2) Calculate the transformation of the PFM file: In the MPCDI protocol, the PFM format is defined as the position P of each pixel within the raster. o (x o y o The corresponding projection content position P f (x p y p ), P o With affine transformation matrices A and Pf The relationship is as follows:

[0026] P f =P o A

[0027] Expanding on this:

[0028] This allows us to obtain the information stored for each point within the grating in the PFM, and generate the PFM file.

[0029] Furthermore, in step S4, the Alpha file is specifically generated through the following steps:

[0030] The alpha generator reads the file storing P1 and P3 information. Alpha values ​​are integers between 0 and 255, representing brightness values. For the entire projection plane, the brightness of the unprojected areas is adjusted to 0, making these areas appear black during projection. For the projection area, the brightness of non-overlapping areas is set to 255. For the left, right, upper, and lower sub-grid areas that form overlapping areas, a blending strategy is used to adjust the brightness. The specific blending strategy is as follows:

[0031] For the horizontally overlapping region, assuming the x-coordinate range of the horizontally overlapping region is [a, b], and c is a point within this region, the coordinates of point c are normalized to obtain x:

[0032] x = (ca) / (ba)

[0033] Where 0 ≤ x ≤ 1;

[0034] For the left sub-grid region of the overlapping area, the brightness calculation of the overlapping part adopts any one of the following four fusion algorithms:

[0035] Linear fusion algorithm:

[0036] f(x) = x

[0037] Exponential fusion algorithm:

[0038]

[0039] In the formula, α is a variable parameter, and the fusion effect can be adjusted by adjusting the value of α;

[0040] Square root fusion algorithm:

[0041]

[0042] Cosine fusion algorithm:

[0043]

[0044] For the right sub-grid region of the overlapping area, the brightness of the overlapping part is 1-f(x);

[0045] For the overlapping regions in the vertical direction, the horizontal coordinates in the above calculation process are replaced with the corresponding vertical coordinates. The left sub-grid region of the overlapping region is replaced with the upper sub-grid region of the overlapping region, and the right sub-grid region of the overlapping region is replaced with the lower sub-grid region of the overlapping region, thus completing the brightness calculation of the overlapping regions in the vertical direction. After the brightness calculations of the overlapping regions in the horizontal and vertical directions are completed, an Alpha file is generated.

[0046] A multi-projector planar blending projection correction system based on Unreal Engine, employing any of the aforementioned multi-projector planar blending projection correction methods based on Unreal Engine, includes: a switch, a projection system host, a projection system slave, a projector, and a projection plane;

[0047] The switch is an Ethernet switch used to enable network communication between the projection system host and all projection system slaves. The projection system host is responsible for opening the Unreal Engine projection project and corresponding plugins, sending projection information to each projection system slave, and calibrating the sub-projection area of ​​the corresponding projector. The projection system slaves are responsible for calibrating the sub-projection area of ​​the corresponding projector and enabling the receiving end of the Unreal Engine projection plugin for projection. The projector is connected to the corresponding projection system host and projection system slaves and displays the projected content. The projection plane is the plane on which the projection is displayed.

[0048] The beneficial effects of this invention are:

[0049] (1) The present invention can select different blending strategies during projection to meet the projection needs under different lighting and shadow environments. At the same time, the projection area of ​​the present invention is customized during projection and has the characteristic of being able to project onto irregular geometric planes, which can meet special projection requirements (e.g., projection onto a trapezoidal plane).

[0050] (2) The present invention does not require professional calibration personnel during adjustment, and the projector does not need to be placed in a precise position. Users only need to calibrate according to fixed principles to obtain a good projection blending effect, which saves labor costs.

[0051] (3) This invention reduces the hardware dependence of the planar fusion projection system and reduces the projection cost.

[0052] (4) This invention is developed based on the open-source software Unreal Engine. The system logic is easy to understand and easy to expand. Furthermore, it can be further developed into curved surface fusion projection based on the extension of curved surface projection using the MPCDI protocol. Related developers can customize a more powerful projection fusion system based on this invention. Attached Figure Description

[0053] Figure 1 This is a flowchart illustrating the multi-projector planar fusion projection correction method based on Unreal Engine according to the present invention.

[0054] Figure 2 This is a flowchart illustrating step S1 of the present invention.

[0055] Figure 3 This is a flowchart illustrating step S2 of the present invention.

[0056] Figure 4 This is a hardware schematic diagram of the multi-projector planar fusion projection correction system based on Unreal Engine according to the present invention. Detailed Implementation

[0057] The present invention will be described in detail below with reference to the accompanying drawings and preferred embodiments. The objectives and effects of the present invention will become clearer as a result. The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0058] like Figure 1 As shown, the multi-projector planar blending projection correction method based on Unreal Engine specifically includes the following steps:

[0059] S1: Open the partitioned project using Unreal Engine, divide the entire projection plane, obtain the position P1 and content of each sub-projection area on the entire projection plane, and arrange multiple projectors to the corresponding positions of the sub-projection areas. This is achieved through the following sub-steps:

[0060] (1.1) The grid is created using Unreal Engine. The grid is constructed using the UMG component of Unreal Engine, with the UMG component used as grid points (visually small square blocks). These grid points are connected according to a grid pattern (each grid point is only connected to its adjacent grid points on the top, bottom, left, and right sides). The coordinates of the grid points are output to a txt file. The entire projection plane of length L and width W is divided into an m*n grid, for example, a 3*5 grid, resulting in (m+1)*(n+1) grid points. Clearly, the x-coordinate of each grid point is... i The value range is 0, L / m, 2L / m, ..., iL / m, ..., L, and the ordinate is y. j The value range of is 0, W / n, 2W / n, ..., jW / n, ..., W, to obtain the grid point coordinates Q. i,j =(x i ,y j ), i=0,1,2,…,m, j=0,1,2,…,n.

[0061] (1.2) Specify that the k-th sub-grid region occupies mk *n k grid (where 0 < m k < m, 0 < n k < n), each sub-grid area is initially located in the upper left corner of the entire projection plane. Then, the position of each sub-grid area is moved by dragging, so that the sub-grid areas cover the entire projection plane and every two adjacent sub-grid areas contain overlapping grids. Specifically, when in use, the user can adjust the position of the sub-grids and the size of the overlapping area according to actual needs; the overlapping area will be automatically generated in this step, and this overlapping area is the area where the fusion projection is required. Export the coordinates of all grid points within and on the edge of all sub-grid areas to a txt file for storage, and denote it as P1. As Figure 2 shown, in this embodiment, in the projection plane divided into a 3*5 grid, the two generated 3*3 sub-grid areas are initially located in the upper left corner of the entire projection plane. Then, the position of the sub-grid areas is moved by dragging, so that the two 3*3 sub-grid areas are located on the left and right sides of the entire projection plane, that is, they reach the specified positions; the overlapping area of the two sub-grids is the middle 3*1 grid.

[0062] (1.3) Arrange multiple projectors so that the projection area of each projector is assigned a different sub-grid area. The projection area corresponding to each projector is a sub-projection area, that is, the content of the sub-grid area is a mapping of the sub-projection area (the sub-grid area is a part of the entire projection plane in terms of content, and the sub-projection area is a part of the entire projection plane in terms of physical area). At this time, P1 is the position of each sub-projection area in the entire projection plane. The projector only needs to ensure that the projection is in a general orientation. For example, in the above example, the two projectors are located on the left and right sides, and the sub-projection areas have an overlap of not less than one-third.

[0063] S2: Open the calibration project using the Unreal Engine and output it to the projector. The projector projects onto the corresponding sub-projection area, determine the grid ratio to be used, generate calibration points, and calibrate each sub-projection area one by one. After calibration, record the positions of the calibration points of each sub-projection area as P2 and export them to a txt file for storage. Specifically, it is implemented through the following sub-steps:

[0064] (2.1) Use the Unreal Engine to create a calibration project. The calibration project is created using the UMG component of the Unreal Engine. Use the components in UMG as grid points (the visual effect is relatively small square blocks), connect these grid points according to the grid style, and add the function of dragging with the left mouse button and the function of moving with the keyboard arrow keys after selection to the grid points. Export the coordinates of the grid points to a txt file, that is, generate the calibration project. Open the calibration project using the Unreal Engine, and use the projector to project the calibration interface onto the corresponding sub-projection area. Each projector needs to project its own calibration interface.

[0065] (2.2) Initialize all calibration interfaces and determine the mesh ratio used by each calibration interface: Input the mesh ratio within each specified sub-mesh region into the calibration project of the Unreal Engine corresponding to the projector, that is, the k-th sub-mesh region occupies m k *n k The grid is generated for each calibration interface, which generates a grid covering the entire sub-projection area in a corresponding proportion, and generates movable calibration points at the vertices of each grid.

[0066] (2.3) such as Figure 3 As shown, manually click and drag with the left mouse button to move the calibration point's position significantly, then select the calibration point with the right mouse button and use the arrow keys on the keyboard to fine-tune its position, making the calibration point's position more accurate. When calibrating overlapping areas, special attention should be paid to ensuring that the calibration points in the overlapping areas completely overlap, otherwise ghosting will occur in the overlapping areas. After the overlapping areas are calibrated, the mesh blocks in the non-blended projection areas should be adjusted to distribute evenly as much as possible to ensure a good projection effect.

[0067] (2.4) After all the calibration points have been adjusted, output the position P2 of the calibration point of each projector's sub-projection area in the calibration interface to a txt file for storage.

[0068] S3: Based on the position P1 of each sub-projection region in the entire projection plane and the position P2 of the calibration point of each sub-projection region, calculate the absolute position P3 of the calibration point of all sub-projection regions in the entire projection plane.

[0069] In this system, P2 uses the calibration interface itself as the reference system, while P1 uses the entire projection plane as the reference system. When calculating the transformation, it is only necessary to calculate the offset vector of each sub-grid region reference system relative to the entire projection plane reference system based on P1, and add this offset vector to the P2 position to obtain the absolute position P3 of the calibration point.

[0070] S4: Generates the PFM file needed for each projector, and the transparency masking alpha file required for projection.

[0071] The absolute position P3 of the calibration point and the position P1 of each sub-projection region in the entire projection plane are fed into the PFM format generator to generate the PFM file required by each projector according to the affine transformation method. This is specifically achieved through the following sub-steps:

[0072] (4.1) The PFM format generator reads the txt file storing P1 and P3 information, and divides each grid into two triangles, one in the upper right corner and one in the lower left corner, by drawing a diagonal line from the upper left corner to the lower right corner. These two triangles are used as the basic unit T of the affine transformation. Based on the P3 corresponding to the T1 generated by P1, T3 is generated, and the affine matrix composed of T1 and T3 is obtained. The affine matrix is ​​calculated for each pair of triangles. For an m1*n1 subgrid region, 2*m1*n1 affine transformation matrices A are obtained, which are used to calculate the transformation of the PFM file in the future.

[0073] (4.2) Calculate the transformation of the PFM file. PFM format is a simple image information storage format, mainly consisting of three parts: format, resolution, and raster information. In the MPCDI protocol, PFM format is defined as the position P of each pixel within the raster. o (x o y o The corresponding projection content position P f (x p y p The information stored for each point in the grating stored in the PFM can be obtained according to the following formula.

[0074] P f =P o A

[0075] Expanding on this:

[0076] Since the projected content does not exceed the projectable area, there will be a projected area without any projected content. The position of this part of the projected content will use the default value, and the color of this part of the projected area will be adjusted to full black using an Alpha file later.

[0077] Simultaneously, the absolute position P3 of the calibration point and the position P1 of each sub-projection region in the entire projection plane are fed into the Alpha generator. Based on the overlapping area calculated from P1 and P3 and the specified blending strategy, the required transparency masking Alpha file for the projection is generated. The specific operation is as follows:

[0078] The alpha file is in PNG format, and each pixel has only one integer value between 0 and 255 as its alpha value, representing the brightness of that pixel; the larger the value, the brighter the pixel. For the entire projection plane, the brightness of the unprojected area is first adjusted to 0, making this area completely black during projection. For the projected area, the brightness of non-overlapping areas is 255. For the left, right, top, and bottom sub-grid areas that form overlapping areas, a blending strategy is used to adjust the brightness. The general principle is that the brightness decreases from the center of the sub-grid area towards the edge. The specific blending strategy is as follows:

[0079] For the horizontally overlapping region, assuming the x-coordinate range of the horizontally overlapping region is [a, b], and c is a point within this region, to obtain the alpha value of point c, we first normalize the coordinates of point c to obtain x:

[0080] x = (ca) / (ba)

[0081] Where 0 ≤ x ≤ 1.

[0082] For the left sub-grid region of the overlapping area, the brightness calculation of the blended region of the projector uses any one of the following four blending algorithms:

[0083] Linear fusion algorithm:

[0084] f(x) = x

[0085] Exponential fusion algorithm:

[0086]

[0087] In the formula, α is a variable parameter, and the fusion effect can be adjusted by adjusting the value of α;

[0088] Square root fusion algorithm:

[0089]

[0090] Cosine fusion algorithm:

[0091]

[0092] For the right sub-grid region of the overlapping area, the luminance meter of the projector's blended region is 1-f(x).

[0093] For the vertically fused region, assuming the ordinate range of the vertically overlapping region is [g, h], and l is a point within this region, the coordinates of point l are normalized to obtain y:

[0094] y = (lg) / (hg)

[0095] Where 0 ≤ y ≤ 1;

[0096] For the upper sub-grid region of the overlapping area, the brightness calculation of the overlapping part adopts any one of the following four fusion algorithms:

[0097] Linear fusion algorithm:

[0098] f(y) = y

[0099] Exponential fusion algorithm:

[0100]

[0101] Square root fusion algorithm:

[0102]

[0103] Cosine fusion algorithm:

[0104]

[0105] For the sub-grid regions of the overlapping area, the brightness of the overlapping part is 1-f(y).

[0106] Once the brightness calculations for the overlapping areas in both the horizontal and vertical directions are complete, an Alpha file is generated. In practical applications, the best-performing fusion algorithm can be selected based on the hardware and lighting conditions.

[0107] S5: Import the PFM file and Alpha file into the Unreal Engine's nDisplay editor for configuration; this is achieved through the following sub-steps:

[0108] (5.1) Open the Unreal Engine project to be projected, open the nDisplay 3D configuration editor, and modify the three main parts of components, clusters, and output mapping to make them compatible with the hardware and network connection used for projection.

[0109] (5.2) Drag the configured nDisplay asset into the project to be projected. The projection cone and projection image will be generated automatically. Adjust the nDisplay asset until the projection requirements are met.

[0110] S6: Projecting project content using Unreal Engine. This is achieved through the following sub-steps:

[0111] (6.1)Reference Figure 4 Enable the SwitchboardListener plugin in all host machines connected to the projection system;

[0112] (6.2) Open Switchboard in Unreal Engine and specify the nDisplay profile. Click Connect to start projection.

[0113] In summary, this invention first leverages the support of the nDisplay module in Unreal Engine for the MPCDI projection protocol, and then creates a calibration project using the UMG component in Unreal Engine. Based on this, and using the definitions of PFM files and affine transformation rules in P1, P3, and MPCDI, a PFM file for projection mapping is directly generated. Similarly, based on the definitions of Alpha files and blending strategies in P1, P3, and MPCDI, an Alpha file for adjusting brightness is directly generated, thereby achieving multi-projector planar blending projection correction based on Unreal Engine.

[0114] like Figure 4 As shown, the multi-projector planar blending projection correction system based on Unreal Engine includes: a switch, a projection system host, a projection system slave, a projector, and a projection plane.

[0115] A switch refers to an Ethernet switch, which is used to enable network communication between the host and all slave devices of a projection system.

[0116] The projection system host is responsible for opening the projection project and corresponding plugins in Unreal Engine, sending projection information to each projection system slave, and calibrating the sub-projection area of ​​the corresponding projector.

[0117] The slave unit of the projection system is responsible for calibrating the sub-projection area of ​​the corresponding projector and enabling the receiver of the Unreal Engine projection plugin to project.

[0118] The projector is connected to the corresponding projection system host and projection system slave, and displays the projected content. It is preferable that all projectors are of the same model to improve the projection effect.

[0119] The projection plane is the plane on which the projection is displayed; it can be a screen or other projectable plane.

[0120] In practical applications, after obtaining the PFM and Alpha files, open the Unreal Engine project to be projected on the projection system host, open the nDisplay editor, configure the network addresses of the computers participating in the projection on the local area network, the position of the projection screen corresponding to each projector in the Unreal Engine project, and import the corresponding PFM and Alpha files for each projection screen. After configuration, save the contents of the nDisplay editor as an asset file. On the projection system host, open the SwitchBoard in the project to be projected, import the asset file saved in the nDisplay editor, and then open the SwitchBoardListener on the projection system host and each projection system slave to ensure that each computer receives the projection information and can forward it to the corresponding projector through the connection cable. Finally, click "Connect All" in the SwitchBoard on the projection system host to start the projection.

[0121] This invention, based on Unreal Engine and the Multi-Projector Collaborative Projection Protocol (MPCDI), develops a planar blending projection correction method and projection system for multiple projectors. Compared to currently available custom-designed projection blending systems, this invention eliminates the need for additional measuring instruments and hardware, such as infrared rangefinders, custom adjustment brackets, and custom single-source input / multi-source output devices. It requires only one or more hosts with NVIDIA graphics cards running Windows and multiple projectors. Blended projection can be achieved at the desired planar location through manual calibration, and the projection blending quality of this invention is comparable to that of custom planar projection blending systems. Therefore, this invention significantly reduces the hardware dependence of planar blending projection systems and lowers projection costs.

[0122] It will be understood by those skilled in the art that the above descriptions are merely preferred examples of the invention and are not intended to limit the invention. Although the invention has been described in detail with reference to the foregoing examples, those skilled in the art can still modify the technical solutions described in the foregoing examples or make equivalent substitutions for some of the technical features. All modifications and equivalent substitutions made within the spirit and principles of the invention should be included within the scope of protection of the invention.

Claims

1. A multi-projector planar fusion projection correction method based on Unreal Engine, characterized in that, Includes the following steps: S1: Use Unreal Engine to divide the entire projection plane and obtain several sub-mesh regions. Arrange the projectors and sub-mesh regions one by one to obtain the position P1 and content of the sub-projection region corresponding to each projector in the entire projection plane, and store it. S2: Open the calibration project using Unreal Engine, use a projector to project the calibration interface onto the corresponding sub-projection area, determine the grid ratio used for each calibration interface, generate the points to be calibrated and calibrate the sub-projection areas one by one, and after calibration, store the position P2 of the calibration point in each sub-projection area in the calibration interface. S3: Based on P1 and P2, obtain the absolute position P3 of the calibration points of all sub-projection regions on the entire projection plane; S4: Input P3 and P1 into the PFM format generator to generate the PFM file required by each projector according to the affine transformation method; input P3 and P1 into the Alpha generator to generate the transparency masking Alpha file required for projection according to the overlapping area and blending strategy calculated by P1 and P3. S5: Input the PFM file and Alpha file into the Unreal Engine's nDisplay editor for configuration; S6: Project project content using Unreal Engine.

2. The multi-projector planar fusion projection correction method based on Unreal Engine according to claim 1, characterized in that, Step S1 is specifically implemented through the following sub-steps: (1.1) Using Unreal Engine, the projected plane with length L and width W is divided into an m*n grid, resulting in (m+1)*(n+1) grid points, Q i,j =(x i ,y j ), i = 0, 1, 2, ..., m, j = 0, 1, 2, ..., n; x-coordinate of grid point i The value range is 0, L / m, 2L / m, ..., iL / m, ..., L, and the ordinate is y. j The value range is 0, W / n, 2W / n, ..., jW / n, ..., W; (1.2) Specify that the k-th sub-grid area occupies an m k *n k grid, 0 < m k < m, 0 < n k < n. Move the position of each sub-grid area by dragging so that the sub-grid areas cover the entire projection plane and there is an overlapping area between every two adjacent sub-grid areas, that is, reach the specified position; the overlapping area between sub-grid areas is the area that needs to be fused for projection; store the coordinates of all grid points inside and on the edge of all sub-grid areas; (1.3) Arrange multiple projectors so that the projection area of ​​the projector corresponds one-to-one with the sub-grid area. The projection area corresponding to each projector is a sub-projection area. The sub-grid area is a part of the entire projection plane in terms of content. The sub-projection area is a part of the entire projection plane in terms of physical area. Record the position of each sub-projection area in the entire projection plane as P1 and store it.

3. The multi-projector planar fusion projection correction method based on Unreal Engine according to claim 1, characterized in that, Step S2 is specifically implemented through the following sub-steps: (2.1) Use Unreal Engine to create a calibration project and open it. Use a projector to project the calibration interface onto the corresponding sub-projection area. Each projector projects its own calibration interface. (2.2) Initialize all calibration interfaces, determine the mesh ratio used by each calibration interface, that is, the mesh ratio inside each sub-mesh region, generate a mesh covering the entire sub-projection region with the corresponding ratio for each calibration interface, and generate movable calibration points at the vertices of each mesh. (2.3) Manually click and drag the point to be calibrated with the left mouse button to move the position of the point to be calibrated significantly. Then select the point to be calibrated with the right mouse button and use the arrow keys on the keyboard to fine-tune the position of the point to be calibrated. The points to be calibrated in the overlapping areas between the sub-projection areas need to be completely overlapping. After the overlapping areas are calibrated, adjust the grid blocks in the non-overlapping areas to distribute them evenly. (2.4) After all the calibration points have been adjusted, the calibration points of each sub-grid area are obtained. The position P2 of the calibration point of each projector's sub-grid area in the calibration interface is output and stored.

4. The multi-projector planar fusion projection correction method based on Unreal Engine according to claim 1, characterized in that, The specific steps of step S3 are as follows: P2 takes the calibration interface as the reference system, and P1 takes the entire projection plane as the reference system. When calculating the transformation, the offset vector of the reference system of each sub-grid region relative to the reference system of the entire projection plane is calculated according to P1. This offset vector is added to the position of P2 to obtain the absolute position P3 of the calibration point.

5. The multi-projector planar fusion projection correction method based on Unreal Engine according to claim 1, characterized in that, In step S4, the generation of the PFM file is specifically achieved through the following sub-steps: (4.1) The PFM format generator reads the file storing P1 and P3 information, divides each grid into two triangles along the diagonal, one at the upper right corner and one at the lower left corner, and uses these two triangles as the basic unit T of the affine transformation. Based on the P3 at the corresponding position of T1 generated by P1, T3 is generated, resulting in an affine matrix composed of T1 and T3. The affine matrix is ​​calculated for each pair of triangles. For an m1*n1 subgrid region, 2*m1*n1 affine transformation matrices A will be obtained. (4.2) Calculate the transformation of the PFM file: In the MPCDI protocol, the PFM format is defined as the position P of each pixel within the raster. o (x o ,y o The corresponding projection content position P f (x p ,y p ), P o With affine transformation matrices A and P f The relationship is as follows: P f =P o A Expanding on this: This allows us to obtain the information stored for each point within the grating in the PFM, and generate the PFM file.

6. The multi-projector planar fusion projection correction method based on Unreal Engine according to claim 1, characterized in that, In step S4, the Alpha file is generated through the following steps: The alpha generator reads the file storing P1 and P3 information. Alpha values ​​are integers between 0 and 255, representing brightness values. For the entire projection plane, the brightness of the unprojected areas is adjusted to 0, making these areas appear black during projection. For the projection area, the brightness of non-overlapping areas is set to 255. For the left, right, upper, and lower sub-grid areas that form overlapping areas, a blending strategy is used to adjust the brightness. The specific blending strategy is as follows: For the horizontally overlapping region, assuming the x-coordinate range of the horizontally overlapping region is [a, b], and c is a point within this region, the coordinates of point c are normalized to obtain x: x = (ca) / (ba) Where 0 ≤ x ≤ 1; For the left sub-grid region of the overlapping area, the brightness calculation of the overlapping part adopts any one of the following four fusion algorithms: Linear fusion algorithm: f(x) = x Exponential fusion algorithm: In the formula, α is a variable parameter, and the fusion effect can be adjusted by adjusting the value of α; Square root fusion algorithm: Cosine fusion algorithm: For the right sub-grid region of the overlapping area, the brightness of the overlapping part is 1(x); For the overlapping regions in the vertical direction, the horizontal coordinates in the above calculation process are replaced with the corresponding vertical coordinates. The left sub-grid region of the overlapping region is replaced with the upper sub-grid region of the overlapping region, and the right sub-grid region of the overlapping region is replaced with the lower sub-grid region of the overlapping region, thus completing the brightness calculation of the overlapping regions in the vertical direction. After the brightness calculations of the overlapping regions in the horizontal and vertical directions are completed, an Alpha file is generated.

7. A multi-projector planar blending projection correction system based on Unreal Engine, employing the multi-projector planar blending projection correction method based on Unreal Engine as described in any one of claims 1-6, characterized in that, include: Switch, projection system host, projection system slave, projector, projection plane; The switch is an Ethernet switch used to enable network communication between the projection system host and all projection system slaves. The projection system host is responsible for opening the Unreal Engine projection project and corresponding plugins, sending projection information to each projection system slave, and calibrating the sub-projection area of ​​the corresponding projector. The projection system slaves are responsible for calibrating the sub-projection area of ​​the corresponding projector and enabling the receiving end of the Unreal Engine projection plugin for projection. The projector is connected to the corresponding projection system host and projection system slaves and displays the projected content. The projection plane is the plane on which the projection is displayed.

Images