A naked-eye stereoscopic video reverse mapping generation method for a special-shaped screen

Abstract

This invention provides a method for generating naked-eye stereoscopic video using reverse mapping for irregularly shaped screens, relating to the technical fields of irregularly shaped display screen adaptation and Web graphics rendering. The method includes: modeling the irregularly shaped screen as a set of display surfaces; establishing an off-axis projection camera from the viewpoint to each display surface; calculating the local basis vectors and view frustum boundaries of the screen surface from the vertices of the display surface and the viewpoint; assembling the off-axis projection matrix and the camera pose matrix to obtain the reverse perspective mapping relationship of the display surface; and projecting the source material planar texture onto each display surface according to the reverse perspective mapping relationship. This invention can generate unfolded images or videos that can be directly played on irregularly shaped screens, thereby improving the automation level, geometric consistency, preview export consistency, and video export stability of content generation under different irregularly shaped screen forms.

Description

Technical Field

[0001] This invention relates to the technical field of irregularly shaped display screen adaptation and Web graphics rendering, and in particular to a method for generating naked-eye stereoscopic video by reverse mapping for irregularly shaped screens. Background Technology

[0002] Naked-eye 3D irregular-shaped screens are typically composed of several flat screens, angled screens, cylindrical screens, curved screens, or irregularly spliced ​​screens. These screens are not conventional rectangular flat displays; when viewed from a specific observation point, the image on the screen needs to undergo perspective distortion beforehand to visually create a prominent sense of three-dimensional space or a perspective relationship that matches the actual screen structure.

[0003] The existing content production methods mainly include the following categories: The first category is to manually stretch, crop, and splice materials face by face in 2D design software or video post-production software; the second category is to pre-set templates for fixed screen shapes, such as those that only adapt to tri-fold screens, four-sided W-shaped screens, or single cube corner screens; the third category is to directly use a regular perspective camera to render 3D scenes and then take screenshots, and then split the screenshots face by face; the fourth category is to rely on the playback end or hardware end to perform simple scaling, cropping, or matrix transformation on the input video.

[0004] The above solutions can achieve certain results with fixed screens and fixed materials, but when the screen shape, screen size, screen angle, viewing distance, viewing height, or output resolution changes, manual recalibration is usually required, and the preview results are prone to inconsistencies with the final exported results. Especially when converting ordinary videos into naked-eye stereoscopic videos adapted for irregularly shaped screens, it is also necessary to solve programmatic issues such as frame-by-frame video decoding, frame rate consistency, encoder compatibility, face-by-face rendering, and multi-face splicing output.

[0005] Specifically, the existing technology has the following drawbacks:

[0006] (1) Screen model solidification: Existing templates usually only correspond to a certain screen shape. They cannot use a unified data structure to describe any number, angle, size or surface discretization of irregular screen surfaces, resulting in limited adaptability.

[0007] (2) Disconnect between viewpoint calculation and image distortion: Existing solutions often process each screen surface with two-dimensional stretching or simple perspective transformation, without constructing an off-axis projection matrix based on the spatial relationship between the best observation point, the three-dimensional vertices of the screen surface and the material plane. This can easily lead to edge misalignment, screen crosstalk, depth perception deviation, or distortion failure after the viewing point changes.

[0008] (3) Lack of geometric boundary calculation for material adaptation: The aspect ratio of ordinary pictures or videos is often different from the visible projection range of irregular screens at the best viewpoint. If only manual scaling or uniform scaling is used, the image will not be fully covered, over-cropped, or the center will be offset.

[0009] (4) Inconsistent preview and export logic: Some tools use one rendering logic during preview and another clipping or stretching logic during export, resulting in inconsistent results when adjusting parameters with the final output file.

[0010] (5) Insufficient stability of video export: Converting ordinary video to video for irregular screen requires frame-by-frame seeking, texture update, face-by-face rendering, encoding and encapsulation. If source frame rate estimation, encoder capability detection, last frame seek timeout and timestamp alignment are not processed, jitter, freezing, export failure or poor player compatibility may occur. Summary of the Invention

[0011] To address the shortcomings of existing technologies, this invention proposes a method for generating naked-eye stereoscopic video by reverse mapping for irregularly shaped screens. Specifically, it is a method that automatically converts ordinary images or videos into naked-eye stereoscopic unfolding materials suitable for display on irregularly shaped screens based on the spatial geometry of the irregularly shaped screen, the optimal viewing point, and the off-axis projection matrix.

[0012] The technical problem to be solved by this invention is to provide a reproducible software method that enables a computer to automatically calculate the reverse perspective mapping relationship of each screen surface based on the three-dimensional geometric data of the irregularly shaped screen, the optimal observation point parameters, and ordinary image or video materials, and generate unfolded images or videos that can be directly played on the irregularly shaped screen surface by surface, thereby improving the automation level, geometric consistency, preview and export consistency, and video export stability of content generation under different irregularly shaped screen forms.

[0013] In a first aspect, the present invention provides a method for generating naked-eye stereoscopic video inverse mapping for irregularly shaped screens, comprising the following steps:

[0014] S1. Define the following parameters: source material parameters, vertices of each display surface of the irregularly shaped screen, viewpoint coordinates, virtual material plane parameters, and output layout parameters;

[0015] S2. Model the irregularly shaped screen as a set of display surfaces, each of which includes its spatial vertex coordinates and in-plane orientation; set the observation point at the center of the overall width of the irregularly shaped screen.

[0016] S3. Attach the source material to the irregularly shaped screen. By deforming the source material, make it fill all visible areas of all visible display surfaces from the viewing point angle to obtain the material's planar texture.

[0017] S4. Establish an off-axis projection camera from the observation point to each display surface. Calculate the local basis vectors of the screen surface and the view frustum boundary from the vertex of the display surface and the observation point. Then assemble the off-axis projection matrix and the camera pose matrix to obtain the reverse perspective mapping relationship of the display surface.

[0018] Perform perspective deformation on the source material's planar texture according to the inverse perspective mapping relationship;

[0019] S5. Perform face-by-face rendering to generate a single-face unfolded diagram;

[0020] S6. Perform timing and encoding parameter pre-analysis on the video source material to generate a set of alternative encoding parameters; establish media timing mapping relationship according to frame index to realize random addressing decoding of video; update the source material attachment and execute step S5 face-by-face rendering, and generate the output file after encoding;

[0021] S7, Preview, Debugging, and Exception Handling.

[0022] In a second aspect, the present invention provides a computer device including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method described in the first aspect.

[0023] Thirdly, the present invention provides a computer program product that, when executed by a processor, implements the steps of the method described in the first aspect.

[0024] This invention provides a reproducible software method that enables a computer to automatically calculate the reverse perspective mapping relationship of each screen surface based on the three-dimensional geometric data of the irregularly shaped screen, the optimal observation point parameters, and ordinary image or video materials, and generate unfolded images or videos that can be directly played on the irregularly shaped screen surface by surface, thereby improving the automation level, geometric consistency, preview and export consistency, and video export stability of content generation under different irregularly shaped screen forms. Attached Figure Description

[0025] Figure 1 This is a flowchart of a method for generating naked-eye stereoscopic video in reverse mapping for irregularly shaped screens, as disclosed in this invention.

[0026] Figure 2 This is a geometric model and optimal viewpoint diagram of a four-sided W-shaped screen. Where G: irregular screen geometry, E: observation point, h: observation point height, S1, S2, S3, S4: display surfaces, p a p b p c p d : Corner point, d: Horizontal distance from the viewer to the front of the screen.

[0027] Figure 3 This is for the process of rendering, unfolding, and exporting face by face. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Steps S1, S2… in the described embodiments of the present invention do not limit the scope of execution of the present invention; the various models, simulation environments, and software described in the present invention are not considered as the only limiting methods of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0029] In this invention, computer device / equipment / system refers to a related entity applied to a computer, such as hardware, a combination of hardware and software, software, or software in execution. More specifically, for example, software includes, but is not limited to, a process running on a processor, a processor, an object, executable software, an execution thread, a program, and / or a computer. Furthermore, an application program or script running on a server, and the server itself, can also be software. One or more software programs may be in an execution process and / or thread, and the software may be localized on one computer and / or distributed across two or more computers, and may be run on various computer-readable media.

[0030] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other.

[0031] In a first aspect, the present invention provides an embodiment of a naked-eye stereoscopic video reverse mapping generation method for irregularly shaped screens. The irregularly shaped screen is abstracted as a spatial geometric set composed of several display surfaces, the viewer's eye point is abstracted as a spatial observation point, and ordinary images or videos are used as texture data located on a virtual material plane. For each display surface, an off-axis projection matrix is ​​calculated based on the observation point and the three-dimensional vertices of the display surface, so that when the rendered single-sided image is played on a real screen and viewed by the observation point, it is aligned with the expected material plane or three-dimensional visual content. Finally, the images of each display surface are arranged into horizontal rows, vertical rows, image sets, or multiple files for output according to the requirements of the playback device.

[0032] like Figure 1 As shown, the specific process can be as follows:

[0033] S1. Input data and parameter definition;

[0034] Source material M: Ordinary image file or ordinary video file, containing metadata such as image width and height, video duration, video frame rate or an estimable frame rate.

[0035] The geometry G of the irregular screen consists of N display surfaces S_i. Each display surface can be represented by quadrilateral vertices, triangular mesh vertices, or polygonal patches after surface discretization. For quadrilateral display surfaces, at least the lower left point p_a, the lower right point p_b, the upper left point p_c, and the upper right point p_d are included.

[0036] Observation point E: Determined by parameters such as viewing distance, viewing height, and horizontal offset, it represents the eye coordinates at the optimal viewing position.

[0037] Virtual material plane P: Defined by parameters such as depth, offsetX / offsetY, scale, horizontal stretch (stretchX), vertical stretch (stretchY), and background color (backgroundColor).

[0038] Output Layout L: Defines the arrangement of images on each display surface in the final output canvas, the resolution of each surface, the total output resolution, and the video encoding specifications.

[0039] S2, Geometric modeling of irregularly shaped screens;

[0040] The system models the irregularly shaped screen as a set of display surfaces G={S_1,S_2,...,S_N}. Each display surface stores its spatial vertex coordinates and in-plane orientation. For the four-sided W-shaped screen embodiment in the current project, all four screens are squares with a side length of s, the angle between adjacent screens is 90 degrees, the bottom edge is flush with the ground, and the vertices of the polygonal lines are constructed as follows:

[0041] Let step = s / sqrt(2), the five bottom edge breakpoints are (-2step, 0, 0), (-step, 0, -step), (0, 0, 0), (step, 0, -step), and (2step, 0, 0). The i-th screen surface is formed by extending s vertically from the i-th breakpoint and the (i+1)-th breakpoint to obtain four corner points. The vertical half-height of the screen is s / 2, and the ground height is -s / 2.

[0042] The coordinates of observation point E are determined in the world coordinate system; these coordinates use the same coordinate system as the vertex of the display surface to ensure consistency in subsequent projection calculations. In the four-sided W-shaped screen embodiment, in E = (0, -s / 2 + h, d), h is the viewer's eye height (height from the ground), and d is the horizontal distance from the viewer to the front of the screen.

[0043] Spatial Coordinate System and Orientation Setting: To ensure that the vertices of each display surface, the observation point, and the material plane can be calculated within the same mathematical framework, the system establishes a unified world coordinate system. Origin O: For irregularly shaped screens with spliced ​​polygons, the origin is taken as the geometric center point (0, 0, 0) of the bottom edge of the screen polygon; for single-sided or curved screens, the origin is taken as the midpoint of the bottom edge of the main display surface or the center of the screen's bezel. The X-axis is positive from left to right along the direction of the main polygon at the bottom edge of the screen; the Y-axis is positive vertically upward along the screen surface, with the ground reference plane typically taken as y = -s / 2; the Z-axis extends positive from the screen plane towards the viewer's side, satisfying the right-handed coordinate system.

[0044] For any irregularly shaped screen, the aforementioned fixed W-shaped point array can be replaced by a set of facets formed by screen calibration files, CAD data, manually entered parameters, or on-site measurement data. If the screen is curved, it can first be discretized into multiple small planar facets according to curvature and error threshold, and then subsequent projection calculations can be performed on each facet.

[0045] S3. Determining the optimal observation point and material plane;

[0046] like Figure 2 As shown, the observation point E is generated based on the screen size and viewing parameters. In the four-sided W-shaped screen embodiment, the observation point is E=(0, -s / 2+h, d), where h is the viewer's eye height and d is the viewing distance.

[0047] The source material plane P is used to hold ordinary images or video textures. The system attaches the source material as a texture to the source material plane and constructs the basic plane size based on the aspect ratio of the source material: the basic width is 6×aspect×scale, the basic height is 6×scale, and then the stretchX, stretchY, offsetX, offsetY and depth parameters are superimposed.

[0048] The source material M refers to the ordinary image or video file to be converted, which is the original visual content that the user ultimately wants to play on the irregularly shaped screen. The virtual material plane P is a rectangular carrier surface located in three-dimensional space, on which the source material is attached as a texture. Each display surface S_i is the actual position of the physical screen in three-dimensional space; the viewer at the observation point E views the image through each screen surface. The system establishes an off-axis projection camera from E to each screen surface, projecting the texture of the material plane onto the unfolded image of each screen surface according to the reverse perspective relationship. The material plane serves as a unified "content source," and the unfolded image of each screen surface is an image slice that should be displayed on each physical screen surface after observing the material plane from point E.

[0049] The parameters in the basic planar dimensions are as follows: aspect is the aspect ratio of the source material; scale is the overall scaling multiplier, controlling the display size of the material in virtual space; stretchX / stretchY are the horizontal / vertical non-uniform stretching multipliers, ensuring the material fills the entire visible screen area at the optimal viewing point; offsetX / offsetY are the horizontal / vertical offsets of the material plane's center, used to correct image off-center; depth is the depth position of the material plane along the Z-axis, affecting the degree of perspective distortion. The constant 6 in the formula determines the default magnitude of the material plane. Superimposing the above parameters yields the final position and size of the plane in 3D space.

[0050] To reduce manual parameter tuning, a one-click adaptation calculation can be performed: Rays are emitted from the observation point E to the corners of all display surfaces. The intersection points of each ray with the material plane P are calculated, and the minimum x, maximum x, minimum y, and maximum y of the intersection point set are counted to obtain the projection boundary width, height, and center point. Based on this, the system updates offsetX, offsetY, stretchX, and stretchY to ensure that ordinary materials cover the entire visible range of the target screen at the optimal observation point. If the ray is parallel to the material plane, the intersection point is located in the opposite direction of the observation point, or the calculation result is a non-finite value, the original parameters are maintained and the system enters the error message process.

[0051] S4. Calculation of the off-axis projection matrix of the display surface;

[0052] This step performs reverse perspective mapping. Conventional perspective imaging projects a 3D scene onto an image plane; this invention performs the reverse operation—given the position of the physical screen in 3D space and the observation point E, it calculates what kind of perspective distortion image should be displayed on the material plane so that the viewer at point E sees a visual effect consistent with the content of the material plane. The off-axis projection matrix is ​​the mathematical expression of this reverse mapping, eliminating the need to manually derive 2D distortion parameters for each irregularly shaped screen.

[0053] Specifically, for each display surface S_i and its corresponding observation point E in the display surface set G, the calculation path is as follows: the local basis vectors of the screen surface and the asymmetric view cone boundary are obtained from the three-dimensional vertices of the screen surface and the observation point, and then the off-axis projection matrix and the camera pose matrix are assembled to obtain the reverse perspective mapping relationship of the display surface.

[0054] For each display surface S_i, calculate the off-axis projection camera using the three non-collinear vertices p_a, p_b, and p_c of that surface and the observation point E. The specific steps are as follows:

[0055] (1) Calculate the horizontal basis vector of the screen v_r=normalize(p_b-p_a), the vertical basis vector of the screen v_u=normalize(p_c-p_a), and the screen normal vector v_n=normalize(cross(v_r,v_u)). If the direction of the normal vector is away from the observation point, then reverse v_n.

[0056] (2) Calculate the vectors from the observation point to the three vertices of the screen: v_a = p_a - E, v_b = p_b - E, v_c = p_c - E.

[0057] (3) Calculate the directed distance d_s = -dot(v_a,v_n) from the observation point to the screen. If d_s is less than the preset near-zero threshold, the camera on the display surface is deemed unusable, the surface is skipped, or a geometric parameter abnormality is indicated.

[0058] (4) Calculate the asymmetric view cone boundary based on the near clipping plane: l=dot(v_r,v_a)×near / d_s, r=dot(v_r,v_b)×near / d_s, b=dot(v_u,v_a)×near / d_s, t=dot(v_u,v_c)×near / d_s.

[0059] Among them, the near clipping plane is the effective imaging plane closest to the observation point.

[0060] (5) Construct a perspective projection matrix using l, r, t, b, near, far, and form a camera pose matrix using v_r, v_u, v_n, so that the camera is located at the observation point E and facing the current display surface.

[0061] With the off-axis projection matrix described above, the system does not need to manually derive the two-dimensional deformation parameters for each irregularly shaped screen. Instead, it can directly obtain the reverse perspective mapping of the display surface from the screen space vertices and the observation point.

[0062] In the above calculations:

[0063] `normalize(v)` divides vector `v` by its magnitude to obtain a unit vector; `dot(a, b)` represents the dot product of vectors `a` and `b`; `cross(a, b)` represents the cross product of vectors. These operations can also be equivalently implemented using conventional trigonometric functions and vector component formulas.

[0064] l (left), r (right), b (bottom), and t (top) represent the left, right, bottom, and top boundaries of the asymmetric perspective frustum with the viewpoint E as its vertex and the current screen as its projection target, respectively; near is the distance to the near clipping plane, and far is the distance to the far clipping plane; v_r is the camera's rightward direction, v_u is the camera's upward direction, and v_n is the camera's forward direction. These three elements form the camera pose matrix, which positions the virtual camera at E and faces the current display surface.

[0065] S5, face-by-face rendering and unfolded diagram generation;

[0066] like Figure 3 As shown, an off-screen renderer and final output canvas are created. For image footage, the image is loaded as a texture; for video footage, the current video frame is drawn onto an intermediate canvas and converted into a dynamic texture. A 3D scene containing only the footage planes is constructed.

[0067] In a 3D scene containing only the source material plane, there is only a textured rectangular plane (i.e., the virtual source material plane P), which does not contain any screen-based solid models. The spatial position of this plane is determined by depth, offsetX, offsetY, scale, stretchX, and stretchY, and the source material texture is attached to the surface of this plane.

[0068] Then, the following process is executed sequentially for each display surface:

[0069] (1) Clear the single-sided rendering area and set the background color.

[0070] (2) Call the parameters of the off-axis projection camera corresponding to the display surface.

[0071] (3) Render the material plane scene to obtain the reverse perspective image that should be displayed on the display surface at the best viewing point. Specifically, step S4 obtains the off-axis projection matrix, which defines the projector parameters, including the viewpoint, orientation, and frustum range of the virtual camera; using these projector parameters, rasterize the material plane scene.

[0072] (4) Write the single-sided rendering result into a specified area in the output layout L. For example, in the four-sided W-shaped screen embodiment, four 1272×1272 single-sided images are horizontally spliced ​​into an unfolded canvas of 5088×1272.

[0073] The rendering result is the reverse perspective image that the display surface should show at the optimal viewing point E. The reverse perspective mapping is reflected in the projection matrix—the renderer uses the off-axis projection matrix to deform the material plane and draw it onto the frame buffer, and the output result is a single-sided unfolded image after reverse perspective correction.

[0074] The preview and export modules use the same set of display surface geometry, the same observation point, and the same off-axis projection calculations to avoid discrepancies between the preview and export results due to different algorithm branches.

[0075] The role of preview and export in this step: When the user adjusts parameters in the preview interface, the system executes the above face-by-face rendering process in real time to generate preview frames; after the user confirms the parameters, the export module generates the final PNG image or MP4 video frame with the exact same display surface geometry G, observation point E, material plane P and off-axis projection calculation link, avoiding deviations between preview and export due to different algorithm branches.

[0076] S6. Video frame-by-frame export and encoding / packaging;

[0077] The technical path for this step includes: reading video metadata and estimating the frame rate → constructing candidate encoder configurations → calculating time points by frame number and performing seek decoding on video elements → updating textures and performing step S5 (face-by-face rendering) → encapsulating the expanded canvas into a VideoFrame and sending it to the VideoEncoder → generating the output file through an MP4 multiplexer after all frames are encoded. Specifically:

[0078] When the source material is video, the system generates a naked-eye 3D video for irregularly shaped screens according to the following steps:

[0079] Read video metadata, attempt to estimate the effective frame rate of the source video by counting media time changes within a short time window using requestVideoFrameCallback, and snap it to one of the commonly used frame rates such as 24, 25, 30, 50, and 60; if the estimation fails, the default frame rate is used.

[0080] Construct candidate encoder configurations based on output resolution and export specifications. For smooth output, prioritize H.264; for high-quality output, prioritize H.265 / HEVC. If the browser's capabilities do not support it, downgrade according to the candidate order.

[0081] Calculate the target time point based on frameIndex / fps and execute a seek with a timeout fallback. If the target time is close to the end of the video, limit the target time to the range of duration-ε to avoid the last frame seek not returning for a long time.

[0082] Where: seek refers to the operation of the video player jumping to a specified timestamp and requesting the decoding of the corresponding frame; duration refers to the total playback duration (in seconds) of the source video, used to determine the boundary of the last frame; seek with timeout fallback means that the system sets a maximum waiting time after initiating seek. If the decoded frame is not returned within the timeout period, a fallback strategy is executed (such as skipping the frame, reusing the previous frame, or terminating the export and providing a prompt), to avoid the last frame seek blocking for a long time and causing the entire export task to freeze. When the target time is close to the end of the video, the system limits the target time to the range of duration - ε (ε is a very small time margin).

[0083] Once the decoded frame is available, draw the frame onto the intermediate canvas, update the texture, and perform face-by-face rendering (6.5) to obtain the current unfolded frame of the irregular screen.

[0084] The expanded canvas is encapsulated as a VideoFrame, a microsecond-level timestamp is set corresponding to the frame number, and then sent to the VideoEncoder at fixed keyframe intervals.

[0085] After all frames are encoded, the encoder is flushed, and the MP4 multiplexer writes the data to the video track and generates an MP4 file.

[0086] The aforementioned video link enables ordinary videos to be converted into irregularly shaped screen unfolded videos frame by frame on the timeline, while maintaining consistent frame rate, timestamp, and encoding encapsulation, thus reducing the risk of video export jitter and freezing.

[0087] S7, Preview, Debugging, and Error Handling;

[0088] Two types of previews are provided: Overview mode for viewing the spatial structure, face numbering, and viewpoint position of irregularly shaped screens; and Viewpoint Simulation mode for placing the camera at the optimal viewpoint to directly verify the visual results seen by the viewer under the current parameters. Features include zooming, returning to the front view, locking viewpoint parameters, scaling the source material, horizontal / vertical offset, one-click adaptation, and background fill color settings within the preview mode.

[0089] Exception handling includes: refusing to import when the file type is not an image or video; providing a prompt when image or video decoding fails; prompting to switch to a browser that supports WebCodecs when encoder capability detection fails; not overwriting the original parameters when the geometric projection calculation yields an invalid value; and continuing the fallback process when frame-by-frame seek does not complete within the time limit to avoid blocking the entire export task.

[0090] In a second aspect, the present invention provides an embodiment of a computer device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method described in the first aspect.

[0091] Thirdly, the present invention provides a computer program product embodiment, which, when executed by a processor, implements the steps of the method described in the first aspect.

Claims

1. A method for generating naked-eye stereoscopic video by reverse mapping for irregularly shaped screens, characterized in that, Includes the following steps: S1. Define the following parameters: source material parameters, vertices of each display surface of the irregularly shaped screen, viewpoint coordinates, virtual material plane parameters, and output layout parameters; S2. Model the irregularly shaped screen as a set of display surfaces, each of which includes its spatial vertex coordinates and in-plane orientation; set the observation point at the center of the overall width of the irregularly shaped screen. S3. Attach the source material to the irregularly shaped screen. By deforming the source material, make it fill all visible areas of all visible display surfaces from the viewing point angle to obtain the material's planar texture. S4. Establish an off-axis projection camera from the observation point to each display surface. Calculate the local basis vectors of the screen surface and the view frustum boundary from the vertex of the display surface and the observation point. Then assemble the off-axis projection matrix and the camera pose matrix to obtain the reverse perspective mapping relationship of the display surface. Perform perspective distortion on the planar texture of the material according to the reverse perspective mapping relationship; S5. Perform face-by-face rendering to generate a single-face unfolded diagram; S6. Perform timing and encoding parameter pre-analysis on the video source material to generate a set of encoding candidate parameters; establish media timing mapping relationship by frame index to realize random addressing decoding of video; update the material planar texture and execute step S5 face-by-face rendering, and generate the output file after encoding; S7, Preview, Debugging, and Exception Handling.

2. The method according to claim 1, characterized in that, In step S1 The vertices of the quadrilateral display face include: bottom left point p_a, bottom right point p_b, top left point p_c, and top right point p_d; The coordinates of the observation point are determined by the viewing distance, viewing height, and horizontal offset; The virtual material's planar parameters include: depth, center offset, scaling, horizontal stretching, vertical stretching, and background color; Output layout parameters include: the arrangement position of each display surface image in the final output canvas, single-surface resolution, total output resolution, and video encoding specifications.

3. The method according to claim 1, characterized in that, In step S2, for the W-shaped irregular screen composed of four square screens, the side length is s and the angle between adjacent screens is 90 degrees, the five bottom edge breakpoints of the irregular screen are (-2step,0,0), (-step,0,-step), (0,0,0), (step,0,-step), (2step,0,0), and the coordinates of the observation point are (0, -s / 2+h,d); where step=s / sqrt(2), h is the observer's eye height, and d is the horizontal distance from the observer to the front of the screen.

4. The method according to claim 1, characterized in that, In step S3, the attachment method includes: Rays are emitted from the observation point to the corners of all display surfaces. The intersection points of each ray with all display surfaces are calculated. The minimum x, maximum x, minimum y, and maximum y of the intersection point set are counted to obtain the width, height, and center point of the projection boundary. Based on this, the size of the source material is adjusted and attached.

5. The method according to claim 1, characterized in that, The specific steps of step S4 include: Calculate the horizontal basis vector, vertical basis vector, and normal vector of the display surface, ensuring that the normal vector points in the direction of the observation point; Calculate the vectors from the observation point to the three vertices of the screen; Calculate the directed distance from the observation point to the screen and filter the valid distances; Calculate the asymmetric cone boundary based on the nearest effective imaging plane from the observation point; Using an asymmetric view cone boundary, a perspective projection matrix is ​​constructed by combining the farthest and nearest effective imaging planes, and a camera attitude matrix is ​​formed according to the different orientations of the camera, so that the camera is located at the observation point and facing the current display surface.

6. The method according to claim 1, characterized in that, In step S5, the specific steps of rendering include: Clear the single-sided rendering area and set the background color; Call the parameters of the off-axis projection camera corresponding to the display surface; Using the viewpoint, orientation, and frustum range of the off-axis projection projector obtained in step S4, rasterize the scene of the virtual material plane. Write the single-sided rendering result to a specified area in the output layout.

7. The method according to claim 1, characterized in that, In step S6, the specific steps for implementing random addressing decoding of video include: Read video metadata and estimate the frame rate, snap it to a common frame rate, construct encoder candidate configurations, calculate time points by frame number, and perform seek decoding with timeout fallback on video elements; if the target time point is close to the end of the video, limit it to a very small time margin.

8. The method according to claim 1, characterized in that, In step S7 The preview mode includes viewing the spatial structure, face numbering, and observation point location of the irregularly shaped screen; it also includes viewing the visual results seen by an observer located at the observation point. The parameter adjustments include zooming in preview mode, returning to the front view, locking viewpoint parameters, scaling the footage, horizontal / vertical offset, one-click adaptation, and setting the background fill color. The exception handling includes: refusing to import when the file type is not an image or video; providing a prompt when image or video decoding fails; prompting to change the browser when encoder capability detection fails; not overwriting the original parameters when the geometric projection calculation yields an invalid value; and continuing the fallback process when decoding is not completed within the time limit.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the method according to any one of claims 1-8.

10. A computer program product, characterized in that, When executed by a processor, the computer program implements the steps of the method described in any one of claims 1-8.

Images