360-degree light field LED lamp post screen advertising machine viewpoint fitting coding display method and system

Abstract

The invention relates to the technical field of display control, in particular to a 360-degree light field LED lamp post screen advertising machine viewpoint fitting coding display method and system, and the method comprises the steps: obtaining a single-viewpoint input image and display parameters; inputting the image into a single-viewpoint reconstruction network based on U-Net, outputting three-dimensional Gaussian parameters, mapping the three-dimensional Gaussian parameters into Gaussian ellipsoid attributes, and collecting the Gaussian ellipsoid attributes into a Gaussian ellipsoid set; distributing viewpoint numbers for the sub-pixels according to the display parameters to form an index matrix; a virtual viewpoint is generated under the off-axis camera model according to the viewpoint number, and angle constraint is applied; rendering blocks are divided in a self-adaptive mode, effective sub-pixels are screened out, sub-pixel coding identifications containing viewpoint numbers and rendering block identifications are generated and sorted, and a sub-pixel sorting sequence is obtained; and carrying out parallel Gaussian sputtering on the GPU according to the rendering blocks based on the Gaussian ellipsoid set, directly outputting a target light field image and generating frame buffer data. According to the invention, the problem that the multi-view fitting generation and sub-pixel interleaving coding efficiency is low under the condition that the lamp post screen only has single-view input content is solved.

Description

Technical Field

[0001] This invention relates to the field of display control technology, and in particular to a display method and system for viewpoint fitting encoding of a 360-degree light field LED light pole screen advertising machine. Background Technology

[0002] 360-degree light field LED light pole screen advertising machines typically employ self-stereoscopic structures such as gratings / cylindrical lenses. By encoding the sub-pixels of the display panel through multi-viewpoint interlacing, viewers from different directions receive light from different viewpoints to achieve parallax and stereoscopic effects. Existing solutions often rely on multi-camera arrays to acquire content or to generate multi-viewpoint renderings of the scene, and then interlacing the images of each viewpoint according to the display parameters at the sub-pixel level. However, multi-camera acquisition is costly and complex to deploy, and multi-viewpoint full-resolution rendering and encoding result in a huge amount of computation and video memory usage. Furthermore, due to the sub-pixel viewpoint mapping caused by grating tilt, there may still be a large number of redundant sub-pixel calculations that do not participate in the final light field presentation. At the same time, when fitting viewpoints based on single-viewpoint content, if the virtual viewpoint span is large, display artifacts such as geometric inconsistencies and occlusion errors are prone to occur, making it difficult to stably achieve high-quality light field display in high-resolution, real-time outdoor advertising scenarios such as light pole screens. Summary of the Invention

[0003] In view of the above technical problems, the present invention provides a display method and system for viewpoint fitting encoding of 360-degree light field LED light pole screen advertising machine, aiming to solve the problems of low efficiency of multi-viewpoint fitting generation and sub-pixel interleaving encoding, large computation and storage overhead, and poor consistency of large viewing angle fitting of 360-degree light field LED light pole screen advertising machine under the condition of only having single viewpoint input content.

[0004] Other features and advantages of this disclosure will become apparent from the following detailed description, or may be learned in part by practice of this disclosure.

[0005] According to one aspect of the present invention, a display method for viewpoint fitting encoding of a 360-degree light field LED light pole screen advertising machine is proposed, the method comprising: Acquire a single-viewpoint input image and acquire display parameters corresponding to the display panel. The display parameters include the number of viewpoints, the tilt angle of the grating relative to the vertical direction, the width of the sub-pixel covered by the grating unit in the horizontal direction, and the distance between the display panel and the grating. The single-viewpoint input image is input into a single-viewpoint reconstruction network based on the U-Net architecture, and three-dimensional Gaussian representation parameters corresponding to each pixel of the single-viewpoint input image are output pixel by pixel; the three-dimensional Gaussian representation parameters are mapped to the attributes of the corresponding three-dimensional Gaussian ellipsoids, and the three-dimensional Gaussian ellipsoids are collected to form a set of Gaussian ellipsoids. The viewpoint number of each sub-pixel in the target light field image is determined based on the display parameters to obtain a sub-pixel viewpoint index matrix. The viewpoint number is used to indicate the viewpoint that the sub-pixel of the target light field image should adopt. Based on the viewpoint number, multiple virtual viewpoints are generated under the off-axis camera model, and constraints are applied to the multiple virtual viewpoints so that the observation directions of the multiple virtual viewpoints are all within a preset angle range corresponding to the input viewpoint direction of the single viewpoint input image. The rendering block size is adaptively determined based on the number of viewpoints and the target light field image is divided into several rendering blocks, so that each rendering block contains only the effective sub-pixels that participate in the final light field rendering. Based on each of the effective sub-pixels, a sub-pixel encoding identifier is generated, which includes the viewpoint number, the rendering block identifier, and the row and column positions of the sub-pixel. The sub-pixel encoding identifiers are sorted according to the viewpoint number and the rendering block identifier to obtain a sub-pixel sorting sequence. Based on the Gaussian ellipsoid set, Gaussian sputtering rendering is performed on the sub-pixel sorting sequence. During rendering, the rendering block is used as a parallel processing unit to perform viewpoint-related projection, depth sorting, and opacity-based blending accumulation on the Gaussian ellipsoid set in parallel on the graphics processor. The target light field image is obtained directly without rendering the complete viewpoint image. The frame buffer data of the display panel is generated based on the target light field image.

[0006] Furthermore, when obtaining the sub-pixel viewpoint index matrix, the process includes: The vertical distance from the preset observation position to the grating plane, the spacing between the display panel and the grating, and the cross-sectional width of the grating unit are obtained from the display parameters. Based on the tilt angle of the grating relative to the vertical direction, the vertical distance from the observation position to the grating plane, the spacing between the display panel and the grating, and the cross-sectional width of the grating unit, the width of the sub-pixel covered by the grating unit in the horizontal direction is calculated. Calculate the horizontal distance of the sub-pixel relative to the left boundary of the leftmost grating unit, and obtain the relative offset of the sub-pixel within the period of the grating unit by taking the remainder of the horizontal distance with respect to the width of the sub-pixel covered by the grating unit in the horizontal direction; The width of the sub-pixels covered by the grating unit in the horizontal direction is divided equally according to the number of viewpoints to obtain the sub-interval width; the relative offset is assigned to an interval based on the sub-interval width, and the ratio of the relative offset to the sub-interval width is rounded up to obtain the viewpoint number, thereby forming the sub-pixel viewpoint index matrix.

[0007] Furthermore, the single-viewpoint reconstruction network outputs a set of parameters bound to each pixel of the single-viewpoint input image, and maps the parameter set to the attributes of the corresponding Gaussian ellipsoid in the Gaussian ellipsoid set. The attributes of the Gaussian ellipsoid include at least: opacity for representing transparency or occlusion relationship, three-dimensional position offset and depth for inferring three-dimensional position from pixel coordinates, scale and rotation for describing the shape of the Gaussian ellipsoid, and color for describing appearance. The single-viewpoint reconstruction network encodes the parameter set to generate the Gaussian ellipsoid set during inference.

[0008] Furthermore, during Gaussian sputtering rendering, the following are included: For each Gaussian ellipsoid in the set of Gaussian ellipsoids, determine the three-dimensional position mean vector, scale vector, rotation parameter, color parameter, and opacity; A scale matrix is ​​constructed based on the scale vector, and a rotation matrix is ​​constructed based on the rotation parameters. The three-dimensional covariance characterization of the Gaussian ellipsoid is obtained using the scale matrix and the rotation matrix. For the virtual viewpoint, a local linear approximation from three-dimensional space to two-dimensional imaging plane is determined based on view transformation and projection transformation, and the three-dimensional covariance representation is projected into a two-dimensional covariance representation accordingly to obtain the influence range of the projection ellipse of the Gaussian ellipsoid on the two-dimensional imaging plane. When performing color synthesis on the same sub-pixel, multiple Gaussian ellipsoids affecting the sub-pixel are sorted according to the depth order associated with the virtual viewpoint, and an opacity-based blending accumulation is performed in a manner that gradually decreases the forward transmittance. The transparency state is updated during the blending process until the sub-pixel colors of each rendering block are output and written to the frame buffer, thus obtaining the target light field image.

[0009] Furthermore, the specific steps involved in dividing the rendering blocks include: A rasterization method based on rendering blocks is adopted to divide the imaging plane into multiple rendering blocks and bind the set of Gaussian ellipsoids to the rendering blocks; When the number of viewpoints is less than a threshold, a preset base rendering block size is used. When the number of viewpoints increases, the rendering block size is increased. The determination of the rendering block size is based at least on the number of viewpoints and the viewpoint distribution, so that the number of effective sub-pixels contained in each rendering block can match the number of threads or scheduling strategy of the graphics processor's parallel thread block.

[0010] Furthermore, obtaining the sub-pixel sorting sequence includes: Based on the sub-pixel viewpoint index matrix, each valid sub-pixel of the target light field image is assigned a viewpoint number and a sub-pixel sorting index matrix is ​​generated. The sub-pixel sorting index matrix is ​​used to record the correspondence between each valid sub-pixel and its sub-pixel position in the target light field image, and is used to determine the linear index or write address corresponding to each valid sub-pixel. After dividing the target light field image into rendering blocks, the rendering block identifier is determined based on the rendering block where the effective sub-pixel is located, and the row and column positions of the sub-pixel are determined based on the row and column numbers of the effective sub-pixel in the target light field image. Each valid subpixel is encoded as a subpixel encoding identifier that includes the viewpoint number, the rendering block identifier, the column number, and the row number; The sub-pixel encoding identifiers are sorted according to the viewpoint number and the rendering block identifier, so that sub-pixels with the same viewpoint number are continuous in the data structure and sub-pixels with different viewpoint numbers and different rendering block identifiers are evenly distributed in the data structure. The sorted results are saved as a one-dimensional sequence arranged in the order of the viewpoint number and the rendering block identifier; The one-dimensional sequence is interpolated or padded to fill the thread block capacity corresponding to each rendering block identifier under each viewpoint number, and the row and column positions of the padded pseudo-subpixels are marked as invalid to indicate that the pseudo-subpixels do not exist in the actual light field rendering, so that the rendering blocks can be rendered independently and in parallel in a single thread block; wherein, after the color calculation of the effective subpixels is completed, the output color of the effective subpixels is written to the corresponding subpixel position of the target light field image according to the linear index or write address to form the target light field image.

[0011] Furthermore, Gaussian sputtering rendering also includes: Within the same rendering block, the effective sub-pixels are prioritized based on their contribution to the final light field rendering. The contribution is determined based on whether the effective sub-pixel is necessary for the light field rendering, the position of the effective sub-pixel within the rendering block, and the sampling density related to the viewpoint number. When the graphics processor performs parallel rendering, threads are allocated to high-priority effective sub-pixels first, and their sub-pixel color calculations are completed in advance, while the calculation frequency of low-priority effective sub-pixels is reduced or their processing is delayed.

[0012] According to another aspect of the present invention, a display system for viewpoint fitting encoding of a 360-degree light field LED light pole screen advertising machine is provided, comprising: The parameter acquisition module is used to acquire a single viewpoint input image and acquire display parameters corresponding to the display panel. The display parameters include the number of viewpoints, the tilt angle of the grating relative to the vertical direction, the width of the sub-pixel covered by the grating unit in the horizontal direction, and the distance between the display panel and the grating. The 3D Gaussian reconstruction module is used to input the single-viewpoint input image into a single-viewpoint reconstruction network based on the U-Net architecture, output 3D Gaussian representation parameters that correspond one-to-one with each pixel of the single-viewpoint input image, map the 3D Gaussian representation parameters to the attributes of the corresponding 3D Gaussian ellipsoids, and collect the 3D Gaussian ellipsoids to form a Gaussian ellipsoid set. The viewpoint index generation module is used to determine the viewpoint number of each sub-pixel in the target light field image based on the display parameters, and obtain a sub-pixel viewpoint index matrix. The viewpoint number is used to indicate the viewpoint that the sub-pixel of the target light field image should adopt. The virtual viewpoint generation and constraint module is used to generate multiple virtual viewpoints under the off-axis camera model based on the viewpoint number, and to apply constraints to the multiple virtual viewpoints so that the observation directions of the multiple virtual viewpoints are all within a preset angle range corresponding to the input viewpoint direction of the single viewpoint input image. The rendering block partitioning module is used to adaptively determine the rendering block size according to the number of viewpoints and divide the target light field image into several rendering blocks, so that each rendering block contains only the effective sub-pixels that participate in the final light field rendering. The encoding and layout module is used to generate a sub-pixel encoding identifier based on each of the effective sub-pixels, including the viewpoint number, the rendering block identifier, and the row and column positions of the sub-pixels, and to sort the sub-pixel encoding identifiers according to the viewpoint number and the rendering block identifier to obtain a sub-pixel sorting sequence; The rendering module is used to perform Gaussian sputtering rendering on the sub-pixel sorting sequence based on the Gaussian ellipsoid set. During rendering, the rendering block is used as a parallel processing unit to perform viewpoint-related projection, depth sorting, and opacity-based blending accumulation on the Gaussian ellipsoid set in parallel on the graphics processor. The target light field image is obtained directly without rendering the complete viewpoint image, and the frame buffer data of the display panel is generated based on the target light field image.

[0013] The technical solution of the present invention has the following beneficial effects: Compared to existing solutions that require generating complete multi-viewpoint images and then interleaving them, this invention achieves rapid construction from a single viewpoint to a renderable 3D representation by outputting a 3D Gaussian representation pixel by pixel through a single-viewpoint reconstruction network and aggregating them into a Gaussian ellipsoid set. It establishes a sub-pixel viewpoint index matrix based on display parameters, ensuring a one-to-one correspondence between sub-pixels of the target light field image and viewpoints, thus conforming to the encoding rules of the light pole screen raster structure from the source. Under an off-axis virtual viewpoint model, a preset angle range constraint is applied to the virtual viewpoint, reducing inconsistencies and artifacts caused by large-viewpoint extrapolation. Furthermore, it employs adaptive rendering block partitioning based on the number of viewpoints and sorts sub-pixel encoding identifiers according to viewpoint numbers and rendering block identifiers to generate a sub-pixel sorting sequence, enabling efficient scheduling of rendering on the graphics processor using rendering blocks as parallel units. Based on Gaussian sputtering rendering, it directly performs projection, depth sorting, and transparency mixing accumulation on the required effective sub-pixels, directly obtaining the target light field image and generating frame buffer data without rendering a complete viewpoint image, thereby significantly reducing redundant pixel calculations and memory usage, and improving real-time performance and engineering deployability. Attached Figure Description

[0014] Figure 1 This is a flowchart illustrating a viewpoint fitting encoding display method for a 360-degree light field LED light pole screen advertising machine, as described in this specification. Figure 2 This is a structural block diagram of a 360-degree light field LED light pole screen advertising machine viewpoint fitting encoding display system according to an embodiment of this specification; Figure 3 This specification describes a terminal device for implementing a viewpoint fitting encoding method for a 360-degree light field LED light pole screen advertising machine, as exemplified in this specification. Figure 4 This specification describes a computer-readable storage medium containing a viewpoint fitting code for a 360-degree light field LED light pole screen advertising machine, as exemplified in this specification. Detailed Implementation

[0015] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided to make this disclosure more comprehensive and complete, and to fully convey the concept of the example embodiments to those skilled in the art. The described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a full understanding of embodiments of this disclosure. However, those skilled in the art will recognize that the technical solutions of this disclosure can be practiced with one or more of the specific details omitted, or other methods, components, systems, steps, etc., can be employed. In other instances, well-known technical solutions are not shown or described in detail to avoid obscuring various aspects of this disclosure.

[0016] Furthermore, the accompanying drawings are merely illustrative of this disclosure. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor systems and / or microcontroller systems.

[0017] This invention provides a display method for viewpoint fitting encoding of a 360-degree light field LED light pole screen advertising machine. (Refer to...) Figure 1 The diagram shown is a flowchart illustrating a viewpoint fitting and encoding display method for a 360-degree light field LED light pole screen advertising machine according to an embodiment of the present invention. This method can be applied to electronic devices such as personal computers and servers. The method can be executed by a system, which can be implemented by software and / or hardware. Specifically, the method may include the following steps S101~S107: In step S101, a single-viewpoint input image is acquired, and display parameters corresponding to the display panel are acquired. The display parameters include the number of viewpoints, the tilt angle of the grating relative to the vertical direction, the width of the sub-pixel covered by the grating unit in the horizontal direction, and the distance between the display panel and the grating.

[0018] The single-viewpoint input image can be a single-frame 2D image obtained from the content creation end or the camera acquisition end. Its resolution is preferably correlated with the target output resolution of the display panel. For example, the single-viewpoint input image can be scaled, cropped, or filled according to the pixel grid of the display panel to ensure that subsequent viewpoint allocation and rendering by sub-pixel can be aligned to the row and column coordinates of the display panel. The single-viewpoint input image can be stored in RGB format, and each pixel is conventionally composed of three sub-pixels (e.g., arranged in R, G, B). The acquisition of the single-viewpoint input image is not limited to static images, video frames, or rendered images from the advertising material library; it can also be images captured in real-time by the camera, as long as it can be used as input for the subsequent single-viewpoint reconstruction network.

[0019] Simultaneously, step S101 requires obtaining display parameters corresponding to the display panel. These display parameters include at least the number of viewpoints N, the tilt angle θ of the grating relative to the vertical direction, the sub-pixel width Pe covered by the grating unit in the horizontal direction, and the distance Lg between the display panel and the grating. The grating can be a columnar lens array or an oblique grating structure, used to project the light field encoded image on the display panel to multiple different viewing directions, so that different viewing positions correspond to different viewpoint content. The number of viewpoints N is used to characterize the number of discrete viewpoints that the display panel can provide; the tilt angle θ is used to characterize the tilt of the grating stripes (or columnar lens axis) relative to the vertical direction; the distance Lg is used to characterize the mounting spacing between the light-emitting surface of the display panel and the grating plane; Pe, also known as "line count" or "period coverage width," is used to characterize the number of sub-pixels (or equivalent sub-pixel width) covered by a single grating unit in the horizontal direction. In one implementation, Pe can be directly given by hardware calibration parameters; in another implementation, Pe can be calculated based on the vertical distance L from the preset observation position to the grating plane and the cross-sectional width Pl of the grating unit, and the calculation relationship is as follows: Where L represents the vertical distance from the preset observation position to the grating plane, Pl represents the cross-sectional width of the grating unit, and Lg represents the distance from the display panel to the grating. This represents the tilt angle of the grating relative to the vertical direction. The display parameters obtained in this way provide a fundamental constraint for subsequent calculations of the relative offset of sub-pixels within the grating period and for further determination of the sub-pixel viewpoint number, thereby ensuring that the target light field image can accurately match the physical optical parameters of the display panel.

[0020] In step S102, the single-viewpoint input image is input into a single-viewpoint reconstruction network based on the U-Net architecture, and three-dimensional Gaussian representation parameters corresponding to each pixel of the single-viewpoint input image are output pixel by pixel; the three-dimensional Gaussian representation parameters are mapped to the attributes of the corresponding three-dimensional Gaussian ellipsoids, and the three-dimensional Gaussian ellipsoids are collected to form a Gaussian ellipsoid set.

[0021] The single-viewpoint reconstruction network outputs a set of parameters bound to each pixel of the single-viewpoint input image, and maps the parameter set to the attributes of the corresponding Gaussian ellipsoid in the Gaussian ellipsoid set. The attributes of the Gaussian ellipsoid include at least: opacity for representing transparency or occlusion relationship, three-dimensional position offset and depth for inferring three-dimensional position from pixel coordinates, scale and rotation for describing the shape of the Gaussian ellipsoid, and color for describing appearance. The single-viewpoint reconstruction network encodes the parameter set to generate the Gaussian ellipsoid set during inference.

[0022] Based on the geometric parameters of the display panel and the grating structure, each sub-pixel in the target light field image needs to have its corresponding viewpoint number determined to form a sub-pixel viewpoint index matrix. This index matrix establishes a one-to-one correspondence between the sub-pixel position and the viewpoint to be used, ensuring that subsequent fitting and rendering only require the viewpoint corresponding to that sub-pixel, thereby guaranteeing that the light field encoding matches the hardware's light emission direction. Specifically, the sub-pixels of the target light field image can be represented by triplet arrays. This is represented as follows: where i is the row number of the sub-pixel, j is the column number of the sub-pixel, and k is the color channel number (e.g., k=1,2,3 correspond to R, G, and B sub-pixels respectively), with the left boundary of the leftmost raster unit as the horizontal distance reference zero point. This is based on the tilt angle of the raster relative to the vertical direction. Calculate any sub-pixel The horizontal distance from the left boundary to the left boundary of the leftmost raster unit And further obtain the relative offset of the sub-pixel within the grating period. The calculation relationship is as follows:

[0023] In the above formula, "3" is used to represent a counting unit where a pixel contains three sub-pixels in the horizontal direction. The modulo operator represents the width of the subpixels covered by the raster unit in the horizontal direction, where mod is the modulo operation used to... The relative position is converted to a single grating unit period.

[0024] After obtaining the relative offset Then, the grating unit covers the sub-pixel width in the horizontal direction. Divide the sub-interval width d into equal parts according to the number of viewpoints N, and then... To determine the viewpoint number n, interval assignment is performed, where the relationship between d and n is as follows:

[0025] in, This is a floor function used to round up relative offsets. The mapping is to discrete viewpoint numbers; the viewpoint number n can be limited to an integer from 1 to N, used to indicate the viewpoint source from which the sub-pixel of the target light field image should be used. This applies to all sub-pixels within the target light field image. By calculating and recording the viewpoint number of each pixel, the subpixel viewpoint index matrix (which can also be understood as the viewpoint index part in the subpixel sorting index matrix) can be obtained, thus providing a basis for subsequent organization of subpixels by viewpoint and spatial blocks, and for accurately matching the relationship between subpixels and viewpoints during rendering.

[0026] In step S103, the viewpoint number of each sub-pixel in the target light field image is determined according to the display parameters to obtain a sub-pixel viewpoint index matrix. The viewpoint number is used to indicate the viewpoint that the sub-pixel of the target light field image should adopt.

[0027] Furthermore, when obtaining the sub-pixel viewpoint index matrix, the process includes: The vertical distance from the preset observation position to the grating plane, the spacing between the display panel and the grating, and the cross-sectional width of the grating unit are obtained from the display parameters. Based on the tilt angle of the grating relative to the vertical direction, the vertical distance from the observation position to the grating plane, the spacing between the display panel and the grating, and the cross-sectional width of the grating unit, the width of the sub-pixel covered by the grating unit in the horizontal direction is calculated. Calculate the horizontal distance of the sub-pixel relative to the left boundary of the leftmost grating unit, and obtain the relative offset of the sub-pixel within the period of the grating unit by taking the remainder of the horizontal distance with respect to the width of the sub-pixel covered by the grating unit in the horizontal direction; The width of the sub-pixels covered by the grating unit in the horizontal direction is divided equally according to the number of viewpoints to obtain the sub-interval width; the relative offset is assigned to an interval based on the sub-interval width, and the ratio of the relative offset to the sub-interval width is rounded up to obtain the viewpoint number, thereby forming the sub-pixel viewpoint index matrix.

[0028] In step S104, based on the viewpoint number, multiple virtual viewpoints are generated under the off-axis camera model, and constraints are applied to the multiple virtual viewpoints so that the observation directions of the multiple virtual viewpoints are all within a preset angle range of the input viewpoint direction corresponding to the single viewpoint input image.

[0029] To ensure a clear and controllable geometric basis for subsequent multi-view projections based on the Gaussian ellipsoid set, multiple virtual viewpoints need to be constructed using the viewpoint numbers as indexes. These virtual viewpoints are essentially virtual camera viewpoints at the rendering end, representing the imaging geometry under different viewing directions. The off-axis camera model describes situations where the imaging plane of the virtual viewpoint is no longer strictly coaxial with its optical axis (i.e., the principal point / projection center is offset relative to the imaging plane, or the imaging frustum is laterally offset), thus better reflecting the off-axis observation patterns caused by light field propagation and raster / optical encoding. A parallel multi-view rendering module is constructed based on this off-axis camera model. This module can efficiently map from multiple viewpoints to light field image output based on the geometric properties of the input viewpoint and the Gaussian distribution parameters without changing the single-viewpoint input image acquisition method. Furthermore, it supports flexible adjustments to the off-axis viewpoint to enhance the algorithm's adaptability and versatility.

[0030] When generating the multiple virtual viewpoints, the viewpoint number can be mapped to the viewpoint sequence number of the virtual viewpoint, and corresponding viewpoint pose parameters and off-axis projection parameters can be configured for each viewpoint sequence number: the viewpoint pose parameters are used to determine the position and orientation of the virtual viewpoint in three-dimensional space (for example, to perform discrete displacement in the horizontal direction relative to the input viewpoint and apply corresponding yaw angle changes to cover the required viewpoint distribution), and the off-axis projection parameters are used to determine the offset relationship of the imaging frustum on the imaging plane, so that the imaging center of the virtual viewpoint and the imaging plane form a projection geometry consistent with off-axis observation. Subsequently, viewpoint constraints are applied to the multiple virtual viewpoints: an input viewpoint direction corresponding to the single viewpoint input image is set, and a preset angle range is given in advance; the observation direction of each virtual viewpoint is calculated (which can be understood as the optical axis direction of the virtual viewpoint or the main observation direction from the virtual viewpoint to the scene), and the angle between the observation direction and the input viewpoint direction is limited to the preset angle range; when a virtual viewpoint does not meet the constraints, its angle can be clipped (projecting the observation direction back into the allowed angle cone), the pose of the virtual viewpoint can be resampled, or the mapping relationship between the viewpoint number and the pose can be recalibrated, so that the output viewpoint participating in the rendering is always in the limited field of view near the input viewpoint, thereby ensuring that the viewpoint of the output viewpoint image and the viewpoint of the input viewpoint image are kept within a certain range.

[0031] The reason for adopting the above constraints is that light field-oriented displays do not require indiscriminate reconstruction of the entire scene 360°, but rather emphasize obtaining a consistent and stable viewpoint appearance within the target field of view. If the field of view is too large, mismatches are more likely to occur in the positions of the 3D Gaussian points across viewpoints, leading to blurry or inconsistent rendering results. By limiting the viewing angle range of the virtual viewpoint, the set of Gaussian ellipsoids inferred by the network can be more concentrated on the side facing the camera within the target field of view, enhancing the learnability of the local structure and reducing background interference, thereby obtaining better viewpoint consistency and rendering quality within the preset angle range.

[0032] In step S105, the size of the rendering block is adaptively determined according to the number of viewpoints and the target light field image is divided into several rendering blocks, so that each rendering block contains only effective sub-pixels that participate in the final light field rendering.

[0033] Specifically, when dividing the rendering blocks, the following steps are taken: using a rasterization method based on rendering blocks to divide the imaging plane into multiple rendering blocks and binding the rendering blocks to the Gaussian ellipsoid set; using a preset base rendering block size when the number of viewpoints is less than a threshold, increasing the rendering block size when the number of viewpoints increases, and determining the rendering block size is based at least on the number of viewpoints and the viewpoint distribution, so that the number of effective sub-pixels contained in each rendering block can match the number of threads or scheduling strategy of the graphics processor's parallel thread block.

[0034] Specifically, in step S105, to ensure stable scheduling granularity and controllable computation / storage overhead during subsequent parallel rendering on the graphics processor, the imaging plane corresponding to the target light field image needs to be divided into several rectangular rendering blocks (which can be understood as image block units, such as tiles), and the rendering blocks are used as the basic units for parallel processing and data organization. Here, effective sub-pixels refer to sub-pixels that actually participate in the final light field rendering, need to be assigned values ​​and written to the frame buffer under the viewpoint interleaving rules defined by the sub-pixel viewpoint index matrix; in contrast, invalid sub-pixels are sub-pixel positions that do not need to participate in the display output under the current number and distribution of viewpoints. By organizing by rendering blocks, rendering calculations can be initiated only for effective sub-pixels within the coverage area of ​​each rendering block, and invalid sub-pixels can be skipped, thereby avoiding the redundant overhead caused by indiscriminate calculation of the entire viewpoint image or the entire sub-pixel grid; at the same time, rendering blocks are naturally adapted to the thread block scheduling mode of the graphics processor, so that rendering blocks and thread blocks form a relatively fixed mapping relationship, which facilitates parallel computing and memory access optimization.

[0035] When dividing specific rendering blocks, a rasterization method based on rendering blocks is adopted. The imaging plane is divided into regular grids according to preset block width and height, and a set of data to be rendered is established for each rendering block. The Gaussian ellipsoid set is bound to the rendering block, but instead of copying all Gaussian ellipsoids to each rendering block, the Gaussian ellipsoid indices or references that may affect the effective sub-pixels within a certain rendering block are added to the block's association list based on the influence range of the Gaussian ellipsoids after projection onto the imaging plane from the current virtual viewpoint. This way, during subsequent rendering, each rendering block only needs to traverse its association list to complete the color evaluation of the effective sub-pixels within that block, reducing access to irrelevant Gaussian ellipsoids. Simultaneously, within the rendering block, the sub-pixel positions covered by the block can be filtered and marked according to the sub-pixel viewpoint index matrix, forming a set of effective sub-pixels for that block. This ensures that the thread only performs calculations on these effective sub-pixels and writes them back to the corresponding positions after completion, guaranteeing that the calculated content within the rendering block is consistent with the final light field rendering requirements.

[0036] The rendering block size adopts an adaptive strategy based on the number of viewpoints: when the number of viewpoints is less than a preset threshold, the basic rendering block size can be used to roughly match the number of effective sub-pixels in each block with the capacity of the graphics processor's thread blocks; however, as the number of viewpoints increases, viewpoint interleaving leads to a sparser distribution of effective sub-pixels in the same spatial area. If a smaller basic rendering block size is still used, problems such as insufficient number of effective sub-pixels in each rendering block, decreased thread utilization, and increased block management and scheduling overhead due to a sharp increase in the number of rendering blocks will occur. Therefore, the rendering block size is increased when the number of viewpoints increases, so that a single rendering block covers a larger area, thereby statistically containing more effective sub-pixels and reducing the total number of rendering blocks and the scale of related intermediate data; the determination of the rendering block size is at least combined with the number of viewpoints and viewpoint distribution, so that the number of effective sub-pixels in each rendering block is more in line with the number of threads or scheduling strategy of the thread block, ensuring that the generation of the target light field image can be completed stably and efficiently when the rendering block is used as a parallel processing unit.

[0037] In step S106, a subpixel encoding identifier is generated based on each of the valid subpixels, including the viewpoint number, the rendering block identifier, and the row and column positions of the subpixels. The subpixel encoding identifiers are sorted according to the viewpoint number and the rendering block identifier to obtain a subpixel sorting sequence.

[0038] Specifically, obtaining the sub-pixel sorting sequence includes: assigning a viewpoint number to each valid sub-pixel of the target light field image based on the sub-pixel viewpoint index matrix and generating a sub-pixel sorting index matrix, wherein the sub-pixel sorting index matrix is ​​used to record the correspondence between each valid sub-pixel and its sub-pixel position in the target light field image, and is used to determine the linear index or write address corresponding to each valid sub-pixel; after dividing the target light field image into rendering blocks, determining the rendering block identifier based on the rendering block where the valid sub-pixel is located, and determining the sub-pixel row and column positions based on the row and column numbers of the valid sub-pixel in the target light field image; encoding each valid sub-pixel as a sub-pixel encoding identifier containing the viewpoint number, the rendering block identifier, the column number, and the row number; and sorting the sub-pixels according to the viewpoint number and the rendering block identifier. The subpixel encoding identifiers are sorted to ensure that subpixels with the same viewpoint number are continuous in the data structure and that subpixels with different viewpoint numbers and different rendering block identifiers are evenly distributed in the data structure. The sorted result is saved as a one-dimensional sequence arranged in the order of the viewpoint number and the rendering block identifier. The one-dimensional sequence is interpolated or padded to fill the thread block capacity corresponding to each rendering block identifier under each viewpoint number, and the row and column positions of the padded pseudo-subpixels are marked as invalid to indicate that the pseudo-subpixels do not exist in the actual light field rendering, so that the rendering blocks can be rendered independently and in parallel in a single thread block. After the color calculation of the effective subpixels is completed, the output color of the effective subpixels is written to the corresponding subpixel position of the target light field image according to the linear index or write address to form the target light field image.

[0039] In step S106, to organize the effective sub-pixels in the target light field image that need to participate in the final light field rendering into a data structure suitable for parallel execution by the graphics processor, it is necessary to first assign a viewpoint number to each effective sub-pixel according to the sub-pixel viewpoint index matrix, and generate a sub-pixel sorting index matrix. The sub-pixel sorting index matrix is ​​used to record the correspondence between each effective sub-pixel and its sub-pixel position in the target light field image, so that after the sub-pixel color calculation is completed, the linear index or write address of the effective sub-pixel can be determined according to the correspondence and written back to the correct sub-pixel position. Furthermore, after dividing the target light field image into rendering blocks, for each valid sub-pixel, a rendering block identifier is determined based on the rendering block it belongs to, and the row and column positions of the sub-pixel are determined based on its row number and column number. In one implementation, the linear index or write address can be determined by the "row number, column number and color channel number" to indicate the position to be written in the frame buffer, while the sub-pixel sorting index matrix is ​​used to ensure that the "valid sub-pixel - write-back position" mapping is not lost after filtering out invalid sub-pixels, thereby ensuring the correctness of subsequent write-backs.

[0040] Subsequently, each valid sub-pixel is encoded as a sub-pixel encoding identifier. The sub-pixel encoding identifier includes at least the viewpoint number, the rendering block identifier, the column number, and the row number, thus constructing a unique identifier for each valid sub-pixel. In a specific implementation, each sub-pixel can be encoded in the form of "(viewpoint, tileID, columnnumber, rownumber)" to achieve efficient organization and management of sub-pixels and prepare for parallel computing. Sorting is performed based on the sub-pixel encoding identifier: the sub-pixel encoding identifiers are sorted according to the viewpoint number and the rendering block identifier, ensuring that sub-pixels with the same viewpoint number remain continuous in the data structure, while ensuring that sub-pixels with different viewpoint numbers and different rendering block identifiers are evenly distributed overall, thereby improving data locality and reducing the probability of parallel access conflicts. After sorting, the result is saved as a one-dimensional sequence in the order of viewpoint number and rendering block identifier, forming the sub-pixel sorting sequence. Subsequent rendering can be batch-scheduled according to this one-dimensional sequence, thus avoiding the redundant process of generating a complete viewpoint image for each viewpoint and then interleaving it.

[0041] After obtaining the one-dimensional sequence, in order to enable each rendering block under each viewpoint number to be rendered independently and in parallel as a single thread block in the graphics processor, the one-dimensional sequence needs to be interpolated or padded to fill the thread block capacity corresponding to each rendering block identifier under each viewpoint number; for example, to enable each rendering block to "fill a CUDA block" during scheduling, thereby achieving a stable mapping of a single thread block. For the pseudo-subpixels obtained by padding, their row and column positions need to be marked as invalid (e.g., marked as (-1,-1)) to indicate that the pseudo-subpixel does not exist in the actual light field rendering, so that it can be quickly skipped in the rendering kernel without participating in the actual color write-back. The above organization method enables each rendering block to be rendered independently and in parallel within a single thread block, and allows each thread to focus on the calculation task of a single subpixel, thereby making full use of the parallel computing power of the graphics processor while maintaining the mapping relationship from the viewpoint to the light field subpixel. Furthermore, since this step has explicitly marked invalid subpixels and filtered them at the data level, subsequent rendering only needs to calculate and write back the valid subpixels, thereby reducing the computational burden and storage occupation caused by redundant subpixels from a mechanism perspective.

[0042] In step S107, based on the Gaussian ellipsoid set, the sub-pixel sorting sequence is subjected to Gaussian sputtering rendering. During rendering, the rendering block is used as a parallel processing unit to perform viewpoint-related projection, depth sorting, and opacity-based blending accumulation on the Gaussian ellipsoid set in parallel on the graphics processor. The target light field image is obtained directly without rendering the complete viewpoint image, and the frame buffer data of the display panel is generated based on the target light field image.

[0043] The Gaussian sputtering rendering process includes: determining a three-dimensional position mean vector, scale vector, rotation parameter, color parameter, and opacity for each Gaussian ellipsoid in the Gaussian ellipsoid set; constructing a scale matrix based on the scale vector and a rotation matrix based on the rotation parameter; and obtaining a three-dimensional covariance representation of the Gaussian ellipsoid using the scale matrix and the rotation matrix; for the virtual viewpoint, determining a local linear approximation from three-dimensional space to a two-dimensional imaging plane based on view transformation and projection transformation, and projecting the three-dimensional covariance representation into a two-dimensional covariance representation accordingly to obtain the influence range of the projected ellipse of the Gaussian ellipsoid on the two-dimensional imaging plane; when performing color synthesis on the same sub-pixel, sorting multiple Gaussian ellipsoids affecting the sub-pixel according to the depth order related to the virtual viewpoint, and performing opacity-based blending accumulation by gradually attenuating forward transmittance, updating the opacity state during the blending process, until the sub-pixel colors of each rendering block are output and written to the frame buffer to obtain the target light field image.

[0044] Furthermore, during Gaussian sputtering rendering, the method further includes: prioritizing the effective sub-pixels within the same rendering block based on their contribution to the final light field rendering, wherein the contribution is determined based on whether the effective sub-pixel is necessary for the light field rendering, the position of the effective sub-pixel within the rendering block, and the sampling density related to the viewpoint number; and during parallel rendering by the graphics processor, allocating threads to high-priority effective sub-pixels first and completing their sub-pixel color calculations in advance, while reducing the calculation frequency or delaying the processing of low-priority effective sub-pixels. In this step, the sub-pixel sorting sequence obtained in the previous step is used as the basis for rendering scheduling. The light field target image is divided into several rendering tiles, and the rendering tiles are used as parallel processing units on the graphics processor. Since the light field image essentially only needs to extract target sub-pixels from a large number of viewpoints and fill them into the final image, if the complete viewpoint image is rendered one viewpoint at a time in the traditional way, it will lead to the waste of most of the rendered sub-pixels being discarded. Therefore, this step adopts a strategy of only rendering the sub-pixels actually needed for the final light field image, so as to directly generate the target light field image without rendering the complete viewpoint image and significantly reduce the computation and memory burden. To support block-level parallelism and data locality in rendering, each sub-pixel to be rendered is encoded with a unique identifier based on its viewpoint, tile, column, and row numbers. These identifiers are then sorted according to viewpoint and tile dimensions, ensuring that sub-pixels corresponding to the same viewpoint (and the same tile) remain continuous within the data structure. This facilitates the batch binding of sub-pixels from each tile to a CUDA block for independent parallel execution. When the number of valid sub-pixels for certain tile / viewpoint combinations is insufficient to fill a CUDA block, interpolation / padding can be used to fill the gaps. The row and column of the filled sub-pixels are marked as (-1, -1) to indicate that the sub-pixel does not exist and the thread can skip the actual computation. Through this encoding, sorting, and block-level parallel organization, each thread focuses on the computation of a single sub-pixel, while filtering out unnecessary redundant sub-pixels reduces memory access conflicts and improves throughput.

[0045] In specific Gaussian sputtering rendering calculations, the scene point cloud is represented by a set of Gaussian ellipsoids, each Gaussian ellipsoid being determined by its position. Scale vector Rotation vector Color c (which can be represented by a spherical harmonic function) and opacity Its distribution in three-dimensional space can be represented by isoparametric characterization as follows:

[0046] Where the covariance matrix For easier optimization, it can be decomposed into a combination of the scaling matrix S and the rotation matrix R:

[0047] For viewpoint-dependent projection, given the view transformation W and projection transformation at a given viewpoint, the affine approximation Jacobian matrix J of the view transformation is used to project the 3D Gaussian onto the 2D plane, yielding the screen space covariance:

[0048] After viewpoint projection is completed, each Gaussian ellipsoid is bound to the rendering blocks it affects, and depth sorting is performed in conjunction with viewpoint depth information; this sorting, together with tile binding, is used to ensure subsequent blending and accumulation in depth order and to accelerate the rasterization process.

[0049] Within each rendering block, and in the thread corresponding to each sub-pixel, an opacity-based blending accumulation is performed on the Gaussian ellipsoid covering that sub-pixel based on the depth sorting result. Color can be selected according to... The hybrid rendering pattern is accumulated from front to back:

[0050] Where N represents the Gaussian set that contributes to that sub-pixel. These represent the contributions of the i-th Gaussian to the color and opacity of the sub-pixel, respectively. Through the above mixing and accumulation, the color values ​​of each sub-pixel in the target light field image can be directly obtained under tile parallelism. To further address resource bottlenecks, priority can be assigned based on the contribution of sub-pixels to the final light field image, combined with dynamic thread allocation, ensuring quality for critical sub-pixels and reducing unnecessary overhead in low-priority areas. Finally, the sub-pixel results output from each rendering block are written to the target light field image buffer according to the sub-pixel layout of the target light field image, and frame buffer data corresponding to the display panel (e.g., frame buffers organized according to the RGB sub-pixel order of the display panel) is generated accordingly for subsequent display output.

[0051] Furthermore, within the same rendering block, the effective sub-pixels can be prioritized to ensure that sub-pixels more critical to the final light field rendering are prioritized when the parallel resources of the graphics processor are limited. The contribution can be determined by considering the following factors: first, whether the effective sub-pixel is necessary for the light field rendering (e.g., whether it corresponds to a real viewpoint-carrying sub-pixel rather than a filler or invalid marker sub-pixel); second, the position of the effective sub-pixel within the rendering block (e.g., sub-pixels near the rendering block boundary or high-frequency detail areas can be given higher priority to reduce the risk of artifacts at inter-block stitching or details); and third, the sampling density related to the viewpoint number (e.g., increasing priority in viewpoint areas with denser viewpoint interweaving or more sensitive observation). When the graphics processor performs parallel rendering, threads can be allocated according to the priority: threads are allocated to high-priority effective sub-pixels first and their sub-pixel color calculations are completed first; for low-priority effective sub-pixels, methods such as reducing the calculation frequency, delaying processing, or skipping subsequent contribution evaluation after meeting preset termination conditions can be adopted, thereby improving the rendering stability and subjective display effect of key areas within the overall computing budget, while keeping the scheme of parallel rendering by rendering block and write-back by sub-pixel unchanged.

[0052] In one embodiment, during the rendering cycle of each virtual viewpoint, the intermediate cache is dynamically allocated on demand based on the number of Gaussian ellipsoids that the virtual viewpoint needs to process. After binning and sorting operations are completed, the intermediate results and unused memory blocks generated by the binning and sorting operations are immediately reclaimed, and a delayed reclamation strategy is adopted to ensure that memory blocks are released only when they are no longer accessed. During the sorting process, the memory blocks used to store sorting keys and their associated data are reclaimed in a hierarchical manner, and memory fragmentation is reduced through a memory pool reuse mechanism. Furthermore, the lifecycle-managed memory reclamation further combines asynchronous memory management and streaming data processing, enabling the reclamation operation to be executed in parallel with the rendering task. The intermediate data includes at least geometric state data, image state data, and binning and sorting state data, and by increasing the tile size in step S5 to reduce the number of Gaussian ellipsoid-associated tiles, the memory usage of the binning and sorting state data is further reduced.

[0053] In one embodiment, to adapt to the memory pressure brought by multiple virtual viewpoints and high-resolution target light field images, the intermediate cache involved in the rendering process is dynamically allocated on demand during the rendering cycle of each virtual viewpoint. That is, the corresponding amount of video memory space is dynamically requested based on the number of Gaussian ellipsoids that the virtual viewpoint needs to process. This allows the memory usage of different virtual viewpoints to vary with the actual load fluctuations, rather than reserving a fixed upper limit of memory for all virtual viewpoints. Furthermore, by finely controlling the memory lifecycle of the rendering process, the memory related to the "viewpoint-rendering block" processing can be requested, used, and promptly reclaimed at the viewpoint granularity after processing, thereby reducing long-term occupation and improving video memory utilization.

[0054] Furthermore, after completing the binning and sorting operations, the intermediate results and unused memory blocks generated by these operations are immediately reclaimed to prevent intermediate data from residing in video memory for extended periods. To ensure safe release, a delayed reclamation strategy is adopted, releasing memory blocks only when it is confirmed that they are no longer accessed, thereby reducing the risk of access conflicts and avoiding rendering anomalies caused by erroneous releases. In particular, a layered reclamation mechanism is implemented for memory blocks used to store sorting keys and their associated data during the sorting process: after the sorting stage is completed, the key-value cache and its associated data cache corresponding to the completed stage are reclaimed first, and layered release continues in subsequent stages. Combined with a memory pool reuse mechanism, the reclaimed memory blocks are quickly reused for the processing of the next virtual viewpoint or the next rendering block, reducing fragmentation problems caused by frequent allocation / release. To further improve the reliability of reclamation, modern memory management techniques such as reference counting and smart pointers can be combined to ensure that each memory block is only reclaimed after its reference count reaches zero, thus preventing memory leaks or access conflicts from a mechanism perspective.

[0055] In the specific implementation, the intermediate data includes at least geometric state data, image state data, and binning and sorting state data. Geometric state data stores parameters of the Gaussian reconstruction kernel (e.g., depth, scale, color), image state data stores data related to the final rendered image, and binning and sorting state data encodes, bins, and sorts the Gaussian ellipsoids; its memory consumption is directly related to the number of ellipsoids and the number of associated rendering blocks. To reduce the peak memory usage of binning and sorting state data, after calculating the range information of each rendering block based on key values ​​(e.g., storing the Gaussian start and end indices corresponding to each rendering block), the intermediate results generated during the sorting process can be discarded, thereby reducing invalid resident memory. Simultaneously, the lifecycle-managed memory reclamation further combines asynchronous memory management and streaming data processing, enabling the reclamation operation to be executed in parallel with the rendering task, reducing the performance bottleneck introduced by memory management. Furthermore, by increasing the rendering block size in step S105, the number of rendering blocks associated with a single Gaussian ellipsoid can be reduced, thereby reducing the overhead of binning and sorting state data in the "rendering block dimension" and further reducing video memory usage.

[0056] Based on the same line of thought, such as Figure 2 As shown, a display system for a 360-degree light field LED light pole screen advertising machine with viewpoint fitting encoding is provided, comprising: The parameter acquisition module 201 is used to acquire a single viewpoint input image and acquire display parameters corresponding to the display panel. The display parameters include the number of viewpoints, the tilt angle of the grating relative to the vertical direction, the width of the sub-pixel covered by the grating unit in the horizontal direction, and the distance between the display panel and the grating. The 3D Gaussian reconstruction module 202 is used to input the single-viewpoint input image into a single-viewpoint reconstruction network based on the U-Net architecture, output 3D Gaussian representation parameters that correspond one-to-one with each pixel of the single-viewpoint input image, map the 3D Gaussian representation parameters to the attributes of the corresponding 3D Gaussian ellipsoids, and collect the 3D Gaussian ellipsoids to form a Gaussian ellipsoid set. The viewpoint index generation module 203 is used to determine the viewpoint number of each sub-pixel in the target light field image based on the display parameters, and obtain a sub-pixel viewpoint index matrix. The viewpoint number is used to indicate the viewpoint that the sub-pixel of the target light field image should adopt. The virtual viewpoint generation and constraint module 204 is used to generate multiple virtual viewpoints under the off-axis camera model based on the viewpoint number, and to apply constraints to the multiple virtual viewpoints so that the observation directions of the multiple virtual viewpoints are all within a preset angle range corresponding to the input viewpoint direction of the single viewpoint input image. The rendering block division module 205 is used to adaptively determine the size of the rendering block according to the number of viewpoints and divide the target light field image into several rendering blocks, so that each rendering block contains only the effective sub-pixels that participate in the final light field rendering. The encoding and wiring module 206 is used to generate a sub-pixel encoding identifier based on each of the effective sub-pixels, including the viewpoint number, the rendering block identifier, and the row and column position of the sub-pixel, and sort the sub-pixel encoding identifiers according to the viewpoint number and the rendering block identifier to obtain a sub-pixel sorting sequence; The rendering module 207 is used to perform Gaussian sputtering rendering on the sub-pixel sorting sequence based on the Gaussian ellipsoid set. During rendering, the rendering block is used as a parallel processing unit to perform viewpoint-related projection, depth sorting and opacity-based blending accumulation on the Gaussian ellipsoid set in parallel on the graphics processor. The target light field image is obtained directly without rendering the complete viewpoint image, and the frame buffer data of the display panel is generated based on the target light field image.

[0057] Compared to existing solutions that require generating complete multi-viewpoint images and then interleaving them, this system rapidly constructs a renderable 3D representation from a single viewpoint by outputting a 3D Gaussian representation pixel by pixel through a single-viewpoint reconstruction network and aggregating them into a Gaussian ellipsoid set. It establishes a sub-pixel viewpoint index matrix based on display parameters, ensuring a one-to-one correspondence between sub-pixels of the target light field image and viewpoints, thus conforming to the encoding rules of the light pole screen raster structure from the source. Under an off-axis virtual viewpoint model, a preset angle range constraint is applied to the virtual viewpoint, reducing inconsistencies and artifacts caused by large-viewpoint extrapolation. Furthermore, it employs adaptive rendering block partitioning based on the number of viewpoints and sorts sub-pixel encoding identifiers according to viewpoint numbers and rendering block identifiers to generate a sub-pixel sorting sequence, enabling efficient scheduling of rendering on the graphics processor using rendering blocks as parallel units. Based on Gaussian sputtering rendering, it directly performs projection, depth sorting, and transparency mixing accumulation on the required effective sub-pixels, directly obtaining the target light field image and generating frame buffer data without rendering a complete viewpoint image. This significantly reduces redundant pixel calculations and memory usage, improving real-time performance and engineering deployability.

[0058] The specific details of each module / unit in the above system have been described in detail in the implementation method section. For any undisclosed details, please refer to the implementation method section, and therefore will not be repeated here.

[0059] Based on the same idea, this specification also provides a display device for viewpoint fitting encoding of a 360-degree light field LED light pole screen advertising machine, such as... Figure 3 As shown.

[0060] The display device for the 360-degree light field LED light pole screen advertising machine with viewpoint fitting encoding can be the terminal device or server provided in the above embodiments.

[0061] The display device for a 360-degree light field LED light pole screen advertising machine with viewpoint fitting encoding can vary significantly due to differences in configuration or performance. It may include one or more processors 301, memory 302, and buses. The memory 302 may store one or more application programs or data. The memory 302 may include readable media in the form of volatile storage units, such as random access memory (RAM) and / or cache memory units, i.e., plug-in external hard drives, smart media cards (SMC), secure digital cards (SD), flash cards, etc., and may further include read-only storage units. The application programs stored in the memory 302 may include one or more program modules (not shown in the figure). Such program modules include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Furthermore, the processor 301 can be configured to communicate with the memory 302 and execute a series of computer-executable instructions in the memory 302 on the display device of the 360-degree light field LED light pole screen advertising machine viewpoint fitting encoding. The display device of the 360-degree light field LED light pole screen advertising machine viewpoint fitting encoding may also include one or more power supplies 303, one or more wired or wireless network interfaces 304, one or more I / O interfaces (input / output interfaces) 305, one or more external devices 306 (e.g., keyboards), and may also communicate with one or more devices that enable user interaction with the device, and / or with any device that enables the device to communicate with one or more other computing devices (e.g., routers, network switches, etc.). This communication can be performed through the I / O interface 305. Furthermore, the device can also communicate with one or more networks (e.g., local area networks (LANs)) through the wired or wireless interface 304.

[0062] Figure 3 Only a display device with viewpoint fitting encoding for a 360-degree light field LED light pole screen advertising machine with components is shown. Those skilled in the art will understand that... Figure 3 The structure shown does not constitute a limitation on the display device for viewpoint fitting encoding of 360-degree light field LED light pole screen advertising machine, and may include fewer or more components than shown, or combine certain components, or have different component arrangements.

[0063] Specifically, in this embodiment, the display device for viewpoint fitting encoding of the 360-degree light field LED light pole screen advertising machine includes a memory and one or more programs, wherein one or more programs are stored in the memory, and one or more programs may include one or more modules, and each module may include a series of computer-executable instructions for the 360-degree light field LED light pole screen advertising machine viewpoint fitting encoding display device, and is configured to be executed by one or more processors. The one or more programs include computer-executable instructions for performing the following: Acquire a single viewpoint input image and acquire display parameters corresponding to the display panel. The display parameters include at least the number of viewpoints, the tilt angle of the grating relative to the vertical direction, the width of the sub-pixel covered by the grating unit in the horizontal direction, and the distance between the display panel and the grating. The single-viewpoint input image is input into a single-viewpoint reconstruction network based on the U-Net architecture, and three-dimensional Gaussian representation parameters corresponding to each pixel of the single-viewpoint input image are output pixel by pixel; the three-dimensional Gaussian representation parameters are mapped to the attributes of the corresponding three-dimensional Gaussian ellipsoids, and the three-dimensional Gaussian ellipsoids are collected to form a set of Gaussian ellipsoids. The viewpoint number of each sub-pixel in the target light field image is determined based on the display parameters to obtain a sub-pixel viewpoint index matrix. The viewpoint number is used to indicate the viewpoint that the sub-pixel of the target light field image should adopt. Based on the viewpoint number, multiple virtual viewpoints are generated under the off-axis camera model, and constraints are applied to the multiple virtual viewpoints so that the observation directions of the multiple virtual viewpoints are all within a preset angle range corresponding to the input viewpoint direction of the single viewpoint input image. The rendering block size is adaptively determined based on the number of viewpoints and the target light field image is divided into several rendering blocks, so that each rendering block contains only the effective sub-pixels that participate in the final light field rendering. Based on each of the effective sub-pixels, a sub-pixel encoding identifier is generated, which includes the viewpoint number, the rendering block identifier, and the row and column positions of the sub-pixel. The sub-pixel encoding identifiers are sorted according to the viewpoint number and the rendering block identifier to obtain a sub-pixel sorting sequence. Based on the Gaussian ellipsoid set, Gaussian sputtering rendering is performed on the sub-pixel sorting sequence. During rendering, the rendering block is used as a parallel processing unit to perform viewpoint-related projection, depth sorting, and opacity-based blending accumulation on the Gaussian ellipsoid set in parallel on the graphics processor. The target light field image is obtained directly without rendering the complete viewpoint image. The frame buffer data of the display panel is generated based on the target light field image.

[0064] Based on the same idea, exemplary embodiments of the present invention also provide a computer-readable storage medium having a program product stored thereon capable of implementing the methods described above in this specification. In some possible embodiments, various aspects of this disclosure can also be implemented as a program product including program code, which, when the program product is run on a terminal device, causes the terminal device to perform the steps according to the various exemplary embodiments of this disclosure described in the "Exemplary Methods" section above.

[0065] refer to Figure 4 As shown, a program 400 for implementing the above-described method according to an exemplary embodiment of the present disclosure is described. This program may employ a portable compact disc read-only memory (CD-ROM) and include program code, and may run on a terminal device, such as a personal computer. However, the program product of the present disclosure is not limited thereto. In this document, a readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, system, or device.

[0066] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: electrical connections with one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0067] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of sending, propagating, or transmitting programs for use by or in conjunction with an instruction execution system, system, or device.

[0068] Program code for performing the operations of this disclosure can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, CSS, and HTML, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0069] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, terminal system, or network device, etc.) to execute the method according to the exemplary embodiments of this disclosure.

[0070] Furthermore, the above figures are merely illustrative representations of the processes included in the methods according to exemplary embodiments of this disclosure, and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Additionally, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0071] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to exemplary embodiments of this disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0072] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and embodiments are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the claims.

[0073] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A display method for viewpoint fitting encoding of a 360-degree light field LED light pole screen advertising machine, characterized in that, The method includes: Acquire a single-viewpoint input image and acquire display parameters corresponding to the display panel. The display parameters include the number of viewpoints, the tilt angle of the grating relative to the vertical direction, the width of the sub-pixel covered by the grating unit in the horizontal direction, and the distance between the display panel and the grating. The single-viewpoint input image is input into a single-viewpoint reconstruction network based on the U-Net architecture, and three-dimensional Gaussian representation parameters corresponding to each pixel of the single-viewpoint input image are output pixel by pixel; the three-dimensional Gaussian representation parameters are mapped to the attributes of the corresponding three-dimensional Gaussian ellipsoids, and the three-dimensional Gaussian ellipsoids are collected to form a set of Gaussian ellipsoids. The viewpoint number of each sub-pixel in the target light field image is determined based on the display parameters to obtain a sub-pixel viewpoint index matrix. The viewpoint number is used to indicate the viewpoint that the sub-pixel of the target light field image should adopt. Based on the viewpoint number, multiple virtual viewpoints are generated under the off-axis camera model, and constraints are applied to the multiple virtual viewpoints so that the observation directions of the multiple virtual viewpoints are all within a preset angle range corresponding to the input viewpoint direction of the single viewpoint input image. The rendering block size is adaptively determined based on the number of viewpoints and the target light field image is divided into several rendering blocks, so that each rendering block contains only the effective sub-pixels that participate in the final light field rendering. Based on each of the effective sub-pixels, a sub-pixel encoding identifier is generated, which includes the viewpoint number, the rendering block identifier, and the row and column positions of the sub-pixel. The sub-pixel encoding identifiers are sorted according to the viewpoint number and the rendering block identifier to obtain a sub-pixel sorting sequence. Based on the Gaussian ellipsoid set, Gaussian sputtering rendering is performed on the sub-pixel sorting sequence. During rendering, the rendering block is used as a parallel processing unit to perform viewpoint-related projection, depth sorting, and opacity-based blending accumulation on the Gaussian ellipsoid set in parallel on the graphics processor. The target light field image is obtained directly without rendering the complete viewpoint image. The frame buffer data of the display panel is generated based on the target light field image.

2. The display method for viewpoint fitting encoding of a 360-degree light field LED light pole screen advertising machine according to claim 1, characterized in that, When obtaining the sub-pixel viewpoint index matrix, the following steps are included: The vertical distance from the preset observation position to the grating plane, the spacing between the display panel and the grating, and the cross-sectional width of the grating unit are obtained from the display parameters. Based on the tilt angle of the grating relative to the vertical direction, the vertical distance from the observation position to the grating plane, the spacing between the display panel and the grating, and the cross-sectional width of the grating unit, the width of the sub-pixel covered by the grating unit in the horizontal direction is calculated. Calculate the horizontal distance of the sub-pixel relative to the left boundary of the leftmost grating unit, and obtain the relative offset of the sub-pixel within the period of the grating unit by taking the remainder of the horizontal distance with respect to the width of the sub-pixel covered by the grating unit in the horizontal direction; The width of the sub-pixels covered by the grating unit in the horizontal direction is divided equally according to the number of viewpoints to obtain the sub-interval width; the relative offset is assigned to an interval based on the sub-interval width, and the ratio of the relative offset to the sub-interval width is rounded up to obtain the viewpoint number, thereby forming the sub-pixel viewpoint index matrix.

3. The display method for viewpoint fitting encoding of a 360-degree light field LED light pole screen advertising machine according to claim 1, characterized in that, The single-viewpoint reconstruction network outputs a set of parameters bound to each pixel of the single-viewpoint input image, and maps the parameter set to the attributes of the corresponding Gaussian ellipsoid in the Gaussian ellipsoid set. The attributes of the Gaussian ellipsoid include at least: opacity for representing transparency or occlusion relationship, three-dimensional position offset and depth for inferring three-dimensional position from pixel coordinates, scale and rotation for describing the shape of the Gaussian ellipsoid, and color for describing appearance. The single-viewpoint reconstruction network encodes the parameter set to generate the Gaussian ellipsoid set during inference.

4. The display method for viewpoint fitting encoding of a 360-degree light field LED light pole screen advertising machine according to claim 1, characterized in that, When performing Gaussian sputtering rendering, the following are included: For each Gaussian ellipsoid in the set of Gaussian ellipsoids, determine the three-dimensional position mean vector, scale vector, rotation parameter, color parameter, and opacity; A scale matrix is ​​constructed based on the scale vector, and a rotation matrix is ​​constructed based on the rotation parameters. The three-dimensional covariance characterization of the Gaussian ellipsoid is obtained using the scale matrix and the rotation matrix. For the virtual viewpoint, a local linear approximation from three-dimensional space to two-dimensional imaging plane is determined based on view transformation and projection transformation, and the three-dimensional covariance representation is projected into a two-dimensional covariance representation accordingly to obtain the influence range of the projection ellipse of the Gaussian ellipsoid on the two-dimensional imaging plane. When performing color synthesis on the same sub-pixel, multiple Gaussian ellipsoids affecting the sub-pixel are sorted according to the depth order associated with the virtual viewpoint, and an opacity-based blending accumulation is performed in a manner that gradually decreases the forward transmittance. The transparency state is updated during the blending process until the sub-pixel colors of each rendering block are output and written to the frame buffer, thus obtaining the target light field image.

5. The display method for viewpoint fitting encoding of a 360-degree light field LED light pole screen advertising machine according to claim 1, characterized in that, When dividing rendering blocks, the specific steps include: A rasterization method based on rendering blocks is adopted to divide the imaging plane into multiple rendering blocks and bind the set of Gaussian ellipsoids to the rendering blocks; When the number of viewpoints is less than a threshold, a preset base rendering block size is used. When the number of viewpoints increases, the rendering block size is increased. The determination of the rendering block size is based at least on the number of viewpoints and the viewpoint distribution, so that the number of effective sub-pixels contained in each rendering block can match the number of threads or scheduling strategy of the graphics processor's parallel thread block.

6. The display method for viewpoint fitting encoding of a 360-degree light field LED light pole screen advertising machine according to claim 1, characterized in that, When obtaining the sub-pixel sorting sequence, the following is included: Based on the sub-pixel viewpoint index matrix, each valid sub-pixel of the target light field image is assigned a viewpoint number and a sub-pixel sorting index matrix is ​​generated. The sub-pixel sorting index matrix is ​​used to record the correspondence between each valid sub-pixel and its sub-pixel position in the target light field image, and is used to determine the linear index or write address corresponding to each valid sub-pixel. After dividing the target light field image into rendering blocks, the rendering block identifier is determined based on the rendering block where the effective sub-pixel is located, and the row and column positions of the sub-pixel are determined based on the row and column numbers of the effective sub-pixel in the target light field image. Each valid subpixel is encoded as a subpixel encoding identifier that includes the viewpoint number, the rendering block identifier, the column number, and the row number; The sub-pixel encoding identifiers are sorted according to the viewpoint number and the rendering block identifier, so that sub-pixels with the same viewpoint number are continuous in the data structure and sub-pixels with different viewpoint numbers and different rendering block identifiers are evenly distributed in the data structure. The sorted results are saved as a one-dimensional sequence arranged in the order of the viewpoint number and the rendering block identifier; The one-dimensional sequence is interpolated or padded to fill the thread block capacity corresponding to each rendering block identifier under each viewpoint number, and the row and column positions of the padded pseudo-subpixels are marked as invalid to indicate that the pseudo-subpixels do not exist in the actual light field rendering, so that the rendering blocks can be rendered independently and in parallel in a single thread block; wherein, after the color calculation of the effective subpixels is completed, the output color of the effective subpixels is written to the corresponding subpixel position of the target light field image according to the linear index or write address to form the target light field image.

7. The display method for viewpoint fitting encoding of a 360-degree light field LED light pole screen advertising machine according to claim 1, characterized in that, When performing Gaussian sputtering rendering, it also includes: Within the same rendering block, the effective sub-pixels are prioritized based on their contribution to the final light field rendering. The contribution is determined based on whether the effective sub-pixel is necessary for the light field rendering, the position of the effective sub-pixel within the rendering block, and the sampling density related to the viewpoint number. When the graphics processor performs parallel rendering, threads are allocated to high-priority effective sub-pixels first, and their sub-pixel color calculations are completed in advance, while the calculation frequency of low-priority effective sub-pixels is reduced or their processing is delayed.

8. A display system for viewpoint fitting and encoding of a 360-degree light field LED light pole screen advertising machine, characterized in that, include: The parameter acquisition module is used to acquire a single viewpoint input image and acquire display parameters corresponding to the display panel. The display parameters include the number of viewpoints, the tilt angle of the grating relative to the vertical direction, the width of the sub-pixel covered by the grating unit in the horizontal direction, and the distance between the display panel and the grating. The 3D Gaussian reconstruction module is used to input the single-viewpoint input image into a single-viewpoint reconstruction network based on the U-Net architecture, output 3D Gaussian representation parameters that correspond one-to-one with each pixel of the single-viewpoint input image, map the 3D Gaussian representation parameters to the attributes of the corresponding 3D Gaussian ellipsoids, and collect the 3D Gaussian ellipsoids to form a Gaussian ellipsoid set. The viewpoint index generation module is used to determine the viewpoint number of each sub-pixel in the target light field image based on the display parameters, and obtain a sub-pixel viewpoint index matrix. The viewpoint number is used to indicate the viewpoint that the sub-pixel of the target light field image should adopt. The virtual viewpoint generation and constraint module is used to generate multiple virtual viewpoints under the off-axis camera model based on the viewpoint number, and to apply constraints to the multiple virtual viewpoints so that the observation directions of the multiple virtual viewpoints are all within a preset angle range corresponding to the input viewpoint direction of the single viewpoint input image. The rendering block partitioning module is used to adaptively determine the rendering block size according to the number of viewpoints and divide the target light field image into several rendering blocks, so that each rendering block contains only the effective sub-pixels that participate in the final light field rendering. The encoding and layout module is used to generate a sub-pixel encoding identifier based on each of the effective sub-pixels, including the viewpoint number, the rendering block identifier, and the row and column positions of the sub-pixels, and to sort the sub-pixel encoding identifiers according to the viewpoint number and the rendering block identifier to obtain a sub-pixel sorting sequence; The rendering module is used to perform Gaussian sputtering rendering on the sub-pixel sorting sequence based on the Gaussian ellipsoid set. During rendering, the rendering block is used as a parallel processing unit to perform viewpoint-related projection, depth sorting, and opacity-based blending accumulation on the Gaussian ellipsoid set in parallel on the graphics processor. The target light field image is obtained directly without rendering the complete viewpoint image, and the frame buffer data of the display panel is generated based on the target light field image.

Images