Method of rendering and apparatus performance the same

Abstract

A rendering method is disclosed. A rendering method according to one embodiment may include the operation of obtaining a 2D Gaussian splat from a 3D Gaussian splat, the operation of generating a proxy polygon having the same center coordinates as the 2D Gaussian splat, the operation of storing the color value of the 2D Gaussian splat in a Gaussian texture, and the operation of rendering the 3D Gaussian splat to the proxy polygon by applying the Gaussian texture.

Description

Technology Field

[0001] The following disclosure relates to a method of rendering and an apparatus for performing the same. Background Technology

[0003] To render a Gaussian splat, the fragment shader must calculate an exponential function for each pixel. Additionally, an environment capable of using fragment shaders is required to render the Gaussian splat.

[0004] A 3D Gaussian distribution or Gaussian function can be used as a primitive to represent volume information in 3D space. Using a 3D Gaussian distribution or Gaussian function, 3D space can be reconstructed and images can be synthesized.

[0005] The background technology described above is possessed or acquired by the inventor in the process of deriving the content of the disclosure of the present application, and cannot necessarily be considered as prior art disclosed to the general public prior to the filing of this application. The problem to be solved

[0007] One embodiment may use a Gaussian texture to render a 3D Gaussian splat.

[0008] One embodiment may use a Gaussian function to generate a Gaussian texture.

[0009] One embodiment can replace the operation of a Gaussian function by rendering a 3D Gaussian splat using a Gaussian texture.

[0010] However, technical challenges are not limited to the technical challenges described above, and other technical challenges may exist. means of solving the problem

[0012] A rendering method according to one embodiment may include the operation of obtaining a 2D Gaussian splat from a 3D Gaussian splat, the operation of creating a proxy polygon having the same center coordinates as the 2D Gaussian splat, the operation of storing the color value of the 2D Gaussian splat in a Gaussian texture, and the operation of rendering the 3D Gaussian splat to the proxy polygon by applying the Gaussian texture.

[0013] The above method further includes the operation of generating the Gaussian texture using a Gaussian function, wherein the Gaussian function is defined based on a Gaussian distribution with a first coordinate as the center coordinate, and the range of the two-dimensional coordinates of the Gaussian function may be limited to values ​​between 0 and 1.

[0014] The above Gaussian distribution may have the same standard deviation value on the x-axis and the standard deviation value on the y-axis of the above Gaussian distribution.

[0015] The operation of generating the above Gaussian texture may include the operation of determining the resolution of the above Gaussian texture and the operation of calculating the Gaussian function value for each pixel of the above Gaussian texture.

[0016] The operation of determining the resolution of the above Gaussian texture may include the operation of determining the resolution of the above Gaussian texture based on the above 2D Gaussian splat.

[0017] The operation of generating the proxy polygon may further include the operation of calculating the eigenvector of the 2D Gaussian splat and the operation of generating the proxy polygon having the same eigenvector as the eigenvector.

[0018] The color value of the above 2D Gaussian splat may be the value of a spherical harmonics function calculated for the above 2D Gaussian splat.

[0019] A rendering device according to one embodiment includes a processor and a memory for storing instructions, and when the instructions are executed by the processor, the device may obtain a 2D Gaussian splat from a 3D Gaussian splat, create a proxy polygon with the same center coordinates as the 2D Gaussian splat, store the color values ​​of the 2D Gaussian splat in a Gaussian texture, and apply the Gaussian texture to render the 3D Gaussian splat to the proxy polygon.

[0020] The above instructions may generate the Gaussian texture using a Gaussian function, and the Gaussian function may be defined based on a Gaussian distribution with a first coordinate as the center coordinate, and the range of the two-dimensional coordinates of the Gaussian function may be limited to values ​​between 0 and 1.

[0021] The above Gaussian distribution may have the same standard deviation value on the x-axis and the standard deviation value on the y-axis of the above Gaussian distribution.

[0022] The above instructions may determine the resolution of the Gaussian texture and calculate the Gaussian function value for each pixel of the Gaussian texture.

[0023] The above instructions may be used to determine the resolution of the Gaussian texture based on the 2D Gaussian splat.

[0024] The above instructions may be used to calculate the eigenvectors of the 2D Gaussian splat and to generate proxy polygons having eigenvectors identical to the eigenvectors.

[0025] The color value of the above 2D Gaussian splat may be the value of a spherical harmonics function calculated for the above 2D Gaussian splat. Brief explanation of the drawing

[0027] FIG. 1 is a drawing for explaining a rendering system according to one embodiment. FIG. 2 is a drawing for explaining a Gaussian texture according to one embodiment. FIG. 3 is a diagram illustrating a proxy polygon according to one embodiment. FIG. 4 is a flowchart illustrating a rendering method according to one embodiment. FIG. 5 is a flowchart illustrating a method for a rendering device to generate a Gaussian texture according to one embodiment. FIG. 6 is a schematic block diagram of an electronic device according to one embodiment. Specific details for implementing the invention

[0028] Specific structural or functional descriptions of the embodiments are disclosed for illustrative purposes only and may be modified and implemented in various forms. Accordingly, actual implementations are not limited to the specific embodiments disclosed, and the scope of this specification includes modifications, equivalents, or substitutions included in the technical concept described by the embodiments.

[0029] Terms such as "first" or "second" may be used to describe various components, but these terms should be interpreted solely for the purpose of distinguishing one component from another. For example, the first component may be named the second component, and similarly, the second component may be named the first component.

[0030] When it is stated that a component is "connected" to another component, it should be understood that it may be directly connected to or coupled with that other component, or that there may be other components in between.

[0031] Singular expressions include plural expressions unless the context clearly indicates otherwise. In this document, phrases such as “A or B,” “at least one of A and B,” “at least one of A or B,” “A, B or C,” “at least one of A, B and C,” and “at least one of A, B, or C” may each include any one of the items listed together with the corresponding phrase, or all possible combinations thereof. In this specification, terms such as “comprising” or “having” are intended to designate the existence of the described feature, number, step, action, component, part, or combination thereof, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0032] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as generally understood by those skilled in the art. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology, and should not be interpreted in an ideal or overly formal sense unless explicitly defined in this specification.

[0033] Hereinafter, embodiments will be described in detail with reference to the attached drawings. In the description with reference to the attached drawings, identical components are given the same reference numeral regardless of the drawing number, and redundant descriptions thereof will be omitted.

[0035] FIG. 1 is a drawing for explaining a rendering system according to one embodiment.

[0036] Referring to FIG. 1, according to one embodiment, the system (100) may include a rendering device (110) and a display (130).

[0037] The rendering device (100) can render an image. For example, the rendering device (100) can render a Gaussian splat. The rendering device (100) can transmit the rendered image to a display (130).

[0038] The display (130) can obtain a rendered image from the rendering device (100). The display (130) can display the rendered image on a screen.

[0039] The rendering device (100) can generate a Gaussian texture using a Gaussian function. The Gaussian function may be defined based on a Gaussian distribution.

[0041] [Mathematical Formula 1]

[0043]

[0045] The Gaussian distribution can be a multivariate normal distribution. The Gaussian distribution can be a one-dimensional probability distribution. A one-dimensional probability distribution can be represented by Equation 1. The Gaussian distribution can retain the characteristic of following a Gaussian distribution even after applying an affine transform. The Gaussian distribution When applying an affine transformation of It can be.

[0047] [Mathematical Formula 2]

[0049]

[0051] The multivariate normal distribution can be extended to two dimensions or more. A multivariate normal distribution of two dimensions or more is the covariance matrix and center location It can be defined through Mathematical Equation 2, which includes . When the multivariate normal distribution is defined in three dimensions, can be a 3-dimensional vector, and can be a 3 x 3 matrix.

[0053] [Mathematical Formula 3]

[0055]

[0057] The Gaussian function can be represented as shown in Equation 3.

[0058] A Gaussian distribution or Gaussian function in three dimensions can be used as a primitive to represent volume information in three-dimensional space.

[0059] Gaussian splat primitives can be converted into 3D Gaussian splats in camera (or view) space or 2D Gaussian splats in screen space. The orientation (rotation value) of the Gaussian splat ) and model-to-world matrix , world-to-view matrix , and projection matrix You can convert Gaussian splat primitives into 2D and / or 3D Gaussian splats using this.

[0060] Transformation matrix It can transform a 3D Gaussian splat from the space of Gaussian splat primitives to camera space (or view space). Transformation matrix silver , , and rotation value It can be calculated in. Perspective projection from view space to screen space can be transformed into a 2D Gaussian distribution using the Jacobian.

[0062] [Mathematical Formula 4]

[0064]

[0066] A 3D Gaussian splat can store color information in the form of spherical harmonics coefficients. Color information in the form of spherical harmonics coefficients can represent directional colors. The color of a 3D Gaussian splat can be calculated using the coefficients of the 3D Gaussian splat and the view direction. The spherical harmonics coefficients Let it be, and the normal vector of the pixel to be rendered If so, the color can be calculated by a spherical harmonic formula such as mathematical formula 4.

[0068] FIG. 2 is a drawing for explaining a Gaussian texture according to one embodiment.

[0069] Referring to FIG. 2, according to one embodiment, a Gaussian texture (200) may be required to render a 3D Gaussian splat. The Gaussian texture (200) may be used to replace the operation of a Gaussian function during the rendering process of the 3D Gaussian splat.

[0071] [Mathematical Formula 5]

[0073]

[0075] A Gaussian texture (200) can be generated using a Gaussian function. The Gaussian function can be defined based on a Gaussian distribution. If the Gaussian function is defined in the range [0, 1], the center of the Gaussian distribution can be set to the first coordinate. For example, the center of the Gaussian distribution can be set to (0.5, 0.5). The values ​​of the standard deviation of the x-axis and the standard deviation of the y-axis of the Gaussian distribution can be the same.

[0076] The range of the 2D coordinates of a Gaussian function can be limited to values ​​between 0 and 1. For example, a Gaussian function can be defined in the range where any coordinate (u, v) is [0, 1]. The standard deviation value at any coordinate (u, v) of the Gaussian function is equal in the u and v directions. When set to , the Gaussian function can be defined as in Equation 5.

[0077] The Gaussian texture (200) can be created as a grayscale image using an 8-bit unsigned integer type or a 32-bit floating-point type. The Gaussian texture (200) can be created by calculating a Gaussian function pixel by pixel.

[0079] [Mathematical Formula 6]

[0081]

[0083] [Mathematical Formula 7]

[0085]

[0087] [Mathematical Formula 8]

[0089]

[0091] A Gaussian function may have a transformation formula. When mapping a Gaussian function to a rectangular area centered at the origin with a width of w and a height of h, the transformation formula of the Gaussian function can be defined as Equation 6. The inverse transformation formula of the Gaussian function can be defined as Equation 7. When mapping a Gaussian texture (200) generated based on the transformation and inverse transformation formulas of the Gaussian function to a rectangular area centered at the origin with a width of w and a height of h, the value of the Gaussian texture (200) mapped to the (x, y) coordinate can be defined as Equation 8. Equation 8 is the standard deviation value It may be a Gaussian function. When mapping a Gaussian texture (200) to an arbitrary rectangular area, a Gaussian distribution can be created that has standard deviation values ​​according to the width and height of the rectangle being mapped.

[0093] [Mathematical Formula 9]

[0095]

[0097] [Mathematical Formula 10]

[0099]

[0101] The result of calculating the values ​​on the x-axis in mathematical formula 8 can be expressed as mathematical formula 5.

[0102] The proxy polygon may be a rectangle on which a Gaussian texture (200) is mapped. At the edges of the proxy polygon (e.g., the proxy polygon (300) in FIG. 3), the value of the Gaussian function It can be calculated when.

[0103] In mathematical formula 9 When substituting , Equation 9 can be expressed as Equation 10. In Equation 10 When the value of is 1 / 2, 1 / 3, or 1 / 4, the result of Equation 10 can be 0.135, 0.111, and 0.000335, respectively. When representing the screen's RGB values ​​with 8 bits, the smallest RGB value can be 1 / 256 = 0.00391. In the case where α is 1 / 4, the value of Equation 10 is smaller than the smallest RGB value, so it can be ignored on the screen. Therefore, of the proxy polygon to render the 3D Gaussian splat If you set it to 1 / 4, you can prevent a decrease in rendering quality.

[0105] [Mathematical Formula 11]

[0107]

[0109] The texel (texture pixel) value at any coordinate (u, v) of the Gaussian texture (200) can be calculated using Equation 11.

[0110] The Gaussian texture (200) may have a resolution. The resolution of the Gaussian texture (200) may exhibit aliasing. Aliasing may refer to a phenomenon in which staircasing, distortion, and / or unnecessary patterns occur during the processing of digital images, etc. The resolution of the Gaussian texture (200) may be determined based on a 2D Gaussian splat. A 2D Gaussian splat may be used to reduce the aliasing of the resolution of the Gaussian texture (200). For example, the horizontal and vertical lengths of the 2D Gaussian splat may be estimated, and the resolution of the Gaussian texture (200) may be determined by doubling the estimated horizontal and vertical lengths, respectively. A mipmap function may be used to reduce the aliasing of the resolution of the Gaussian texture (200).

[0111] The Gaussian texture (200) can be applied equally to all Gaussians. Only one Gaussian texture (200) can be created and used in the rendering system (100).

[0112] The center coordinates (or center point) of a 3D Gaussian splat are and It can be calculated in view space using matrices. The center coordinates of a 3D Gaussian splat are It can be written as.

[0113] 2D covariance matrix of a 3D Gaussian splat is the covariance matrix in projection space Calculate, It can be obtained by extracting the 2 x 2 matrix at the top left of.

[0115] FIG. 3 is a diagram illustrating a proxy polygon according to one embodiment.

[0116] Referring to FIG. 3, according to one embodiment, a proxy polygon (300) may be required to apply a Gaussian texture (200).

[0117] Proxy polygons (300) may be necessary to render a 3D Gaussian splat on the screen. Proxy polygons (300) can be created as quad polygons in the shape of rectangles formed by combining two triangles.

[0118] A 2D Gaussian splat can be obtained based on the on-screen center coordinates, orientation, and size of a 3D Gaussian splat. A 2D Gaussian splat can be obtained by projecting the 3D Gaussian splat onto the screen.

[0119] The center of the proxy polygon (300) may be the same as the center of the 2D Gaussian splat. The orientation of the proxy polygon (300) may be set parallel to the two axes of the 2D Gaussian splat.

[0120] Proxy polygons can have eigenvectors identical to the eigenvectors of a 2D Gaussian splat.

[0121] The axes of the 2D Gaussian splat are the covariance matrix It can be obtained using the two eigenvectors of. Two eigenvalues ​​corresponding to each of the two eigenvectors of The standard deviation of this Gaussian function It can be.

[0123] [Mathematical Formula 12]

[0125]

[0127] [Mathematical Formula 13]

[0129]

[0131] Matrix of 2D Gaussian Splat It can be expressed as in mathematical formula 12. Eigenvalues ​​of a 2D Gaussian splat represented by a matrix like this It can be calculated using Equation 13. The eigenvectors (x, y) of a 2D Gaussian splat are It can satisfy. The slope of the axis of a 2D Gaussian splat is It can be obtained as.

[0133] [Mathematical Formula 14]

[0135]

[0137] The x-axis and y-axis of the proxy polygon (300) may be parallel to the axes of the 2D Gaussian splat. The center coordinates of the proxy polygon (300) may be the origin. A Gaussian function calculated in a coordinate system in which the center coordinates of the proxy polygon (300) are the origin and the x-axis and y-axis directions are parallel to the two sides of the proxy polygon (300) may be defined by Equation 14.

[0138] When mapping a Gaussian texture (200) to a rectangular area with a width of w and a height of h, the standard deviation values ​​in the x-axis and y-axis directions at When 1 / 4, the horizontal and vertical lengths of the proxy polygon (300) By doing so, you can map the Gaussian function to the exact location.

[0139] The horizontal length of the proxy polygon (300) is , the vertical length is It can be. The direction of the proxy polygon (300) is in the horizontal direction. person It can become a square rotated counterclockwise by that amount.

[0140] A 2D Gaussian splat can have a shape that is symmetrical in all directions. Any of the vertices of a 2D Gaussian splat can be the origin of a Gaussian texture.

[0141] The color of a 2D Gaussian splat can be defined by a spherical harmonics function. A spherical harmonics function can be defined by 16 coefficients. To calculate the spherical harmonics function value, 16 equations may need to be calculated for each pixel. The spherical harmonics function value of a 2D Gaussian splat can be pre-calculated and stored in a Gaussian texture (200). In this case, by using a texture lookup, the color value of the 2D Gaussian splat can be obtained without calculating the equation for each pixel.

[0143] FIG. 4 is a flowchart illustrating a rendering method according to one embodiment.

[0144] Referring to FIG. 4, according to one embodiment, operations 410 to 490 may be operations performed by the rendering device (110) described with reference to FIG. 1 to 3.

[0145] According to one embodiment, operations 410 to 490 may be understood to be performed in a processor (e.g., processor (630) of FIG. 6) of a rendering device (110) described with reference to FIG. 1 (e.g., electronic device (600) of FIG. 6).

[0146] In operation 410, the rendering device (110) can obtain a 2D Gaussian splat from a 3D Gaussian splat.

[0147] In operation 430, the rendering device (110) can generate a Gaussian texture using a Gaussian function.

[0148] In operation 450, the rendering device (110) can generate a proxy polygon with the same center coordinates as the 2D Gaussian splat.

[0149] In operation 470, the rendering device (110) can store the color values ​​of the 2D Gaussian splat in a Gaussian texture.

[0150] In operation 490, the rendering device (110) can render a 3D Gaussian splat to a proxy polygon by applying a Gaussian texture.

[0151] Operations 410 through 490 may be performed sequentially, but are not limited thereto. For example, two or more operations may be performed in parallel.

[0153] FIG. 5 is a flowchart illustrating a method for a rendering device to generate a Gaussian texture according to one embodiment.

[0154] Referring to FIG. 5, according to one embodiment, operations 510 to 590 may be operations performed by the rendering device (110) described with reference to FIG. 1 to 4.

[0155] In operation 510, the rendering device (110) can generate a Gaussian texture (200) using a Gaussian function.

[0156] In operation 530, the rendering device (110) can calculate the on-screen center coordinates, orientation, and size of the 3D Gaussian splat.

[0157] In operation 550, the rendering device (110) can obtain a 2D Gaussian splat based on the on-screen center coordinates, orientation, and size of the 3D Gaussian splat.

[0158] In operation 570, the rendering device (110) can calculate the color values ​​of a 2D Gaussian splat using a spherical harmonics function.

[0159] In operation 590, the rendering device (100) can store the color values ​​of the 2D Gaussian splat in a Gaussian texture.

[0161] FIG. 6 is a schematic block diagram of an electronic device according to one embodiment.

[0162] Referring to FIG. 6, according to one embodiment, an electronic device (600) (e.g., a rendering device (110) of FIG. 1) may include a memory (610) and a processor (630).

[0163] The memory (610) can store instructions (or programs) executable by the processor (630). For example, the instructions may include instructions for executing the operation of the processor (630) and / or the operation of each component of the processor (630).

[0164] The memory (610) may include one or more computer-readable storage media. The memory (610) may include non-volatile storage devices (e.g., magnetic hard disc, optical disc, floppy disc, flash memory, EPROM (electrically programmable memories), EEPROM (electrically erasable and programmable)).

[0165] The memory (610) may be a non-transitory medium. The term "non-transitory" may indicate that the storage medium is not implemented by a carrier wave or a propagated signal. However, the term "non-transitory" should not be interpreted as meaning that the memory (610) is immobile.

[0166] The processor (630) can process data stored in memory (610). The processor (630) can execute computer-readable code (e.g., software) stored in memory (610) and instructions triggered by the processor (630).

[0167] The processor (630) may be a data processing device implemented in hardware having a circuit having a physical structure for executing desired operations. For example, the desired operations may include code or instructions included in a program.

[0168] For example, a data processing device implemented in hardware may include a microprocessor, a central processing unit, a processor core, a multi-core processor, a multiprocessor, an Application-Specific Integrated Circuit (ASIC), and a Field Programmable Gate Array (FPGA).

[0169] The processor (630) can cause the electronic device (600) to perform one or more operations by executing code and / or instructions stored in memory (610). The operations performed by the electronic device (600) may be substantially the same as the operations performed by the rendering device (100) described with reference to FIGS. 1 through 6. Such redundant descriptions are omitted.

[0171] The embodiments described above may be implemented as hardware components, software components, and / or combinations of hardware and software components. For example, the devices, methods, and components described in the embodiments may be implemented using a general-purpose computer or a special-purpose computer, such as, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. The processing unit may execute an operating system (OS) and software applications executed on said operating system. Additionally, the processing unit may access, store, manipulate, process, and generate data in response to the execution of the software. For ease of understanding, the processing unit may be described as being used as a single unit, but those skilled in the art will understand that the processing unit may include multiple processing elements and / or multiple types of processing elements. For example, the processing unit may include multiple processors or one processor and one controller. In addition, other processing configurations, such as parallel processors, are also possible.

[0172] Software may include computer programs, code, instructions, or a combination of one or more of these, and may configure a processing unit to operate as desired or instruct the processing unit independently or collectively. Software and / or data may be stored on any type of machine, component, physical device, virtual equipment, computer storage medium, or device so as to be interpreted by the processing unit or to provide instructions or data to the processing unit. Software may be distributed over networked computer systems and stored or executed in a distributed manner. Software and data may be stored on computer-readable recording media.

[0173] The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may store program instructions, data files, data structures, etc., either individually or in combination, and the program instructions recorded on the medium may be those specifically designed and configured for the embodiment or those known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, and flash memory. Examples of program instructions include machine code, such as that generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc.

[0174] The hardware device described above may be configured to operate as one or more software modules to perform the operation of the embodiment, and vice versa.

[0175] Although the embodiments have been described above with reference to the limited drawings, those skilled in the art can apply various technical modifications and variations based thereon. For example, suitable results may be achieved even if the described techniques are performed in a different order than described, and / or if the components of the described system, structure, device, circuit, etc. are combined or assembled in a form different from described, or replaced or substituted by other components or equivalents.

[0176] Therefore, other implementations, other embodiments, and equivalents to the claims also fall within the scope of the claims set forth below.

Claims

Claim 1 A method for rendering, comprising: obtaining a 2D Gaussian splat from a 3D Gaussian splat; generating a proxy polygon having the same center coordinates as the 2D Gaussian splat; storing the color values ​​of the 2D Gaussian splat in a Gaussian texture; and rendering the 3D Gaussian splat to the proxy polygon by applying the Gaussian texture.

Images