Three-dimensional data processing method and device, image rendering method and device, equipment and medium

Abstract

The invention provides a three-dimensional data processing method, relates to the technical field of artificial intelligence, in particular to the technical fields of computer vision, deep learning, large models, augmented reality and the like, and can be applied to scenes of three-dimensional reconstruction and the like. According to the specific implementation scheme, according to attribute information of a plurality of three-dimensional Gaussian sputtering units of a three-dimensional Gaussian sputtering model to be processed, a plurality of first covariance matrixes used for the plurality of three-dimensional Gaussian sputtering units are determined, and the first covariance matrixes comprise a plurality of pieces of first initial matrix data; obtaining a plurality of first processed matrix data of the plurality of first covariance matrixes according to the plurality of first initial matrix data of the plurality of first covariance matrixes; and obtaining a first multi-channel image for the to-be-processed three-dimensional Gaussian sputtering model according to a plurality of pieces of first processed matrix data of the plurality of first covariance matrixes. The invention further provides an image rendering method and device, electronic equipment and a storage medium.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technologies, and particularly to the fields of computer vision, deep learning, large models, augmented reality, etc., and can be applied to scenarios such as three-dimensional reconstruction. More specifically, the present disclosure provides a three-dimensional data processing method, an image rendering method, a device, an electronic device, and a storage medium. Background Art

[0002] With the development of artificial intelligence technologies, the application of three-dimensional rendering technologies has become increasingly widespread. As a technology for object reconstruction, visualization, and rendering, 3D Gaussian Splatting (3DGS) can efficiently process complex volume data, improve the quality of rendering results, and is easy to integrate into the rendering pipeline of modern engines, and has broad application prospects in fields such as medicine, science, games, and industrial design. Summary of the Invention

[0003] The present disclosure provides a three-dimensional data processing method, an image rendering method, a device, a device, and a storage medium.

[0004] According to one aspect of the present disclosure, there is provided a three-dimensional data processing method, the method including: determining, according to the respective attribute information of a plurality of three-dimensional Gaussian splatting units of a three-dimensional Gaussian splatting model to be processed, a plurality of first covariance matrices for the plurality of three-dimensional Gaussian splatting units, wherein the first covariance matrix includes a plurality of first initial matrix data; obtaining, according to the plurality of first initial matrix data of each of the plurality of first covariance matrices, a plurality of first processed matrix data of each of the plurality of first covariance matrices; and obtaining, according to the plurality of first processed matrix data of each of the plurality of first covariance matrices, a first multi-channel image for the three-dimensional Gaussian splatting model to be processed, wherein the first multi-channel image includes a plurality of first pixels, and the plurality of first pixel channel values of the first pixel are determined according to the plurality of first processed matrix data of the first covariance matrix.

[0005] According to one aspect of the present disclosure, there is provided an image rendering method, the method comprising: obtaining a target image of a three-dimensional Gaussian sputtering model to be processed according to at least one of a plurality of multi-channel images for the three-dimensional Gaussian sputtering model to be processed; displaying the target image on a visual interface, wherein the three-dimensional Gaussian sputtering model to be processed includes a plurality of three-dimensional Gaussian sputtering units, the plurality of first multi-channel images include a first multi-channel image, the first multi-channel image is obtained according to a plurality of first processed matrix data of a plurality of first covariance matrices for the plurality of three-dimensional Gaussian sputtering units, the first multi-channel image includes a plurality of first pixels, the plurality of first pixel channel values of the first pixels are determined according to the plurality of first processed matrix data of the first covariance matrix, the plurality of first processed matrix data of the plurality of first covariance matrices are obtained according to the plurality of first initial matrix data of the plurality of first covariance matrices, and the plurality of first covariance matrices are obtained according to the attribute information of the plurality of three-dimensional Gaussian sputtering units respectively.

[0006] According to another aspect of the present disclosure, there is provided a three-dimensional data processing device, the device comprising: a first determination module, configured to determine a plurality of first covariance matrices for a plurality of three-dimensional Gaussian sputtering units according to the attribute information of the plurality of three-dimensional Gaussian sputtering units of the three-dimensional Gaussian sputtering model to be processed, wherein the first covariance matrix includes a plurality of first initial matrix data; a first obtaining module, configured to obtain a plurality of first processed matrix data of the plurality of first covariance matrices according to the plurality of first initial matrix data of the plurality of first covariance matrices; a second obtaining module, configured to obtain a first multi-channel image for the three-dimensional Gaussian sputtering model to be processed according to the plurality of first processed matrix data of the plurality of first covariance matrices, wherein the first multi-channel image includes a plurality of first pixels, and the plurality of first pixel channel values of the first pixels are determined according to the plurality of first processed matrix data of the first covariance matrix.

[0007] According to another aspect of the present disclosure, there is provided an image rendering apparatus, comprising: a rendering module configured to obtain a target image of a three-dimensional Gaussian sputtering model to be processed according to at least one of a plurality of multi-channel images for the three-dimensional Gaussian sputtering model to be processed; a display module configured to display the target image on a visual interface, wherein the three-dimensional Gaussian sputtering model to be processed includes a plurality of three-dimensional Gaussian sputtering units, the plurality of first multi-channel images include a first multi-channel image, the first multi-channel image is obtained according to a plurality of first processed matrix data of a plurality of first covariance matrices for the plurality of three-dimensional Gaussian sputtering units, the first multi-channel image includes a plurality of first pixels, the plurality of first pixel channel values of the first pixels are determined according to the plurality of first processed matrix data of the first covariance matrix, the plurality of first processed matrix data of the plurality of first covariance matrices are obtained according to the plurality of first initial matrix data of the plurality of first covariance matrices, and the plurality of first covariance matrices are obtained according to the attribute information of the plurality of three-dimensional Gaussian sputtering units respectively.

[0008] According to another aspect of the present disclosure, there is provided an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method provided by the present disclosure.

[0009] According to another aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions for causing a computer to execute the method provided by the present disclosure.

[0010] According to another aspect of the present disclosure, there is provided a computer program product, comprising a computer program which, when executed by a processor, implements the method provided by the present disclosure.

[0011] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The drawings are used to better understand the solution and do not constitute a limitation to the present disclosure. Among them:

[0013] Figure 1 is a schematic diagram of an exemplary system architecture to which a three-dimensional data processing method and apparatus according to an embodiment of the present disclosure can be applied;

[0014] Figure 2 is a flowchart of a three-dimensional data processing method according to an embodiment of the present disclosure;

[0015] Figure 3A Schematic diagram of a first multi-channel image according to an embodiment of the present disclosure;

[0016] Figure 3B Schematic diagram of a second multi-channel image according to an embodiment of the present disclosure;

[0017] Figure 4 Schematic flowchart of an image rendering method according to an embodiment of the present disclosure;

[0018] Figure 5 Schematic diagram of a target image according to an embodiment of the present disclosure;

[0019] Figure 6 Block diagram of a three-dimensional data processing device according to an embodiment of the present disclosure;

[0020] Figure 7 Block diagram of an image rendering device according to another embodiment of the present disclosure; and

[0021] Figure 8 Block diagram of an electronic device that can apply the three-dimensional data processing method and / or the image rendering method according to an embodiment of the present disclosure. Detailed implementation manners

[0022] The following describes exemplary embodiments of the present disclosure with reference to the accompanying drawings. Various details of the embodiments of the present disclosure are included to facilitate understanding, and they should be considered merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0023] Three-dimensional Gaussian sputtering can combine the advantages of discrete and continuous representations, can overcome the limitations of three-dimensional reconstruction technology in terms of noise and rendering quality, and can effectively improve the rendering speed. However, when using the differentiable three-dimensional Gaussian sputtering technology, a large amount of storage resources and memory resources are required. For example, a differentiable three-dimensional Gaussian sputtering model can include millions of three-dimensional Gaussian sputtering units, which require a large amount of storage resources and memory resources. The three-dimensional Gaussian sputtering unit can also be referred to as a three-dimensional Gaussian sputtering sheet or a three-dimensional Gaussian element. For example, a three-dimensional Gaussian sputtering unit can be an ellipsoid, and its projection on a two-dimensional plane can be an ellipse.

[0024] The polygon file format (PLY) file that can parse the three-dimensional Gaussian sputtering model can be used to obtain the attribute information of the three-dimensional Gaussian sputtering model. A structured buffer can be set in the video memory of the graphics processing unit (GPU). The attribute information can be stored in this structured buffer. The attribute information can include a rotation attribute matrix and a scaling attribute matrix. During rendering, the covariance matrix required for rendering can be determined according to the rotation attribute matrix and the scaling attribute matrix.

[0025] To implement the rendering of the three-dimensional Gaussian sputtering model, it is necessary to dynamically load and parse the polygon file format file. This file can be parsed in plain code, but it is not a rendering-friendly file format. It is difficult for the graphics processing unit to use this file for efficient rendering and shading.

[0026] In the attribute information of the three-dimensional Gaussian sputtering model, most of the data are spherical harmonic function coefficients. These spherical harmonic function coefficients occupy a large amount of video memory space. When the graphics processing unit obtains the attribute information, it will use a large amount of video memory bandwidth, resulting in a large bandwidth resource overhead. In addition, for rendering, the graphics processing unit also needs to frequently calculate the covariance matrix, resulting in a large computing power overhead of the graphics processing unit, and thus it is difficult to effectively use the hardware resources of the graphics processing unit.

[0027] To fully improve the rendering efficiency of the three-dimensional Gaussian sputtering model, the present disclosure provides a three-dimensional data processing method, which will be described below.

[0028] Figure 1 It is a schematic diagram of an exemplary system architecture to which the three-dimensional data processing method and apparatus according to an embodiment of the present disclosure can be applied. It should be noted that Figure 1 The figure shown is only an example of the system architecture to which the embodiments of the present disclosure can be applied, to help those skilled in the art understand the technical content of the present disclosure, but it does not mean that the embodiments of the present disclosure cannot be used in other devices, systems, environments or scenarios.

[0029] As Figure 1 shown, the system architecture 100 according to this embodiment may include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, etc.

[0030] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Terminal devices 101, 102, and 103 can be various electronic devices with a display screen and supporting web browsing, including but not limited to smartphones, tablets, laptop computers, and desktop computers, etc.

[0031] Server 105 can be a server that provides various services. For example, it can be a background management server (only for example) that supports the websites browsed by users using terminal devices 101, 102, and 103. The background management server can analyze and process data such as user requests received, and feedback the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal devices.

[0032] It should be noted that the three-dimensional data processing method provided by the embodiments of the present disclosure can generally be executed by server 105. Correspondingly, the three-dimensional data processing device provided by the embodiments of the present disclosure can generally be set in server 105. The three-dimensional data processing method provided by the embodiments of the present disclosure can generally be executed by terminal devices 101, 102, and 103. Correspondingly, the three-dimensional data processing device provided by the embodiments of the present disclosure can generally be set in terminal devices 101, 102, and 103. The three-dimensional data processing method provided by the embodiments of the present disclosure can also be executed by a server or a server cluster that is different from server 105 and can communicate with terminal devices 101, 102, 103 and / or server 105. Correspondingly, the three-dimensional data processing device provided by the embodiments of the present disclosure can also be set in a server or a server cluster that is different from server 105 and can communicate with terminal devices 101, 102, 103 and / or server 105.

[0033] It can be understood that the system architecture of the present disclosure has been described above, and the method of the present disclosure will be described below.

[0034] Figure 2 is a flowchart of a three-dimensional data processing method according to an embodiment of the present disclosure.

[0035] As Figure 2 shown, the method 200 may include operation S210 to operation S230.

[0036] In operation S210, according to the respective attribute information of multiple three-dimensional Gaussian sputtering units of the three-dimensional Gaussian sputtering model to be processed, multiple first covariance matrices for the multiple three-dimensional Gaussian sputtering units are determined.

[0037] In the embodiments of the present disclosure, the three-dimensional Gaussian sputtering model to be processed can be constructed based on three-dimensional Gaussian sputtering technology. The three-dimensional Gaussian sputtering model to be processed can include three-dimensional Gaussian sputtering units. The attribute information of the three-dimensional Gaussian sputtering units can include multiple attribute sub-information. The multiple attribute sub-information includes: position sub-information, opacity sub-information, color sub-information, rotation sub-information, and scaling sub-information, etc. For example, the position sub-information can indicate the center (position) μ of the three-dimensional Gaussian sputtering unit. The opacity sub-information can indicate the opacity α of the three-dimensional Gaussian sputtering unit. The rotation sub-information can be implemented as a rotation attribute matrix. The scaling sub-information can be implemented as a scaling attribute matrix. The color sub-information can represent the color c within the three-dimensional Gaussian sputtering unit.

[0038] In the embodiments of the present disclosure, according to the rotation sub-information and the scaling sub-information, the first covariance matrix of the three-dimensional Gaussian sputtering unit can be determined. The first covariance matrix can be a three-dimensional (3D) covariance matrix. The first covariance matrix can be a symmetric matrix and can include multiple first initial matrix data. It can be understood that the first initial matrix data can be a numerical value. Each three-dimensional Gaussian sputtering unit corresponds to a first covariance matrix.

[0039] In operation S220, according to the multiple first initial matrix data of each of the multiple first covariance matrices, multiple first processed matrix data of each of the multiple first covariance matrices are obtained.

[0040] In the embodiments of the present disclosure, data compression can be performed on the multiple first initial matrix data of the first covariance matrix to obtain the first processed matrix data of the first covariance matrix. For example, the number of the first processed matrix data can be less than or equal to the number of the first initial matrix data. Also for example, the data volume of the first processed matrix data can be less than or equal to the data volume of the first initial matrix data.

[0041] In operation S230, according to the multiple first processed matrix data of each of the multiple first covariance matrices, a first multi-channel image for the three-dimensional Gaussian sputtering model to be processed is obtained.

[0042] In the embodiments of the present disclosure, the pixels of the multi-channel image can include multiple pixel channel values. For example, the data type of the pixel channel values can be 32-bit floating-point number (FP32) or 32-bit integer (int32). That is, in the video memory, the capacity of the video memory space for the pixel channel values can be 32 bits (bit).

[0043] In the embodiments of the present disclosure, the first multi-channel image may include a plurality of first pixels. The plurality of first pixel channel values of the first pixels may be determined according to a plurality of first processed matrix data of the first covariance matrix. For example, one first processed matrix data may be used as one first pixel channel value. Another example is that, as described above, the capacity of the video memory space for the pixel channel value may be 32 bits. If the data volume of the first processed matrix data is 16 bits, two first processed matrix data may also be spliced into the first pixel channel value.

[0044] Through the embodiments of the present disclosure, the pixels in the multi-channel image are determined according to one or more attribute sub-information of the three-dimensional Gaussian sputtering unit. Thus, one or more attribute sub-information of the three-dimensional Gaussian sputtering model can be converted into a multi-channel image for storage. The graphics processing unit can be used to efficiently process based on the multi-channel image, and the graphics processing unit can also be used to efficiently render and color based on the multi-channel image. In addition, before rendering, the first covariance matrix is calculated, and the first pixels are obtained according to the data in the first covariance matrix, which can reduce the computational resource overhead of the graphics processing unit during the rendering process and improve the utilization rate of the computational resources of the graphics processing unit.

[0045] It can be understood that the method of the present disclosure has been described above, and the first covariance matrix of the present disclosure will be described below.

[0046] In some embodiments, in some implementation manners of the above operation S210, determining a plurality of first covariance matrices for a plurality of three-dimensional Gaussian sputtering units according to the respective attribute information of the plurality of three-dimensional Gaussian sputtering units of the three-dimensional Gaussian sputtering model to be processed includes: multiplying the rotation attribute matrix, the scaling attribute matrix, the transposed scaling attribute matrix, and the transposed rotation attribute matrix of the three-dimensional Gaussian sputtering unit to obtain the first covariance matrix for the three-dimensional Gaussian sputtering unit. The transposed scaling attribute matrix is obtained by transposing the scaling attribute matrix.

[0047] For example, the first covariance matrix can be determined by the following formula:

[0048] (Formula 1)

[0049] can be the first covariance matrix for a three-dimensional Gaussian sputtering unit. can be the rotation attribute matrix of the three-dimensional Gaussian sputtering unit. can be the scaling attribute matrix of the three-dimensional Gaussian sputtering unit. can be the transposed rotation attribute matrix of the three-dimensional Gaussian sputtering unit. can be the transposed scaling attribute matrix of the three-dimensional Gaussian sputtering unit.

[0050] It can be understood that the method for determining the first covariance matrix has been described above, and the method for obtaining multiple first processed matrix data will be described below.

[0051] In some embodiments, in some implementations of the above operation S220, obtaining multiple first processed matrix data for each of the multiple first covariance matrices based on the multiple first initial matrix data for each of the multiple first covariance matrices includes: determining multiple first to-be-processed matrix data for each of the multiple first covariance matrices from the multiple first initial matrix data for each of the multiple first covariance matrices. For example, the first covariance matrix can be a 3×3 matrix. In one example, the first covariance matrix can be:

[0052] (Equation 2)

[0053] ~ There can be 9 first initial matrix data. The first covariance matrix can be a symmetric matrix. The first initial matrix data is equal to the first initial matrix data . The first initial matrix data is equal to the first initial matrix data . The first initial matrix data is equal to the first initial matrix data . Thus, there are duplicate data in the first covariance matrix. The duplicate data can be removed to obtain multiple first to-be-processed matrix data of the first covariance matrix, which will be further described below.

[0054] In some embodiments, multiple first initial matrix data located on the matrix diagonal of the first covariance matrix and multiple first initial matrix data located above the matrix diagonal of the first covariance matrix can be determined as multiple first to-be-processed matrix data of the first covariance matrix. For example, the first initial coordinate data , the first initial coordinate data and the first initial coordinate data are located on the matrix diagonal of the first covariance matrix and can be used as multiple first to-be-processed matrix data. The first initial coordinate data , the first initial coordinate data and the first initial coordinate data can be located above the matrix diagonal of the first covariance matrix and can also be used as multiple first to-be-processed matrix data. That is, 6 first to-be-processed matrix data can be obtained.

[0055] In some other embodiments, multiple first initial matrix data located on the matrix diagonal of the first covariance matrix and multiple first initial matrix data located below the matrix diagonal of the first covariance matrix may also be determined as multiple first matrix data to be processed of the first covariance matrix. For example, the first initial coordinate data , the first initial coordinate data and the first initial coordinate data are located on the matrix diagonal of the first covariance matrix and may be used as multiple first matrix data to be processed. The first initial coordinate data , the first initial coordinate data and the first initial coordinate data may be located below the matrix diagonal of the first covariance matrix and may also be used as multiple first matrix data to be processed. That is, 6 first matrix data to be processed may also be obtained.

[0056] It can be understood that some methods for determining the first matrix data to be processed are described above, and below, some methods for determining the first processed matrix data will be described.

[0057] In some embodiments, in some implementation manners of the above operation S220, obtaining multiple first processed matrix data of each of the multiple first covariance matrices according to the multiple first initial matrix data of each of the multiple first covariance matrices further includes: performing data type conversion on the multiple first matrix data to be processed of each of the multiple first covariance matrices to obtain multiple first processed matrix data of each of the multiple first covariance matrices.

[0058] For example, the data type of the first matrix data to be processed may be 32-bit floating-point numbers. The 32-bit floating-point numbers may be converted into 16-bit floating-point numbers to obtain the first processed matrix data. Through the embodiments of the present disclosure, multiple first matrix data to be processed are screened out from the multiple first initial matrix data, reducing the number of matrix data, and effectively reducing the occupied video memory space. Performing data type conversion on the first data to be processed can effectively reduce the data volume of the matrix data and further effectively utilize storage resources. It can be understood that the data type conversion performed on the first data to be processed may be a first data type conversion to convert floating-point numbers with higher precision into floating-point numbers with lower precision.

[0059] It can be understood that the methods for determining the first processed matrix data of the present disclosure are described above, and below, the methods for determining the first multi-channel image will be described.

[0060] In some embodiments, in some implementations of the above operation S230, obtaining a first multi-channel image for a three-dimensional Gaussian sputtering model to be processed based on multiple first processed matrix data of each of the multiple first covariance matrices includes: obtaining multiple matrix fusion data for the first covariance matrix based on the multiple first processed matrix data of the first covariance matrix. The matrix fusion data is obtained by fusing the multiple first processed matrix data. For example, as described above, the data type of the first processed matrix data is 16-bit floating point, and the data volume is 16 bits. The capacity of the video memory space for the first pixel can be 32 bits. Two first processed matrix data can be concatenated into a 32-bit matrix fusion data. There can be 6 first processed matrix data and 3 matrix fusion data.

[0061] In some embodiments, in some implementations of the above operation S230, obtaining a first multi-channel image for a three-dimensional Gaussian sputtering model to be processed based on multiple first processed matrix data of each of the multiple first covariance matrices further includes: using the multiple matrix fusion data for the first covariance matrix as multiple first pixel channel values for the first covariance matrix respectively to obtain a first pixel for the first covariance matrix. For example, 3 matrix fusion data can be used as 3 first pixel channel values respectively to obtain a first pixel. It can be understood that the first pixel for the first covariance matrix can be the first pixel for a three-dimensional Gaussian key unit. Through the embodiments of the present disclosure, the valid data in the first covariance matrix can be converted into first pixel channel values. Thus, a first multi-channel image can be obtained to facilitate the graphics processing unit to efficiently obtain data from the first multi-channel image.

[0062] It can be understood that the multi-channel image of the present disclosure has been described above by taking the first multi-channel image as an example. However, the present disclosure is not limited thereto, and the number of multi-channel images for the three-dimensional Gaussian sputtering model to be processed can be multiple, which will be described below.

[0063] In some embodiments, the multi-channel image can be a texture image. The multi-channel image can be a red-green-blue-alpha (RGBA) image. The data volume of each pixel channel value of the multi-channel image can be 32 bits.

[0064] In some embodiments, the attribute information of the three-dimensional Gaussian sputtering unit further includes position sub-information, color sub-information, and opacity sub-information. For example, the position sub-information includes multiple initial coordinate data. The color sub-information can include multiple first initial spherical harmonic data. Below, an example will be given where the color sub-information includes multiple components of zero-order spherical harmonic.

[0065] In some embodiments, the above method may further include: obtaining a second multi-channel image for the three-dimensional Gaussian sputtering model to be processed according to the respective multiple initial coordinate data and respective color sub-information of the multiple three-dimensional Gaussian sputtering units. The second multi-channel image includes multiple second pixels. The multiple second pixel channel values of the second pixels are determined according to the multiple initial coordinate data of the three-dimensional Gaussian sputtering units, the multiple first initial spherical harmonic data, and the opacity sub-information.

[0066] For example, according to the multiple first initial spherical harmonic data and the opacity sub-information, fusion sub-information can be obtained. In one example, the data type of the multiple first initial spherical harmonic data can be converted to obtain multiple first converted spherical harmonic data. The multiple first converted spherical harmonic data are subjected to splicing processing to obtain first spherical harmonic fusion data. The opacity sub-information can be an opacity value, which can be a 32-bit floating-point number. By converting the data type of the opacity value, a converted opacity value can be obtained. The first spherical harmonic fusion data and the converted opacity value are spliced to obtain a color opacity fusion value as the fusion sub-information. It can be understood that the data type conversion performed on the first initial spherical harmonic data and the opacity value can be a second data type conversion, which can be quantization to convert a floating-point number to a fixed-point number. The fixed-point number can be an 8-bit integer.

[0067] For example, the respective multiple initial coordinate data and the fusion sub-information of the three-dimensional Gaussian sputtering units are used as the multiple second pixel channel values for the three-dimensional Gaussian sputtering units to obtain second pixels for the three-dimensional Gaussian sputtering units. The data volume of the initial coordinate data can be 32 bits. The multiple initial coordinate data include the initial coordinate data in the first direction (X), the initial coordinate data in the second direction (Y), and the initial coordinate data in the third direction (Z). The initial coordinate data in the first direction (X), the initial coordinate data in the second direction (Y), and the initial coordinate data in the third direction (Z) can be used as the red pixel channel value, the green pixel channel value, and the blue pixel channel value respectively. The color opacity fusion value can be used as the transparency pixel channel value. Through the embodiments of the present disclosure, the position, color, and opacity of the three-dimensional Gaussian sputtering unit can be converted into pixel channel values of pixels, so that the graphics processing unit can efficiently obtain position data, color data, and opacity data from the second multi-channel image, which helps to improve the rendering efficiency of the graphics processing unit.

[0068] It can be understood that the above description of the present disclosure is made by taking the multiple first initial spherical harmonic data including multiple components of the zero-order spherical harmonic as an example. However, the present disclosure is not limited thereto. The color sub-information may include multiple components of the second-order spherical harmonic or multiple components of the first-order spherical harmonic, and may also include multiple components of higher-order spherical harmonics. Below, the second-order spherical harmonic will be taken as an example for description.

[0069] In some embodiments, the color sub-information includes a plurality of first initial spherical harmonic data and a plurality of second initial spherical harmonic data. The above method may further include: obtaining at least one third multi-channel image for the three-dimensional Gaussian sputtering model to be processed according to the plurality of second initial spherical harmonic data of each of the plurality of three-dimensional Gaussian sputtering units. The third multi-channel image includes a plurality of third pixels. The plurality of third pixel channel values of the third pixels are determined according to the plurality of second initial spherical harmonic data of the three-dimensional Gaussian sputtering units.

[0070] For example, taking the color sub-information of the three-dimensional Gaussian sputtering unit including a plurality of components of the second-order spherical harmonic as an example, the plurality of components of the second-order spherical harmonic may be 27. The plurality of components of the second-order spherical harmonic includes a plurality of first initial spherical harmonic data and a plurality of second initial spherical harmonic data. The data type of the second initial spherical harmonic data may be 32-bit floating-point numbers. A plurality of first initial spherical harmonic data can be determined from the plurality of components of the second-order spherical harmonic. The above description of the first initial spherical harmonic data also applies to this embodiment, and the present disclosure will not repeat it here. It can also be understood that the components of the spherical harmonic can be the spherical harmonic function coefficients.

[0071] For example, data type conversion can be performed on the plurality of second initial spherical harmonic data of the three-dimensional Gaussian sputtering unit to obtain a plurality of second converted spherical harmonic data of the three-dimensional Gaussian sputtering unit. In one example, the second initial spherical harmonic data can be quantized from 32-bit floating-point numbers to 8-bit integers (int8) to obtain the second converted spherical harmonic data. There may be 24 second initial spherical harmonic data, and there may also be 24 second converted spherical harmonic data.

[0072] For example, according to the plurality of second converted spherical harmonic data of the three-dimensional Gaussian sputtering unit, a plurality of second spherical harmonic fusion data of the three-dimensional Gaussian sputtering unit are obtained. The second spherical harmonic fusion data is obtained by fusing the plurality of second converted spherical harmonic data. In one example, the data volume of the second converted spherical harmonic data may be 8 bits. Three second converted spherical harmonic data can be concatenated into one second spherical harmonic fusion data. There may be 24 second converted spherical harmonic data, and 8 second spherical harmonic fusion data.

[0073] For example, multiple second spherical harmonic fusion data of a three-dimensional Gaussian sputtering unit are respectively used as multiple third pixel channel values to obtain at least one third pixel for the three-dimensional Gaussian sputtering unit. In one example, 4 second spherical harmonic fusion data can be used as 4 third pixel channel values to obtain one third pixel. Thus, based on 8 spherical harmonic fusion data, 2 third pixels can be obtained. These 2 third pixels can be respectively located in 2 third multi-channel images. That is, in the case where multiple initial spherical harmonic data are components of the second-order spherical harmonic, based on a three-dimensional Gaussian sputtering model to be processed, 2 third multi-channel images can be obtained. It can be understood that the first spherical harmonic fusion data is stored in the second pixel of the above-mentioned second multi-channel image. Through the embodiments of the present disclosure, multiple components of the spherical harmonic of the three-dimensional Gaussian sputtering unit can be converted into pixel channel values of pixels, so that the graphics processing unit can efficiently obtain color data from the second multi-channel graphics and the third multi-channel images, which helps to further improve the rendering efficiency of the graphics processing unit

[0074] It can be understood that the above 32-bit floating-point number can be an unsigned 32-bit floating-point number. The above 8-bit integer can be an unsigned 8-bit integer (uint8).

[0075] It can be understood that the three-dimensional data processing method has been described above. Next, the multi-channel images of the present disclosure will be described in conjunction with Figure 3A and Figure 3B to illustrate the multi-channel images of the present disclosure.

[0076] Figure 3A is a schematic diagram of a first multi-channel image according to an embodiment of the present disclosure.

[0077] As shown in FIG. 3, the first multi-channel image ti31 may include multiple first pixels. The multiple first pixel channel values of the first pixels are obtained according to multiple first processed matrix data in the first covariance matrix.

[0078] Figure 3B is a schematic diagram of a second multi-channel image according to an embodiment of the present disclosure.

[0079] As Figure 3B shown, the second multi-channel image ti32 may include multiple second pixels. The multiple second pixel channel values of the second pixels are determined according to multiple initial coordinate data, multiple first initial spherical harmonic data, and opacity sub-information of the three-dimensional Gaussian sputtering unit.

[0080] As Figure 3A and Figure 3B shown, the multi-channel image is mainly used to store one or more attribute sub-information of each of multiple three-dimensional Gaussian sputtering units. The graphics processing unit can quickly obtain the data required for rendering from the first multi-channel image ti31 and the second multi-channel image ti32.

[0081] It can be understood that the method of the present disclosure has been described above, and the image rendering method of the present disclosure will be described below.

[0082] Figure 4 It is a schematic flowchart of an image rendering method according to an embodiment of the present disclosure.

[0083] As Figure 4 shown, the method 400 may include operation S440 and operation S450.

[0084] In operation S440, a target image of the three-dimensional Gaussian sputtering model to be processed is obtained according to at least one of a plurality of multi-channel images for the three-dimensional Gaussian sputtering model to be processed.

[0085] In the embodiment of the present disclosure, the three-dimensional Gaussian sputtering model to be processed includes a plurality of three-dimensional Gaussian sputtering units, the plurality of first multi-channel images include a first multi-channel image, the first multi-channel image is obtained according to a plurality of first processed matrix data of a plurality of first covariance matrices for the plurality of three-dimensional Gaussian sputtering units, the first multi-channel image includes a plurality of first pixels, the plurality of first pixel channel values of the first pixels are determined according to the plurality of first processed matrix data of the first covariance matrix, the plurality of first processed matrix data of the plurality of first covariance matrices are obtained according to the plurality of first initial matrix data of the plurality of first covariance matrices, and the plurality of first covariance matrices are obtained according to the respective attribute information of the plurality of three-dimensional Gaussian sputtering units. For example, the first multi-channel image can be obtained according to the above method 200.

[0086] In operation S450, the target image is displayed on the visual interface.

[0087] Through the embodiment of the present disclosure, image rendering of the three-dimensional Gaussian sputtering model can be performed based on multiple channels, and the rendered target image can be displayed on the visual interface. The graphics processing unit can be used to efficiently sample the multi-channel image and convert the pixel channel values of the pixels into data required for rendering.

[0088] It can be understood that the image rendering method of the present disclosure has been described above, and the image rendering method of the present disclosure will be further described below.

[0089] In some embodiments, a radix sort can be performed on the plurality of three-dimensional Gaussian sputtering units to obtain a sorting result. Based on the sorting result, the respective indexes of the plurality of three-dimensional Gaussian sputtering units are determined for efficient rendering.

[0090] In some embodiments, the multiple multi-channel images further include a second multi-channel image, which is obtained based on the respective multiple initial coordinate data, the respective color sub-information, and the respective opacity sub-information of the multiple three-dimensional Gaussian sputtering units. The second multi-channel image includes multiple second pixels. It can be understood that the above description of the second multi-channel image also applies to the second multi-channel image involved in the image rendering method, and the present disclosure will not elaborate herein.

[0091] In some embodiments, in some implementations of the above operation S410, obtaining the target image of the three-dimensional Gaussian sputtering model to be processed based on at least one of the multiple multi-channel images for the three-dimensional Gaussian sputtering model to be processed includes: obtaining multiple position data respectively for the multiple three-dimensional Gaussian sputtering units, multiple color data respectively for the multiple three-dimensional Gaussian sputtering units, and multiple opacity data respectively for the multiple three-dimensional Gaussian sputtering units according to the multiple second pixels respectively within the multiple three-dimensional Gaussian sputtering units. For example, as described above, when determining the multiple second pixel channel values of the second pixels, the initial coordinate data in the first direction (X), the initial coordinate data in the second direction (Y), and the initial coordinate data in the third direction (Z) are respectively used as the red pixel channel value, the green pixel channel value, and the blue pixel channel value. Thus, the red pixel channel value, the green pixel channel value, and the blue pixel channel value of the second pixel can be used as the position data for the three-dimensional Gaussian sputtering unit. It can be understood that when generating the second pixel, no data type conversion is performed on the initial coordinate data. Thus, the position data obtained from the second pixel is lossless and has a high precision. The position components in the first direction, the second direction, and the third direction in the position data can be 32-bit floating-point numbers. Also for example, the second pixel further includes second pixel channel values determined according to the color transparency fusion value. The color transparency fusion value can be obtained from the second pixel. By splitting the color transparency fusion value, a first spherical harmonic fusion data and a converted opacity value can be obtained. By further splitting the first spherical harmonic fusion data, multiple first converted spherical harmonic data can be obtained. By performing data type conversion on the converted opacity value and the multiple first converted spherical harmonic data, an opacity value and multiple first initial spherical harmonic data can be obtained. According to the multiple first initial spherical harmonic data, color data can be obtained. The opacity value can be used as the opacity data.

[0092] In some embodiments, in some implementations of the above operation S410, obtaining the target image of the three-dimensional Gaussian sputtering model to be processed based on at least one of the multiple multi-channel images for the three-dimensional Gaussian sputtering model to be processed further includes: obtaining a rendering result according to the multiple position data respectively for the multiple three-dimensional Gaussian sputtering units and the multiple first pixels respectively within the multiple three-dimensional Gaussian sputtering units, which will be further described below.

[0093] In some embodiments, obtaining a rendering result based on multiple position data respectively for multiple three-dimensional Gaussian sputtering units and multiple first pixels respectively within the multiple three-dimensional Gaussian sputtering units includes: obtaining multiple first covariance matrices respectively for the multiple first pixels within the multiple three-dimensional Gaussian sputtering units. For example, as described above, 3 matrix fusion data are used as 3 first pixel channel values. Thus, 3 matrix fusion data can be obtained from the first pixels. The 3 matrix fusion data are split to obtain 6 first processed matrix data. The data volume of the first processed matrix data can be 16 bits. The data type of the first processed matrix data is converted to convert 16-bit floating-point numbers to 32-bit floating-point numbers to obtain first matrix data to be processed. Based on the 6 first matrix data to be processed, a first covariance matrix can be obtained. Thus, decoding of multiple first pixel channel values of the first pixels can be achieved, and fast image rendering can be realized based on a graphics processing unit with less precision loss.

[0094] In some embodiments, obtaining a rendering result based on multiple position data respectively for multiple three-dimensional Gaussian sputtering units and multiple first pixels respectively within the multiple three-dimensional Gaussian sputtering units includes: determining multiple second covariance matrices respectively for the multiple three-dimensional Gaussian sputtering units based on the multiple first covariance matrices and the multiple position data.

[0095] For example, multiple Jacobian matrices respectively for the multiple three-dimensional Gaussian sputtering units are determined based on the multiple position data. In one example, a Jacobian matrix for a three-dimensional Gaussian sputtering unit can be determined according to set camera parameters and position data:

[0096] (Formula Three)

[0097] and are the first focal length and the second focal length in the camera parameters respectively. (X, Y, Z) are the position data, which are the initial coordinate data in the first direction, the initial coordinate data in the second direction, and the initial coordinate data in the third direction respectively as described above.

[0098] For example, multiple second covariance matrices are determined based on the multiple Jacobian matrices, multiple perspective transformation matrices, and multiple first covariance matrices respectively for the multiple three-dimensional Gaussian sputtering units. In one example, by multiplying the Jacobian matrix, perspective transformation matrix, first covariance matrix, transposed perspective transformation matrix, and transposed Jacobian matrix for a three-dimensional Gaussian sputtering unit, a second covariance matrix can be obtained. The second covariance matrix can be obtained through the following formula:

[0099] (Formula Four)

[0100] It can be the first covariance matrix for a three-dimensional Gaussian sputtering unit. It can be the second covariance matrix for the three-dimensional Gaussian sputtering unit. It can be the Jacobian matrix for a three-dimensional Gaussian sputtering unit. It can be the perspective transformation matrix for a three-dimensional Gaussian sputtering unit. It can be the transposed Jacobian matrix. It can be the transposed perspective transformation matrix.

[0101] In some embodiments, obtaining a rendering result based on multiple position data respectively for multiple three-dimensional Gaussian sputtering units and multiple first pixels respectively within multiple three-dimensional Gaussian sputtering units includes: determining multiple projection coordinate data for multiple three-dimensional Gaussian sputtering units according to multiple second covariance matrices. For example, the second covariance matrix can be a two-dimensional (2D) covariance matrix. According to the second covariance matrix, the major axis radius, coverage range, etc. of the three-dimensional Gaussian sputtering unit on the imaging plane can be determined to obtain the projection coordinate data of the three-dimensional Gaussian sputtering unit on the imaging plane.

[0102] In some embodiments, obtaining a rendering result based on multiple position data respectively for multiple three-dimensional Gaussian sputtering units and multiple first pixels respectively within multiple three-dimensional Gaussian sputtering units includes: obtaining a rendering result according to multiple projection coordinate data, multiple color data, and multiple opacity data. For example, according to the center position of the three-dimensional Gaussian sputtering unit and the coordinates of multiple points in three-dimensional space, multiple relative position data can be obtained. According to the multiple relative position data and the inverse matrix of the second covariance matrix, the exponential part of the Gaussian distribution can be determined. In addition, according to the exponential part of the Gaussian distribution and the opacity data, the transparency data to be rendered of the three-dimensional Gaussian sputtering unit can be determined. According to the transparency data to be rendered and the color data, the color data to be rendered of the three-dimensional Gaussian sputtering unit can be determined, and Gaussian sputtering rendering is performed based on the alpha-blending technology to obtain the target image.

[0103] Next, operation S450 can be executed to display the target image on the visual interface.

[0104] It can be understood that the above has described the present disclosure by taking the color data including the first initial spherical harmonic data as an example. However, the present disclosure is not limited thereto, and the color data can also include the second initial spherical harmonic data, which will be described below.

[0105] In some embodiments, the multiple multi-channel images further include at least one third multi-channel image. The at least one third multi-channel image is obtained based on the multiple second initial spherical harmonic data of the respective multiple three-dimensional Gaussian sputtering units. The above description of the third multi-channel image also applies to this embodiment, and the present disclosure will not repeat it here.

[0106] In some embodiments, obtaining the multiple position data, the multiple color data, and the multiple opacity data respectively for the multiple three-dimensional Gaussian sputtering units based on the multiple second pixels respectively in the multiple three-dimensional Gaussian sputtering units includes: obtaining the multiple position data and the multiple opacity data respectively for the multiple three-dimensional Gaussian sputtering units based on the multiple second pixels respectively in the multiple three-dimensional Gaussian sputtering units. Obtaining the multiple color data respectively for the multiple three-dimensional Gaussian sputtering units based on the multiple second pixels and the multiple third pixels respectively in the multiple three-dimensional Gaussian sputtering units. The color data includes multiple first initial spherical harmonic data and multiple second initial spherical harmonic data.

[0107] For example, the third pixel channel value of the third pixel is obtained based on the second spherical harmonic fusion data. The second spherical harmonic fusion data can be obtained from the third pixel. By splitting the second spherical harmonic fusion data, multiple second transformed spherical harmonic data can be obtained. By performing data type conversion on the multiple second transformed spherical harmonic data, multiple second initial spherical harmonic data are obtained.

[0108] For another example, based on the transparency data to be rendered and the color data including multiple first initial spherical harmonic data and multiple second initial spherical harmonic data, the color data to be rendered of the three-dimensional Gaussian sputtering unit can be determined, and Gaussian sputtering rendering is performed based on the alpha blending technique to obtain the target image.

[0109] Next, Figure 5 the target image of the present disclosure will be further described.

[0110] Figure 5 is a schematic diagram of the target image according to an embodiment of the present disclosure.

[0111] As Figure 5 shown, the target image i50 can be rendered by a graphics processing unit based on the first multi-channel image, the second multi-channel image, and the at least one third multi-channel image for the three-dimensional Gaussian sputtering model to be processed. The target image i50 can be displayed on a visual interface.

[0112] It can be understood that the method of the present disclosure has been described above, and the apparatus of the present disclosure will be described below.

[0113] Figure 6It is a block diagram of a three-dimensional data processing device according to an embodiment of the present disclosure.

[0114] As Figure 6 shown, the device 600 may include a first determination module 610, a first acquisition module 620, and a second acquisition module 630.

[0115] The first determination module 610 is configured to determine, according to the attribute information of each of the multiple three-dimensional Gaussian sputtering cells of the three-dimensional Gaussian sputtering model to be processed, multiple first covariance matrices for the multiple three-dimensional Gaussian sputtering cells. The first covariance matrix includes multiple first initial matrix data.

[0116] The first acquisition module 620 is configured to obtain, according to the multiple first initial matrix data of each of the multiple first covariance matrices, multiple first processed matrix data of each of the multiple first covariance matrices.

[0117] The second acquisition module 630 is configured to obtain, according to the multiple first processed matrix data of each of the multiple first covariance matrices, a first multi-channel image for the three-dimensional Gaussian sputtering model to be processed. The first multi-channel image includes multiple first pixels, and the multiple first pixel channel values of the first pixels are determined according to the multiple first processed matrix data of the first covariance matrix.

[0118] In some embodiments, the attribute information of the three-dimensional Gaussian sputtering cell includes a rotation attribute matrix and a scaling attribute matrix. The first determination module includes: a first multiplication sub-module, configured to multiply the rotation attribute matrix, the scaling attribute matrix, the transposed scaling attribute matrix, and the transposed rotation attribute matrix of the three-dimensional Gaussian sputtering cell to obtain a first covariance matrix for the three-dimensional Gaussian sputtering cell. The transposed scaling attribute matrix is obtained by transposing the scaling attribute matrix, and the transposed rotation attribute matrix is obtained by transposing the rotation attribute matrix.

[0119] In some embodiments, the first acquisition module includes: a first determination sub-module, configured to determine, from the multiple first initial matrix data of each of the multiple first covariance matrices, multiple first to-be-processed matrix data of each of the multiple first covariance matrices; a first acquisition sub-module, configured to perform data type conversion on the multiple first to-be-processed matrix data of each of the multiple first covariance matrices to obtain multiple first processed matrix data of each of the multiple first covariance matrices.

[0120] In some embodiments, the first determination sub-module includes: a first determination unit, configured to determine the multiple first initial matrix data located on the matrix diagonal of the first covariance matrix and the multiple first initial matrix data located above the matrix diagonal of the first covariance matrix as the multiple first to-be-processed matrix data of the first covariance matrix.

[0121] In some embodiments, the first determination sub-module includes: a second determination unit configured to determine, as multiple first matrix data to be processed of the first covariance matrix, multiple first initial matrix data located on the matrix diagonal of the first covariance matrix and multiple first initial matrix data located below the matrix diagonal of the first covariance matrix.

[0122] In some embodiments, the second acquisition module includes: a second acquisition sub-module configured to obtain, based on multiple first processed matrix data of the first covariance matrix, multiple matrix fusion data for the first covariance matrix. The matrix fusion data is obtained by fusing multiple first processed matrix data. A third acquisition sub-module configured to, respectively, use the multiple matrix fusion data for the first covariance matrix as multiple first pixel channel values for the first covariance matrix to obtain a first pixel for the first covariance matrix.

[0123] In some embodiments, the attribute information of the three-dimensional Gaussian sputtering unit further includes position sub-information, color sub-information, and opacity sub-information. The position sub-information includes multiple initial coordinate data, and the color sub-information includes multiple first initial spherical harmonic data. The above apparatus further includes: a third acquisition module configured to obtain, based on the multiple initial coordinate data of each of the multiple three-dimensional Gaussian sputtering units, the color sub-information of each of the multiple three-dimensional Gaussian sputtering units, and the opacity sub-information of each of the multiple three-dimensional Gaussian sputtering units, a second multi-channel image for the three-dimensional Gaussian sputtering model to be processed. The second multi-channel image includes multiple second pixels, and the multiple second pixel channel values of the second pixels are determined based on the multiple initial coordinate data of the three-dimensional Gaussian sputtering unit, the multiple first initial spherical harmonic data, and the opacity sub-information.

[0124] In some embodiments, the third acquisition module includes: a second determination sub-module configured to determine fusion sub-information based on the multiple first initial spherical harmonic data and the opacity sub-information. A fourth acquisition sub-module configured to, respectively, use the multiple initial coordinate data of the three-dimensional Gaussian sputtering unit and the fusion sub-information as multiple second pixel channel values for the three-dimensional Gaussian sputtering unit to obtain a second pixel for the three-dimensional Gaussian sputtering unit.

[0125] In some embodiments, the attribute information of the three-dimensional Gaussian sputtering unit further includes color sub-information, and the color sub-information includes multiple first initial spherical harmonic data and multiple second initial spherical harmonic data. The above apparatus further includes: a fourth acquisition module configured to obtain, based on the multiple second initial spherical harmonic data of each of the multiple three-dimensional Gaussian sputtering units, at least one third multi-channel image for the three-dimensional Gaussian sputtering model to be processed. The third multi-channel image includes multiple third pixels, and the multiple third pixel channel values of the third pixels are determined based on the multiple second initial spherical harmonic data of the three-dimensional Gaussian sputtering unit.

[0126] In some embodiments, the fourth obtaining module includes: a fifth obtaining sub-module, configured to perform data type conversion on multiple second initial spherical harmonic data of a three-dimensional Gaussian sputtering unit to obtain multiple second converted spherical harmonic data of the three-dimensional Gaussian sputtering unit. A sixth obtaining sub-module, configured to obtain multiple second spherical harmonic fusion data of the three-dimensional Gaussian sputtering unit according to the multiple second converted spherical harmonic data of the three-dimensional Gaussian sputtering unit. The second spherical harmonic fusion data is obtained by fusing the multiple second converted spherical harmonic data. A seventh obtaining sub-module, configured to use the multiple second spherical harmonic fusion data of the three-dimensional Gaussian sputtering unit as multiple third pixel channel values respectively to obtain at least one third pixel for the three-dimensional Gaussian sputtering unit.

[0127] Figure 7 It is a block diagram of an image rendering apparatus according to another embodiment of the present disclosure.

[0128] As Figure 7 shown, the apparatus 700 may include a rendering module 740 and a display module 750.

[0129] The rendering module 740 is configured to obtain a target image of a three-dimensional Gaussian sputtering model to be processed according to at least one of multiple multi-channel images for the three-dimensional Gaussian sputtering model to be processed.

[0130] The display module 750 is configured to display the target image on a visual interface.

[0131] In some embodiments, the three-dimensional Gaussian sputtering model to be processed includes multiple three-dimensional Gaussian sputtering units, the multiple first multi-channel images include a first multi-channel image, the first multi-channel image is obtained according to multiple first processed matrix data of multiple first covariance matrices for the multiple three-dimensional Gaussian sputtering units, the first multi-channel image includes multiple first pixels, the multiple first pixel channel values of the first pixels are determined according to the multiple first processed matrix data of the first covariance matrix, the multiple first processed matrix data of the multiple first covariance matrices are obtained according to the multiple first initial matrix data of the multiple first covariance matrices, and the multiple first covariance matrices are obtained according to the attribute information of the multiple three-dimensional Gaussian sputtering units respectively.

[0132] In some embodiments, the multiple multi-channel images further include a second multi-channel image, which is obtained based on the multiple initial coordinate data, the multiple first initial spherical harmonic data, and the opacity sub-information of each of the multiple three-dimensional Gaussian sputtering units. The second multi-channel image includes multiple second pixels. The rendering module includes: an eighth obtaining sub-module, configured to obtain multiple position data respectively for the multiple three-dimensional Gaussian sputtering units, multiple color data respectively for the multiple three-dimensional Gaussian sputtering units, and multiple opacity data respectively for the multiple three-dimensional Gaussian sputtering units according to the multiple second pixels respectively in the multiple three-dimensional Gaussian sputtering units. The position data includes the multiple initial coordinate data, the color data includes the multiple first initial spherical harmonic data, and the opacity data corresponds to the opacity sub-information. A ninth obtaining sub-module, configured to obtain a rendering result according to the multiple position data respectively for the multiple three-dimensional Gaussian sputtering units and the multiple first pixels respectively in the multiple three-dimensional Gaussian sputtering units.

[0133] In some embodiments, the ninth obtaining sub-module includes: a first obtaining unit, configured to obtain multiple first covariance matrices respectively for the multiple three-dimensional Gaussian sputtering units according to the multiple first pixels respectively in the multiple three-dimensional Gaussian sputtering units. A third determining unit, configured to determine multiple second covariance matrices respectively for the multiple three-dimensional Gaussian sputtering units according to the multiple first covariance matrices and the multiple position data. A fourth determining unit, configured to determine multiple projection coordinate data for the multiple three-dimensional Gaussian sputtering units according to the multiple second covariance matrices. A second obtaining unit, configured to obtain a rendering result according to the multiple projection coordinate data, the multiple color data, and the multiple opacity data.

[0134] In some embodiments, the third determining unit includes: a first determining sub-unit, configured to determine multiple Jacobian matrices respectively for the multiple three-dimensional Gaussian sputtering units according to the multiple position data. A second determining sub-unit, configured to determine multiple second covariance matrices according to the multiple Jacobian matrices respectively for the multiple three-dimensional Gaussian sputtering units, the multiple perspective transformation matrices, and the multiple first covariance matrices.

[0135] In some embodiments, the first determining sub-unit is further configured to: multiply the Jacobian matrix, the perspective transformation matrix, the first covariance matrix, the transposed perspective transformation matrix, and the transposed Jacobian matrix for the three-dimensional Gaussian sputtering unit to obtain the second covariance matrix.

[0136] In some embodiments, the multiple multi-channel images include at least one third multi-channel image, which is obtained based on the multiple second initial spherical harmonic data of each of the multiple three-dimensional Gaussian sputtering units. The color data further includes the multiple second initial spherical harmonic data.

[0137] In the technical solution of the present disclosure, the processing of the collection, storage, use, processing, transmission, provision, and disclosure of the user's personal information complies with the provisions of relevant laws and regulations and does not violate public order and good customs.

[0138] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0139] Figure 8 FIG. shows a schematic block diagram of an exemplary electronic device 800 that can be used to implement the embodiments of the present disclosure. The electronic device is intended to represent various forms of digital computers, such as, for example, a laptop computer, a desktop computer, a workbench, a personal digital assistant, a server, a blade server, a mainframe computer, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as, for example, a personal digital processor, a cellular phone, a smartphone, a wearable device, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely exemplary and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0140] As Figure 8 shown, the device 800 includes a computing unit 801, which can execute various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other through a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0141] Multiple components in the device 800 are connected to the I / O interface 805, including: an input unit 806, such as a keyboard, a mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a disk, an optical disc, etc.; and a communication unit 809, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the device 800 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0142] The computing unit 801 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 801 executes the various methods and processes described above, such as the three-dimensional data processing method and / or the image rendering method. For example, in some embodiments, the three-dimensional data processing method and / or the image rendering method can be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as the storage unit 808. In some embodiments, part or all of the computer program can be loaded and / or installed onto the device 800 via the ROM 802 and / or the communication unit 809. When the computer program is loaded into the RAM 803 and executed by the computing unit 801, one or more steps of the three-dimensional data processing method and / or the image rendering method described above can be executed. Alternatively, in other embodiments, the computing unit 801 can be configured to execute the three-dimensional data processing method and / or the image rendering method by any other suitable means (e.g., by means of firmware).

[0143] The various embodiments of the systems and techniques described above in this specification can be implemented in digital electronic circuitry, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard parts (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: being implemented in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which may be a special-purpose or general-purpose programmable processor that can receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0144] The program code for implementing the methods of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the program codes, when executed by the processor or controller, cause the functions / operations specified in the flowchart and / or block diagram to be implemented. The program code can be executed entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine, or entirely on the remote machine or server.

[0145] In the context of this disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory (EPROM) or flash memory, an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0146] To provide for interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a cathode ray tube (CRT) monitor or a liquid crystal display (LCD)) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can also be used to provide for interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, speech input, or tactile input).

[0147] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which the user can interact with an implementation of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0148] A computer system may include a client and a server. The client and the server are generally far apart from each other and typically interact via a communication network. The relationship between the client and the server is generated by computer programs that run on the respective computers and have a client-server relationship with each other.

[0149] It should be understood that various forms of the processes shown above may be used, with steps reordered, added, or deleted. For example, the steps recited in this disclosure may be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and no limitation is imposed herein.

[0150] The above specific embodiments do not constitute a limitation on the protection scope of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the protection scope of this disclosure.

Claims

1. A three-dimensional data processing method, comprising: Determine a plurality of first covariance matrices for a plurality of three-dimensional Gaussian sputtering units according to respective attribute information of a plurality of three-dimensional Gaussian sputtering units of the three-dimensional Gaussian sputtering model to be processed, wherein the first covariance matrix includes a plurality of first initial matrix data; Obtaining a plurality of first processed matrix data of each of the plurality of first covariance matrices according to a plurality of first initial matrix data of each of the plurality of first covariance matrices; According to the multiple first processed matrix data of each of the multiple first covariance matrices, a first multi-channel image for the three-dimensional Gaussian sputtering model to be processed is obtained, wherein the first multi-channel image includes multiple first pixels, and the multiple first pixel channel values ​​of the first pixels are determined according to the multiple first processed matrix data of the first covariance matrix.

2. The method according to claim 1, wherein: The attribute information of the three-dimensional Gaussian sputtering unit includes a rotation attribute matrix and a scaling attribute matrix. The determining of a plurality of first covariance matrices for a plurality of three-dimensional Gaussian sputtering units according to respective attribute information of a plurality of three-dimensional Gaussian sputtering units of the three-dimensional Gaussian sputtering model to be processed comprises: The rotation property matrix, the scaling property matrix, the transposed scaling property matrix and the transposed rotation property matrix of the three-dimensional Gaussian sputtering unit are multiplied to obtain the first covariance matrix for the three-dimensional Gaussian sputtering unit, wherein the transposed scaling property matrix is ​​obtained by transposing the scaling property matrix, and the transposed rotation property matrix is ​​obtained by transposing the rotation property matrix.

3. The method according to claim 1, wherein: The obtaining, according to the plurality of first initial matrix data of each of the plurality of first covariance matrices, a plurality of first processed matrix data of each of the plurality of first covariance matrices comprises: Determining a plurality of first to-be-processed matrix data of each of the plurality of first covariance matrices from a plurality of first initial matrix data of each of the plurality of first covariance matrices; Data type conversion is performed on the plurality of first to-be-processed matrix data of each of the plurality of first covariance matrices to obtain the plurality of first processed matrix data of each of the plurality of first covariance matrices.

4. The method according to claim 3, wherein: The determining, from the plurality of first initial matrix data of each of the plurality of first covariance matrices, a plurality of first to-be-processed matrix data of each of the plurality of first covariance matrices comprises: Determine the plurality of first initial matrix data located on the matrix diagonal of the first covariance matrix and the plurality of first initial matrix data located above the matrix diagonal of the first covariance matrix as the plurality of first to-be-processed matrix data of the first covariance matrix; or, A plurality of the first initial matrix data located on the matrix diagonal of the first covariance matrix and a plurality of the first initial matrix data located below the matrix diagonal of the first covariance matrix are determined as a plurality of the first to-be-processed matrix data of the first covariance matrix.

5. The method according to claim 1, wherein: The step of obtaining a first multi-channel image for the three-dimensional Gaussian sputtering model to be processed according to the plurality of first processed matrix data of each of the plurality of first covariance matrices comprises: Obtaining a plurality of matrix fusion data for the first covariance matrix according to the plurality of the first processed matrix data of the first covariance matrix, wherein the matrix fusion data is obtained by fusing the plurality of the first processed matrix data; The plurality of matrix fusion data used for the first covariance matrix are respectively used as a plurality of first pixel channel values ​​used for the first covariance matrix to obtain the first pixels used for the first covariance matrix.

6. The method according to claim 1, wherein: The attribute information of the three-dimensional Gaussian sputtering unit also includes position sub-information, color sub-information and opacity sub-information, wherein the position sub-information includes a plurality of initial coordinate data, and the color sub-information includes a plurality of first initial spherical harmonic data. Also includes: According to the multiple initial coordinate data of each of the multiple three-dimensional Gaussian sputtering units, the color sub-information of each of the multiple three-dimensional Gaussian sputtering units, and the opacity sub-information of each of the multiple three-dimensional Gaussian sputtering units, a second multi-channel image for the three-dimensional Gaussian sputtering model to be processed is obtained, wherein the second multi-channel image includes multiple second pixels, and the multiple second pixel channel values ​​of the second pixels are determined based on the multiple initial coordinate data of the three-dimensional Gaussian sputtering unit, the multiple first initial spherical harmonic data, and the opacity sub-information.

7. The method according to claim 6, wherein: The step of obtaining a second multi-channel image for the three-dimensional Gaussian sputtering model to be processed according to the multiple initial coordinate data of each of the multiple three-dimensional Gaussian sputtering units and the multiple color sub-information of each of the multiple three-dimensional Gaussian sputtering units comprises: Determine fusion sub-information according to the plurality of first initial spherical harmonic data and the opacity sub-information; The multiple initial coordinate data and the fusion sub-information of the three-dimensional Gaussian sputtering unit are respectively used as multiple second pixel channel values ​​for the three-dimensional Gaussian sputtering unit to obtain the second pixel for the three-dimensional Gaussian sputtering unit.

8. The method according to claim 1, wherein: The attribute information of the three-dimensional Gaussian sputtering unit also includes color sub-information, and the color sub-information includes a plurality of first initial spherical harmonic data and a plurality of second initial spherical harmonic data. Also includes: Based on the multiple second initial spherical harmonic data of each of the multiple three-dimensional Gaussian sputtering units, at least one third multi-channel image for the three-dimensional Gaussian sputtering model to be processed is obtained, wherein the third multi-channel image includes multiple third pixels, and the multiple third pixel channel values ​​of the third pixels are determined based on the multiple second initial spherical harmonic data of the three-dimensional Gaussian sputtering unit.

9. The method according to claim 8, wherein: The step of obtaining at least one third multi-channel image for the three-dimensional Gaussian sputtering model to be processed according to the multiple initial spherical harmonic data of each of the multiple three-dimensional Gaussian sputtering units comprises: Performing data type conversion on a plurality of the second initial spherical harmonic data of the three-dimensional Gaussian sputtering unit to obtain a plurality of second converted spherical harmonic data of the three-dimensional Gaussian sputtering unit; According to the plurality of the second converted spherical harmonic data of the three-dimensional Gaussian sputtering unit, a plurality of second spherical harmonic fusion data of the three-dimensional Gaussian sputtering unit is obtained, wherein the second spherical harmonic fusion data is obtained by fusing the plurality of the second converted spherical harmonic data; The plurality of the second spherical harmonic fusion data of the three-dimensional Gaussian sputtering unit are respectively used as a plurality of third pixel channel values ​​to obtain at least one of the third pixels used for the three-dimensional Gaussian sputtering unit.

10. An image rendering method, comprising: Obtaining a target image of the three-dimensional Gaussian sputtering model to be processed according to at least one of a plurality of multi-channel images for the three-dimensional Gaussian sputtering model to be processed; Displaying the target image in a visual interface, Among them, the three-dimensional Gaussian sputtering model to be processed includes multiple three-dimensional Gaussian sputtering units, multiple first multi-channel images include a first multi-channel image, the first multi-channel image is obtained based on multiple first processed matrix data of each of the multiple first covariance matrices for the multiple three-dimensional Gaussian sputtering units, the first multi-channel image includes multiple first pixels, multiple first pixel channel values ​​of the first pixels are determined based on multiple first processed matrix data of the first covariance matrix, multiple first processed matrix data of each of the multiple first covariance matrices are obtained based on multiple first initial matrix data of each of the multiple first covariance matrices, and multiple first covariance matrices are obtained based on attribute information of each of the multiple three-dimensional Gaussian sputtering units.

11. The method according to claim 10, wherein: The plurality of multi-channel images further include a second multi-channel image, wherein the second multi-channel image is obtained according to a plurality of initial coordinate data of each of the plurality of three-dimensional Gaussian sputtering units, a plurality of first initial spherical harmonic data of each of the plurality of three-dimensional Gaussian sputtering units, and the opacity sub-information of each of the plurality of three-dimensional Gaussian sputtering units, and the second multi-channel image includes a plurality of second pixels. The step of obtaining a target image of the three-dimensional Gaussian sputtering model to be processed according to at least one of the plurality of multi-channel images for the three-dimensional Gaussian sputtering model to be processed comprises: According to a plurality of second pixels respectively used in the plurality of three-dimensional Gaussian sputtering units, a plurality of position data respectively used in the plurality of three-dimensional Gaussian sputtering units, a plurality of color data respectively used in the plurality of three-dimensional Gaussian sputtering units, and a plurality of opacity data respectively used in the plurality of three-dimensional Gaussian sputtering units are obtained, wherein the position data includes a plurality of the initial coordinate data, the color data includes a plurality of the first initial spherical harmonic data, and the opacity data corresponds to the opacity sub-information; The rendering result is obtained according to the plurality of position data respectively used for the plurality of three-dimensional Gaussian sputtering units and the plurality of first pixels respectively used in the plurality of three-dimensional Gaussian sputtering units.

12. The method according to claim 11, wherein: The obtaining of the rendering result according to the plurality of position data respectively used for the plurality of three-dimensional Gaussian sputtering units and the plurality of first pixels respectively used in the plurality of three-dimensional Gaussian sputtering units comprises: According to a plurality of first pixels respectively used in a plurality of the three-dimensional Gaussian sputtering units, a plurality of first covariance matrices respectively used in the plurality of the three-dimensional Gaussian sputtering units are obtained; Determine a plurality of second covariance matrices respectively used for a plurality of the three-dimensional Gaussian sputtering units according to a plurality of the first covariance matrices and a plurality of the position data; Determining a plurality of projection coordinate data for a plurality of the three-dimensional Gaussian sputtering units according to a plurality of the second covariance matrices; The rendering result is obtained according to the plurality of projection coordinate data, the plurality of color data and the plurality of opacity data.

13. The method according to claim 12, wherein: Determining a plurality of second covariance matrices respectively used for a plurality of the three-dimensional Gaussian sputtering units according to a plurality of the first covariance matrices and a plurality of the position data comprises: Determine a plurality of Jacobian matrices respectively used for a plurality of the three-dimensional Gaussian sputtering units according to the plurality of the position data; A plurality of the second covariance matrices are determined according to a plurality of Jacobian matrices, a plurality of viewing angle transformation matrices and a plurality of the first covariance matrices respectively used for a plurality of the three-dimensional Gaussian sputtering units.

14. The method according to claim 13, wherein: Determining a plurality of the second covariance matrices according to a plurality of Jacobian matrices, a plurality of perspective transformation matrices and a plurality of the first covariance matrices respectively used for a plurality of the three-dimensional Gaussian sputtering units comprises: The Jacobian matrix for the three-dimensional Gaussian sputtering unit, the perspective transformation matrix, the first covariance matrix, the transposed perspective transformation matrix and the transposed Jacobian matrix are multiplied to obtain the second covariance matrix.

15. The method according to claim 11, wherein: The plurality of multi-channel images include at least one third multi-channel image, and the at least one third multi-channel image is obtained based on a plurality of second initial spherical harmonic data of each of the plurality of three-dimensional Gaussian sputtering units. The color data also includes a plurality of second initial spherical harmonic data.

16. A three-dimensional data processing device, comprising: A first determination module, configured to determine a plurality of first covariance matrices for a plurality of three-dimensional Gaussian sputtering units of a three-dimensional Gaussian sputtering model to be processed according to respective attribute information of the plurality of three-dimensional Gaussian sputtering units, wherein the first covariance matrix includes a plurality of first initial matrix data; A first obtaining module, configured to obtain a plurality of first processed matrix data of each of the plurality of first covariance matrices according to a plurality of first initial matrix data of each of the plurality of first covariance matrices; A second acquisition module is used to obtain a first multi-channel image for the three-dimensional Gaussian sputtering model to be processed based on the multiple first processed matrix data of each of the multiple first covariance matrices, wherein the first multi-channel image includes multiple first pixels, and the multiple first pixel channel values ​​of the first pixels are determined based on the multiple first processed matrix data of the first covariance matrix.

17. An image rendering device, comprising: A rendering module, used for obtaining a target image of the three-dimensional Gaussian sputtering model to be processed according to at least one of the multiple multi-channel images for the three-dimensional Gaussian sputtering model to be processed; A display module, used to display the target image on a visual interface, Among them, the three-dimensional Gaussian sputtering model to be processed includes multiple three-dimensional Gaussian sputtering units, multiple first multi-channel images include a first multi-channel image, the first multi-channel image is obtained based on multiple first processed matrix data of each of the multiple first covariance matrices for the multiple three-dimensional Gaussian sputtering units, the first multi-channel image includes multiple first pixels, multiple first pixel channel values ​​of the first pixels are determined based on multiple first processed matrix data of the first covariance matrix, multiple first processed matrix data of each of the multiple first covariance matrices are obtained based on multiple first initial matrix data of each of the multiple first covariance matrices, and multiple first covariance matrices are obtained based on attribute information of each of the multiple three-dimensional Gaussian sputtering units.

18. An electronic device, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 15.

19. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 15.

20. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 15.

Images