Gpu accelerated 3d gaussian splatting three-dimensional coordinate high-precision real-time picking method and system

Abstract

The application provides a GPU accelerated 3D Gaussian splash three-dimensional coordinate high-precision real-time picking method, system, device and storage medium, including the following steps: generating a plurality of Gaussian ellipsoids based on the data information of the target scene, projecting the plurality of Gaussian ellipsoids into a two-dimensional screen through a view projection matrix; calculating the projection coordinates of the plurality of Gaussian ellipsoids, calculating the depth values of the plurality of Gaussian ellipsoids according to the projection coordinates; converting the depth values into color values, and calculating a target color value according to the color values; picking up the target color value, and calculating the three-dimensional coordinates of the plurality of Gaussian ellipsoids based on the target color value. The application can improve the picking precision and efficiency of the three-dimensional coordinates.

Description

Technical Field

[0001] This invention relates to the fields of computer graphics and 3D reconstruction technology, specifically to a GPU-accelerated 3D Gaussian splashing 3D coordinate high-precision real-time picking method, system, device and storage medium. Background Technology

[0002] As a cutting-edge technology for real-time 3D rendering, 3D Gaussian Splatting (3DGS) faces the core challenge of not only efficiently rendering large-scale Gaussian points, but also accurately picking the coordinates of the rendered model. This step directly determines the realism and smoothness of the interactive experience.

[0003] Current mainstream 3DGS picking solutions generally adopt CPU-based picking algorithms. The core logic of traditional CPU picking is to complete coordinate picking on the CPU side through software algorithms (such as ray casting and bounding box hierarchy detection): the CPU needs to traverse all Gaussian points, emit query rays for each Gaussian point or perform collision detection in the bounding box hierarchy, and finally determine the picked 3D coordinates by calculating the "nearest point". However, existing picking solutions have the following drawbacks:

[0004] First, computational efficiency is insufficient to meet the parallel requirements of large-scale Gaussian points. 3DGS typically represents a complex scene using millions to tens of millions of discrete Gaussian points, while the CPU's serial computing mode cannot support such a massive amount of computation. Taking a scene containing 1 million Gaussian points as an example, the CPU needs to traverse it one by one and perform ray picking, with a single frame processing time often exceeding 200ms, far exceeding the hard requirement of 30ms for real-time rendering.

[0005] Secondly, the accuracy of occlusion detection is insufficient, easily leading to false "penetration" judgments. In complex scenes, when the Gaussian point is located at the edge of an occluder or nearly parallel to the surface of the scene geometry, the CPU's ray casting may incorrectly determine that the Gaussian ellipsoid has not been picked up due to insufficient sampling points or floating-point precision errors. For example, the actual intersection point of the ray and the occluder may be misjudged as "non-intersecting" due to errors, resulting in errors in the picked coordinates. Summary of the Invention

[0006] To overcome the shortcomings of the prior art, this invention provides a GPU-accelerated 3D Gaussian splash three-dimensional coordinate high-precision real-time picking method and system, which can solve the problem of penetration misjudgment in the existing three-dimensional coordinate picking and improve the picking accuracy and picking efficiency.

[0007] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0008] The first aspect of this invention discloses a three-dimensional coordinate picking method based on GPU and 3D Gaussian splashing, comprising the following steps:

[0009] Several Gaussian ellipsoids are generated based on the data information of the target scene, and the several Gaussian ellipsoids are projected onto a two-dimensional screen through a view projection matrix;

[0010] Calculate the projected coordinates of several Gaussian ellipsoids, and calculate the depth values ​​of several Gaussian ellipsoids based on the projected coordinates;

[0011] The depth value is converted into a color value, and the target color value is calculated based on the color value;

[0012] Pick the target color value, and calculate the three-dimensional coordinates of several Gaussian ellipsoids based on the target color value.

[0013] Furthermore, the step of calculating the projected coordinates of several Gaussian ellipsoids, calculating the depth values ​​of several Gaussian ellipsoids based on the projected coordinates, and storing the depth values ​​in a pre-created depth texture further includes:

[0014] Calculate the projected coordinates of several Gaussian ellipsoids based on their model coordinates and view projection matrices.

[0015] Furthermore, the step of calculating the projected coordinates of the plurality of Gaussian ellipsoids and calculating the depth values ​​of the plurality of Gaussian ellipsoids based on the projected coordinates further includes:

[0016] Create a depth buffer in the CPU's video memory and store the depth value in the depth buffer;

[0017] Create a depth texture in the GPU and send the data in the depth buffer to the depth texture.

[0018] Furthermore, storing the depth value into the depth buffer further includes:

[0019] The world coordinates of several Gaussian ellipsoids are converted into camera space coordinates using a view matrix;

[0020] Calculate the depth sort of all Gaussian ellipsoids based on the three-dimensional coordinates and matrix vectors of the centers of several Gaussian ellipsoids;

[0021] The depth values ​​are sorted according to their depth and stored in the depth buffer.

[0022] Furthermore, the step of converting the depth value into a color value and calculating the target color value based on the color value further includes:

[0023] In the GPU's shader, the depth value is converted into a color value;

[0024] Calculate the color compensation value based on the color value;

[0025] Calculate the target color value based on the color compensation value.

[0026] Furthermore, the step of picking the target color value and calculating the three-dimensional coordinates of several Gaussian ellipsoids based on the target color value also includes:

[0027] A selection operation is performed on the interface where the target color value is located. Based on the texture sampling function and the screen coordinates of the click position corresponding to the selection operation on the interface, the target color value corresponding to the click position is obtained.

[0028] Based on the target color value of the clicked location, calculate the target depth value corresponding to the clicked location, and determine the three-dimensional coordinates corresponding to the clicked location based on the target depth value.

[0029] Furthermore, the step of calculating the target depth value corresponding to the click location based on the target color value of the click location, and determining the three-dimensional coordinates corresponding to the click location based on the target depth value, further includes:

[0030] Calculate normalized device coordinates based on the screen coordinates of the clicked location and the target depth value;

[0031] Based on the inverse of the projection matrix, the normalized device coordinates are converted into camera space coordinates;

[0032] The camera spatial coordinates are converted into three-dimensional coordinates based on the inverse of the view matrix.

[0033] The second aspect of this invention discloses a three-dimensional coordinate picking system based on GPU and 3D Gaussian splashing, comprising:

[0034] The preprocessing module is used to generate several Gaussian ellipsoids based on the data information of the target scene, and project the several Gaussian ellipsoids onto a two-dimensional screen through a view projection matrix;

[0035] A depth value calculation module is used to calculate the projected coordinates of several Gaussian ellipsoids and calculate the depth value of several Gaussian ellipsoids based on the projected coordinates.

[0036] The target color value calculation module is used to convert the depth value into a color value and calculate the target color value based on the color value;

[0037] The three-dimensional coordinate calculation module is used to pick up the target color value and calculate the three-dimensional coordinates of several Gaussian ellipsoids based on the target color value.

[0038] A third aspect of the present invention discloses an electronic device, including a memory and a processor, wherein the memory is used to store a program and the processor is used to execute the program to implement the method as described in any of the first aspects.

[0039] The fourth aspect of the present invention discloses a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method as described in any of the first aspects.

[0040] Due to the application of the above technical solution, the present invention has the following advantages compared with the prior art:

[0041] 1. This application improves the color accuracy of depth texture by dynamically calculating the color accuracy compensation value of depth texture, thereby dynamically calculating the depth value of each Gaussian ellipsoid in the two-dimensional screen space, and calculating the three-dimensional coordinates based on the depth value. This can completely solve the problem of penetration misjudgment caused by the distribution characteristics of Gaussian ellipsoids in traditional picking methods, and improve the picking accuracy of three-dimensional coordinates, so that the single-point picking accuracy of 3DGS is improved from a deviation of tens of meters or no picking at all to centimeter-level accuracy.

[0042] 2. This application calculates the three-dimensional coordinates of the Gaussian ellipsoid by executing the calculation on the GPU, giving full play to the advantages of GPU parallel computing. The CPU only undertakes the tasks of initialization configuration and data transmission, which can significantly reduce the CPU computing load, improve the calculation efficiency and picking efficiency of coordinates, and improve the response speed. This improves the single-point picking speed of 3DGS from more than 100ms in CPU calculation mode to less than 30ms.

[0043] To make the above and other objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0044] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0045] Figure 1 This is a flowchart of a GPU-accelerated 3D Gaussian splash three-dimensional coordinate high-precision real-time picking method provided in an embodiment of this application;

[0046] Figure 2 The image shown is the original image obtained by the high-precision real-time 3D coordinate picking method for GPU acceleration provided in the embodiments of this application.

[0047] Figure 3 This is a pickup effect diagram obtained by the GPU-accelerated 3D Gaussian splash three-dimensional coordinate high-precision real-time pickup method provided in the embodiments of this application. Detailed Implementation

[0048] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. In addition, the accompanying drawings of the present invention are only simple illustrations and are not depictions based on actual dimensions, as stated in advance.

[0049] In this invention, it should be noted that the terms "upper," "lower," "inner," "outer," "forward," "backward," "between," "near," and "farthest" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. These terms are used only for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. It should also be noted that, unless otherwise explicitly specified and limited, the terms "installation" and "connection" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a direct connection or an indirect connection. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0050] It should be understood that while terms such as "first," "second," and "third" may be used in this document to describe various components or signals, these components or signals should not be limited by these terms. These terms are primarily used to distinguish one component from another, or one signal from another. Furthermore, the term "or" as used herein should, as appropriate, include any combination of one or more of the related listed items.

[0051] Example 1: Refer to Figure 1 This application provides a GPU-accelerated method for high-precision real-time acquisition of 3D Gaussian splash coordinates, including:

[0052] Step 101: Generate several Gaussian ellipsoids based on the data information of the target scene, and project the several Gaussian ellipsoids onto a two-dimensional screen through a view projection matrix.

[0053] Specifically, the target scene is the scene to be reconstructed in 3D and whose coordinates are picked. The data information can be 3D point cloud data. The Gaussian ellipsoid is a mathematical model of a regular rotating ellipsoid used to approximate the shape of the Earth (or other spherical bodies). A customized ellipsoid model that fits the needs of the scene can be obtained using existing technologies. For example, the shape features of the target objects in the scene can be extracted by data fitting, the ellipsoid parameters that conform to the scene can be calculated, and finally the Gaussian ellipsoid model can be constructed. This application will not describe it in detail here.

[0054] The view projection matrix is ​​a core mathematical tool in computer graphics that converts points in 3D space into 2D screen coordinates. It consists of two parts: the "view matrix" and the "projection matrix," which together complete the mapping from a 3D scene to a 2D image.

[0055] In this embodiment, the view projection matrix is ​​calculated using left multiplication to ensure the efficiency and accuracy of the coordinate transformation. The specific calculation formula is: VP = V,

[0056] Where V is the view matrix, which is used to convert the world coordinate system of the Gaussian ellipsoid into a left-handed coordinate system with the camera as the origin. It is constructed by the camera's position, orientation (target point), and up vector (usually the direction perpendicular to the line of sight).

[0057] P is the projection matrix, used to convert 3D coordinates in camera space into clip space coordinates, determining the perspective effect of the scene.

[0058] Step 102: Calculate the projected coordinates of several Gaussian ellipsoids, and calculate the depth values ​​of several Gaussian ellipsoids based on the projected coordinates.

[0059] In this embodiment, the projected coordinates of several Gaussian ellipsoids are calculated based on the model coordinates and view projection matrix of several Gaussian ellipsoids.

[0060] The specific calculation formula is as follows: The formula for calculating the projected coordinates of the Gaussian ellipsoid is: P' = ;

[0061] Where P' represents the projected coordinates of the Gaussian ellipsoid. The three-dimensional coordinates of the center of the Gaussian ellipsoid are: For view projection matrix;

[0062] The formula for calculating the depth value of a Gaussian ellipsoid is: depth = ( ;

[0063] in, Let P' be the value of P' in the z-component. Let P' be the value of P' on the w component.

[0064] This step also includes:

[0065] Create a depth buffer in the CPU's video memory and store the depth value in the depth buffer;

[0066] Create a depth texture in the GPU and send the data in the depth buffer to the depth texture.

[0067] Specifically, the depth buffer is a data structure in computer graphics used to store the depth value of each pixel. The depth value is a normalized value stored in the depth buffer. The depth texture is a special texture pre-created in the GPU to store the data in the depth buffer. The size of the depth texture is exactly the same as the screen resolution (e.g., 1920×1080). A GPU is a microprocessor specifically designed for performing image and graphics-related computations on personal computers, workstations, game consoles, and some mobile devices (such as tablets and smartphones).

[0068] By using the above method, color value picking and depth value calculation can be performed entirely on the GPU, giving full play to the advantages of GPU parallel computing. The CPU only undertakes the tasks of initialization configuration and data transmission, which significantly reduces the CPU computing load and improves computing efficiency.

[0069] Optionally, the depth buffer can be created on the CPU using the WebGL interface createTexture. After compiling and executing the shader using WebGL to write the depth value of each Gaussian ellipsoid into the depth buffer, the data information of the depth buffer on the CPU can be copied to the depth texture on the GPU using a GPU API (such as OpenGL's glTexImage2D or DirectX's UpdateSubresource).

[0070] The step of "storing the depth value into the depth buffer" further includes:

[0071] The world coordinates of several Gaussian ellipsoids are converted into camera space coordinates using a view matrix;

[0072] Calculate the depth sort of all Gaussian ellipsoids based on the three-dimensional coordinates and matrix vectors of the centers of several Gaussian ellipsoids;

[0073] The depth values ​​are sorted according to their depth and stored in the depth buffer.

[0074] The depth in the depth sorting is the projected length of the Gaussian ellipsoid along the camera's optical axis, and its calculation formula is: z = ,

[0075] Where z is the projected length of the Gaussian ellipsoid along the camera's optical axis, and P is the three-dimensional coordinate of the center of the Gaussian ellipsoid. The view matrix is ​​oriented by the vector (Z-axis).

[0076] Optionally, the depth sorting order is from largest to smallest. By calculating the depth sorting, the drawing order of each Gaussian ellipsoid in the scene can be determined. The depth values ​​are written to the depth buffer according to the depth sorting to avoid Gaussians in the distance from occluding nearby Gaussians.

[0077] Step 103: Convert the depth value into a color value, and calculate the target color value based on the color value.

[0078] Specifically, this step includes:

[0079] Step 1031: In the GPU shader, the depth value is converted into a color value;

[0080] In this embodiment, the color value is formatted as an RGBA vector. In computer graphics, color computing, or programming, an RGBA vector typically refers to a mathematical structure that represents the four channel values ​​of RGBA in vector form.

[0081] Since depth values ​​cannot be directly picked up in the GPU, they need to be converted into color values ​​before the depth value can be calculated. The specific conversion formula is as follows: ;

[0082] in, For color values, The depth value is a vector formatted as an RGBA vector, and both the color value and the depth value are in the form of an RGBA vector.

[0083] Step 1032: Calculate the color compensation value based on the color value;

[0084] In the process of converting depth values ​​to color values, some color values ​​are lost, which leads to coordinate errors during coordinate picking. In order to improve the accuracy of coordinate picking, in this embodiment, the color value of each coordinate is dynamically compensated by calculating an accuracy compensation value to avoid the loss of color values.

[0085] The formula for calculating the color compensation value is: ;

[0086] Step 1033: Calculate the target color value based on the color compensation value.

[0087] The calculation formula is: ;

[0088] in, In order to be in The accuracy enhancement value after compensation, which is the target color value. The color value for the decimal part. for The value of the y component, for In the value of the z component, for The value of the w component.

[0089] Through the above steps, the rounding error of the high-order channel can be dynamically compensated to the adjacent low-order channel, thereby solving the quantization distortion problem caused by traditional 8-bit channel storage, improving the color accuracy of depth texture, and thus improving the picking accuracy of 3D coordinates.

[0090] Step 104: Pick the target color value and calculate the three-dimensional coordinates of several Gaussian ellipsoids based on the target color value.

[0091] Specifically, this step includes:

[0092] Step 1041: Perform a selection operation on the interface where the target color value is located, and obtain the target color value corresponding to the click position based on the screen coordinates of the click position in the interface corresponding to the selection operation and the texture sampling function.

[0093] The selection operation can be generated by the mouse or the keyboard, and this application embodiment does not limit the way the selection operation is generated. For example, the selection operation can be generated by the user clicking a certain position on the interface with the mouse. Accordingly, the click position corresponding to the selection operation on the current interface is the click position of the user's mouse on the current interface.

[0094] For example, if the screen coordinates of the current mouse click position are (u, v), the color value at that position can be read from the depth texture using a texture sampling function (such as texture(depthTexture, vec2(u, v)) in GLSL).

[0095] Step 1042: Calculate the target depth value corresponding to the clicked position based on the target color value of the clicked position, and determine the three-dimensional coordinates corresponding to the clicked position based on the target depth value.

[0096] Specifically, the target depth value can be calculated using the following formula: ;

[0097] = ;

[0098] in, For color values, For normalized color values, for The value of the r component. for The value in the g component, for The value in the b component, for The value of component a. The target depth value.

[0099] Based on the above embodiments and calculation methods, the target depth value can be made the same as the depth value calculated in step 102, thereby avoiding errors in calculating the three-dimensional coordinates corresponding to the click position. Compared with the existing technology of directly reading and picking coordinate information on the CPU, this method can make full use of GPU resources, save CPU memory, and reduce data transmission and CPU processing time, greatly improving the accuracy and efficiency of coordinate picking.

[0100] Specifically, this step also includes:

[0101] Step 1042.1: Calculate normalized device coordinates based on the screen coordinates of the clicked location and the target depth value;

[0102] The specific calculation formula is: X ndc = ;

[0103] Y ndc = - Z ndc = ;

[0104] Step 1042.2: Based on the inverse of the projection matrix, convert the normalized device coordinates into camera space coordinates;

[0105] The specific calculation formula is: P c = ;

[0106] Step 1042.3: Based on the inverse of the view matrix, convert the camera spatial coordinates into three-dimensional coordinates.

[0107] The specific calculation formula is: P w = ;

[0108] Where (u,v) are the screen coordinates of the current mouse click position, u is the horizontal coordinate, v is the vertical coordinate, w is the viewport width, h is the viewport height, P is the projection matrix, V is the view matrix, and P c P represents the camera's spatial coordinates. w For world space coordinates.

[0109] Reference Figure 2 and Figure 3 This application provides a comparison of the original image and the resulting image using a GPU-accelerated 3D Gaussian splash three-dimensional coordinate high-precision real-time picking method. The method provided in this application has high accuracy in picking three-dimensional coordinates. It should be noted that the image used in this application embodiment is a landscape image that can be seen everywhere by the roadside.

[0110] Example 2: This application also provides a GPU-accelerated 3D Gaussian splash three-dimensional coordinate high-precision real-time picking system, including:

[0111] The preprocessing module is used to generate several Gaussian ellipsoids based on the data information of the target scene, and project the several Gaussian ellipsoids onto a two-dimensional screen through a view projection matrix;

[0112] A depth value calculation module is used to calculate the projected coordinates of several Gaussian ellipsoids and calculate the depth value of several Gaussian ellipsoids based on the projected coordinates.

[0113] The target color value calculation module is used to convert the depth value into a color value and calculate the target color value based on the color value;

[0114] The three-dimensional coordinate calculation module is used to pick up the target color value and calculate the three-dimensional coordinates of several Gaussian ellipsoids based on the target color value.

[0115] It should be noted that the module division in the above embodiments is illustrative and represents only one logical functional division. In actual implementation, other division methods are possible. For example, two or more functions can be integrated into one processing module. The integrated module can be implemented in hardware or as a software functional unit.

[0116] Example 3: This application also provides an electronic device, including a memory and a processor. The memory is used to store a program, and the processor is used to execute the program to implement the method described in Example 1.

[0117] Example 4: This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the method described in Example 1.

[0118] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented using software programs, implementation can be, in whole or in part, in the form of a computer program product. This computer program product includes one or more computer-executable instructions. When these computer-executable instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer-executable instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer-executable instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device containing one or more servers, data centers, etc., that can be integrated with the medium. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a solid-state disk (SSD), etc.

[0119] Specific embodiments have been used to illustrate the principles and implementation methods of this invention. The descriptions of the embodiments above are only for the purpose of helping to understand the method and core ideas of this invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this invention. Therefore, the content of this specification should not be construed as a limitation of this invention.

Claims

1. A GPU-accelerated method for high-precision real-time acquisition of 3D Gaussian splash coordinates, characterized in that, Includes the following steps: Several Gaussian ellipsoids are generated based on the data information of the target scene, and the several Gaussian ellipsoids are projected onto a two-dimensional screen through a view projection matrix; Calculate the projected coordinates of several Gaussian ellipsoids, and calculate the depth values ​​of several Gaussian ellipsoids based on the projected coordinates; The step of calculating the projected coordinates of several Gaussian ellipsoids and calculating the depth values ​​of several Gaussian ellipsoids based on the projected coordinates further includes: Create a depth buffer in the CPU's video memory and store the depth value in the depth buffer; Create a depth texture in the GPU and send the data in the depth buffer to the depth texture; The depth value is converted into a color value, and the target color value is calculated based on the color value; Pick the target color value, and calculate the three-dimensional coordinates of several Gaussian ellipsoids based on the target color value.

2. The GPU-accelerated 3D Gaussian splash three-dimensional coordinate high-precision real-time picking method according to claim 1, characterized in that, The step of calculating the projected coordinates of several Gaussian ellipsoids and calculating the depth values ​​of several Gaussian ellipsoids based on the projected coordinates further includes: Calculate the projected coordinates of several Gaussian ellipsoids based on their model coordinates and view projection matrices.

3. The GPU-accelerated 3D Gaussian splash three-dimensional coordinate high-precision real-time picking method according to claim 1, characterized in that, The step of storing the depth value into the depth buffer further includes: The world coordinates of several Gaussian ellipsoids are converted into camera space coordinates using a view matrix; Calculate the depth sort of all Gaussian ellipsoids based on the three-dimensional coordinates and matrix vectors of the centers of several Gaussian ellipsoids; The depth values ​​are sorted according to their depth and stored in the depth buffer.

4. The GPU-accelerated 3D Gaussian splash three-dimensional coordinate high-precision real-time picking method according to claim 1, characterized in that, The step of converting the depth value into a color value and calculating the target color value based on the color value further includes: In the GPU's shader, the depth value is converted into a color value; Calculate the color compensation value based on the color value; Calculate the target color value based on the color compensation value.

5. The GPU-accelerated 3D Gaussian splash three-dimensional coordinate high-precision real-time picking method according to claim 1, characterized in that, The step of picking up the target color value and calculating the three-dimensional coordinates of several Gaussian ellipsoids based on the target color value further includes: A selection operation is performed on the interface where the target color value is located. Based on the texture sampling function and the screen coordinates of the click position corresponding to the selection operation on the interface, the target color value corresponding to the click position is obtained. Based on the target color value of the clicked location, calculate the target depth value corresponding to the clicked location, and determine the three-dimensional coordinates corresponding to the clicked location based on the target depth value.

6. The GPU-accelerated 3D Gaussian splash three-dimensional coordinate high-precision real-time picking method according to claim 5, characterized in that, The step of calculating the target depth value corresponding to the clicked location based on the target color value of the clicked location, and determining the three-dimensional coordinates corresponding to the clicked location based on the target depth value, further includes: Calculate normalized device coordinates based on the screen coordinates of the clicked location and the target depth value; Based on the inverse of the projection matrix, the normalized device coordinates are converted into camera space coordinates; The camera spatial coordinates are converted into three-dimensional coordinates based on the inverse of the view matrix.

7. A GPU-accelerated 3D Gaussian splash three-dimensional coordinate high-precision real-time acquisition system, characterized in that, include: The preprocessing module is used to generate several Gaussian ellipsoids based on the data information of the target scene, and project the several Gaussian ellipsoids onto a two-dimensional screen through a view projection matrix; A depth value calculation module is used to calculate the projected coordinates of several Gaussian ellipsoids, and to calculate the depth value of several Gaussian ellipsoids based on the projected coordinates. The calculation of the projected coordinates of several Gaussian ellipsoids and the calculation of the depth value of several Gaussian ellipsoids based on the projected coordinates further includes: Create a depth buffer in the CPU's video memory and store the depth value in the depth buffer; Create a depth texture in the GPU and send the data in the depth buffer to the depth texture; The target color value calculation module is used to convert the depth value into a color value and calculate the target color value based on the color value; The three-dimensional coordinate calculation module is used to pick up the target color value and calculate the three-dimensional coordinates of several Gaussian ellipsoids based on the target color value.

8. An electronic device, characterized in that, It includes a memory and a processor, the memory being used to store a program, and the processor being used to execute the program to implement the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 6.

Images