Three-dimensional data processing method, device, equipment, storage medium and program product

Abstract

This application discloses a method, apparatus, device, storage medium, and program product for processing three-dimensional data, relating to the field of server technology. The method includes: in response to a distribution controller receiving a processing request for three-dimensional data, obtaining task identification information corresponding to the three-dimensional data from the processing request, the task identification information including at least a data type identifier and a tree structure identifier; determining, based on the tree structure identifier, whether the three-dimensional data requires tree structure loading processing; if so, invoking a tree structure traversal unit to load the three-dimensional data; and in response to the completion of three-dimensional data loading, determining the data type of the three-dimensional data based on the data type identifier, and invoking a neural rendering unit and / or an arithmetic calculation unit to process the three-dimensional data based on the data type. This method can improve the processing efficiency of three-dimensional data.

Description

Technical Field

[0001] This application relates to the field of server technology, and in particular to a method, apparatus, device, storage medium, and program product for processing three-dimensional data. Background Technology

[0002] 3D scenes are typically constructed from one or more types of 3D data. These types of 3D data can include neural network-based 3D data (i.e., neural network 3D data) and triangular mesh data. Currently, the demand for rendering mixed types of 3D data is growing.

[0003] In related technologies, 3D graphics processing mainly relies on image processors and accelerators. For example, image processors process traditional triangular mesh data, while dedicated accelerators process neural network 3D data. However, image processors and accelerators are designed for single data types and are difficult to efficiently process multiple types of mixed 3D data simultaneously, resulting in low processing efficiency for 3D data. Summary of the Invention

[0004] This application provides a method, apparatus, device, storage medium, and program product for processing three-dimensional data, in order to at least solve the problem of low processing efficiency of three-dimensional data in related technologies.

[0005] On one hand, this application provides a method for processing three-dimensional data, applied to a three-dimensional computing accelerator. The three-dimensional computing accelerator includes an allocation controller, a tree structure traversal unit, a neural rendering unit, and an arithmetic calculation unit. The processing method includes:

[0006] In response to the distribution controller receiving a processing request for 3D data, the task identification information corresponding to the 3D data is obtained from the processing request. The task identification information includes at least a data type identifier and a tree structure identifier.

[0007] The allocation controller determines whether the 3D data needs tree structure loading processing based on the tree structure identifier. If so, the allocation controller calls the tree structure traversal unit to load the 3D data.

[0008] In response to the completion of 3D data loading, the data type of the 3D data is determined by the allocation controller based on the data type identifier, and the neural rendering unit and / or arithmetic calculation unit are invoked to process the 3D data according to the data type.

[0009] On the other hand, this application provides a three-dimensional data processing apparatus applied to a three-dimensional computing accelerator. The three-dimensional computing accelerator includes an allocation controller, a tree structure traversal unit, a neural rendering unit, and an arithmetic calculation unit. The processing apparatus includes:

[0010] The acquisition module is used to respond to the processing request for three-dimensional data received by the allocation controller, and to obtain the task identification information corresponding to the three-dimensional data from the processing request. The task identification information includes at least the data type identifier and the tree structure identifier.

[0011] The determination module is used to determine whether the 3D data needs tree structure loading processing based on the tree structure identifier through the allocation controller. If so, the 3D data is loaded by calling the tree structure traversal unit through the allocation controller.

[0012] The processing module, in response to the completion of 3D data loading, determines the data type of the 3D data by means of the allocation controller based on the data type identifier, and calls the neural rendering unit and / or arithmetic calculation unit to process the 3D data according to the data type.

[0013] This application provides a method, apparatus, device, storage medium, and program product for processing three-dimensional data. The processing method includes: in response to a distribution controller receiving a processing request for three-dimensional data, obtaining task identification information corresponding to the three-dimensional data from the processing request, the task identification information including at least a data type identifier and a tree structure identifier; determining, based on the tree structure identifier, whether the three-dimensional data requires tree structure loading processing; if so, calling a tree structure traversal unit to load the three-dimensional data; and in response to the completion of three-dimensional data loading, determining the data type of the three-dimensional data based on the data type identifier, and calling a neural rendering unit and / or an arithmetic calculation unit to process the three-dimensional data based on the data type. In this embodiment, when 3D data requires tree structure loading processing, the allocation controller can call the tree structure traversal unit to load the 3D data. Furthermore, the allocation controller can call the neural rendering unit and / or arithmetic calculation unit to process the 3D data based on the data type identifier. Thus, the processing device can determine whether the 3D data needs tree structure acceleration through the tree structure identifier, and call the neural rendering unit and / or arithmetic calculation unit to process different types of 3D data based on the data type identifier. In other words, the automatic control of the data processing flow is achieved through the data type identifier and the tree structure identifier, breaking through the limitation of processing a single data type in the prior art, solving the task scheduling problem of mixed 3D data, and thus improving the processing efficiency of 3D data. Attached Figure Description

[0014] To more clearly illustrate the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0015] Figure 1A schematic diagram of the structure of the three-dimensional computing accelerator provided in the embodiments of this application. Figure 1 ;

[0016] Figure 2 The flowchart of the three-dimensional data processing method provided in the embodiments of this application Figure 1 ;

[0017] Figure 3 A schematic diagram of the task identification information provided in the embodiments of this application. Figure 1 ;

[0018] Figure 4 A schematic diagram of the three-dimensional data processing method provided in the embodiments of this application. Figure 2 ;

[0019] Figure 5 A schematic diagram of the neural rendering unit provided in the embodiments of this application. Figure 1 ;

[0020] Figure 6 This application provides a schematic diagram of the data processing of the index calculation unit in an embodiment of the index. Figure 1 ;

[0021] Figure 7 Data processing illustration of the lookup unit provided in the embodiments of this application Figure 1 ;

[0022] Figure 8 A schematic diagram of the feature hash table provided in the embodiments of this application. Figure 1 ;

[0023] Figure 9 Illustration of multiple resolution features provided in embodiments of this application Figure 1 ;

[0024] Figure 10 A schematic diagram of the arithmetic calculation unit provided in the embodiments of this application. Figure 1 ;

[0025] Figure 11 This application provides a schematic diagram of data processing for a vector multiply-accumulate unit in an embodiment of the present application. Figure 1 ;

[0026] Figure 12 This is a schematic diagram of the structure of the three-dimensional data processing device provided in the embodiments of this application;

[0027] Figure 13 A schematic diagram of the structure of the electronic device provided in this application. Detailed Implementation

[0028] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.

[0029] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0030] 3D scenes are typically constructed from one or more types of 3D data. These 3D data types can include neural network-based 3D data and triangular mesh data. Currently, the demand for rendering hybrid 3D data is increasing; for example, in autonomous driving, there is a need for real-time rendering of hybrid scenes that incorporate dynamic environmental information generated by neural networks and static road models.

[0031] In related technologies, 3D graphics processing mainly relies on image processors and accelerators. For example, image processors process traditional triangular mesh data, while dedicated accelerators process neural network 3D data. However, image processors and accelerators are designed for single data types and are difficult to efficiently process multiple types of mixed 3D data simultaneously, resulting in low processing efficiency for 3D data.

[0032] Therefore, how to process triangular mesh data, neural network 3D data, and hybrid 3D data represented by both to improve the processing efficiency of 3D data is a technical problem that urgently needs to be solved.

[0033] To address the aforementioned technical problems, this application proposes a 3D computing accelerator capable of processing 3D neural network data, triangular mesh data, and hybrid representations of both. Through a modular 3D computing accelerator, a shared cache mechanism, and configurable computing units, it achieves efficient collaborative processing of heterogeneous data. Optionally, as... Figure 1 As shown, the task scheduler distributes computing tasks to different 3D computing accelerators for execution, and the different 3D computing accelerators exchange and share data through a shared cache.

[0034] Optionally, the 3D computing accelerator includes an allocation controller, an arithmetic computation unit, a tree structure traversal unit, and a neural rendering unit. The arithmetic computation unit, tree structure traversal unit, and neural rendering unit are used to perform 3D data rendering computation tasks. The allocation controller is used to schedule tasks among the various computing units within the 3D computing accelerator, and the various computing units exchange and share data through caching. The specific process of processing 3D data through the 3D computing accelerator may include: in response to the allocation controller receiving a 3D data processing request, obtaining the task identification information corresponding to the 3D data from the processing request; the task identification information includes at least a data type identifier and a tree structure identifier; determining, based on the tree structure identifier, whether the 3D data requires tree structure loading processing; if so, calling the tree structure traversal unit to load the 3D data; and in response to the completion of 3D data loading, determining the data type of the 3D data based on the data type identifier, and calling the neural rendering unit and / or the arithmetic computation unit to process the 3D data based on the data type.

[0035] In this embodiment, when 3D data requires tree structure loading processing, the allocation controller can call the tree structure traversal unit to load the 3D data. Furthermore, the allocation controller can call the neural rendering unit and / or arithmetic calculation unit to process the 3D data based on the data type identifier. Thus, the processing device can determine whether the 3D data needs tree structure acceleration through the tree structure identifier, and call the neural rendering unit and / or arithmetic calculation unit to process different types of 3D data based on the data type identifier. In other words, the automatic control of the data processing flow is achieved through the data type identifier and the tree structure identifier, breaking through the limitation of processing a single data type in the prior art, solving the task scheduling problem of mixed 3D data, and thus improving the processing efficiency of 3D data.

[0036] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0037] The specific application environment architecture or specific hardware architecture on which the execution of the combined 3D data processing method depends is described here.

[0038] Figure 2 The flowchart of the three-dimensional data processing method provided in the embodiments of this application Figure 1 The execution entity for this three-dimensional data processing method can be a three-dimensional computing accelerator. For example... Figure 2 As shown, the processing method for this three-dimensional data includes:

[0039] S201. In response to the distribution controller receiving a processing request for three-dimensional data, the task identification information corresponding to the three-dimensional data is obtained from the processing request. The task identification information includes at least a data type identifier and a tree structure identifier.

[0040] In this embodiment, the three-dimensional data can be the three-dimensional data to be processed in a three-dimensional scene. This three-dimensional data can be triangular mesh data, neural network three-dimensional data, or a mixture of both.

[0041] Optionally, the task identification information is used to indicate the type of 3D data and the processing flow. In some embodiments, the task identification information includes at least a data type identifier and a tree structure identifier. The data type identifier indicates the type of 3D data. The tree structure identifier indicates whether the 3D data requires tree structure loading processing.

[0042] In some embodiments, such as Figure 3 As shown, the task identification information may also include 3D data address information and tree structure data address information. Specifically, the data type identifier is the identifier information corresponding to the data type identifier bit; the tree structure identifier is the identifier information corresponding to the tree structure identifier bit; the 3D data address information is the address information corresponding to the 3D data address bit, which includes the address of the 3D data in memory; and the tree structure data address information is the address information corresponding to the tree structure data address bit, which includes the address of the tree structure data in memory.

[0043] For example, such as Figure 3 As shown, the first two bits of the task identification information are data type identifier bits, indicating the type of 3D data to be processed. Optionally, the value of the data type identifier bit includes 0, 1, 2, and 3; the corresponding 3D data types are 0: neural network 3D data, 1: triangular mesh data, 2: hybrid 3D data, and 3: other 3D data. The third bit of the task identification information is the tree structure identifier bit, indicating whether the processed 3D data uses tree structure acceleration. The values ​​of the tree structure identifier bit have the following meanings: 0: no tree structure acceleration; 1: tree structure acceleration. Bits 4-16 of the task identification information are tree structure data address bits, used to store the memory address of the tree acceleration structure data. Bits 17-32 of the task identification information are 3D data address bits, used to store the memory address of the 3D data to be processed.

[0044] S202. The allocation controller determines whether the 3D data needs tree structure loading processing based on the tree structure identifier. If so, the allocation controller calls the tree structure traversal unit to load the 3D data.

[0045] In some embodiments, the allocation controller determines whether the 3D data requires tree structure loading processing based on the tree structure identifier. This includes: if the tree structure identifier is a fifth preset value, the allocation controller determines that the 3D data requires tree structure loading processing; in this case, the tree structure traversal unit is invoked to perform tree structure traversal calculation to load the 3D data. If the tree structure identifier is a sixth preset value, the allocation controller determines that the 3D data does not require tree structure loading processing; in this case, the tree structure traversal unit is skipped, and the 3D data is loaded directly. In this embodiment, the values ​​of the fifth and sixth preset values ​​are not specifically limited.

[0046] For example, the values ​​of the fifth and sixth preset values ​​can be 1 and 0, respectively. If the tree structure identifier is 1, the allocation controller determines that the 3D data requires tree structure loading processing; in this case, the tree structure traversal unit is called to perform tree structure traversal calculation to load the 3D data. If the tree structure identifier is 0, the allocation controller determines that the 3D data does not require tree structure loading processing; in this case, the tree structure traversal unit is skipped, and the 3D data is loaded directly.

[0047] In some embodiments, the task identification information further includes spatial point information of the 3D data, 3D data address information, and tree structure data address information; loading 3D data by calling the tree structure traversal unit through the allocation controller includes: loading the tree structure data corresponding to the tree structure traversal unit from the data storage unit according to the tree structure data address information through the tree structure traversal unit; determining the address offset information corresponding to the 3D data according to the spatial point information of the 3D data and the tree structure data through the tree structure traversal unit; determining the memory address information corresponding to the 3D data according to the address offset information and the 3D data address information through the tree structure traversal unit, and loading the 3D data from the data storage unit according to the memory address information.

[0048] The spatial point information represents the attribute information of the spatial point corresponding to the 3D data. Optionally, the spatial point information includes spatial point coordinates and view direction information. For example, the spatial point is point P, and the spatial point coordinates include the X-axis coordinates, Y-axis coordinates, and Z-axis coordinates of point P in 3D space. The view direction information represents the view direction information of the spatial point. Optionally, the view direction information includes polar angle information and azimuth angle information.

[0049] In some embodiments, the tree structure traversal unit includes a root node, intermediate nodes, and leaf nodes. Correspondingly, the tree structure traversal unit determines the address offset information corresponding to the 3D data based on the spatial point information of the 3D data and the tree structure data. This includes: dividing multiple triangular facet data in the tree structure data according to multiple node levels in the tree structure traversal unit to obtain a root node, intermediate nodes, and leaf nodes, where one triangular facet data corresponds to a rendering unit in the 3D scene; traversing multiple triangular facet data in the tree structure data in the order of root node, intermediate node, and leaf node according to the spatial point coordinates and view direction information in the spatial point information of the 3D data to determine the target triangular facet data corresponding to the spatial point information of the 3D data; and obtaining the address offset information corresponding to the 3D data from the target triangular facet data through the tree structure traversal unit.

[0050] In this embodiment of the application, since rendering requires calculating the intersection points of a large number of rays (such as camera rays and shadow rays) with geometric elements such as triangular meshes in the scene, the amount of computation increases linearly with the number of triangular faces. Among them, the tree structure traversal unit can divide the space by hierarchical bounding boxes, quickly eliminate non-intersecting geometric regions, greatly reduce the number of faces that need to be accurately intersected, and thus improve the loading efficiency of 3D data.

[0051] S203. In response to the completion of 3D data loading, the data type of the 3D data is determined by the allocation controller based on the data type identifier, and the neural rendering unit and / or arithmetic calculation unit are invoked to process the 3D data according to the data type of the 3D data.

[0052] In some embodiments, the allocation controller determines the data type of the three-dimensional data based on the data type identifier, including: if the data type identifier is a first preset value, the allocation controller determines that the data type of the three-dimensional data is neural network three-dimensional data; if the data type identifier is a second preset value, the allocation controller determines that the data type of the three-dimensional data is triangular mesh data; if the data type identifier is a third preset value, the allocation controller determines that the data type of the three-dimensional data is a hybrid three-dimensional data of neural network three-dimensional data and triangular mesh data; if the data type identifier is a fourth preset value, the allocation controller determines that the data type of the three-dimensional data is other types of three-dimensional data.

[0053] In this embodiment, the values ​​of the first, second, third, and fourth preset values ​​are not specifically limited. For example, the first, second, third, and fourth preset values ​​can be 0, 1, 2, and 3. If the data type identifier is 0, the allocation controller determines that the data type of the 3D data is neural network 3D data; if the data type identifier is 1, the allocation controller determines that the data type of the 3D data is triangular mesh data; if the data type identifier is 2, the allocation controller determines that the data type of the 3D data is a hybrid 3D data of neural network 3D data and triangular mesh data; if the data type identifier is 3, the allocation controller determines that the data type of the 3D data is other types of 3D data.

[0054] In some embodiments, the allocation controller invokes a neural rendering unit and / or an arithmetic computation unit to process the 3D data according to the data type of the 3D data, including: if the data type of the 3D data is neural network 3D data, the allocation controller first invokes the neural rendering unit to process the 3D data, and then invokes the arithmetic computation unit to process the 3D data; if the data type of the 3D data is triangular mesh data, the allocation controller invokes the arithmetic computation unit to process the 3D data; if the data type of the 3D data is hybrid 3D data, the allocation controller invokes both the neural rendering unit and the arithmetic computation unit to process the 3D data; if the data type of the 3D data is other types of 3D data, the allocation controller invokes the arithmetic computation unit to process the 3D data.

[0055] It should be noted that the 3D computing accelerator may include one or more arithmetic computing units. The aforementioned arithmetic computing unit, the first arithmetic computing unit, and the second arithmetic computing unit may be the same arithmetic computing unit within the 3D computing accelerator, or they may be different arithmetic computing units within the 3D computing accelerator. In this embodiment, the allocation controller can allocate 3D data computing tasks based on the load status of the multiple arithmetic computing units.

[0056] In some embodiments, such as Figure 4 As shown, the arithmetic calculation unit includes a first arithmetic calculation unit and a second arithmetic calculation unit; correspondingly, the neural rendering unit and the arithmetic calculation unit are called by the allocation controller to process the three-dimensional data, including: calling the first arithmetic calculation unit to process triangular mesh data by the allocation controller, and calling the neural rendering unit to process neural network three-dimensional data; the second arithmetic calculation unit is called by the allocation controller to fuse the triangular mesh data processed by the first arithmetic calculation unit and the neural network three-dimensional data processed by the neural rendering unit to obtain fused three-dimensional data, and then processing the fused three-dimensional data.

[0057] For example, such as Figure 4As shown, when the data type flag is 0, the 3D data is a neural network 3D data type. The controller calls the neural rendering unit and the arithmetic calculation unit to perform the neural 3D rendering calculation task. When the data type flag is 1, the 3D data is a triangular mesh data type. The controller calls the arithmetic calculation unit to perform the 3D mesh data rendering calculation task. When the data type flag is 2, the 3D data is a mixed representation of neural network 3D data and triangular mesh data. The controller calls the first arithmetic calculation unit, the neural rendering unit, and the second arithmetic calculation unit to perform the rendering calculation task for the mixed representation 3D data. When the data type flag is 3, the controller calls the arithmetic calculation unit to perform the rendering task for other types of 3D representation data.

[0058] It should be noted that the 3D computing accelerator may also include a caching module; wherein, the neural rendering unit and arithmetic computing unit in the 3D computing accelerator can interact with 3D data through the caching module.

[0059] For example, the data type of the 3D data is hybrid 3D data. The allocation controller calls the first arithmetic calculation unit to process the triangular mesh data, and then calls the neural rendering unit to process the neural network 3D data. The triangular mesh data processed by the first arithmetic calculation unit can be stored in a cache module, and the neural network 3D data processed by the neural rendering unit can also be stored in a cache module. In this case, the second arithmetic calculation unit can obtain the triangular mesh data processed by the first arithmetic calculation unit and the neural network data processed by the neural rendering unit from the cache module, fuse the two sets of data to obtain hybrid 3D data, and then process the hybrid 3D data.

[0060] In some embodiments, such as Figure 5 As shown, the neural rendering unit includes an encoding unit and a color calculation unit; the task identification information also includes spatial point information of the 3D data; correspondingly, calling the neural rendering unit to process 3D data may include the following steps S11 to S13:

[0061] S11. Call the encoding unit to obtain the spatial point coordinates from the spatial point information of the three-dimensional data.

[0062] S12. The spatial feature data corresponding to the spatial point coordinates is determined by the encoding unit. The spatial feature data includes the feature data of the eight vertices corresponding to the spatial voxel where the spatial point coordinates are located, as well as the weights corresponding to the vertex feature data.

[0063] In some embodiments, such as Figure 5As shown, the encoding unit includes an index calculation unit, a table lookup unit, and an interpolation calculation unit; correspondingly, determining the spatial feature data corresponding to the coordinates of a spatial point through the encoding unit may include the following steps (1) to (3):

[0064] (1) Determine the index information corresponding to the coordinates of the spatial point through the index calculation unit.

[0065] Optionally, the encoding unit includes calculation modules such as an index calculation unit, a table lookup unit, and an interpolation calculation unit, used to realize the function of converting spatial point coordinates into feature encoding calculations. The encoding unit first receives the sequence of intersection points between rays and the mesh model from the buffer, stores it in a FIFO (First In First Out) buffer unit, and then enters the index calculation unit.

[0066] The index calculation unit reads a point coordinate from the FIFO buffer unit, uses the previous point as the starting point of the ray, determines the line connecting the next adjacent point as the ray direction, and performs point sampling on the ray between the two points to obtain a series of spatial sample point coordinates. The sampling method here can be uniform sampling, which can be implemented using a multiply-accumulate tree to calculate the index of the sample point coordinates. The calculation of sample points can also be implemented using non-uniform sampling methods such as importance sampling. Optionally, such as... Figure 6 As shown, the index calculation unit can control the scaling of spatial point coordinates through the scale parameter, control the position offset of each coordinate component (x, y, z coordinates) in memory through the offset, and calculate the index position of each coordinate component in the spatial location storage table.

[0067] In some embodiments, spatial voxels can be hierarchically divided according to different resolutions, that is, the same spatial range is divided into voxels of different sizes. Combining multiple resolution division methods can achieve representation of space at different granularities. Optionally, determining the index information corresponding to the spatial point coordinates through the index calculation unit includes: dividing the three-dimensional space according to multiple preset resolution levels through the index calculation unit to obtain spatial voxels corresponding to each of the multiple preset resolution levels; and determining the index information of the spatial point coordinates at the multiple preset resolution levels based on the spatial voxels corresponding to each of the multiple preset resolution levels through the index calculation unit.

[0068] For example, the preset resolution levels include L1 resolution level, L2 resolution level, L3 resolution level, and L4 resolution level. At this time, the index calculation unit determines the index information of the spatial point coordinates under L1 resolution level, L2 resolution level, L3 resolution level, and L4 resolution level.

[0069] (2) By using the lookup table unit to determine the eight vertex feature data corresponding to the spatial voxel where the spatial point coordinates are located, based on the index information corresponding to the spatial point coordinates.

[0070] Optionally, the table lookup unit determines the eight vertex feature data corresponding to the spatial voxel where the spatial point coordinates are located based on the index information corresponding to the spatial point coordinates. This includes: performing hash processing on the index information corresponding to the spatial point coordinates through the table lookup unit to obtain the target hash index information corresponding to the spatial point coordinates; and obtaining the eight vertex feature data corresponding to the spatial voxel where the spatial point coordinates are located from the pre-stored correspondence between the hash index information and the vertex feature data based on the target hash index information through the table lookup unit.

[0071] The lookup unit, based on spatial point coordinates, can obtain the position index of the corresponding feature data in the hash table through a single lookup calculation unit. The lookup calculation unit can perform hash processing on the index information corresponding to the spatial point coordinates using the following hash calculation function to obtain the target hash index information corresponding to the spatial point coordinates.

[0072]

[0073] in, The symbol represents a bitwise XOR operation, where a bit is set to 1 if the corresponding binary bits of two numbers are different, and to 0 if the corresponding binary bits are the same. Where P... jk Represents the three coordinate components of a point in space. Represents large prime numbers, where =1, =2654435761, =80545961. mod represents the modulo operation, and N represents the maximum number of features stored in the hash table.

[0074] For example, such as Figure 7 As shown, the spatial point coordinate index includes the indexes corresponding to each coordinate component (x, y, z coordinates). First, the indexes corresponding to each coordinate component are XORed bit by bit through the lookup table unit, and then the hash value of the output result is determined to obtain the hash index information.

[0075] Optionally, such as Figure 8 As shown, the correspondence between pre-stored hash index information and vertex feature data can be represented by a feature hash table. For each spatial point, the pre-computed features of the eight vertices of its spatial voxel are stored in the hash table. Each hash entry in the feature hash table defines a structure including a hash index ID and the memory locations of the corresponding eight vertex feature data.

[0076] In some embodiments, spatial voxels can be hierarchically divided according to different resolutions. Accordingly, the lookup table unit determines the eight vertex feature data corresponding to the spatial voxel where the spatial point coordinates are located based on the index information corresponding to the spatial point coordinates. This includes: hashing the index information of the spatial point coordinates at multiple preset resolution levels using the lookup table unit to obtain multiple target hash index information corresponding to the spatial point coordinates; and obtaining the eight vertex feature data corresponding to the spatial voxels at multiple preset resolution levels from the pre-stored correspondence between the hash index information and vertex feature data using the lookup table unit based on the multiple target hash index information.

[0077] In this embodiment, the number of preset resolution levels is not specifically limited and can be set and modified as needed. For example, the preset resolution levels include L1 resolution level, L2 resolution level, L3 resolution level, and L4 resolution level. The lookup table unit determines the eight vertex feature data corresponding to the spatial voxel where the spatial point coordinates are located at L1 resolution level based on the index information at L1 resolution level; the same applies to L2 resolution level; L3 resolution level; and L4 resolution level.

[0078] (3) The interpolation calculation unit determines the weights of the feature data of the eight vertices based on the coordinates of the spatial points.

[0079] Optionally, the interpolation calculation unit can use the index of the spatial point to look up a table and calculate the weight value corresponding to each level of feature of that point.

[0080] S13. Using the color calculation unit, determine the rendered 3D data corresponding to the spatial point coordinates based on the eight vertex feature data corresponding to the spatial voxel where the spatial point coordinates are located and the weights corresponding to the vertex feature data.

[0081] Optionally, such as Figure 5As shown, the color calculation unit includes a color calculation subunit and a color aggregation unit. Accordingly, the color calculation unit determines the rendered 3D data corresponding to the spatial point coordinates based on the eight vertex feature data corresponding to the spatial voxel where the spatial point coordinates are located and the weights corresponding to the vertex feature data. This includes: performing weighted calculations through the color calculation subunit based on the eight vertex feature data corresponding to the spatial voxel where the spatial point coordinates are located and the weights corresponding to the vertex feature data to obtain the hidden feature information corresponding to the spatial point coordinates; and processing the hidden feature information corresponding to the spatial point coordinates through the color aggregation unit to obtain the rendered 3D data corresponding to the spatial point coordinates.

[0082] For example, features at multiple resolutions of the same spatial point are sequentially connected to form the final feature of that spatial point. For instance... Figure 9 As shown, based on the feature data and corresponding weights of the eight vertices of the same spatial point at resolution levels L1 to L4, the color calculation unit can obtain the final implicit feature corresponding to the sampling point through weighted calculation and store it in the cache unit.

[0083] In some embodiments, such as Figure 10 As shown, the arithmetic calculation unit includes multiple arithmetic calculation sub-units and a cache unit; correspondingly, processing three-dimensional data through the arithmetic calculation unit includes: processing the three-dimensional data through multiple arithmetic calculation sub-units to obtain processed three-dimensional data; and storing the processed three-dimensional data in the cache unit.

[0084] The arithmetic calculation unit can perform basic arithmetic calculations, while also supporting scalar and vector calculations, enabling numerical calculations for 3D data rendering and processing such as meshes.

[0085] Optionally, such as Figure 10 As shown, the arithmetic calculation subunit includes a vector multiplication and addition unit and a scalar multiplication and addition unit; correspondingly, the three-dimensional data is processed by multiple arithmetic calculation subunits to obtain the processed three-dimensional data, including: processing the vector data in the three-dimensional data by the vector multiplication and addition unit in the arithmetic calculation subunit, and processing the scalar data in the three-dimensional data by the scalar multiplication and addition unit in the arithmetic calculation subunit to obtain the processed three-dimensional data.

[0086] The scalar multiply-accumulate unit performs numerical multiply-accumulate calculations. The vector multiply-accumulate unit performs vector multiply-accumulate calculations. Furthermore, the vector multiply-accumulate unit is configurable in dimension and can perform 2-dimensional, 4-dimensional, 8-dimensional, 16-dimensional, and 128-dimensional vector multiply-accumulate calculation tasks respectively.

[0087] In some embodiments, the state of each configuration switch in the vector multiply-accumulate unit is determined according to the dimension of the vector data in the three-dimensional data, so as to realize multiply-accumulate calculations on vector data of different dimensions. Optionally, the vector multiply-accumulate unit includes multiple configuration switches; correspondingly, the vector data in the three-dimensional data is processed by the vector multiply-accumulate unit in the arithmetic calculation subunit, including: determining the configuration instruction corresponding to the vector multiply-accumulate unit according to the dimension of the vector data in the three-dimensional data by the vector multiply-accumulate unit in the arithmetic calculation subunit, the configuration instruction being used to indicate the switch state corresponding to each of the multiple configuration switches; configuring the switch state corresponding to each of the multiple configuration switches according to the configuration instruction, so as to realize multiply-accumulate calculations on vector data of different dimensions.

[0088] Optionally, the input to the vector multiply-accumulate unit includes configuration instructions and vector data, wherein the configuration instructions are used to control the processing flow of vector data by controlling the configuration switch, so as to realize multiply-accumulate calculations on vector data of different dimensions.

[0089] For example, such as Figure 11 As shown, when performing vector operations with a dimension of 2, the configuration switch after every two vector data points is in the open state, and the calculation result is directly written to the output register. At this time, the configuration switches after data a1-b2, data a3-b4, data a5-b6, data a7-b8, data a9-b10, data a11-b12, data a13-b14, and data a15-b16 are all in the open state, while the other configuration switches are in the closed state.

[0090] When performing vector operations with a dimension of 4, the configuration switch after every four vector data points is in the open state, and the calculation result is directly written to the output register. At this time, the configuration switches after data a1-b4, data a5-b8, data a9-b12, and data a13-b16 are all in the open state, while the other configuration switches are in the closed state.

[0091] When performing vector operations with a dimension of 8, the configuration switch after every 8 vector data points is in the open state, and the calculation result is directly written to the output register. At this time, the configuration switches after data a1-b8 and data a9-b16 are in the open state, while the other configuration switches are in the closed state.

[0092] When performing vector operations with a dimension of 16, the configuration switches after data a1-data b2, data a3-data b4, data a5-data b6, data a7-data b8, data a9-data b10, data a11-data b12, data a13-data b14, data a15-data b16, data a1-data b4, data a5-data b8, data a9-data b12, and data a13-data b16 are all in the off state.

[0093] In this embodiment, the data processing capability of the arithmetic calculation subunit is improved by controlling the vector data processing flow through the control configuration switch to perform multiplication and addition calculations on vector data of different dimensions.

[0094] This application provides a method for processing three-dimensional data: In response to a distribution controller receiving a processing request for three-dimensional data, the method obtains task identification information corresponding to the three-dimensional data from the processing request. The task identification information includes at least a data type identifier and a tree structure identifier. Based on the tree structure identifier, the distribution controller determines whether the three-dimensional data needs tree structure loading processing. If so, the distribution controller calls a tree structure traversal unit to load the three-dimensional data. In response to the completion of three-dimensional data loading, the distribution controller determines the data type of the three-dimensional data based on the data type identifier. Based on the data type of the three-dimensional data, the distribution controller calls a neural rendering unit and / or an arithmetic calculation unit to process the three-dimensional data. In this embodiment, when 3D data requires tree structure loading processing, the allocation controller can call the tree structure traversal unit to load the 3D data. Furthermore, the allocation controller can call the neural rendering unit and / or arithmetic calculation unit to process the 3D data based on the data type identifier. Thus, the processing device can determine whether the 3D data needs tree structure acceleration through the tree structure identifier, and call the neural rendering unit and / or arithmetic calculation unit to process different types of 3D data based on the data type identifier. In other words, the automatic control of the data processing flow is achieved through the data type identifier and the tree structure identifier, breaking through the limitation of processing a single data type in the prior art, solving the task scheduling problem of mixed 3D data, and thus improving the processing efficiency of 3D data.

[0095] Figure 12 This is a schematic diagram of a three-dimensional data processing device provided in an embodiment of this application. The three-dimensional data processing device is applied to a three-dimensional computing accelerator, which includes an allocation controller, a tree structure traversal unit, a neural rendering unit, and an arithmetic calculation unit, such as… Figure 12 As shown, the device includes:

[0096] The acquisition module 1201 is used to respond to the processing request of the three-dimensional data received by the allocation controller, and to obtain the task identification information corresponding to the three-dimensional data from the processing request. The task identification information includes at least the data type identifier and the tree structure identifier.

[0097] The determination module 1202 is used to determine whether the 3D data needs tree structure loading processing based on the tree structure identifier through the allocation controller. If so, the 3D data is loaded by calling the tree structure traversal unit through the allocation controller.

[0098] The processing module 1203 is used to respond to the completion of 3D data loading by determining the data type of the 3D data through the allocation controller based on the data type identifier, and calling the neural rendering unit and / or arithmetic calculation unit to process the 3D data according to the data type of the 3D data.

[0099] In some embodiments, the processing module 1203 determines the data type of the three-dimensional data according to the data type identifier through the allocation controller, including: if the data type identifier is a first preset value, then the allocation controller determines that the data type of the three-dimensional data is neural network three-dimensional data; if the data type identifier is a second preset value, then the allocation controller determines that the data type of the three-dimensional data is triangular mesh data; if the data type identifier is a third preset value, then the allocation controller determines that the data type of the three-dimensional data is a hybrid three-dimensional data of neural network three-dimensional data and triangular mesh data; if the data type identifier is a fourth preset value, then the allocation controller determines that the data type of the three-dimensional data is other types of three-dimensional data.

[0100] In some embodiments, the processing module 1203 calls a neural rendering unit and / or an arithmetic calculation unit to process the three-dimensional data according to the data type of the three-dimensional data, including: if the data type of the three-dimensional data is neural network three-dimensional data, then the neural rendering unit is called to process the three-dimensional data first through the allocation controller, and then the arithmetic calculation unit is called to process the three-dimensional data; if the data type of the three-dimensional data is triangular mesh data, then the arithmetic calculation unit is called to process the three-dimensional data through the allocation controller; if the data type of the three-dimensional data is mixed three-dimensional data, then the neural rendering unit and the arithmetic calculation unit are called to process the three-dimensional data through the allocation controller; if the data type of the three-dimensional data is other types of three-dimensional data, then the arithmetic calculation unit is called to process the three-dimensional data through the allocation controller.

[0101] In some embodiments, the arithmetic calculation unit includes a first arithmetic calculation unit and a second arithmetic calculation unit; the processing module 1203 calls the neural rendering unit and the arithmetic calculation unit to process three-dimensional data through the allocation controller, including: calling the first arithmetic calculation unit to process triangular mesh data through the allocation controller, and calling the neural rendering unit to process neural network three-dimensional data; calling the second arithmetic calculation unit through the allocation controller to fuse the triangular mesh data processed by the first arithmetic calculation unit and the neural network three-dimensional data processed by the neural rendering unit to obtain fused three-dimensional data, and processing the fused three-dimensional data.

[0102] In some embodiments, the task identification information further includes spatial point information of the 3D data, 3D data address information, and tree structure data address information; the processing module 1203 loads the 3D data by calling the tree structure traversal unit through the allocation controller, including: loading the tree structure data corresponding to the tree structure traversal unit from the data storage unit according to the tree structure data address information through the tree structure traversal unit; determining the address offset information corresponding to the 3D data according to the spatial point information of the 3D data and the tree structure data through the tree structure traversal unit; determining the memory address information corresponding to the 3D data according to the address offset information and the 3D data address information through the tree structure traversal unit, and loading the 3D data from the data storage unit according to the memory address information.

[0103] In some embodiments, the processing module 1203 determines the address offset information corresponding to the 3D data based on the spatial point information and tree structure data of the 3D data through a tree structure traversal unit, including: dividing multiple triangular facet data in the tree structure data according to multiple node levels in the tree structure traversal unit to obtain root nodes, intermediate nodes, and leaf nodes, wherein one triangular facet data corresponds to a rendering unit in the 3D scene; traversing multiple triangular facet data in the tree structure data in the order of root node, intermediate node, and leaf node according to the spatial point coordinates and view direction information in the spatial point information of the 3D data through the tree structure traversal unit to determine the target triangular facet data corresponding to the spatial point information of the 3D data; and obtaining the address offset information corresponding to the 3D data from the target triangular facet data through the tree structure traversal unit.

[0104] In some embodiments, the neural rendering unit includes an encoding unit and a color calculation unit; the processing module 1203 calls the neural rendering unit to process three-dimensional data, including: calling the encoding unit to obtain the spatial point coordinates in the spatial point information of the three-dimensional data; determining the spatial feature data corresponding to the spatial point coordinates through the encoding unit, the spatial feature data including the eight vertex feature data corresponding to the spatial voxel where the spatial point coordinates are located and the weights corresponding to the vertex feature data; and determining the rendered three-dimensional data corresponding to the spatial point coordinates through the color calculation unit based on the eight vertex feature data corresponding to the spatial voxel where the spatial point coordinates are located and the weights corresponding to the vertex feature data.

[0105] In some embodiments, the encoding unit includes an index calculation unit, a table lookup unit, and an interpolation calculation unit; the processing module 1203 determines the spatial feature data corresponding to the spatial point coordinates through the encoding unit, including: determining the index information corresponding to the spatial point coordinates through the index calculation unit; determining the eight vertex feature data corresponding to the spatial voxel where the spatial point coordinates are located through the table lookup unit based on the index information corresponding to the spatial point coordinates; and determining the weights corresponding to each of the eight vertex feature data based on the spatial point coordinates through the interpolation calculation unit.

[0106] In some embodiments, the processing module 1203 determines the eight vertex feature data corresponding to the spatial voxel where the spatial point coordinates are located based on the index information corresponding to the spatial point coordinates through the lookup table unit, including: performing hash processing on the index information corresponding to the spatial point coordinates through the lookup table unit to obtain the target hash index information corresponding to the spatial point coordinates; and obtaining the eight vertex feature data corresponding to the spatial voxel where the spatial point coordinates are located from the pre-stored correspondence between the hash index information and the vertex feature data based on the target hash index information through the lookup table unit.

[0107] In some embodiments, the processing module 1203 determines the index information corresponding to the spatial point coordinates through the index calculation unit, including: dividing the three-dimensional space according to multiple preset resolution levels through the index calculation unit to obtain spatial voxels corresponding to each of the multiple preset resolution levels; and determining the index information of the spatial point coordinates under the multiple preset resolution levels through the index calculation unit based on the spatial voxels corresponding to each of the multiple preset resolution levels.

[0108] In some embodiments, the processing module 1203 determines the eight vertex feature data corresponding to the spatial voxel where the spatial point coordinates are located based on the index information corresponding to the spatial point coordinates through a lookup table unit. This includes: performing hash processing on the index information of the spatial point coordinates at multiple preset resolution levels through a lookup table unit to obtain multiple target hash index information corresponding to the spatial point coordinates; and obtaining the eight vertex feature data corresponding to the spatial voxel at multiple preset resolution levels from the correspondence between the pre-stored hash index information and vertex feature data based on the multiple target hash index information through a lookup table unit.

[0109] In some embodiments, the color calculation unit includes a color calculation subunit and a color aggregation unit; the processing module 1203 determines the rendered 3D data corresponding to the spatial point coordinates through the color calculation unit based on the eight vertex feature data corresponding to the spatial voxel where the spatial point coordinates are located and the weights corresponding to the vertex feature data, including: performing weighted calculations based on the eight vertex feature data corresponding to the spatial voxel where the spatial point coordinates are located and the weights corresponding to the vertex feature data through the color calculation subunit to obtain the hidden feature information corresponding to the spatial point coordinates; and processing the hidden feature information corresponding to the spatial point coordinates through the color aggregation unit to obtain the rendered 3D data corresponding to the spatial point coordinates.

[0110] In some embodiments, the arithmetic calculation unit includes multiple arithmetic calculation subunits and a cache unit; the processing module 1203 calls the arithmetic calculation unit to process the three-dimensional data, including: processing the three-dimensional data through multiple arithmetic calculation subunits to obtain processed three-dimensional data; and storing the processed three-dimensional data in the cache unit.

[0111] In some embodiments, the arithmetic calculation subunit includes a vector multiplication and addition unit and a scalar multiplication and addition unit; the processing module 1203 processes the three-dimensional data through multiple arithmetic calculation subunits to obtain processed three-dimensional data, including: processing the vector data in the three-dimensional data through the vector multiplication and addition unit in the arithmetic calculation subunit, and processing the scalar data in the three-dimensional data through the scalar multiplication and addition unit in the arithmetic calculation subunit to obtain processed three-dimensional data.

[0112] In some embodiments, the determining module 1202 determines whether the 3D data needs tree structure loading processing based on the tree structure identifier through the allocation controller, including: if the tree structure identifier is a fourth preset value, then the allocation controller determines that the 3D data needs tree structure loading processing; if the tree structure identifier is a fifth preset value, then the allocation controller determines that the 3D data does not need tree structure loading processing.

[0113] In some embodiments, the 3D computing accelerator further includes a caching module; the neural rendering unit and the arithmetic computing unit in the 3D computing accelerator interact with 3D data through the caching module.

[0114] This application provides a three-dimensional data processing device. When three-dimensional data requires tree structure loading processing, the tree structure traversal unit can be invoked by the allocation controller to load the three-dimensional data. Furthermore, the allocation controller invokes the neural rendering unit and / or arithmetic calculation unit to process the three-dimensional data based on the data type identifier. Thus, this processing device can determine whether the three-dimensional data needs tree structure acceleration through the tree structure identifier, and invoke the neural rendering unit and / or arithmetic calculation unit to process different types of three-dimensional data based on the data type identifier. In other words, the automatic control of the data processing flow is realized through the data type identifier and the tree structure identifier, breaking through the limitation of processing a single data type in the prior art, solving the task scheduling problem of mixed three-dimensional data, and thus improving the processing efficiency of three-dimensional data.

[0115] For a description of the features in the embodiment of the three-dimensional data processing apparatus provided in this application, please refer to the relevant description of the embodiment of the three-dimensional data processing method, which will not be repeated here.

[0116] Figure 13 A schematic diagram of the structure of the electronic device provided in this application. Figure 13 As shown, the electronic device 130 provided in this embodiment includes at least one processor 1301 and a memory 1302. Optionally, the electronic device 130 also includes a communication component 1303. The processor 1301, the memory 1302, and the communication component 1303 are connected via a bus.

[0117] In a specific implementation, at least one processor 1301 executes computer execution instructions stored in memory 1302, causing at least one processor 1301 to execute the above-described three-dimensional data processing method embodiment.

[0118] The specific implementation process of processor 1301 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0119] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the application can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.

[0120] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.

[0121] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0122] Embodiments of this application also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the steps in any of the above embodiments of the three-dimensional data processing method when running.

[0123] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0124] Embodiments of this application also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the above embodiments of the three-dimensional data processing method.

[0125] Embodiments of this application also provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps in any of the above-described embodiments of the three-dimensional data processing method.

[0126] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0127] The foregoing has provided a detailed description of a method, apparatus, device, storage medium, and program product for processing three-dimensional data. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are merely for the purpose of helping to understand the method and its core ideas. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of the claims of this application.

Claims

1. A method for processing three-dimensional data, characterized in that, Applied to a 3D computing accelerator, the 3D computing accelerator includes an allocation controller, a tree structure traversal unit, a neural rendering unit, and an arithmetic calculation unit, the processing method includes: In response to the allocation controller receiving a processing request for three-dimensional data, the task identification information corresponding to the three-dimensional data is obtained from the processing request. The task identification information includes at least a data type identifier and a tree structure identifier. The allocation controller determines whether the 3D data needs tree structure loading processing based on the tree structure identifier. If so, the allocation controller calls the tree structure traversal unit to load the 3D data. In response to the completion of the loading of the three-dimensional data, the allocation controller determines the data type of the three-dimensional data according to the data type identifier, and calls the neural rendering unit and / or the arithmetic calculation unit to process the three-dimensional data according to the data type of the three-dimensional data.

2. The processing method according to claim 1, characterized in that, The step of determining the data type of the three-dimensional data through the allocation controller based on the data type identifier includes: If the data type identifier is a first preset value, then the data type of the three-dimensional data is determined to be neural network three-dimensional data by the allocation controller; If the data type identifier is a second preset value, then the data type of the three-dimensional data is determined to be triangular mesh data by the allocation controller; If the data type identifier is a third preset value, then the data type of the three-dimensional data is determined by the allocation controller to be a hybrid three-dimensional data of neural network three-dimensional data and triangular mesh data; If the data type identifier is a fourth preset value, then the data type of the three-dimensional data is determined to be other types of three-dimensional data by the allocation controller.

3. The processing method according to claim 1, characterized in that, The step of processing the three-dimensional data by calling the neural rendering unit and / or the arithmetic calculation unit according to the data type of the three-dimensional data includes: If the data type of the three-dimensional data is neural network three-dimensional data, then the allocation controller first calls the neural rendering unit to process the three-dimensional data, and then calls the arithmetic calculation unit to process the three-dimensional data. If the data type of the three-dimensional data is triangular mesh data, then the arithmetic calculation unit is invoked through the allocation controller to process the three-dimensional data; If the data type of the three-dimensional data is hybrid three-dimensional data, then the neural rendering unit and the arithmetic calculation unit are invoked by the allocation controller to process the three-dimensional data; If the data type of the three-dimensional data is other types of three-dimensional data, then the arithmetic calculation unit is invoked by the allocation controller to process the three-dimensional data.

4. The processing method according to claim 3, characterized in that, The arithmetic calculation unit includes a first arithmetic calculation unit and a second arithmetic calculation unit; The process of invoking the neural rendering unit and the arithmetic calculation unit through the allocation controller to process the 3D data includes: The allocation controller invokes the first arithmetic calculation unit to process the triangular mesh data, and invokes the neural rendering unit to process the neural network 3D data; The allocation controller calls the second arithmetic calculation unit to fuse the triangular mesh data processed by the first arithmetic calculation unit and the neural network 3D data processed by the neural rendering unit to obtain fused 3D data, and then processes the fused 3D data.

5. The processing method according to claim 3, characterized in that, The neural rendering unit includes an encoding unit and a color calculation unit; The process of calling the neural rendering unit to process the 3D data includes: The encoding unit is invoked to obtain the spatial point coordinates from the spatial point information of the three-dimensional data; The spatial feature data corresponding to the spatial point coordinates is determined by the encoding unit. The spatial feature data includes the feature data of eight vertices corresponding to the spatial voxel where the spatial point coordinates are located, and the weights corresponding to the vertex feature data. The color calculation unit determines the rendered 3D data corresponding to the spatial point coordinates based on the eight vertex feature data corresponding to the spatial voxel where the spatial point coordinates are located and the weights corresponding to the vertex feature data.

6. The processing method according to claim 5, characterized in that, The encoding unit includes an index calculation unit, a table lookup unit, and an interpolation calculation unit; The step of determining the spatial feature data corresponding to the spatial point coordinates through the encoding unit includes: The index calculation unit determines the index information corresponding to the spatial point coordinates. The table lookup unit determines the eight vertex feature data corresponding to the spatial voxel where the spatial point coordinates are located based on the index information corresponding to the spatial point coordinates. The interpolation calculation unit determines the weights corresponding to the feature data of the eight vertices based on the coordinates of the spatial points.

7. The processing method according to claim 5, characterized in that, The color calculation unit includes a color calculation subunit and a color aggregation unit; The step of determining the rendered 3D data corresponding to the spatial point coordinates through the color calculation unit, based on the eight vertex feature data corresponding to the spatial voxel where the spatial point coordinates are located and the weights corresponding to the vertex feature data, includes: The color calculation subunit performs weighted calculations based on the eight vertex feature data corresponding to the spatial voxel where the spatial point coordinates are located and the weights corresponding to the vertex feature data to obtain the hidden feature information corresponding to the spatial point coordinates. The hidden feature information corresponding to the spatial point coordinates is processed by the color aggregation unit to obtain the rendered 3D data corresponding to the spatial point coordinates.

8. The processing method according to claim 3, characterized in that, The arithmetic calculation unit includes multiple arithmetic calculation sub-units and a cache unit; The process of calling the arithmetic calculation unit to process the three-dimensional data includes: The three-dimensional data is processed by the multiple arithmetic calculation subunits to obtain the processed three-dimensional data; The processed 3D data is stored in the cache unit.

9. The processing method according to claim 8, characterized in that, The arithmetic calculation subunit includes a vector multiplication and addition unit and a scalar multiplication and addition unit; The process of processing the three-dimensional data through the plurality of arithmetic calculation subunits to obtain processed three-dimensional data includes: The vector data in the three-dimensional data is processed by the vector multiplication and addition unit in the arithmetic calculation subunit, and the scalar data in the three-dimensional data is processed by the scalar multiplication and addition unit in the arithmetic calculation subunit to obtain the processed three-dimensional data.

10. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the method for processing three-dimensional data as described in any one of claims 1 to 9.

Images