Feature storage method and device, equipment, medium and program product

Abstract

The invention discloses a feature storage method, device and equipment, a medium and a program product, and relates to the technical field of neural rendering and on-chip storage system optimization. The method comprises the following steps: transmitting rays from the center of a camera to each pixel according to camera parameters and an observation direction; the method comprises the following steps: uniformly dividing a to-be-detected three-dimensional space into a plurality of coarse voxels, traversing each coarse voxel along a ray direction, and determining a current coarse voxel and a current fine voxel; for the current coarse voxel, detecting a target vertex which is not subjected to feature vector storage in the current fine voxel, and performing feature interpolation on the features of the target vertex to obtain a target feature vector; and determining a target data fragment and a target storage address according to the target vertex three-dimensional coordinate of the target vertex in the current voxel coordinate system, and storing the target feature vector according to the target data fragment and the target storage address. According to the technical scheme provided by the embodiment of the invention, the problem of feature redundant storage in the neural rendering process is solved, and the on-chip storage overhead is reduced.

Description

Technical Field

[0001] This invention relates to the field of neural rendering and on-chip storage system optimization technology, and in particular to a feature storage method, apparatus, device, medium and program product. Background Technology

[0002] In the rendering process based on the Instant Neural Graphics Primitives (Instant-NGP) algorithm, each sampling point requires the calculation of its feature vector through feature interpolation. Therefore, the feature interpolation process necessitates accessing the feature vectors of the eight vertices of the voxel containing the sampling point. Because the access addresses of feature vectors are highly irregular, this often leads to on-chip SRAM (Static Random Access Memory) cache conflicts and redundant storage issues.

[0003] To alleviate access conflicts in the feature cache, a Vertex-Interleaved Mapping (VIM) approach is proposed, which supports accessing all vertex features associated with the current voxel at once.

[0004] Using vertex staggered mapping can improve the efficiency of feature access and avoid access address conflicts. However, since multiple adjacent voxels share the same vertex, vertex staggered mapping will store the features of the shared vertex multiple times, which still has the problem of redundant feature storage and increases on-chip storage overhead. Summary of the Invention

[0005] This invention provides a feature storage method, apparatus, device, medium, and program product, which solves the problem of redundant feature storage and reduces on-chip storage overhead.

[0006] According to one aspect of the present invention, a feature storage method is provided, the method comprising:

[0007] Based on camera parameters and viewing direction, a ray is transmitted from the camera center to each pixel;

[0008] The three-dimensional space to be detected is uniformly divided into multiple coarse voxels, and each coarse voxel is traversed along the ray direction to determine the current coarse voxel and the current fine voxel; wherein, the current fine voxel is a hierarchical voxel obtained by further uniformly dividing the current coarse voxel.

[0009] For the current bold voxel, detect the target vertices in the current thin voxels that have not been stored as feature vectors, and perform feature interpolation on the features of the target vertices to obtain the target feature vector;

[0010] Based on the three-dimensional coordinates of the target vertex in the current voxel coordinate system, the target data fragments and target storage addresses are determined, and the target feature vectors are stored based on the target data fragments and target storage addresses.

[0011] According to another aspect of the present invention, a feature storage device is provided, the device comprising:

[0012] The ray generation and tracking module is used to transmit rays from the center of the camera to each pixel based on camera parameters and viewing direction.

[0013] A two-level voxel traversal module is used to uniformly divide the three-dimensional space to be detected into multiple coarse voxels, and traverse each of the coarse voxels along the ray direction to determine the current coarse voxel and the current fine voxel; wherein, the current fine voxel is a level voxel obtained by further uniformly dividing the current coarse voxel.

[0014] The target vertex filtering module is used to detect target vertices in the current thin voxels that have not been stored as feature vectors, and to perform feature interpolation on the features of the target vertices to obtain target feature vectors.

[0015] The target feature storage module is used to determine the target data slices and target storage addresses based on the three-dimensional coordinates of the target vertex in the current voxel coordinate system, and to store the target feature vectors based on the target data slices and the target storage addresses.

[0016] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0017] At least one processor; and

[0018] A memory communicatively connected to the at least one processor; wherein,

[0019] The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the feature storage method according to any embodiment of the present invention.

[0020] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the feature storage method described in any embodiment of the present invention.

[0021] According to another aspect of the present invention, a computer program product is provided, the computer program product comprising a computer program that, when executed by a processor, implements the feature storage method described in any embodiment of the present invention.

[0022] The technical solution of this invention achieves two-level traversal of the three-dimensional space to be detected by uniformly dividing the three-dimensional space to be detected into multiple coarse voxels and traversing each coarse voxel along the ray direction to determine the current coarse voxel and the current fine voxel, thereby improving the sampling efficiency of the three-dimensional space to be detected. By detecting the target vertices in the current fine voxels that have not been stored as feature vectors for the current coarse voxel, and performing feature interpolation on the features of the target vertices to obtain the target feature vectors, and storing the features in units of coarse voxels, the repeated storage of common vertices between adjacent fine voxels in the same coarse voxel is avoided, which significantly reduces the redundant storage of feature vectors and saves on-chip storage overhead. By determining the target data fragments and target storage addresses based on the three-dimensional coordinates of the target vertices in the current voxel coordinate system, and storing the target feature vectors based on the target data fragments and target storage addresses, the access conflicts of the feature cache can be reduced, enabling efficient feature access to on-chip features during neural rendering and improving neural rendering efficiency.

[0023] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 This is a flowchart of a feature storage method provided according to Embodiment 1 of the present invention;

[0026] Figure 2 This is a schematic diagram of the real-time neural image primitive algorithm provided in Embodiment 1 of the present invention;

[0027] Figure 3 This is a flowchart of a two-level ray traversal provided in Embodiment 1 of the present invention;

[0028] Figure 4 This is a flowchart of a feature storage method provided according to Embodiment 2 of the present invention;

[0029] Figure 5 This is a histogram showing the uneven distribution of feature vectors according to Embodiment 2 of the present invention;

[0030] Figure 6This is a diagram illustrating the effect of cyclic shifting and storing the feature vector according to Embodiment 2 of the present invention.

[0031] Figure 7 This is a schematic diagram of the cyclic shift storage of feature vectors according to Embodiment 2 of the present invention;

[0032] Figure 8 This is a schematic diagram of the structure of a feature storage device according to Embodiment 3 of the present invention;

[0033] Figure 9 This is a schematic diagram of the structure of an electronic device that implements the feature storage method of the present invention. Detailed Implementation

[0034] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0035] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0036] Example 1

[0037] Figure 1 This is a flowchart illustrating a feature storage method provided in Embodiment 1 of the present invention. This embodiment of the invention is applicable to situations where feature storage is performed during the rendering of a neural radiation field. The method can be executed by a feature storage device, which can be implemented in hardware and / or software, and can be configured in an electronic device that carries feature storage functionality.

[0038] See Figure 1 The feature storage method shown includes:

[0039] S101. Based on the camera parameters and the viewing direction, transmit a ray from the center of the camera to each pixel.

[0040] Camera parameters include intrinsic and extrinsic parameters. Intrinsic parameters include focal length, principal point, and image resolution. Focal length corresponds to the focal length in the width and height directions of the image, respectively. The principal point is the intersection of the camera's optical axis and the image plane. Image resolution characterizes the width and height of the image. Extrinsic parameters include camera position and camera rotation. The viewing direction refers to the direction of light rays from the camera center to a point in the image. The viewing direction defines the camera's perspective of the scene. In neural radiation field rendering, based on the camera parameters and viewing direction, rays are transmitted from the camera center to each pixel in the image (often called ray projection or ray generation). This is typically used to generate a scene representation from the camera's perspective for subsequent neural network training or inference.

[0041] Specifically, based on the camera parameters, the pixel coordinates in the image are first converted to the camera coordinate system, and then to the world coordinate system. Starting from the camera center, light rays are transmitted to each pixel in the image along the viewing direction.

[0042] S102. Divide the three-dimensional space to be detected into multiple coarse voxels, and traverse each coarse voxel along the ray direction to determine the current coarse voxel and the current fine voxel.

[0043] The 3D space to be detected is the spatial range for voxel traversal. Optionally, the 3D space to be detected includes a fixed range pre-defined by the technician, an automatic range based on camera parameters, a pre-calculated range, or a range estimated using a pre-generated depth map. Coarse voxels represent the initial uniform division of the 3D space to be detected. Coarse voxels are hierarchical voxels obtained by initially uniformly dividing the 3D space to be detected. For example, the 3D space to be detected can be uniformly divided into... A bold element, for example The fine voxels are the result of further uniform division of the current coarse voxels. For example, the current coarse voxels can be uniformly divided into... Individual elements, for example The ray direction is the direction of the light ray originating from the camera center and passing through a pixel on the image plane. The current bold voxel is the bold voxel being sampled. The current fine voxel is the fine voxel being sampled from the current bold voxel. The current fine voxel is the hierarchical voxel obtained by further uniformly subdividing the current bold voxel. Optionally, the current bold voxel is the non-empty bold voxel among all bold voxels. Correspondingly, the current fine voxel is the non-empty fine voxel among all fine voxels of the current bold voxel.

[0044] Specifically, the three-dimensional space to be detected is uniformly divided into a first preset number of bold voxels. Along the ray direction, the bold voxels are traversed, and each bold voxel is checked for non-emptiness. If a bold voxel is empty, its adjacent bold voxels are traversed along the ray direction, and each adjacent bold voxel is checked for non-emptiness. If a bold voxel is not empty, it is determined as the current bold voxel, and further divided into a second preset number of fine voxels. Along the ray direction, the fine voxels of the current bold voxel are traversed, and each fine voxel is checked for non-emptiness. If a fine voxel is empty, its adjacent fine voxels are traversed along the ray direction, and each adjacent fine voxel is checked for non-emptiness. If a fine voxel is not empty, it is determined as the current fine voxel. Here, the first preset number is the number of bold voxels in the three-dimensional space to be detected. The second preset number is the number of fine voxels within a single current bold voxel. Optionally, the first preset number and the second preset number can be the same or different.

[0045] In an optional embodiment of the present invention, traversing each coarse voxel along the ray direction to determine the current coarse voxel and the current fine voxel includes: performing a non-empty detection on each coarse voxel along the ray direction; when the coarse voxel is non-empty, determining the coarse voxel as the current coarse voxel, and further uniformly dividing the current coarse voxel into multiple fine voxels; performing a non-empty detection on each fine voxel along the ray direction; when the fine voxel is non-empty, determining the fine voxel as the current fine voxel.

[0046] A non-empty bold voxel can be understood as the existence of a sampling point within the bold voxel, allowing sampling of the bold voxel. A non-empty fine voxel can also be understood as the existence of a sampling point within the fine voxel, allowing sampling of the fine voxel.

[0047] Specifically, along the ray direction, based on the pre-determined state identifiers in the voxel occupancy bitmap, a non-empty check is performed on each bold voxel. If the state identifier is empty, the bold voxel is determined to be empty; if the state identifier is not empty, the bold voxel is determined to be non-empty. When a bold voxel is empty, along the ray direction, the adjacent bold voxels are traversed, and their non-empty status is checked. If a bold voxel is not empty, it is designated as the current bold voxel and further evenly divided into multiple fine voxels. Along the ray direction, based on the pre-determined state identifiers, a non-empty check is performed on each fine voxel. If the state identifier is empty, the fine voxel is determined to be empty; if the state identifier is not empty, the fine voxel is determined to be non-empty. When a fine voxel is empty, along the ray direction, the adjacent bold voxels are traversed, and their non-empty status is checked. If a fine voxel is not empty, it is designated as the current fine voxel. Here, the state identifier is used to identify whether a voxel in a given scene is empty.

[0048] For example, firstly, the entire three-dimensional space to be detected is divided into... There are 1 bold voxels; each bold voxel corresponds to a 1-bit state identifier, used to indicate "empty" or "not empty". The "empty" state indicates that there are no sampling points within the bold voxel, and the ray can directly step to the adjacent bold voxel during traversal without entering the fine voxel level for calculation; the "not empty" state indicates that the bold voxel contains sampling points and needs further subdivision. For the current bold voxel marked as "not empty", it can be further divided into... Each voxel is a fine voxel. Similarly, each fine voxel is represented by a 1-bit state identifier indicating either "empty" or "non-empty". When a fine voxel is "empty", it means there are no sampling points in that region, and the ray can skip that voxel and continue forward. When a fine voxel is "non-empty", it means it contains sampling points. In this case, target vertices that have not yet had their feature vectors stored are extracted from the current fine voxel, and their features are calculated and used for subsequent rendering processing.

[0049] This scheme performs non-empty detection on each coarse voxel along the ray direction. When a coarse voxel is non-empty, it is identified as the current coarse voxel. The current coarse voxel is then further divided into multiple fine voxels, and non-empty detection is performed on each fine voxel along the ray direction. Through the above two-level voxel division and spatial domain jumping mechanism, effective sampling points can be quickly located, significantly reducing the number of traversals of empty regions. This avoids redundant calculations for empty regions, accelerates the ray propagation process, and improves the overall traversal efficiency and rendering efficiency of neural rendering.

[0050] S103. For the current bold voxel, detect the target vertex in the current thin voxel that has not been stored as a feature vector, and perform feature interpolation on the features of the target vertex to obtain the target feature vector.

[0051] The target vertex is the vertex of the current fine voxel that has not yet had its feature vector stored within the current bold voxel. This can be understood as follows: this invention stores feature vectors on a unit of bold voxel. For vertices of fine voxels within the same current bold voxel, the feature vector of common vertices between adjacent fine voxels is stored only once, without needing to be stored repeatedly. The target feature vector is the result of feature difference calculation on the features of the target vertex.

[0052] See the example above, in At that time, a current bold voxel is divided into 64 (i.e., (512) fine voxels. In traditional feature vector storage methods, such as vertex staggered mapping, 512 (i.e., ...) voxels need to be stored. (64) feature vectors. Here, 64 represents that a current bold voxel contains 64 thin voxels; 8 represents the 8 vertices of a thin voxel. However, in the feature vector storage method of this invention, only 125 (i.e., ...) feature vectors need to be stored. The present invention stores the feature vectors of each fine voxel as a whole along the dimension of the fine voxel, reducing the number of feature vectors required to be stored for each fine voxel from 512 to 125, which can significantly reduce redundancy.

[0053] Specifically, using the current bold voxel as the unit, target vertices that have not yet had their feature vectors stored are detected among the vertices of the current thin voxels. Feature interpolation methods such as trilinear interpolation can be used to interpolate the features of the target vertices to obtain the target feature vectors.

[0054] S104. Based on the three-dimensional coordinates of the target vertex in the current voxel coordinate system, determine the target data slices and target storage address, and store the target feature vectors according to the target data slices and target storage address.

[0055] The current voxel coordinate system is a coordinate system constructed based on the current bold voxel and each fine voxel within it. It can be understood as dividing the current bold voxel into a three-dimensional spatial coordinate system with the bottom-left vertex of the current bold voxel as the origin. Based on the side length of each fine voxel, the three-dimensional coordinates in this three-dimensional spatial coordinate system are normalized to obtain the vertex's three-dimensional coordinates. Referring to the example above, the current fine voxel is divided into 64 (i.e.,...) (Number) fine voxels; the lower left vertex is the origin, and the coordinates of the origin are... The coordinate ranges of the x-axis, y-axis, and z-axis are: The target vertex's 3D coordinates are the vertex's 3D coordinates in the current voxel coordinate system.

[0056] The target data slice is a data slice that stores the target feature vector. Optionally, the number of target data slices can be the same as the number of vertices of the current bold voxel. That is, the number of target data slices can be 8. For example, the target data slice can be a Feature Static Random Access Memory Bank (FSRAM Bank). The target storage address is the storage address identifier within a single target data slice. It can be understood that a target data slice can have multiple storage addresses. A storage address can be used to store the feature vector of a vertex. The target storage address is used to store the target feature vector of the target vertex.

[0057] Specifically, the 3D coordinates of the target vertex in the current voxel coordinate system are normalized to a value that is equal to the target vertex coordinates in the voxel coordinate system. Within the specified range, the coordinate values ​​of each 3D coordinate of the target vertex are rounded. The rounded 3D coordinates of the target vertex are treated as three-bit binary numbers and converted to decimal to obtain the target vertex number. The target vertex number is used to determine the corresponding target data slice. The target vertex number and the target vertex's 3D coordinates in the current voxel coordinate system are combined to determine the target storage address. Based on the target data slice and the target storage address, the target feature vector is stored.

[0058] For example, the following formula can be used to convert the three-dimensional coordinates of the target vertex into a decimal target vertex number:

[0059] ;

[0060] In the formula, Number the target vertex; The three-dimensional coordinates of the target vertex.

[0061] Optionally, when performing feature interpolation on the sampling points, the storage addresses of the data slices corresponding to the eight vertices of the current fine voxel where the sampling point is located can be read in parallel at one time. This can avoid access conflicts of data slices and enable the loading of all features in a single cycle.

[0062] Figure 2 This is a schematic diagram of the real-time neural image primitive algorithm. The rendering process based on neural radiation fields mainly includes five key steps. The technical solution of this embodiment mainly optimizes the feature vector storage in step 3 feature interpolation and the sampling process in step 2. Figure 2 As shown, the rendering process based on neural radiation fields includes:

[0063] Step 1: Ray generation and tracing.

[0064] Specifically, based on the camera parameters and the viewing direction, a ray is projected from the center of the camera towards each pixel. Each ray corresponds to a pixel in the image.

[0065] Step 2, sampling.

[0066] Specifically, the 3D space to be inspected is divided into a multi-resolution voxel mesh, that is, the 3D space to be inspected is divided into voxels of different precision, such as coarse voxels and fine voxels. Sampling is performed along each ray within a set range (i.e., within the 3D space to be inspected) to obtain the spatial coordinates of the sampling points on the ray. and observation direction This is used for subsequent feature extraction and volume integral calculation. To improve sampling efficiency, such as... Figure 3As shown, the sampling process employs a two-level traversal: coarse traversal and fine traversal. The coarse traversal iterates through the coarse voxels in the 3D space to be detected; the fine traversal iterates through each fine voxel within the current coarse voxel. The coarse traversal identifies the non-empty current coarse voxels; the fine traversal identifies the non-empty current fine voxels. The purpose of this two-level traversal is to quickly locate effective sampling points, thereby avoiding redundant calculations for empty regions and improving the overall rendering efficiency of the neural radiation field.

[0067] Step 3: Feature interpolation.

[0068] Specifically, after detecting the spatial coordinates of the sampling point, feature interpolation is performed on the features of the eight vertices of the voxel where the sampling point is located to generate feature vectors for the eight vertices corresponding to that sampling point. Then, the feature vectors are stored.

[0069] In existing vertex interleaving mapping methods, the features of the eight vertices of each voxel are directly stored in the corresponding data slice according to the vertex number. This results in redundant storage of common vertices between adjacent voxels. The technical solution of this invention uses coarse voxels as the unit, storing common vertices between adjacent fine voxels only once, thus avoiding redundant feature storage. Specifically, feature mapping can be performed on sampling points. First, the current coarse voxel and current fine voxel where the sampling point is located are determined based on the spatial coordinates of the sampling point. The features of the target vertices among the eight vertices of the current fine voxel where the sampling point is located that have not yet had their feature vectors stored are read. Second, trilinear interpolation is performed on the target vertex feature vectors of each target vertex to obtain the target vertex feature vector of the target vertex corresponding to the sampling point. Then, based on the three-dimensional coordinates of the target vertex, the target data slice and target storage address are determined, and the target feature vector is stored.

[0070] Step 4: Inference based on neural networks.

[0071] Specifically, the target vertex feature vectors corresponding to each sampling point, along with the frequency-encoded observation direction, are input into a multilayer perceptron (MLP) network, which outputs the color of that sampling point. With volume density .

[0072] Step 5: Volume rendering.

[0073] Specifically, the colors of all sampling points on the same ray... With volume density The pixel color corresponding to the ray is calculated by volume integration. The specific formula is as follows:

[0074] ;

[0075] In the formula, represents the pixel color corresponding to the ray; N is the number of all sampling points on the ray. Transmittance; Bulk density; The distance between adjacent sampling points; For color.

[0076] The transmittance can be calculated using the following formula:

[0077] ;

[0078] In the formula, Transmittance; Bulk density; This represents the distance between adjacent sampling points.

[0079] The technical solution of this invention achieves two-level traversal of the three-dimensional space to be detected by uniformly dividing the three-dimensional space to be detected into multiple coarse voxels and traversing each coarse voxel along the ray direction to determine the current coarse voxel and the current fine voxel, thereby improving the sampling efficiency of the three-dimensional space to be detected. By detecting the target vertices in the current fine voxels that have not been stored as feature vectors for the current coarse voxel, and performing feature interpolation on the features of the target vertices to obtain the target feature vectors, and storing the features in units of coarse voxels, the repeated storage of common vertices between adjacent fine voxels in the same coarse voxel is avoided, which significantly reduces the redundant storage of feature vectors and saves on-chip storage overhead. By determining the target data fragments and target storage addresses based on the three-dimensional coordinates of the target vertices in the current voxel coordinate system, and storing the target feature vectors based on the target data fragments and target storage addresses, the access conflicts of the feature cache can be reduced, enabling efficient feature access to on-chip features during neural rendering and improving neural rendering efficiency.

[0080] Example 2

[0081] Figure 4This is a flowchart of a feature storage method provided in Embodiment 2 of the present invention. Based on the above embodiments, this embodiment of the present invention specifies "determining the target data slice and target storage address based on the three-dimensional coordinates of the target vertex in the current voxel coordinate system" as follows: "For each fine voxel of the current coarse voxel, construct the current voxel coordinate system and obtain the three-dimensional coordinates of the target vertex in the current voxel coordinate system; represent the target vertex coordinate values ​​in each dimension of the three-dimensional coordinates of the target vertex in binary form to obtain the binary number of the target vertex in each dimension; extract the least significant bit of the target vertex binary number in each dimension to generate the target vertex category number; extract the remaining significant bits of the target vertex binary number in each dimension to generate the target vertex encoding address; determine the target data slice and the corresponding target storage address in the target data slice based on the target vertex category number and the target vertex encoding address." This method is simple to implement, has low hardware overhead, requires no additional lookup tables or complex logic control, and can quickly achieve feature storage. It is suitable for deployment in large-scale parallel neural rendering accelerators and can effectively improve the bandwidth utilization of feature access and the overall system throughput performance. It should be noted that parts not described in detail in this embodiment of the present invention can be referred to in the descriptions of other embodiments.

[0082] See Figure 4 The feature storage method shown includes:

[0083] S401. Based on camera parameters and viewing direction, transmit a ray from the camera center to each pixel.

[0084] S402. Divide the three-dimensional space to be detected into multiple coarse voxels, and traverse each coarse voxel along the ray direction to determine the current coarse voxel and the current fine voxel.

[0085] Among them, the current fine voxel is the hierarchical voxel obtained by further uniformly dividing the current coarse voxel.

[0086] S403. For the current bold voxel, detect the target vertex in the current thin voxel that has not been stored as a feature vector, and perform feature interpolation on the features of the target vertex to obtain the target feature vector.

[0087] S404. For each fine voxel of the current coarse voxel, construct the current voxel coordinate system and obtain the three-dimensional coordinates of the target vertex in the current voxel coordinate system.

[0088] The current voxel coordinate system is a coordinate system constructed based on the current coarse voxel and the fine voxels within the current coarse voxel. Referring to the example above, the current fine voxel can be divided into 125 (i.e., ...) voxels. (Number) fine voxels; the lower left vertex is the origin, and the coordinates of the origin are... The coordinate ranges of the x-axis, y-axis, and z-axis are: .

[0089] Specifically, the bottom-left vertex of the current bold voxel is used as the origin; the current bold voxel is divided into a three-dimensional spatial coordinate system. Based on the side length of a single thin voxel, the three-dimensional coordinates in the three-dimensional spatial coordinate system are normalized to obtain the current voxel coordinate system. This allows the detection of the target vertex's three-dimensional coordinates within the current voxel coordinate system.

[0090] S405. Represent the target vertex coordinates in each dimension of the three-dimensional coordinate system in binary form to obtain the binary number of the target vertex in each dimension.

[0091] The target vertex coordinates are the coordinates of a single dimension in the three-dimensional coordinate system of the target vertex. For example, the target vertex coordinates could be... The target vertex coordinates are: x-axis: 4; y-axis: 1; z-axis: 2. The target vertex binary number is the binary representation of its coordinates. The number of bits in the target vertex binary number is the same for all dimensions of the same current bold voxel. Referring to the example above, the current thin voxel is divided into 64 (i.e.,...) (Number) fine volumetric elements. Target vertex coordinates are... Correspondingly, the target vertex on the x-axis is 100 in binary; the target vertex on the y-axis is 001 in binary; and the target vertex on the z-axis is 010 in binary.

[0092] Specifically, the highest bit count of each vertex's binary number can be determined based on the maximum value of the target vertex coordinates in the current bold pixel. The target vertex's 3D coordinates in each dimension are then converted to the highest bit count binary number to obtain the target vertex's binary number for each dimension.

[0093] S406. Extract the least significant bit of the target vertex binary number for each dimension and generate the target vertex category number.

[0094] The least significant bit of the target is the least significant bit of the binary representation of the target vertex. Referring to the example above, the target vertex coordinates are... Correspondingly, the least significant bit of the target vertex's binary number along the x-axis is 0; the least significant bit along the y-axis is 1; and the least significant bit along the z-axis is 0. The target vertex category number is used to characterize the vertex category to which the target vertex belongs. This can be understood as follows: the current bold pixel has 8 vertices; the target vertex category number is used to identify which of the 8 vertices the target vertex belongs to.

[0095] Specifically, the least significant bits of the target vertex's binary data in each dimension are extracted. These least significant bits are then arranged according to a preset sorting method to generate a three-digit target binary number. This three-digit target binary number is converted to decimal to obtain the target vertex category number. The preset sorting method refers to the order of the target vertex's three-dimensional coordinates across each dimension. For example, the preset sorting method could include xyz, yxz, zxy, or zyx. The three-digit target binary number is the result of sorting the least significant bits of the target vertex according to the preset sorting method. For example, if the preset sorting method is zyx; the target three-digit binary data is 010; and the target vertex category number is 2.

[0096] S407. Extract the remaining valid bits of the target vertex binary number in each dimension and generate the target vertex encoded address.

[0097] The remaining significant bits of the target are the remaining significant bits in the binary representation of the target vertex, excluding the least significant bit. Referring to the example above, the target vertex coordinates are... Correspondingly, the remaining valid bits of the target vertex binary number along the x-axis are 10; the remaining valid bits along the y-axis are 00; and the remaining valid bits along the z-axis are 01. The target vertex encoding address is used to distinguish different fine voxels belonging to the same target vertex category number. This can be understood as follows: within the same current fine voxel, one target vertex category number corresponds to one target data slice. One target data slice can store feature vectors of multiple vertices with the same target vertex category number but belonging to different fine voxels. At this point, it is necessary to distinguish feature vectors from different fine voxels. The target vertex encoding address is used to distinguish feature vectors of vertices belonging to the same target vertex category number but originating from different fine voxels.

[0098] Specifically, the remaining valid bits of the target in each dimension are arranged according to a preset sorting method to generate the target vertex encoding address. For example, the preset sorting method is zyx; the target vertex encoding address is 010010.

[0099] In an optional embodiment of the present invention, extracting the remaining valid bits of the target vertex binary number in each dimension to generate the target vertex encoded address includes: extracting the remaining valid bits of the target vertex binary number in each dimension to generate the original address of the target vertex; and determining the corresponding target vertex encoded address based on the original address of the target vertex.

[0100] The original address of the target vertex is the address of the target vertex directly generated based on the remaining valid bits of the target vertex's binary number in each dimension. The number of bits in the target vertex encoded address is less than that in the original target vertex address. It can be understood that the target vertex encoded address is a compressed result of the original target vertex address.

[0101] Optionally, the remaining valid bits of the target in each dimension are arranged according to a preset sorting method to generate the original address of the target vertex. The original address bit length and the encoded address bit length of the target vertex are obtained. A preset compression method is used to compress the original address of the target vertex based on the original address bit length and the encoded address bit length to obtain the encoded address of the target vertex. Here, the original address bit length refers to the number of bits in the original address of the target vertex; the encoded address bit length refers to the number of bits in the encoded address of the target vertex. The preset compression method can be a pre-defined method for compressing binary numbers. For example, preset compression methods include direct binary bit concatenation, multi-bit binning compression, floating-point bit truncation, or space-filling curve methods, etc.

[0102] Optionally, there can be a preset correspondence between the original vertex address and the vertex encoded address. Based on the target vertex's original address and the preset correspondence between the original vertex address and the vertex encoded address, the target vertex encoded address corresponding to the target vertex's original address is queried.

[0103] In an optional embodiment of the present invention, before determining the corresponding target vertex encoded address based on the original address of the target vertex, the method includes: obtaining the reference vertex three-dimensional coordinates of each reference vertex within the current bold voxel; for a single reference vertex, representing the reference vertex coordinate values ​​of each dimension of the three-dimensional coordinates of the reference vertex in binary form to obtain the reference vertex binary number for each dimension; for a single reference vertex, extracting the least significant bit of the reference vertex binary number for each dimension to generate a reference vertex category number; for a single reference vertex, extracting the remaining significant bits of the reference vertex binary number to generate the original address of the reference vertex; sorting the original addresses of each reference vertex under the same reference vertex category number in ascending order and calculating the reference vertex address sequence number of each original address under the reference vertex category number; for a single reference vertex, representing the reference vertex address sequence number in binary form to generate the corresponding reference vertex encoded address; and storing each original address of the reference vertex and its corresponding reference vertex encoded address accordingly.

[0104] A reference vertex can be a vertex of a fine voxel within the current bold voxel. The reference vertex's 3D coordinates are its coordinates in the current voxel coordinate system. Each reference vertex coordinate is a single-dimensional coordinate. The reference vertex binary number is the binary representation of its coordinates. The binary numbers of all reference vertices within the same current bold voxel have the same number of bits. The least significant bit is the least significant bit of the reference vertex binary number. The reference vertex category number identifies the vertex category to which the reference vertex belongs. Each reference vertex category number includes... This can be understood as follows: the current bold element has 8 vertices; the reference vertex category number is used to identify which of the 8 vertices the target vertex belongs to. The remaining significant bits of the reference vertex binary number are the remaining significant bits excluding the least significant bit. The original address of the reference vertex is the address of the reference vertex directly generated based on the remaining significant bits of the reference vertex binary numbers for each dimension. The reference vertex address sequence number is used to characterize the sorting of the original addresses of a reference vertex under the same reference vertex category number. This can be understood as follows: the original address of the reference vertex is a binary number; sorting the original addresses of each reference vertex in ascending order yields the position of a particular original address within the set of original addresses, thus enabling bit compression of the original address. The reference vertex encoded address is the binary representation of the reference vertex address sequence number of the original address. The reference vertex encoded address is the result of bit compression of the original address. The number of bits in the reference vertex encoded address is less than that in the original address of the target vertex.

[0105] Specifically, the process involves detecting the 3D coordinates of each reference vertex within the current bold voxel in the current voxel coordinate system. For a single reference vertex, the highest number of bits in the binary representation of each reference vertex in the current bold voxel is determined based on the maximum value of its coordinates. The reference vertex coordinates in each dimension of the 3D voxel are converted to the highest-numbered binary representation, yielding the reference vertex binary representation for each dimension. For a single reference vertex, the least significant bit (LSB) of the reference vertex binary representation for each dimension is extracted. The LSBs are then sorted according to a preset sorting method to generate a three-bit reference binary representation. This three-bit reference binary representation is converted to decimal to obtain the reference vertex category number. For a single reference vertex, the remaining significant bits in each dimension are sorted according to a preset sorting method to generate the original reference vertex address. The original reference addresses under the same category number are sorted in ascending order to obtain the reference vertex address sequence number under the category number. For a single reference vertex, the maximum value of each reference vertex address sequence number is used to determine the number of bits in the encoded address of the reference vertex. The reference vertex address sequence number is converted from decimal to a binary number of coded address bits, generating the corresponding reference vertex coded address. The original addresses and corresponding coded addresses of each reference vertex are stored accordingly. The three reference bits are the sorting results of the least significant bits of the references in each dimension according to a preset sorting method.

[0106] This scheme determines the reference vertex address sequence number under the same reference vertex category number by sorting the original addresses of each reference vertex in ascending order. The reference vertex address sequence number is then represented in binary to generate the corresponding reference vertex encoded address. This achieves bit compression of the original reference vertex address, which can save on-chip address width, improve the access efficiency of data fragmentation and the rendering efficiency of neural radiation fields. Moreover, by storing each original reference vertex address and its corresponding reference vertex encoded address, the efficiency of determining the target vertex encoded address can be improved, thereby improving the storage efficiency of feature vectors.

[0107] S408. Based on the target vertex category number and the target vertex encoding address, determine the target data fragment and the corresponding target storage address in the target data fragment, and store the target feature vector according to the target data fragment and the target storage address.

[0108] Specifically, the corresponding target data fragment can be determined based on the target vertex category number. The target vertex encoding address, or a combination of the target vertex category number and the target vertex encoding address, can be used to determine the corresponding target storage address within the target data fragment. There is a one-to-one correspondence between the target vertex number and the target data fragment. This can be understood as a fixed mapping relationship between vertex category numbers and data fragment numbers. Based on the target vertex category number and the fixed mapping relationship between vertex category numbers and data fragment numbers, the target data fragment corresponding to the target vertex category number can be queried.

[0109] In an optional embodiment of the present invention, determining the target data fragment and the corresponding target storage address in the target data fragment based on the target vertex category number and the target vertex encoding address includes: obtaining the current bold voxel number and the highest bit storage address of the stored feature vector in the target data fragment; performing a cyclic shift on the target vertex category number based on the current bold voxel number to determine the target data fragment number; and accumulating the highest bit storage address and the target vertex encoding address to calculate the target storage address.

[0110] The current bold voxel number is the voxel's number in the 3D space to be detected. It is used to identify the current bold voxel in the 3D space. The highest-order bit storage address is the highest-order bit address among all the storage addresses of the feature vectors already stored in the target data piece. The target data piece number is used to identify the target data piece. The range of the target data piece number is consistent with the range of the target vertex category number.

[0111] However, if the vertex category number is directly mapped to the data slice number according to the above scheme, the number of feature vectors corresponding to different vertex category numbers within a bold voxel will be significantly unbalanced, resulting in an extremely unbalanced distribution of feature vectors. This leads to significant differences in depth between different data slices, ultimately affecting the feasibility of chip layout and routing.

[0112] Referring to the example above, the current fine voxels are evenly divided into 64 (i.e., (Number) fine voxels. If the traditional feature vector storage method is used to directly store the feature vectors of 512 vertices, with a one-to-one correspondence between the vertex category number and a single data slice, and each data slice storing 64 feature vectors, there will be no imbalance in the distribution of feature vectors. However, if the feature vectors are stored in units of coarse voxels as in this scheme, then only the feature vectors of 125 vertices need to be stored, which can avoid the redundant storage problem in coarse voxels. However, if the 125 vertices are divided into 8 vertex category numbers, and a fixed one-to-one correspondence is used between the FID (Feature ID, i.e., the target vertex category number) of the coarse voxel and the FSRAM ID (i.e., the target data slice number) is used (for example, FID 0 always corresponds to FSRAM 0), then this fixed mapping method will lead to an uneven distribution of storage load in each FSRAM Bank (feature static random access memory data slice).

[0113] like Figure 5 As shown, out of 125 vertices, 27 are numbered 0; 18 are numbered 1; 18 are numbered 2; 12 are numbered 3; 18 are numbered 4; 12 are numbered 5; 12 are numbered 6; and 8 are numbered 7. At this point, an imbalanced feature distribution has already occurred when storing the feature vector for a single bold voxel. Assuming there are a total of h bold voxels, then FSRAM 0 needs to store… 1 feature vector, while FSRAM 7 only needs to store 1 feature vector. There are 10 feature vectors. As the number of bold voxels increases, this imbalance will be further aggravated, causing some FSRAM (feature static random access memory) storage resources to be over-occupied, while other FSRAM (feature static random access memory) is idle, thus affecting the overall storage efficiency and access parallelism.

[0114] To address the aforementioned issues, this solution proposes a cyclically shifted mapping strategy between vertex category numbers and data partition numbers. This strategy uses the current bold voxel number to achieve a cyclic shift between vertex category numbers and data partition numbers, ensuring that the same vertex category number for different bold voxels is distributed across different data partitions. This redistributes the feature vectors corresponding to the same vertex category number for different bold voxels across various data partitions, achieving a uniform distribution of voxel features across multiple data partitions, maintaining a balanced depth across each data partition, and reducing routing complexity.

[0115] Specifically, the process involves obtaining the current bold voxel number stored on-chip and the address of the highest-order bit of the feature vector already stored in the target data slice. The target vertex category number and the current bold voxel number are summed, and the result is modulo-calculated to determine the target data slice number. Finally, the address of the highest-order bit and the target vertex encoding address are summed to obtain the target storage address.

[0116] For example, the following formula can be used to sum the target vertex category number and the current bold voxel number, and then perform a modulo operation on the summation result to determine the target data fragment number:

[0117] ;

[0118] In the formula, The target data is segmented and numbered; FID is the target vertex category number; Number the current bold element; Modulo operation.

[0119] like Figure 6 As shown in the figure, different colors represent the feature vector distribution of different bold elements. Take the eight bold elements corresponding to different colors as an example. (i.e., the current bold pixel numbers) are 0, 1, 2, 3, 4, 5, 6, and 7. For the blue bold pixels ( ), For green bold voxels ( ), ,Right now The corresponding features exist. FSRAM, and so on.

[0120] This solution adjusts the fixed mapping relationship between the target vertex category number and the target data segment number based on the current bold voxel number to a cyclic shift mapping relationship between the target vertex category number and the target data segment number. This avoids the problem of uneven distribution of feature vectors among different data segments and enables uniform distribution of voxel features across multiple data segments, ensuring the feasibility of chip layout and routing. By accumulating the highest bit storage address and the target vertex encoding address to calculate the target storage address, the uniqueness of the target storage address within the same data segment is improved, further avoiding access address conflicts between data segments.

[0121] The technical solution of this invention constructs a current voxel coordinate system for each fine voxel of the current coarse voxel, obtains the three-dimensional coordinates of the target vertex in the current voxel coordinate system, represents the target vertex coordinate values ​​in each dimension of the three-dimensional coordinates of the target vertex in binary form, obtains the target vertex binary number in each dimension, extracts the least significant bit of the target vertex binary number in each dimension to generate the target vertex category number, extracts the remaining significant bits of the target vertex binary number in each dimension to generate the target vertex encoding address, and determines the target data fragment and the corresponding target storage address in the target data fragment based on the target vertex category number and the target vertex encoding address. It uses a concise method to determine the target vertex category number and the target vertex encoding address, with low hardware overhead, no need for additional lookup tables or complex logic control, and can quickly realize feature storage. It is suitable for deployment in large-scale parallel neural rendering accelerators and can effectively improve the bandwidth utilization of feature access and the overall system throughput performance.

[0122] Based on the above embodiments, the present invention also provides a preferred embodiment of the feature storage method. The present invention aims to solve the following key problems: 1) how to avoid access address conflicts caused by sampling points accessing voxel vertex features; 2) how to reduce feature storage redundancy caused by multiple voxels sharing vertices; 3) how to maintain the depth of each data slice while maintaining access parallelism, so as to simplify layout and routing.

[0123] Specifically, the current bold elements can be divided into 64 (i.e. Each fine voxel is assigned a vertex category number (0 to 7) to each of its eight vertices. For each vertex of a fine voxel, the least significant bits in its 3D coordinates include the x, y, and z dimensions, used to determine its vertex category number. Therefore, for any fine voxel, its eight vertices are guaranteed to be distributed across different data slices. The features of each fine voxel's eight vertices are divided into eight categories based on their vertex category numbers and stored in eight independent data slices. When calculating the features of the sampling points using a trilinear interpolation method, all eight data slices can be read in parallel at once, avoiding access conflicts and enabling the loading of all features in a single cycle.

[0124] like Figure 7 As shown, for the current bold voxel, a Cartesian coordinate system (i.e., the current voxel coordinate system) can be established, using... This represents the 3D coordinates of 125 vertices. The value of each coordinate ranges from... It can be represented by 3 binary digits: , , .

[0125] First, extract The least significant bit (LSB) is formed in the order of zyx. This allows us to divide the 125 vertices into 8 categories, resulting in vertex category numbers for each category. For example, the coordinates of vertex A are... ,correspond Therefore, vertex category number A is 2 (FID binary 010 corresponds to decimal 2). Optionally, for , and When the value is 0, it means that the corresponding vertex coordinates are even (i.e., 0, 2, and 4); when the value is 1, it means that the corresponding vertex coordinates are odd (i.e., 1 and 3).

[0126] Next, for a certain type of vertex category number, it is... The most significant bit and the middle bit (i.e., the remaining significant bits) are extracted and combined into a 6-bit binary number according to the zyx order. This 6-bit binary number can represent a maximum of 64 numbers. However, in practical applications, as shown in Table 1, the number of vertices of the 8 types in each bold voxel is 27, 18, 18, 12, 18, 12, 12, and 8, respectively. Table 1 Vertex Number Distribution Table

[0127]

[0128] To save address width in data fragmentation, a mapping relationship can be designed to map 6-bit original vertex addresses to 5-bit vertex encoded addresses, ensuring that the 5-bit vertex encoded address corresponding to each class of vertices is unique within the class and that the encoding range is consistent with the number of vertices in each class. For example, the first class has 27 vertices, with an encoding range of 0~26; the second class has 18 vertices, with an encoding range of 0~17. For each vertex... The specific mapping process of vertex 3D coordinates includes the following three steps:

[0129] 1) Extract the remaining significant bits from the vertex 3D coordinates, excluding the least significant bit, and map the remaining significant bits to a ternary index. ,in, .

[0130] 2) In each fixed FID category (by the least significant bit) In the definition, all vertices are arranged according to... Or press The order (if they are the same, then sort by) (Sort by method) and assign consecutive decimal vertex address numbers. n is the number of vertices in this category.

[0131] 3) Convert the decimal number It is represented by 5 bits as the vertex encoding address.

[0132] For example, the specific mapping process from vertex 3D coordinates to vertex category number and vertex code address is shown in Table 2. ) and Table 3 ( As shown in the figure.

[0133] Table 2 Vertex 3D Coordinate Mapping Table

[0134]

[0135] Table 3 Vertex 3D Coordinate Mapping Table

[0136]

[0137] This invention first introduces a vertex type classification storage mechanism, dividing voxel vertices into 8 categories based on their vertex category numbers (0 to 7), with each category mapped to one data slice. The 8 vertex features that a sampling point needs to access can be read in parallel from the 8 data slices at once, improving reading efficiency and eliminating access address conflicts. Second, it introduces a fine voxel-level feature organization method, instead of storing 8 vertex features for each fine voxel separately, it stores the vertex features of each coarse voxel as a whole, reducing the number of feature vectors required to be stored for each coarse voxel from 512 to 125, significantly reducing redundant storage and saving on-chip storage area. Finally, it introduces a data slice number and vertex category number mapping balancing mechanism, avoiding the problem of uneven data slice depth that may result from directly mapping vertex category numbers to data slice numbers. This invention introduces a cyclic shift mapping mechanism between data fragment numbering and vertex category numbering, ensuring consistent depth for each data fragment, reducing wiring conflicts and area waste, and improving on-chip resource utilization efficiency. This scheme supports efficient feature access in high-density neural radiation field inference scenarios, adapts to higher resolutions and more complex topologies, and can be integrated into the feature cache subsystem of a neural rendering chip. In summary, this invention fully combines feature access behavior, voxel topology, and the physical structure characteristics of SRAM (Static Random Access Memory) to construct an efficient feature storage scheme for neural rendering, possessing advantages such as clear structure and ease of implementation.

[0138] Example 3

[0139] Figure 8 This is a schematic diagram of a feature storage device provided in Embodiment 3 of the present invention. This embodiment of the invention is applicable to situations where feature storage is performed during the rendering of a neural radiation field. The device can execute a feature storage method and can be implemented in hardware and / or software. The device can be configured in an electronic device that carries feature storage functionality.

[0140] See Figure 8The feature storage device shown includes: a ray generation and tracing module 801, a two-level voxel traversal module 802, a target vertex filtering module 803, and a target feature storage module 804. The system includes: a ray generation and tracking module 801, used to transmit rays from the camera center to each pixel according to camera parameters and observation direction; a two-level voxel traversal module 802, used to uniformly divide the three-dimensional space to be detected into multiple coarse voxels, and traverse each coarse voxel along the ray direction to determine the current coarse voxel and the current fine voxel; wherein the current fine voxel is a hierarchical voxel obtained by further uniformly dividing the current coarse voxel; a target vertex filtering module 803, used to detect target vertices in the current fine voxels that have not been stored as feature vectors, and to perform feature interpolation on the features of the target vertices to obtain target feature vectors; and a target feature storage module 804, used to determine target data slices and target storage addresses according to the three-dimensional coordinates of the target vertices in the current voxel coordinate system, and to store the target feature vectors according to the target data slices and the target storage addresses.

[0141] The technical solution of this invention achieves two-level traversal of the three-dimensional space to be detected by uniformly dividing the three-dimensional space to be detected into multiple coarse voxels and traversing each coarse voxel along the ray direction to determine the current coarse voxel and the current fine voxel, thereby improving the sampling efficiency of the three-dimensional space to be detected. By detecting the target vertices in the current fine voxels that have not been stored as feature vectors for the current coarse voxel, and performing feature interpolation on the features of the target vertices to obtain the target feature vectors, and storing the features in units of coarse voxels, the repeated storage of common vertices between adjacent fine voxels in the same coarse voxel is avoided, which significantly reduces the redundant storage of feature vectors and saves on-chip storage overhead. By determining the target data fragments and target storage addresses based on the three-dimensional coordinates of the target vertices in the current voxel coordinate system, and storing the target feature vectors based on the target data fragments and target storage addresses, the access conflicts of the feature cache can be reduced, enabling efficient feature access to on-chip features during neural rendering and improving neural rendering efficiency.

[0142] In an optional embodiment of the present invention, the target feature storage module 804 includes: a target vertex three-dimensional coordinate acquisition unit, used to construct a current voxel coordinate system for each fine voxel of the current coarse voxel, and acquire the target vertex three-dimensional coordinates in the current voxel coordinate system; a target vertex binary number generation unit, used to represent the target vertex coordinate values ​​in each dimension of the target vertex three-dimensional coordinates in binary form, to obtain the target vertex binary number in each dimension; a target vertex category number generation unit, used to extract the least significant bit of the target vertex binary number in each dimension, to generate a target vertex category number; a target vertex encoding address generation unit, used to extract the remaining significant bits of the target vertex binary number in each dimension, to generate a target vertex encoding address; and a target storage address determination unit, used to determine the target data fragment and the corresponding target storage address in the target data fragment according to the target vertex category number and the target vertex encoding address.

[0143] In an optional embodiment of the present invention, the target vertex encoded address generation unit includes: a target vertex original address generation subunit, used to extract the target remaining valid bits of the target vertex binary number in each dimension to generate the target vertex original address; and a target vertex encoded address generation subunit, used to determine the corresponding target vertex encoded address based on the target vertex original address; wherein the number of bits in the target vertex encoded address is lower than that in the target vertex original address.

[0144] In an optional embodiment of the present invention, the target vertex encoding address generation unit further includes: a reference vertex 3D coordinate acquisition slave unit, configured to acquire the reference vertex 3D coordinates of each reference vertex within the current bold voxel before determining the corresponding target vertex encoding address based on the original address of the target vertex; a reference vertex binary number generation slave unit, configured to, for a single reference vertex, represent the reference vertex coordinate values ​​of each dimension of the reference vertex 3D coordinates in binary form to obtain the reference vertex binary number for each dimension; and a reference vertex category number generation slave unit, configured to, for a single reference vertex, extract the least significant bit of the reference vertex binary number for each dimension to generate a reference vertex category number; wherein, each reference vertex category number... Includes 1-8; a reference vertex original address generation slave unit, used to extract the remaining valid bits of the reference vertex binary number for a single reference vertex to generate the reference vertex original address; a reference vertex address sequence number calculation slave unit, used to sort the original addresses of each reference vertex under the same reference vertex category number in ascending order and calculate the reference vertex address sequence number of each original address under the reference vertex category number; a reference vertex encoded address generation slave unit, used to represent the reference vertex address sequence number in binary for a single reference vertex to generate the corresponding reference vertex encoded address; and a vertex address correspondence storage slave unit, used to store the original addresses of each reference vertex and the corresponding reference vertex encoded addresses accordingly.

[0145] In an optional embodiment of the present invention, the target storage address determination unit includes: a current bold voxel number acquisition subunit, used to acquire the current bold voxel number and the highest bit storage address of the feature vector already stored in the target data segment; a target data segment number determination subunit, used to perform a cyclic shift on the target vertex category number according to the current bold voxel number to determine the target data segment number; and a target storage address calculation subunit, used to accumulate the highest bit storage address and the target vertex encoding address to calculate the target storage address.

[0146] In an optional embodiment of the present invention, the two-level voxel traversal module 802 includes: a coarse voxel non-empty detection unit, configured to perform non-empty detection on each of the coarse voxels along the ray direction; a current coarse voxel division unit, configured to determine the coarse voxel as the current coarse voxel when the coarse voxel is non-empty, and further uniformly divide the current coarse voxel into a plurality of fine voxels; a fine voxel non-empty detection unit, configured to perform non-empty detection on each of the fine voxels along the ray direction; and a current fine voxel determination unit, configured to determine the fine voxel as the current fine voxel when the fine voxel is non-empty.

[0147] The feature storage device provided in the embodiments of the present invention can execute the feature storage method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of executing the method.

[0148] In the technical solutions of this invention, the acquisition, storage, and application of the target vertex's three-dimensional coordinates in the current voxel coordinate system, the reference vertex's three-dimensional coordinates of each reference vertex in the current bold voxel, the current bold voxel number, and the highest-order bit storage address of the feature vector stored in the target data slice, etc., all comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0149] Example 4

[0150] Figure 9 A schematic diagram of an electronic device 900 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0151] like Figure 9 As shown, the electronic device 900 includes at least one processor 901 and a memory, such as a read-only memory (ROM) 902 or a random access memory (RAM) 903, communicatively connected to the at least one processor 901. The memory stores computer programs executable by the at least one processor. The processor 901 can perform various appropriate actions and processes based on the computer program stored in the ROM 902 or loaded into the RAM 903 from storage unit 908. The RAM 903 can also store various programs and data required for the operation of the electronic device 900. The processor 901, ROM 902, and RAM 903 are interconnected via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0152] Multiple components in electronic device 900 are connected to I / O interface 905, including: input unit 906, such as keyboard, mouse, etc.; output unit 907, such as various types of displays, speakers, etc.; storage unit 908, such as disk, optical disk, etc.; and communication unit 909, such as network card, modem, wireless transceiver, etc. Communication unit 909 allows electronic device 900 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0153] Processor 901 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 901 performs the various methods and processes described above, such as feature storage methods.

[0154] In some embodiments, the feature storage method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 908. In some embodiments, part or all of the computer program may be loaded and / or mounted on electronic device 900 via ROM 902 and / or communication unit 909. When the computer program is loaded into RAM 903 and executed by processor 901, one or more steps of the feature storage method described above may be performed. Alternatively, in other embodiments, processor 901 may be configured to execute the feature storage method by any other suitable means (e.g., by means of firmware).

[0155] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include: implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0156] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0157] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0158] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0159] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0160] A computing system can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system. It addresses the shortcomings of traditional physical hosts and VPS (Virtual Private Server) services, such as high management difficulty and weak business scalability.

[0161] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0162] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A feature storage method characterized by comprising: The method comprises: transmitting a ray from a camera center to each pixel according to camera parameters and an observation direction; uniformly dividing a three-dimensional space to be detected into a plurality of coarse voxels, and traversing each coarse voxel along a ray direction to determine a current coarse voxel and a current fine voxel, wherein the current fine voxel is a hierarchical voxel obtained by further uniformly dividing the current coarse voxel; detecting a target vertex in the current fine voxel which does not store a feature vector for the current coarse voxel, and performing feature interpolation on features of the target vertex to obtain a target feature vector; determining a target data segment and a target storage address according to a target vertex three-dimensional coordinate of the target vertex in a current voxel coordinate system, and storing the target feature vector according to the target data segment and the target storage address.

2. The method of claim 1, wherein, The determining of the target data segment and the target storage address according to the target vertex three-dimensional coordinate of the target vertex in the current voxel coordinate system comprises: constructing a current voxel coordinate system for each fine voxel of the current coarse voxel, and obtaining a target vertex three-dimensional coordinate of the target vertex in the current voxel coordinate system; performing binary representation on target vertex coordinate values of each dimension of the target vertex three-dimensional coordinate to obtain target vertex binary numbers of each dimension; extracting target least significant bits of the target vertex binary numbers of each dimension to generate target vertex category numbers; extracting target remaining significant bits of the target vertex binary numbers of each dimension to generate target vertex encoding addresses; determining a target data segment and a corresponding target storage address in the target data segment according to the target vertex category numbers and the target vertex encoding addresses.

3. The method of claim 2, wherein, The extracting of the target remaining significant bits of the target vertex binary numbers of each dimension to generate the target vertex encoding addresses comprises: extracting target remaining significant bits of the target vertex binary numbers of each dimension to generate target vertex original addresses; determining corresponding target vertex encoding addresses according to the target vertex original addresses, wherein a bit number of the target vertex encoding addresses is lower than that of the target vertex original addresses.

4. The method of claim 3, wherein, Before the determining of the corresponding target vertex encoding addresses according to the target vertex original addresses, the method comprises: obtaining reference vertex three-dimensional coordinates of each reference vertex in the current coarse voxel; performing binary representation on reference vertex coordinate values of each dimension of the reference vertex three-dimensional coordinates to obtain reference vertex binary numbers of each dimension for a single reference vertex; extracting reference least significant bits of the reference vertex binary numbers of each dimension to generate reference vertex category numbers for the single reference vertex, wherein the reference vertex category numbers comprise 1-8; extracting reference remaining significant bits of the reference vertex binary numbers to generate reference vertex original addresses for the single reference vertex; performing ascending order sorting on the reference vertex original addresses of each reference vertex under a same reference vertex category number, and calculating reference vertex address serial numbers of the reference vertex original addresses under the same reference vertex category number. The reference vertex address sequence number is represented in binary form to generate a corresponding reference vertex encoding address for each reference vertex. The reference vertex original address and the corresponding reference vertex encoding address are stored correspondingly.

5. The method of claim 2, wherein, The method comprises the following steps: Obtaining the current coarse voxel number and the highest bit storage address of the stored feature vector in the target data slice; According to the current coarse voxel number, the target vertex category number is circularly shifted to determine the target data slice number; The highest bit storage address and the target vertex encoding address are accumulated to calculate the target storage address.

6. The method of claim 1, wherein, The method comprises the following steps: Non-empty detection is performed on each coarse voxel along the ray direction; When the coarse voxel is not empty, the coarse voxel is determined as the current coarse voxel, and the current coarse voxel is further uniformly divided into a plurality of fine voxels; Non-empty detection is performed on each fine voxel along the ray direction; When the fine voxel is not empty, the fine voxel is determined as the current fine voxel.

7. An attribute storage device characterized by comprising: The device comprises: A ray generation and tracking module for transmitting a ray from the camera center to each pixel according to the camera parameters and the observation direction; A two-level voxel traversal module for uniformly dividing the three-dimensional space to be detected into a plurality of coarse voxels, and traversing each coarse voxel along the ray direction to determine the current coarse voxel and the current fine voxel; wherein the current fine voxel is a level voxel obtained by further uniformly dividing the current coarse voxel; A target vertex screening module for detecting target vertices in the current fine voxel that have not been stored with feature vectors for the current coarse voxel, and performing feature interpolation on the features of the target vertices to obtain a target feature vector; A target feature storage module for determining a target data slice and a target storage address according to the target vertex three-dimensional coordinates of the target vertex in the current voxel coordinate system, and storing the target feature vector according to the target data slice and the target storage address.

8. An electronic device, comprising: The electronic device comprises: At least one processor; and A memory connected in communication with the at least one processor; wherein The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the feature storage method of any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer instructions for enabling the processor to execute the feature storage method of any one of claims 1-6 when executed.

10. A computer program product, characterised in that, The computer program product comprises a computer program, and the computer program, when executed by the processor, implements the feature storage method according to any one of claims 1-6. The computer program product comprises a computer program, and the computer program, when executed by the processor, implements the feature storage method according to any one of claims 1-6.

Images