Nerve radiation field rendering method based on dynamic hash coding

Abstract

The invention discloses a neural radiation field rendering method based on dynamic hash coding, and the method comprises the steps: employing the feature sequence data as the input, calculating the density value and color value of each sampling point through the forward propagation of a neural network, carrying out the volume rendering integral operation according to the ray tracing principle in the direction of a ray, and obtaining the feature sequence data; judging a final color output result of the current pixel point; according to an error value between the color output result and a real image, updating a network parameter weight through a back propagation algorithm, and if the error value is greater than a convergence threshold, continuing to iterate the training process to adjust a feature coding strategy to obtain an optimized neural radiation field model parameter; and after the rendering performance configuration parameters are obtained, optimizing a storage allocation strategy of feature data through a memory pool management mechanism, and if the current memory occupancy rate exceeds a safety threshold, starting a data compression algorithm to reduce the storage space requirement, and obtaining a real-time rendering output result. According to the invention, high-quality real-time rendering of the dynamic scene is realized.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of computer vision, and particularly relates to a neural radiance field rendering method based on dynamic hash coding. BACKGROUND

[0002] As a core technology of three-dimensional scene reconstruction and rendering, the neural radiance field technology plays a crucial role in medical imaging, film production, virtual reality and other fields. Through implicit neural representation method, the technology can reconstruct high-quality three-dimensional scenes from two-dimensional images, providing revolutionary visual modeling capabilities for multiple industries.

[0003] The current mainstream neural radiance field method generally has the problem of excessive consumption of computing resources. The traditional method relies on a fully connected neural network architecture, resulting in a time-consuming training process that is difficult to meet the timeliness requirements of practical applications. Although some improved solutions have made some progress in static scene processing by introducing hash coding technology, these methods still show obvious performance bottlenecks when facing complex dynamic scenes. The existing voxel storage scheme consumes a large amount of video memory resources, making it extremely difficult to process high-resolution scenes.

[0004] The hash collision problem in dynamic scene modeling directly affects the accuracy of feature representation. When multiple spatial positions are mapped to the same hash value, it will cause confusion and loss of feature information. This hash collision phenomenon further exacerbates the efficiency loss in the multi-resolution feature fusion process, because the system needs additional computing resources to handle the conflicting feature mapping relationship. The reduction of multi-resolution feature fusion efficiency not only prolongs the training time, but more importantly, limits the system's modeling ability for complex motion patterns in dynamic scenes, especially when dealing with non-rigid deformation and rapid motion, existing methods often cannot accurately capture and reproduce these dynamic features.

[0005] How to ensure the accuracy of dynamic scene modeling while optimizing the hash coding mechanism and feature fusion strategy to achieve efficient real-time rendering has become a key problem in the development of current neural radiance field technology. SUMMARY

[0006] To solve the problems of low dynamic scene modeling efficiency, high memory occupation and insufficient real-time rendering capability of existing neural radiance field technology, the application provides a neural radiance field rendering method based on dynamic hash coding, comprising:

[0007] Obtain spatial position coordinate data in a dynamic scene, construct a multi-level hash table structure through an adaptive hash mapping algorithm, and if a hash collision phenomenon is detected, start a collision detection mechanism to analyze the spatial distribution characteristics of the collision position, and obtain an optimized hash index mapping relationship table;

[0008] According to the position information in the hash index mapping relationship table, a hierarchical feature fusion algorithm is used to encode and process feature vectors of different resolution levels, and a feature weight distribution mechanism is used to calculate the contribution value of each level feature to determine the integrated feature representation vector after fusion;

[0009] For the integrated feature representation vector, the motion trajectory change pattern of the object in the dynamic scene is analyzed through the spatio-temporal consistency constraint condition, and if the motion amplitude exceeds the preset threshold, a feature update mechanism is triggered to recalculate the feature encoding of the region to obtain the spatio-temporal continuous feature sequence data.

[0010] The feature sequence data is taken as input, and the density value and color value of each sampling point are calculated through neural network forward propagation. According to the principle of ray tracing, volume rendering integral operation is performed along the ray direction to determine the color output result of the current pixel point.

[0011] According to the error value between the color output result and the real image, the network parameter weight is updated through the back propagation algorithm, and if the error value is greater than the convergence threshold, the feature encoding strategy is adjusted by continuing the iterative training process to obtain the optimized neural radiation field model parameters.

[0012] The new view image is predicted and rendered through the neural radiation field model parameters, and a parallel computing architecture is used to process the color calculation task of a plurality of pixel points simultaneously. According to the GPU memory usage, the batch size is dynamically adjusted to determine the optimal rendering performance configuration parameters.

[0013] Based on the rendering performance configuration parameters, the storage allocation strategy of the feature data is optimized through the memory pool management mechanism. If the current memory occupancy rate exceeds the safety threshold, the data compression algorithm is started to reduce the storage space requirement, and the real-time rendering output result is obtained.

[0014] According to the quality evaluation index of the real-time rendering output result, the relationship between rendering accuracy and computing efficiency is dynamically balanced through an adaptive adjustment mechanism. A multi-thread concurrent processing technology is used to speed up the feature fusion calculation process, and the final rendering image that meets the real-time requirement is obtained.

[0015] Preferably, the process of obtaining the optimized hash index mapping relationship table comprises:

[0016] The spatial coordinate data of the target object in the dynamic scene is obtained, the three-dimensional position information is collected in real time through a sensor array and converted into a standardized coordinate format to obtain a spatial coordinate sequence containing X-axis, Y-axis and Z-axis values.

[0017] An adaptive hash mapping function is constructed according to the spatial coordinate sequence, a linear combination of coordinate numerical values is used as a hash key value calculation basis, an initial hash address is determined through a modulo operation, and an index position allocation result of a first-level hash table is obtained;

[0018] If a key value conflict is detected in the hash address calculation process, a collision detection mechanism is started to analyze the spatial distribution characteristics of the collision position, the spatial aggregation degree is judged by calculating the Euclidean distance between the collision coordinate points, and the distribution density parameter of the collision region is obtained;

[0019] According to the distribution density parameter, the hash function is redesigned, the hash table capacity is increased or the hash factor is adjusted to reduce the collision probability, and if the collision density exceeds the preset threshold, a second-level hash table is created, and a hierarchical division scheme of a multi-level index structure is obtained.

[0020] According to the hierarchical division scheme, the collision spatial coordinate data is redistributed to the hash table of the corresponding level, the chain address method is used to process multiple coordinate records under the same hash address, the spatial position information with the same hash value is stored through the chain table structure, and an optimized data storage layout is obtained.

[0021] The K-means clustering algorithm is used to analyze the clustering mode of the spatial coordinates in the multi-level hash table, the coordinate positions of the cluster centers are calculated, and the weight parameters of the hash function are updated, if the clustering result shows that the coordinate distribution is uneven, the hash strategy is adjusted, and the hash mapping parameter configuration based on clustering optimization is obtained.

[0022] The hash mapping parameter configuration is used to reconstruct the index mapping relationship table, and a one-to-one correspondence is established between the original spatial coordinates and the optimized hash addresses, and an optimized hash index mapping relationship table is obtained.

[0023] Preferably, the process of obtaining the fused comprehensive feature representation vector comprises:

[0024] The original multi-resolution image data is obtained, the hash function is used to perform hash calculation on the image pixel coordinates, and a hash index value sequence is obtained.

[0025] According to the hash index value sequence, a hash index mapping table is constructed, the pixel position information corresponding to each hash value is recorded, and the spatial position mapping relationship is determined.

[0026] Through the spatial position mapping relationship, feature extraction is performed on the image regions of different resolution levels, a convolutional neural network is used to extract feature vectors of each level, and a multi-level feature vector set is obtained.

[0027] According to the multi-level feature vector set, a hierarchical feature fusion algorithm is used to calculate a similarity matrix between the feature vectors, and the correlation strength value between the feature vectors is determined.

[0028] If the correlation strength value exceeds a preset threshold, a weight coefficient of the feature vector is calculated by using an attention mechanism, and a contribution degree value of each hierarchical feature is quantified by using a weight distribution mechanism;

[0029] According to the contribution degree value, the hierarchical feature vectors are weighted and fused, and a fused comprehensive feature representation vector is generated by using a vector splicing operation.

[0030] Preferably, the process of obtaining the spatiotemporal continuous feature sequence data comprises:

[0031] An object detection result in a dynamic scene is obtained, a region segmentation algorithm is used to divide a space region occupied by different objects in the scene, and boundary coordinate information of the objects is obtained;

[0032] According to the boundary coordinate information, a comprehensive feature vector of the object region is extracted, a convolutional neural network is used to perform feature encoding processing on the object region, and initial feature representation data is obtained;

[0033] A motion trajectory parameter is calculated by using a change in the position of the object between consecutive frames, and a position coordinate difference analysis method is used to determine a motion amplitude value of the object in a time sequence;

[0034] If the motion amplitude value exceeds a preset threshold, a feature updating mechanism is triggered to re-perform feature encoding calculation on the object region, and an updated feature representation vector is obtained;

[0035] The feature vectors of the same object in consecutive frames are verified for consistency according to a spatiotemporal constraint condition, and the continuity of the object identity is determined by using feature similarity calculation;

[0036] A sliding window method is used to organize the feature vector data in the time sequence, and feature sequence data containing the motion change rule of the object is obtained.

[0037] Preferably, the process of determining the color output result of the current pixel point by performing volume rendering integral operation along the ray direction according to the ray tracing principle comprises:

[0038] The feature sequence data is used as input data of the convolutional neural network, and the density value and the color value of the sampling point are calculated by using forward propagation calculation;

[0039] An integral path of volume rendering is determined according to the ray direction, the density value and the color value are accumulated along the path by using volume rendering integral operation, the integral operation is terminated if the accumulated transparency exceeds a preset threshold, and the final color output of the pixel point is determined by using the rendering result.

[0040] Preferably, the process of obtaining the optimized neural radiance field model parameter comprises:

[0041] The prediction color data output by the neural radiance field model and the corresponding real image color data are acquired, and a mean square error value is calculated to obtain a loss function value of the current training batch;

[0042] According to the loss function value, gradient information of weight parameters of each layer of the neural network is calculated by using a back propagation algorithm to obtain direction and amplitude data of weight updating;

[0043] The weight parameters of the neural network are updated by using a gradient descent algorithm, and if the calculated loss function value is greater than a preset convergence threshold, it is determined that the model training has not reached a convergence state;

[0044] A feature encoding strategy adjustment method is used to determine the encoding weight distribution ratio of the feature extraction layer according to the change trend of the current loss function value;

[0045] The adjusted feature encoding weight distribution ratio is acquired, and the forward propagation output result of the neural radiance field model is recalculated to obtain new prediction color data;

[0046] According to the error comparison result of the new prediction color data and the real image data, the model parameter optimization effect is judged, and if the error value continues to decrease, it is determined that the optimization direction is correct;

[0047] The parameter updating process is repeatedly executed through an iterative training process until the loss function value converges to a preset threshold range, and the finally optimized neural radiance field model parameters are obtained.

[0048] Preferably, the process of determining the optimal rendering performance configuration parameter comprises:

[0049] The spatial position encoding weight and the direction encoding weight of the optimized neural radiance field model parameters are extracted to obtain a scene representation parameter set;

[0050] A view prediction matrix is constructed according to the scene representation parameter set, a three-dimensional coordinate transformation algorithm is used to calculate the light direction vector and the starting position coordinates under the target view angle, and the sampling point distribution under the new view angle is determined;

[0051] The pixel processing task is distributed through a parallel computing architecture, if the available memory of the GPU is greater than a preset threshold, the pixel points are processed in a large batch parallel mode, if the memory is insufficient, the small batch processing mode is switched to, and a pixel-level calculation task distribution scheme is obtained;

[0052] The color calculation process is performed according to the pixel-level calculation task distribution scheme, and the volume rendering integral method is used to accumulate the color value and the transparency value along the light direction to obtain the final color output of each pixel point;

[0053] Real-time detection of GPU memory usage and computing load, if the memory usage exceeds the safety threshold, reduce the batch size, if it is lower than the minimum utilization, increase the batch size, determine the dynamic batch size adjustment strategy;

[0054] Update the performance configuration parameters according to the dynamic batch size adjustment strategy, reassign computing resources using load balancing algorithm, and obtain the optimal rendering performance configuration parameter combination through comprehensive evaluation of rendering time and memory usage;

[0055] Based on the optimal rendering performance configuration parameter combination, execute the final image rendering, complete the final calculation and output of color value through pixel shader, and obtain the new perspective prediction image.

[0056] Preferably, the process of obtaining real-time rendering output result includes:

[0057] Obtain the performance parameter set of the target device, and divide the feature data storage area in the memory pool using the pre-allocation strategy;

[0058] Compare the feedback memory usage with the preset safety threshold, if the memory usage exceeds the threshold, call the LZ4 algorithm to compress the inactive feature data block, release the redundant storage space after compression and update the memory pool allocation table, and input the optimized feature data into the rendering pipeline to generate frame sequence.

[0059] Preferably, the process of obtaining the final rendering image includes:

[0060] Obtain the current frame rendering image data, calculate the definition index, color saturation index and texture detail index of the rendering image, and obtain the comprehensive quality evaluation score;

[0061] According to the comparison result of the comprehensive quality evaluation score and the preset quality threshold, if the quality evaluation score is lower than the preset quality threshold, the adjustment mechanism automatically improves the rendering precision parameter, if the quality evaluation score is higher than the preset quality threshold, the adjustment mechanism reduces the rendering precision parameter, and determines the target rendering precision level of the current frame;

[0062] Obtain the processor utilization and memory usage of the current system, according to the target rendering precision level and system resource state, calculate the optimal thread allocation number through load balancing algorithm, and obtain the multi-thread concurrent processing configuration scheme;

[0063] According to the multi-thread concurrent processing configuration scheme, start parallel processing threads, each thread is responsible for processing image feature data in different regions, and obtain the texture feature vector, color feature vector and geometric feature vector of each region through feature extraction operation;

[0064] The regional feature vectors are weighted and fused, if the variance value of a regional feature vector exceeds a preset variation threshold, the fusion weight coefficient of the corresponding region is increased, and a global feature representation after fusion is obtained;

[0065] Through the rendering engine, a target rendering image is generated according to the global feature representation and a current rendering precision level, the image generation time is monitored in real time, if the generation time exceeds a preset time limit, a precision adjustment mechanism is triggered to reduce the rendering precision parameter of the next frame;

[0066] According to the pixel data of the target rendering image and color space conversion, brightness adjustment and contrast optimization are performed, and a final rendering image output meeting the real-time display requirement is obtained.

[0067] Compared with the prior art, the present application has the following advantages and technical effects:

[0068] The application discloses a dynamic scene real-time rendering method based on a neural radiation field. The application adopts a parallel computing architecture and a memory pool management mechanism to improve rendering efficiency, balances precision and efficiency through adaptive adjustment, realizes high-quality real-time rendering of a dynamic scene, effectively solves the problems of poor rendering effect and low calculation efficiency of a traditional method in a complex dynamic scene, and provides new technical support for virtual reality, augmented reality and other fields. BRIEF DESCRIPTION OF DRAWINGS

[0069] The accompanying drawings, which form a part of this application, are included to provide a further understanding of the application and are incorporated in and constitute a part of this application. The embodiments of this application and their

[0070] Figure 1 The accompanying drawings, which form a part of this application, are included to provide a further understanding of the application and are incorporated in and constitute a part of this application. The embodiments of this application and their DETAILED DESCRIPTION

[0071] It should be noted that the embodiments and features in the present application can be combined with each other without conflict.

[0072] It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a group of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0073] As Figure 1As shown, the embodiment provides a neural radiance field rendering method based on dynamic hash coding, comprising:

[0074] Obtain spatial position coordinate data in a dynamic scene, construct a multi-level hash table structure through an adaptive hash mapping algorithm, if a hash collision phenomenon is detected, start a collision detection mechanism to analyze the spatial distribution characteristics of the collision position, and obtain an optimized hash index mapping relationship table;

[0075] According to the position information in the hash index mapping relationship table, a hierarchical feature fusion algorithm is used to encode and process the feature vectors of different resolution levels, the contribution values of each level feature are calculated through a feature weight distribution mechanism, and a comprehensive feature representation vector after fusion is determined;

[0076] For the comprehensive feature representation vector, analyze the motion trajectory change mode of the object in the dynamic scene through the spatio-temporal consistency constraint condition, if the motion amplitude exceeds the preset threshold, trigger the feature update mechanism to recalculate the feature coding of the region, and obtain the spatio-temporal continuous feature sequence data;

[0077] Take the feature sequence data as input, calculate the density value and color value of each sampling point through neural network forward propagation, perform volume rendering integral operation along the ray direction according to the ray tracing principle, and judge the color output result of the current pixel point;

[0078] According to the error value between the color output result and the real image, update the network parameter weight through the back propagation algorithm, if the error value is greater than the convergence threshold, continue the iteration training process to adjust the feature coding strategy, and obtain the optimized neural radiance field model parameter;

[0079] Predict and render a new view image through the neural radiance field model parameter, use parallel computing architecture to process the color calculation task of a plurality of pixel points at the same time, dynamically adjust the batch size according to the GPU memory usage, and determine the optimal rendering performance configuration parameter;

[0080] Based on the rendering performance configuration parameter, optimize the storage allocation strategy of feature data through the memory pool management mechanism, if the current memory occupancy rate exceeds the safety threshold, start the data compression algorithm to reduce the storage space requirement, and obtain the real-time rendering output result;

[0081] According to the quality evaluation index of the real-time rendering output result, dynamically balance the relationship between rendering accuracy and calculation efficiency through adaptive adjustment mechanism, use multi-thread concurrent processing technology to speed up the feature fusion calculation process, and obtain the final rendering image meeting the real-time requirement.

[0082] Further, the process of obtaining the optimized hash index mapping relationship table comprises:

[0083] The spatial coordinate data of the target object in the dynamic scene is acquired, three-dimensional position information is collected in real time through a sensor array and converted into a standardized coordinate format, and a spatial coordinate sequence containing X-axis, Y-axis and Z-axis values is obtained.

[0084] An adaptive hash mapping function is constructed according to the spatial coordinate sequence, a linear combination of coordinate values is used as a hash key value calculation basis, an initial hash address is determined through a modulo operation, and an index position allocation result of a first-level hash table is obtained.

[0085] If a key value conflict is detected during hash address calculation, a collision detection mechanism is started to analyze the spatial distribution characteristics of the collision position, the spatial aggregation degree is judged by calculating the Euclidean distance between the collision coordinate points, and the distribution density parameter of the collision region is obtained.

[0086] The hash function is redesigned according to the distribution density parameter, the hash table capacity is increased or the hash factor is adjusted to reduce the collision probability, and if the collision density exceeds a preset threshold, a second-level hash table is created, and a hierarchical division scheme of a multi-level index structure is obtained.

[0087] According to the hierarchical division scheme, the collision spatial coordinate data is redistributed to the hash table of the corresponding level, the chain address method is used to process multiple coordinate records under the same hash address, the spatial position information with the same hash value is stored through the linked list structure, and the optimized data storage layout is obtained.

[0088] The clustering mode of the spatial coordinates in the multi-level hash table is analyzed through the K-means clustering algorithm, the coordinate positions of the cluster centers are calculated, and the weight parameters of the hash function are updated, if the clustering result shows that the coordinate distribution is uneven, the hash strategy is adjusted, and the hash mapping parameter configuration based on clustering optimization is obtained.

[0089] The index mapping relationship table is reconstructed using the hash mapping parameter configuration, and a one-to-one correspondence is established between the original spatial coordinates and the optimized hash addresses, and the optimized hash index mapping relationship table is obtained.

[0090] In one possible implementation, the spatial coordinate data of the target object in the dynamic scene is obtained through a multi-sensor array for real-time three-dimensional positioning.

[0091] Specifically, when the moving target in the monitoring area is captured by the laser radar and the depth camera at the same time, the system converts the original sensor data into a standardized three-dimensional coordinate format, and each target object corresponds to a set of spatial position information containing X-axis, Y-axis and Z-axis values.

[0092] For example, the coordinates of target A at a certain moment are (125.6, 89.3, 45.2), and the coordinates of target B are (98.7, 156.4, 38.9). These continuously collected coordinate data constitute a complete spatial coordinate sequence.

[0093] For example, the construction of the adaptive hash mapping function is based on the linear combination calculation of coordinate values. The system converts three-dimensional coordinates into hash key values by using a weighted linear combination method. The X, Y, and Z axis coordinates are weighted and summed by setting weight coefficients, and then the initial storage address is determined by performing a modulo operation on the hash table capacity. When the target coordinates (125.6, 89.3, 45.2) are calculated by the hash function, the address index value is obtained, which directly corresponds to the storage location in the first-level hash table.

[0094] In one embodiment, the hash collision detection mechanism optimizes the storage strategy by analyzing the spatial distribution characteristics of the collision positions. When multiple different spatial coordinates are mapped to the same address after hash calculation, the system automatically starts the collision detection program, calculates the Euclidean distance between these collision coordinate points to judge the degree of spatial aggregation.

[0095] Specifically, if the average distance between the collision coordinate points is less than the preset aggregation threshold, it is considered that there is a high-density spatial distribution in this area, and the hash strategy needs to be adjusted to reduce the collision probability.

[0096] For example, the creation of a multi-level index structure is based on the dynamic evaluation of the collision density parameter. When the collision density of the first-level hash table exceeds the set threshold, the system automatically creates a second-level hash table to disperse the storage pressure. By increasing the hash table capacity or adjusting the hash factor, the system can effectively reduce the collision probability during subsequent data storage while maintaining the stability of query efficiency.

[0097] In one possible implementation, the chain address method processes multiple coordinate records under the same hash address by using a linked list structure to achieve efficient storage. When a certain hash address corresponds to multiple spatial coordinates, the system stores these coordinates in a linked list in chronological order or spatial distance order. Each linked list node contains complete three-dimensional coordinate information and a pointer to the next node.

[0098] For example, the K-means clustering algorithm analyzes the spatial coordinate aggregation pattern in the multi-level hash table, and optimizes the weight parameters of the hash function by calculating the coordinate positions of the cluster centers. When the clustering result shows that the coordinate distribution is obviously uneven, the system adjusts the weight coefficients of the axis coordinates in the hash function according to the cluster center positions, so that the coordinates of similar spatial positions can be more evenly distributed in the hash table.

[0099] In an embodiment, the reconstruction of the index mapping table establishes a one-to-one correspondence between the original spatial coordinates and the optimized hash addresses. The system records the final storage location and access path of each coordinate point through the lookup table, enabling fast spatial coordinate retrieval and update operations, and significantly improving the real-time performance and accuracy of target tracking in dynamic scenarios.

[0100] Further, the process of obtaining the fused comprehensive feature representation vector includes:

[0101] Obtain the original multi-resolution image data, and perform hash calculation on the image pixel coordinates using a hash function to obtain a hash index value sequence;

[0102] According to the hash index value sequence, a hash index mapping table is constructed to record the pixel position information corresponding to each hash value, and the spatial position mapping relationship is determined;

[0103] According to the spatial position mapping relationship, feature extraction is performed on the image regions of different resolution levels, and a convolutional neural network is used to extract feature vectors of each level to obtain a multi-level feature vector set;

[0104] According to the multi-level feature vector set, a similarity matrix between the feature vectors is calculated through a hierarchical feature fusion algorithm to determine the correlation strength value between the feature vectors;

[0105] If the correlation strength value exceeds a preset threshold, the weight coefficient of the feature vector is calculated using an attention mechanism, and the contribution degree value of each level feature is quantified through a weight distribution mechanism;

[0106] According to the contribution degree value, weighted fusion processing is performed on the feature vectors of each level, and a fused comprehensive feature representation vector is generated through vector splicing operation.

[0107] In an embodiment, the calculation process of the hash function on the image pixel coordinates adopts a bit operation combination method of coordinate values, and the row and column coordinates of the pixel are encoded through bit shift operation.

[0108] Specifically, for an image with a resolution of 1920x1080, the pixel coordinate (512, 768) is calculated by the hash function to obtain the index value 3847, which directly points to the corresponding storage location in the hash table. This mapping method can realize fast positioning of pixel position and significantly improve the access efficiency of image processing.

[0109] For example, when constructing the hash index mapping table, the system allocates independent storage space for each hash value to record the detailed location information of the pixels. When the hash value is 3847, the mapping table records that the position corresponds to the pixel point at the 512th row and the 768th column in the original image, and at the same time marks the resolution level to which the pixel belongs as the second level. Through this indexing mechanism, the system can accurately track the hierarchical attribution relationship of each pixel in the multi-resolution image pyramid.

[0110] In one possible implementation, the convolutional neural network uses a multi-scale convolution kernel design when extracting features from different resolution levels. The first level uses a 3x3 convolution kernel to extract detailed features, the second level uses a 5x5 convolution kernel to capture medium-scale features, and the third level uses a 7x7 convolution kernel to obtain global features. The dimension of the feature vector extracted by each level is 256, 512, and 1024 respectively, forming a multi-level feature vector set containing information of different levels of abstraction.

[0111] For example, when calculating the similarity between feature vectors, the hierarchical feature fusion algorithm uses the cosine similarity measurement method. When the similarity value between the first level feature vector and the second level feature vector reaches 0.85, it indicates that there is a strong correlation between the two levels. The system sets the similarity threshold to 0.8, and the feature vector pair that exceeds this threshold will be marked as a high correlation feature combination, which needs to be paid attention to and the weight adjusted.

[0112] In one embodiment, the attention mechanism dynamically allocates weight coefficients according to the correlation strength of the feature vectors. For a feature vector pair with a similarity of 0.85, the first level feature obtains a weight coefficient of 0.6, and the second level feature obtains a weight coefficient of 0.4. The weight allocation follows the principle that the higher the correlation strength, the smaller the difference in weight, ensuring that the high correlation features maintain a balanced contribution in the fusion process.

[0113] For example, during the weighted fusion process, the system linearly combines the feature vectors of the three levels according to the calculated weight coefficients. The first level 256-dimensional feature vector is multiplied by the weight 0.6, the second level 512-dimensional feature vector is multiplied by the weight 0.4, and the third level 1024-dimensional feature vector is multiplied by the weight 0.3. Through the vector splicing operation, a 1792-dimensional comprehensive feature representation vector is generated. This fusion method can fully utilize the complementary information of the features at different levels, and improve the completeness and discriminability of the feature representation.

[0114] In one possible implementation, after receiving the 1792-dimensional comprehensive feature vector, the support vector machine classifier performs non-linear mapping through the radial basis function kernel. The decision boundary learned by the classifier in the training stage can effectively distinguish different classes of feature patterns, and finally output a classification result with a confidence of 0.92, indicating that the feature recognition has high reliability.

[0115] Furthermore, the process of obtaining spatiotemporally continuous feature sequence data includes:

[0116] Obtain object detection results in dynamic scenes, divide the spatial regions occupied by different objects in the scene through region segmentation algorithms, and obtain the boundary coordinate information of the objects;

[0117] The comprehensive feature vector of the object area is extracted based on the boundary coordinate information, and the convolutional neural network is used to perform feature encoding processing on the object area to obtain the initial feature representation data;

[0118] The motion trajectory parameters are calculated by the position change of the object between consecutive frames, and the motion amplitude value of the object in the time series is determined by the position coordinate difference analysis method;

[0119] If the motion amplitude exceeds the preset threshold, the feature update mechanism is triggered to recalculate the feature encoding of the object area to obtain an updated feature representation vector;

[0120] The consistency of the feature vectors of the same object in consecutive frames is verified according to the spatiotemporal constraints, and the continuity of the object identity is determined by feature similarity calculation;

[0121] The sliding window method is used to organize the feature vector data in the time series to obtain the feature sequence data containing the laws of object movement changes.

[0122] In one embodiment, the region segmentation algorithm performs pixel-level classification on the input video frames through a semantic segmentation network, and accurately divides different categories of objects such as pedestrians, vehicles, and buildings in the scene.

[0123] Specifically, the algorithm first performs multi-scale feature extraction on the input image with a resolution of 640×480, and then generates a segmentation mask of the same size as the original image through upsampling and jump connection mechanism. Each object area is assigned a unique identifier and its bounding box coordinate information is recorded.

[0124] For example, when a pedestrian is detected, the system extracts the RGB color histogram, texture features, and shape descriptors of the region as a comprehensive feature vector. The convolutional neural network uses ResNet-50 as the backbone network, and through multi-layer convolution operations, it encodes the 128×128 pixel object region into a 512-dimensional feature vector that contains key information such as the object's appearance, shape, and texture.

[0125] In a possible implementation, the motion trajectory parameter calculation is determined by the change of the object center point coordinates in consecutive three frames of images. Assuming that the coordinates of the object in the t-th frame are (x1, y1), and the coordinates in the t+1-th frame are (x2, y2), the motion amplitude value is calculated by the Euclidean distance. When the motion amplitude exceeds the preset threshold of 15 pixels, it indicates that the object has a significant displacement, and at this time, the feature updating mechanism is triggered to recalculate the feature representation of the object.

[0126] It should be noted that the spatiotemporal constraint condition verification is realized by calculating the cosine similarity of the feature vectors of the same object in consecutive frames. When the similarity value is higher than 0.8, it is considered that the two feature vectors belong to the same object, thereby ensuring the continuity of object identity recognition. This verification mechanism can effectively handle the feature drift problem caused by factors such as light changes and posture changes during the motion of the object.

[0127] In an embodiment, the sliding window method uses a time window with a length of 10 frames to organize the feature sequence data. The window slides on the time axis with a step size of 1, and each window contains the feature vector changes of the object in consecutive 10 frames. By analyzing these consecutive sequences, the system can identify the motion patterns of the object, such as uniform linear motion, accelerated motion, or turning motion, and other types.

[0128] Further, the process of determining the color output result of the current pixel point according to the ray tracing principle and performing volume rendering integral operation along the ray direction includes:

[0129] The feature sequence data is used as the input data of the convolutional neural network, and the density value and color value of the sampling point are calculated by forward propagation;

[0130] According to the ray direction, the integral path of volume rendering is determined, and the density value and color value are accumulated along the path by using the volume rendering integral operation. If the accumulated transparency exceeds the preset threshold, the integral operation is terminated, and the final color output of the pixel point is determined by the rendering result.

[0131] In a possible implementation, the feature column data includes the spatial position information, material attribute parameters, and light response features of the object in the dynamic scene.

[0132] Specifically, when the monitoring camera captures a moving vehicle, the system organizes the three-dimensional coordinates, surface reflectivity, transparency, and other attributes of the vehicle into a feature vector and inputs it into the neural network. The network processes these input data through a multi-layer perceptron structure, and each hidden layer node receives the weighted output of the previous layer during the forward propagation process, and then passes it to the next layer after being processed by the activation function.

[0133] For example, for a moving target in the scene, the sampling point density value reflects the tightness of the object material at that spatial position. When the system detects the metal body area, the density value is usually set to between 0.8 and 0.95, while the density value of the glass window area is controlled in the range of 0.1 to 0.3. The color value is output through the RGB three channels respectively, and the red channel value may be 0.7, the green channel value may be 0.2, and the blue channel value may be 0.1, so as to accurately represent the appearance characteristics of the vehicle.

[0134] It should be noted that the determination of the direction of the ray directly affects the quality effect of volume rendering. The system calculates the ray vector corresponding to each pixel according to the view angle parameters of the virtual camera, and the starting point of the ray is located at the camera position and the direction points to the target area in the scene. The integral path extends along the ray direction, and the path length is dynamically adjusted according to the scene depth range, which is usually set to between 2 meters and 50 meters.

[0135] In an embodiment, the integral algorithm adopts an adaptive step strategy to accumulate values along the path. When the ray passes through the surface of the vehicle, the system samples at intervals of 0.05 meters, and the density value of each sampling point is multiplied by the step length and accumulated into the total density. The accumulation process of the color value considers the forward scattering effect, and the color contribution of the current sampling point is weighted according to the accumulated transparency.

[0136] For example, when the transparency threshold is set to 0.95, it means that the ray has been almost completely blocked. At this time, continuing to integrate has little contribution to the final rendering result, and early termination of the integral process can significantly improve the calculation efficiency. This early termination mechanism can shorten the rendering time by 30% to 50% when processing complex scenes.

[0137] Specifically, the determination of the final color of the pixel point comprehensively considers the contribution of all effective sampling points on the entire ray path. The system normalizes the accumulated RGB color values to ensure that the output values are within the standard range of 0 to 255. This rendering method can maintain the consistency and authenticity of color representation when the vehicle moves under different lighting conditions, providing a high-quality visual data basis for subsequent target tracking and behavior analysis.

[0138] Further, the process of obtaining the optimized neural radiance field model parameters includes:

[0139] Obtain the predicted color data output by the neural radiance field model and the corresponding real image color data, and calculate the mean square error value to obtain the loss function value of the current training batch;

[0140] According to the loss function value, the gradient information of the weight parameters of each layer of the neural network is calculated by using the back propagation algorithm, and the direction and amplitude data of the weight update are obtained;

[0141] The weight parameters of the neural network are updated by a gradient descent algorithm, and if the calculated loss function value is greater than a preset convergence threshold, it is judged that the model training has not reached a convergence state.

[0142] The feature encoding strategy adjustment method is adopted to determine the encoding weight distribution ratio of the feature extraction layer according to the change trend of the current loss function value.

[0143] The adjusted feature encoding weight distribution ratio is obtained, and the forward propagation output result of the neural radiation field model is recalculated to obtain new predicted color data.

[0144] According to the error comparison result of the new predicted color data and the real image data, the model parameter optimization effect is judged, and if the error value continues to decrease, it is determined that the optimization direction is correct.

[0145] The parameter update process is repeatedly executed through the iterative training process until the loss function value converges to the preset threshold range, and the finally optimized neural radiation field model parameters are obtained.

[0146] Specifically, in the training process of the neural radiation field model, the system first calculates the mean square error between the predicted color value and the real image pixel value, for example, when the predicted red, green and blue channel values are (0.75, 0.62, 0.48) and the real values are (0.82, 0.58, 0.51), the calculated loss function value is 0.0038. Based on this error value, the back propagation algorithm starts to update the network parameters, and the weight matrix of the position encoding layer is adjusted by the gradient descent method, and the learning rate is set to 0.0005, and the weight update formula is new weight equal to original weight minus learning rate multiplied by gradient value. The error value of the current iteration 0.0038 is compared with the preset convergence threshold 0.001, since the error value exceeds the threshold, the system automatically triggers the adjustment mechanism of the feature encoding strategy. The frequency parameter of the position encoding is increased from 10 frequency components to 15, and an adaptive weight decay mechanism is introduced, and the regularization coefficient is adjusted from 0.01 to 0.008 to prevent overfitting.

[0147] The direction encoding part adopts spherical harmonic function expansion, and the order is increased from 3 to 4, which enhances the expression ability of the view angle correlation. The number of hidden layer neurons of the multi-layer perception is expanded from 256 to 320, and the activation function remains ReLU unchanged, but Sigmoid activation is added in the last layer to ensure that the color output is within a reasonable range. In the new round of training after parameter adjustment, the loss value is reduced to 0.0025, although it has not yet reached the convergence condition, but the downward trend is obvious, and the system continues to execute iterative optimization until the error converges.

[0148] Further, the process of determining the optimal rendering performance configuration parameter includes:

[0149] Extract the spatial position encoding weights and direction encoding weights of the optimized neural radiation field model parameters to obtain a set of scene representation parameters;

[0150] Construct a perspective prediction matrix based on the scene representation parameter set, use a three-dimensional coordinate transformation algorithm to calculate the light direction vector and starting position coordinates under the target perspective, and determine the sampling point distribution under the new perspective;

[0151] Pixel processing tasks are allocated through a parallel computing architecture. If the GPU available memory is greater than a preset threshold, pixels are processed in a large-batch parallel mode. If the memory is insufficient, the system switches to a small-batch processing mode to obtain a pixel-level computing task allocation solution.

[0152] The color calculation process is performed according to the pixel-level computing task allocation scheme, and the volume rendering integration method is used to accumulate the color value and transparency value along the light direction to obtain the final color output of each pixel;

[0153] Real-time detection of GPU memory usage and computing load. If the memory usage exceeds the safety threshold, the batch size is reduced; if it is below the minimum utilization, the batch size is increased to determine the dynamic batch size adjustment strategy.

[0154] Update performance configuration parameters based on dynamic batch size adjustment strategies, use load balancing algorithms to reallocate computing resources, and obtain the optimal rendering performance configuration parameter combination through comprehensive evaluation of rendering time and memory usage;

[0155] The final image rendering is performed based on the optimal rendering performance configuration parameter combination, and the final calculation and output of the color value are completed through the pixel shader to obtain the new perspective predicted image.

[0156] Specifically, based on the trained neural radiance field model parameters, the system first converts the camera parameter matrix of the target view into a ray equation in three-dimensional space, with each ray corresponding to a pixel point in the output image. Assuming the rendering resolution is 1920x1080, 2073600 rays need to be generated. The system uses CUDA parallel computing architecture to dynamically allocate batch size according to the GPU memory capacity. When an RTX4090 graphics card with 24GB of memory is detected, the initial batch size is set to 65536 rays. The memory usage is monitored in real time, and when the usage exceeds 85%, it is automatically reduced to 32768 rays, and when it is lower than 60%, it is increased to 131072 rays. For each ray, the system performs hierarchical sampling along the ray direction. A hierarchical sampling algorithm is used to uniformly sample 64 points between the near plane 2.0 meters and the far plane 100.0 meters. At the same time, importance sampling is used to additionally sample 128 points in the area with higher density, for a total of 192 sampling points input into the multi-layer perception network. During the forward propagation of the network, the position encoding uses a 10-layer sine-cosine function to map the three-dimensional coordinates to a 60-dimensional feature vector, and the direction encoding uses a 4-layer function to map the view direction to a 24-dimensional vector. After passing through an 8-layer fully connected network, the density value and RGB color value of each sampling point are output. The volume rendering integral uses a numerical integration method to calculate the transmittance according to the Beer-Lambert law, and the final pixel color is obtained by weighted summation. The weight is determined by the product of the density value and the sampling interval. The system uses a performance monitoring module to monitor the rendering frame rate in real time. When the frame rate is lower than 30fps, parameter optimization is triggered, and the number of sampling points is automatically reduced from 192 to 128, or half-precision floating-point calculation is enabled to improve the calculation speed by 2.1 times, ensuring optimal performance configuration while ensuring rendering quality.

[0157] Further, the process of obtaining real-time rendering output results includes:

[0158] Obtain a set of performance parameters of the target device, and divide the feature data storage area in the memory pool using a pre-allocation strategy;

[0159] Compare the feedback memory occupancy rate with the preset safety threshold. If the memory occupancy rate exceeds the threshold, the LZ4 algorithm is called to compress the non-active feature data block, release the redundant storage space after compression, update the memory pool allocation table, and input the optimized feature data into the rendering pipeline to generate a frame sequence.

[0160] Specifically, the system first obtains the current GPU utilization rate of 78%, the frame rate of 45fps, the memory bandwidth occupancy rate of 65% and other rendering performance configuration parameters through the performance monitoring module, dynamically adjusts the allocation strategy of the memory pool based on these parameters, and expands the originally fixed 16MB texture cache pool to 24MB to cope with high-load scenarios. The memory pool management mechanism adopts a hierarchical storage architecture, allocates frequently accessed vertex data to the L1 cache area, stores medium-frequency texture data in the L2 area, and places low-frequency static model data in the L3 area. The LRU algorithm is used to realize intelligent scheduling and replacement of data. When the system detects that the current memory occupancy rate reaches 85%, which exceeds the preset 80% safety threshold, the data compression algorithm is triggered immediately. The improved LZ77 compression algorithm is used to compress the texture data in real time, and the high-resolution texture originally occupying 12MB is compressed to 7.2MB, with a compression ratio of 40%. At the same time, the GPU hardware decompression unit is combined to ensure that the decompression delay is controlled within 0.5ms. During the compression process, the system uses differentiated compression strategies for different regions according to visual importance. Lossless compression is used for the central region to maintain image quality, and lossy compression is used for the edge region to save storage space. Finally, the memory occupancy rate is reduced to 72%, and the frame rate is improved to 60fps, realizing efficient real-time rendering output results.

[0161] Further, the process of obtaining the final rendered image comprises:

[0162] Obtaining current frame rendered image data, calculating the definition index, color saturation index and texture detail index of the rendered image, and obtaining a comprehensive quality evaluation score;

[0163] According to the comparison result of the comprehensive quality evaluation score and the preset quality threshold, if the quality evaluation score is lower than the preset quality threshold, the adjustment mechanism automatically improves the rendering precision parameter, and if the quality evaluation score is higher than the preset quality threshold, the adjustment mechanism reduces the rendering precision parameter, and determines the target rendering precision level of the current frame;

[0164] Obtaining the processor utilization rate and the memory occupancy rate of the current system, calculating the optimal thread allocation number through a load balancing algorithm according to the target rendering precision level and the system resource state, and obtaining a multi-thread concurrent processing configuration scheme;

[0165] According to the multi-thread concurrent processing configuration scheme, starting parallel processing threads, each thread being responsible for processing image feature data of different regions, and obtaining texture feature vectors, color feature vectors and geometric feature vectors of each region through feature extraction operations;

[0166] Performing weighted fusion calculation on the feature vectors of each region, and if the variance value of a certain region feature vector exceeds a preset change threshold, increasing the fusion weight coefficient of the corresponding region, and obtaining a fused global feature representation;

[0167] Through the rendering engine, a target rendering image is generated according to the global feature representation and the current rendering precision level, the image generation time is monitored in real time, and if the generation time exceeds the preset time limit, a precision adjustment mechanism is triggered to reduce the rendering precision parameter of the next frame;

[0168] According to the pixel data of the target rendering image and the color space conversion, brightness adjustment and contrast optimization are performed to obtain the final rendering image output that meets the real-time display requirements.

[0169] Specifically, in real-time rendering quality evaluation, peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) are used as core indicators, and when PSNR is lower than 30 dB or SSIM is less than 0.92, an adaptive adjustment mechanism is triggered. By constructing a dynamic LOD (level of detail) adjustment algorithm based on a PID controller, the error threshold is set to ±2 dB, and the rendering quality data is sampled at a period of 5 ms per frame. When it is detected that the PSNR fluctuation exceeds the threshold, the geometric subdivision level is automatically adjusted from LOD3 to LOD2, and the number of ray tracing samples is reduced from 64 times per pixel to 32 times per pixel, reducing the calculation time from 16.7 ms to 11.2 ms. To accelerate feature fusion, a multi-thread parallel computing architecture based on CUDA is designed, and a 256-dimensional feature vector is decomposed into 8 32-dimensional sub-tasks, achieving a 4.8 times speed improvement through shared memory optimization. A time consistency verification algorithm is used, with the difference between consecutive 3 frames of motion vectors being less than 0.1 pixels as the judgment condition. For static areas, a cache reuse mechanism is enabled to reduce 35% of repeated calculations. Finally, a bicubic interpolation algorithm is used to perform 2 times super-resolution reconstruction on the down-sampled area, and under the premise of maintaining SSIM≥0.89, the overall rendering delay is controlled within 33.3 ms (30 FPS).

[0170] The above is only a preferred specific embodiment of the present application, but the protection scope of the present application is not limited thereto. Any changes or replacements within the technical scope disclosed in the present application can be easily thought of by those skilled in the art, and should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A neural radiation field rendering method based on dynamic hash coding, characterized in that: include: Obtain spatial position coordinate data in dynamic scenes and construct a multi-level hash table structure through an adaptive hash mapping algorithm. If a hash collision is detected, the conflict detection mechanism is activated to analyze the spatial distribution characteristics of the collision position and obtain an optimized hash index mapping relationship table; According to the position information in the hash index mapping relationship table, a hierarchical feature fusion algorithm is used to encode the feature vectors of different resolution levels, and the contribution value of the features of each level is calculated through a feature weight distribution mechanism to determine the fused comprehensive feature representation vector; For the comprehensive feature representation vector, the motion trajectory change pattern of the object in the dynamic scene is analyzed by the spatiotemporal consistency constraint condition. If the motion amplitude exceeds the preset threshold, the feature update mechanism is triggered to recalculate the feature code of the region to obtain spatiotemporally continuous feature sequence data; Taking the feature sequence data as input, the density value and color value of each sampling point are calculated through the forward propagation of the neural network, and the volume rendering integral operation is performed along the ray direction according to the ray tracing principle to determine the color output result of the current pixel point; Based on the error between the color output result and the real image, the network parameter weights are updated through the back propagation algorithm. If the error value is greater than the convergence threshold, the iterative training process is continued to adjust the feature encoding strategy to obtain the optimized neural radiation field model parameters; Predictively render new perspective images using the neural radiance field model parameters, use a parallel computing architecture to simultaneously process color calculation tasks for multiple pixels, dynamically adjust the batch size based on GPU memory usage, and determine the optimal rendering performance configuration parameters; Based on the rendering performance configuration parameters, the storage allocation strategy of the feature data is optimized through the memory pool management mechanism. If the current memory usage exceeds the safety threshold, the data compression algorithm is activated to reduce the storage space requirement and obtain the real-time rendering output result; According to the quality evaluation index of the real-time rendering output result, the relationship between rendering accuracy and computing efficiency is dynamically balanced through an adaptive adjustment mechanism, and multi-threaded concurrent processing technology is used to accelerate the feature fusion calculation process to obtain the final rendered image that meets the real-time requirements.

2. The method according to claim 1, characterized in that The process of obtaining the optimized hash index mapping relationship table includes: Obtain spatial coordinate data of target objects in dynamic scenes, collect three-dimensional position information in real time through the sensor array and convert it into a standardized coordinate format to obtain a spatial coordinate sequence containing X-axis, Y-axis, and Z-axis values; Constructing an adaptive hash mapping function based on the spatial coordinate sequence, using a linear combination of coordinate values ​​as a basis for hash key value calculation, determining an initial hash address through a modulo operation, and obtaining an index position allocation result of a first-level hash table; If a key value conflict is detected during the hash address calculation process, the conflict detection mechanism is activated to analyze the spatial distribution characteristics of the collision location. The degree of spatial aggregation is determined by calculating the Euclidean distance between the conflicting coordinate points, and the distribution density parameter of the conflicting area is obtained. Redesigning the hash strategy of the hash function according to the distribution density parameter, increasing the hash table capacity or adjusting the hash factor to reduce the probability of collision, and creating a second-level hash table if the collision density exceeds a preset threshold, thereby obtaining a hierarchical partitioning scheme for a multi-level index structure; Redistribute conflicting spatial coordinate data to the hash table of the corresponding level according to the hierarchical division scheme, use the chain address method to process multiple coordinate records under the same hash address, and store spatial location information with the same hash value through a linked list structure to obtain an optimized data storage layout; The K-means clustering algorithm is used to analyze the clustering pattern of spatial coordinates in a multi-level hash table. The coordinate positions of the centers of each cluster are calculated and the weight parameters of the hash function are updated. If the clustering results show that the coordinate distribution is uneven, the hashing strategy is adjusted to obtain the hash mapping parameter configuration based on clustering optimization. The hash mapping parameter configuration is used to reconstruct the index mapping relationship table, and a one-to-one correspondence is established between the original space coordinates and the optimized hash addresses to obtain the optimized hash index mapping relationship table.

3. The method according to claim 1, characterized in that The process of obtaining the fused comprehensive feature representation vector includes: Obtain original multi-resolution image data, use a hash function to perform hash calculation on the image pixel coordinates, and obtain a hash index value sequence; Constructing a hash index mapping table based on the hash index value sequence, recording pixel position information corresponding to each hash value, and determining a spatial position mapping relationship; Through the spatial position mapping relationship, feature extraction is performed on image regions at different resolution levels, and a convolutional neural network is used to extract feature vectors at each level to obtain a multi-level feature vector set; Based on the multi-level feature vector set, a similarity matrix between feature vectors is calculated by a hierarchical feature fusion algorithm to determine the correlation strength value between the feature vectors; If the correlation strength value exceeds the preset threshold, the attention mechanism is used to calculate the weight coefficient of the feature vector, and the contribution value of each level feature is quantified through the weight distribution mechanism; According to the contribution values, weighted fusion processing is performed on the feature vectors of each level, and a vector splicing operation is used to generate a fused comprehensive feature representation vector.

4. The method according to claim 1, wherein The process of obtaining spatiotemporal continuous feature sequence data includes: Obtain object detection results in dynamic scenes, divide the spatial regions occupied by different objects in the scene through region segmentation algorithms, and obtain the boundary coordinate information of the objects; Extracting a comprehensive feature vector of the object region based on the boundary coordinate information, performing feature encoding processing on the object region using a convolutional neural network to obtain initial feature representation data; The motion trajectory parameters are calculated by the position change of the object between consecutive frames, and the motion amplitude value of the object in the time series is determined by the position coordinate difference analysis method; If the motion amplitude value exceeds a preset threshold, a feature update mechanism is triggered to recalculate the feature encoding of the object area to obtain an updated feature representation vector; The consistency of the feature vectors of the same object in consecutive frames is verified according to the spatiotemporal constraints, and the continuity of the object identity is determined by feature similarity calculation; The sliding window method is used to organize the feature vector data in the time series to obtain the feature sequence data containing the laws of object movement changes.

5. The method according to claim 1, wherein Based on the principle of ray tracing, volume rendering integral operation is performed along the ray direction. The process of determining the color output result of the current pixel point includes: Using the feature sequence data as input data of a convolutional neural network, the density value and color value of the sampling point are calculated through forward propagation; The integral path of volume rendering is determined according to the direction of the ray. The density and color values ​​are accumulated along the path using the volume rendering integral operation. If the accumulated transparency exceeds the preset threshold, the integral operation is terminated, and the final color output of the pixel is determined by the rendering result.

6. The method according to claim 1, characterized in that The process of obtaining the optimized neural radiation field model parameters includes: Obtain the predicted color data output by the neural radiation field model and the corresponding real image color data, calculate the mean square error value, and obtain the loss function value of the current training batch; Based on the loss function value, a back propagation algorithm is used to calculate the gradient information of the weight parameters of each layer of the neural network to obtain the direction and amplitude data of the weight update; The weight parameters of the neural network are updated using the gradient descent algorithm. If the calculated loss function value is greater than the preset convergence threshold, the model training is judged to have not reached the convergence state; Adopt the feature coding strategy adjustment method to determine the coding weight distribution ratio of the feature extraction layer according to the changing trend of the current loss function value; Obtain the adjusted feature encoding weight distribution ratio, recalculate the forward propagation output result of the neural radiation field model, and obtain new predicted color data; Based on the error comparison between the new predicted color data and the real image data, the optimization effect of the model parameters is judged. If the error value continues to decrease, it is determined that the optimization direction is correct. The parameter update process is repeated through the iterative training process until the loss function value converges to the preset threshold range, and the final optimized neural radiation field model parameters are obtained.

7. The method according to claim 1, characterized in that The process of determining the optimal rendering performance configuration parameters includes: Extract the spatial position encoding weights and direction encoding weights of the optimized neural radiation field model parameters to obtain a set of scene representation parameters; Constructing a perspective prediction matrix based on the scene representation parameter set, calculating the light direction vector and the starting position coordinates under the target perspective using a three-dimensional coordinate transformation algorithm, and determining the sampling point distribution under the new perspective; Pixel processing tasks are allocated through a parallel computing architecture. If the GPU available memory is greater than a preset threshold, pixels are processed in a large-batch parallel mode. If the memory is insufficient, the system switches to a small-batch processing mode to obtain a pixel-level computing task allocation solution. Execute the color calculation process according to the pixel-level computing task allocation scheme, and use the volume rendering integration method to accumulate color values ​​and transparency values ​​along the direction of light to obtain the final color output of each pixel; Real-time detection of GPU memory usage and computing load. If the memory usage exceeds the safety threshold, the batch size is reduced; if it is below the minimum utilization, the batch size is increased to determine the dynamic batch size adjustment strategy. Update performance configuration parameters according to the dynamic batch processing scale adjustment strategy, use the load balancing algorithm to reallocate computing resources, and obtain the optimal rendering performance configuration parameter combination through comprehensive evaluation of rendering time and memory usage; The final image rendering is performed based on the optimal rendering performance configuration parameter combination, and the final calculation and output of the color value are completed through the pixel shader to obtain a new perspective predicted image.

8. The method according to claim 1, characterized in that The process of obtaining real-time rendering output results includes: Obtain the performance parameter set of the target device and use a pre-allocation strategy to divide the feature data storage area in the memory pool; The feedback memory occupancy rate is compared with the preset safety threshold. If the memory occupancy rate exceeds the threshold, the LZ4 algorithm is called to compress the inactive feature data blocks, release the compressed redundant storage space and update the memory pool allocation table. The optimized feature data is input into the rendering pipeline to generate a frame sequence.

9. The method according to claim 1, characterized in that The process of obtaining the final rendered image includes: Obtain the current frame rendered image data, calculate the clarity index, color saturation index and texture detail index of the rendered image, and obtain a comprehensive quality assessment score; Based on a comparison result of the comprehensive quality assessment score and a preset quality threshold, if the quality assessment score is lower than the preset quality threshold, the adjustment mechanism automatically increases the rendering precision parameters; if the quality assessment score is higher than the preset quality threshold, the adjustment mechanism decreases the rendering precision parameters and determines a target rendering precision level for the current frame; Obtain the processor utilization and memory occupancy of the current system, calculate the optimal number of threads to be allocated based on the target rendering accuracy level and system resource status through a load balancing algorithm, and obtain a multi-threaded concurrent processing configuration solution; According to the multi-thread concurrent processing configuration scheme, parallel processing threads are started, each thread is responsible for processing image feature data of different regions, and texture feature vectors, color feature vectors and geometric feature vectors of each region are obtained through feature extraction operations; Perform weighted fusion calculation on the feature vectors of each region. If the variance value of the feature vector of a certain region exceeds the preset change threshold, the fusion weight coefficient of the corresponding region is increased to obtain the fused global feature representation; Generate a target rendered image through a rendering engine based on the global feature representation and the current rendering precision level, monitor the image generation time in real time, and trigger a precision adjustment mechanism to reduce the rendering precision parameters of the next frame if the generation time exceeds a preset time limit; Brightness adjustment and contrast optimization are performed based on the pixel data and color space conversion of the target rendered image to obtain a final rendered image output that meets real-time display requirements.

Images