Urban-level digital twin lightweight rendering method

Abstract

The embodiment of the invention provides a city-level digital twin lightweight rendering method. The method comprises the following steps: acquiring multi-source heterogeneous data; calculating rendering data corresponding to a first rendering channel and a second rendering channel of the multi-source heterogeneous data; rendering multi-level parameters of the rendering data are extracted; establishing a visual rendering response surface model for the rendering multi-level parameters to obtain a rendering response value; performing multi-objective optimization on the rendering response value to obtain a rendering relationship parameter; reading a list corresponding to the rendering relation parameters when the model is loaded according to the rendering relation parameters, performing parameterization configuration on geometry, texture and a shader to obtain a digital twin city model, and converting an original time-consuming real rendering optimization problem into a Kriging model-based rapid prediction and multi-target optimization problem to obtain a digital twin city model. The scientific design of urban-level digital twinning lightweight rendering is realized, the optimization efficiency is remarkably improved, and the robustness of the final scheme is ensured.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a city-level digital twin lightweight rendering method and a city-level digital twin lightweight rendering system, a computer device, and a storage medium. Background Technology

[0002] The core of lightweight rendering for city-level digital twins lies in reducing the burden on massive 3D data through various technical means without affecting the core visual experience, enabling complex city models to run smoothly on various devices such as web pages, mobile phones, and even drones. The realization of this technology relies on a variety of background technologies but also faces some inherent challenges. Lightweight technologies for digital twins can include Level of Detail (LOD) techniques, hybrid geometry and Gaussian splashing representations, data compression and automated polygon reduction, and cloud rendering cluster services. While these lightweight technologies aim to make models run on mobile devices, the scale of a city is virtually limitless. Rendering a city using pure 3D Gaussian splashing (3DGS) technology might require hundreds of millions of Gaussian primitives and tens of gigabytes of video memory. This remains an unbearable burden for the computing power, memory, and bandwidth of mobile GPUs. Summary of the Invention

[0003] In view of the above problems, embodiments of the present invention are proposed to provide a city-level digital twin lightweight rendering method, a city-level digital twin lightweight rendering system, a computer device, and a storage medium to overcome or at least partially solve the above problems.

[0004] To address the aforementioned problems, this invention discloses a lightweight rendering method for city-level digital twins, comprising: Acquire multi-source heterogeneous data; Calculate the rendering data corresponding to the first and second rendering channels of the multi-source heterogeneous data; Extract the rendering multi-level parameters from the rendering data; A visual rendering response surface model is established for the aforementioned multi-level rendering parameters to obtain rendering response values; Multi-objective optimization is performed on the rendered response value to obtain the rendering relationship parameters; When loading the model, the system automatically reads the list corresponding to the rendering relationship parameters and performs parameterized configuration of geometry, texture, and shaders to obtain a digital twin city model.

[0005] Preferably, the multi-source heterogeneous data includes oblique photogrammetry model data, building information model data, geographic information system data, and laser point cloud data. The step of calculating the rendering data corresponding to the first and second rendering channels of the multi-source heterogeneous data includes: The oblique photogrammetry model data, building information model data, geographic information system data, and laser point cloud data are converted into reference signals; The reference signal is input to the first adaptive filter and the second adaptive filter to obtain the main noise source signal and the secondary noise source signal; The secondary noise source signal is processed through error processing to obtain the processed secondary noise source signal; The primary noise source signal and the processed secondary noise source signal are subjected to adaptive weight filtering to obtain the rendering data corresponding to the first rendering channel and the second rendering channel.

[0006] Preferably, the step of extracting the rendering multi-level parameters of the rendering data includes: Extract the geometry layer parameters, texture layer parameters, and shader layer parameters from the rendering data; The geometry layer parameters, texture layer parameters, and shader layer parameters are combined to form a multi-level rendering parameter set.

[0007] Preferably, the step of establishing a visual rendering response surface model for the multi-level rendering parameters to obtain rendering response values ​​includes: A visual rendering response surface model is established based on the Kriging approximation model; Combined sampling is performed on the rendering multi-level parameters, and the initial response value under each set of parameters is obtained in the visual rendering response surface model; Sensitivity analysis is performed using the initial response value to obtain the rendering response value.

[0008] Preferably, the step of performing multi-objective optimization on the rendering response value to obtain rendering relationship parameters includes: Establish a multi-objective optimization function based on the rendering multi-level parameters and rendering response values; The NSGA-II genetic algorithm is used to solve the Pareto optimal solution set on the multi-objective optimization function. The corresponding optimal parameter combination is selected from the Pareto front and determined as the rendering relation parameters.

[0009] Preferably, the step of establishing a multi-objective optimization function based on multi-level rendering parameters and rendering response values ​​includes: A first sub-objective optimization function for maximizing visual quality is established based on multi-level rendering parameters; A second sub-objective optimization function that minimizes rendering overhead is established based on the parameters of multiple rendering levels; A third sub-objective optimization function is established based on the rendering response value to minimize the rendering time; A multi-objective optimization function is established based on the first sub-objective optimization function, the second sub-objective optimization function, and the third sub-objective optimization function.

[0010] This invention discloses a city-level digital twin lightweight rendering system, comprising: The first acquisition module is used to acquire multi-source heterogeneous data; The calculation module is used to calculate the rendering data corresponding to the first rendering channel and the second rendering channel of the multi-source heterogeneous data; The extraction module is used to extract the rendering multi-level parameters of the rendering data; The rendering response value module is used to establish a visual rendering response surface model for the rendering multi-level parameters and obtain the rendering response value. The optimization module is used to perform multi-objective optimization on the rendering response value to obtain rendering relationship parameters. The digital twin city model module is used to automatically read the list corresponding to the rendering relationship parameters when loading the model, and to perform parameterized configuration of geometry, texture, and shaders to obtain the digital twin city model.

[0011] Preferably, the multi-source heterogeneous data includes oblique photogrammetry model data, building information model data, geographic information system data, and laser point cloud data, and the computing module includes: The conversion submodule is used to convert the oblique photogrammetry model data, building information model data, geographic information system data, and laser point cloud data into reference signals; The input submodule is used to input the reference signal to the first adaptive filter and the second adaptive filter to obtain the main noise source signal and the secondary noise source signal; The error processing submodule is used to process the secondary noise source signal to obtain the processed secondary noise source signal. The weighted filtering submodule is used to perform adaptive weighted filtering on the main noise source signal and the processed secondary noise source signal to obtain the rendering data corresponding to the first rendering channel and the second rendering channel.

[0012] This invention also discloses a computer device, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described steps of lightweight rendering of city-level digital twins.

[0013] This invention also discloses a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the above-described steps of lightweight rendering of city-level digital twins.

[0014] The embodiments of the present invention have the following advantages: In this embodiment of the invention, the lightweight rendering method for city-level digital twins includes: acquiring multi-source heterogeneous data; calculating the rendering data corresponding to the first and second rendering channels of the multi-source heterogeneous data; extracting multi-level rendering parameters of the rendering data; establishing a visual rendering response surface model for the multi-level rendering parameters to obtain rendering response values; performing multi-objective optimization on the rendering response values ​​to obtain rendering relationship parameters; and, based on the rendering relationship parameters, reading the list corresponding to the rendering relationship parameters when loading the model, parameterizing the geometry, texture, and shaders to obtain a digital twin city model. This model, trained with a small amount of real sample data, can quickly predict rendering performance (such as frame time) and visual quality (such as structural similarity index) under any combination of parameters, thereby replacing expensive real computations and significantly improving optimization efficiency. This transforms the originally time-consuming real rendering optimization problem into a fast prediction and multi-objective optimization problem based on the Kriging model, realizing a scientific design for lightweight rendering of city-level digital twins, significantly improving optimization efficiency, and ensuring the robustness of the final solution. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a flowchart illustrating the steps of an embodiment of a city-level digital twin lightweight rendering method according to an embodiment of the present invention; Figure 2 This is a structural block diagram of an embodiment of a city-level digital twin lightweight rendering system according to an embodiment of the present invention; Figure 3 This is an internal structural diagram of a computer device according to one embodiment. Detailed Implementation

[0017] To make the technical problems, technical solutions, and beneficial effects solved by the embodiments of the present invention clearer, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are merely illustrative of the present invention and are not intended to limit the present invention.

[0018] In this embodiment of the invention, the preprocessing model corresponding to multi-source heterogeneous data is mapped to a denoising system model. Specifically, firstly, the multi-source heterogeneous data is processed by dual-channel FxLMS rendering data to obtain denoised rendering data; then, the rendering multi-level parameters of the multi-level rendering pipeline composed of geometric meshes, texture maps, and shaders in the denoised rendering data are extracted; a relevant visual rendering response surface model is established to obtain the rendering response value; based on the rendering multi-level parameters and rendering response value, multiple objective optimization functions for visual quality, rendering overhead, and rendering time are set, and then the Pareto optimal solution set is solved to achieve the best combination of visual quality. The optimal combination of rendering Biot parameters is systematically sought to achieve the best balance between lightweight and visual quality. Under the FxLMS framework that combines image noise control and computer graphics rendering pipeline, a dynamic rendering strategy is used to offset the high load, dynamically generate low-precision proxy meshes or reduce the shading rate, and achieve the effect of lightweighting the digital twin city model.

[0019] Reference Figure 1 The diagram illustrates a step flowchart of an embodiment of a lightweight rendering method for city-level digital twins according to an embodiment of the present invention, which may specifically include the following steps: Step 101: Obtain multi-source heterogeneous data; In this embodiment of the invention, the multi-source heterogeneous data may include various types of data such as oblique photogrammetry model data, building information model data, geographic information system data, and laser point cloud data. This embodiment of the invention does not impose too many restrictions on this. This city-level digital twin lightweight rendering method can be applied to terminals. The embodiments of this invention do not limit the specific type of terminal. The operating system of the terminal may include Android, Harmony OS, IOS, Windows Phone, Windows, etc. This invention does not impose too many restrictions on this. Specifically, oblique photogrammetry model data is generated by using multiple sensors (usually a five-lens camera) mounted on the same flight platform to collect images from five different angles, including a vertical one and four oblique ones, and then generating a three-dimensional mesh model through real-scene three-dimensional modeling.

[0020] Building Information Modeling (BIM) data is a parametric model data based on three-dimensional digital technology that integrates geometric, physical, functional, and management information throughout the entire life cycle of a building.

[0021] Geographic Information System (GIS) data is digital information that describes the location, shape, attributes, and interrelationships of geographic spatial entities.

[0022] Laser point cloud data is a dataset that represents the spatial location and attributes of an object's surface in the form of discrete three-dimensional points, obtained through lidar scanning technology.

[0023] Step 102: Calculate the rendering data corresponding to the first rendering channel and the second rendering channel of the multi-source heterogeneous data; In this embodiment of the invention, the multi-source heterogeneous data includes oblique photogrammetry model data, building information model data, geographic information system data, and laser point cloud data. The calculation of the rendering data corresponding to the first and second rendering channels of the multi-source heterogeneous data includes: The oblique photogrammetry model data, building information model data, geographic information system data, and laser point cloud data are converted into reference signals; The reference signal is input to the first adaptive filter and the second adaptive filter to obtain the main noise source signal and the secondary noise source signal; The secondary noise source signal is processed through error processing to obtain the processed secondary noise source signal; The primary noise source signal and the processed secondary noise source signal are subjected to adaptive weight filtering to obtain the rendering data corresponding to the first rendering channel and the second rendering channel.

[0024] The reference signal refers to the raw noise signal extracted from multi-source heterogeneous data that can characterize the current rendering workload, reflecting the complexity of the scene to be rendered at this moment. Specifically, it involves unifying the detailed model of the oblique photogrammetry, the component information of BIM, the geographical range of GIS, and the density of laser point clouds into a numerical sequence. This sequence is dynamically changing in time (with viewpoint movement) and in space (with scene switching). The first adaptive filter refers to the rendering strategy generator for the primary visual region, and the second adaptive filter refers to the rendering strategy generator for the secondary visual region. The primary visual region may include the user's gaze point and the center of the screen, while the secondary visual region may include the edge view and the distant background. This embodiment of the invention does not impose excessive limitations on these aspects. The main noise source signal refers to the computational amount required to directly render all the data after the reference signal is processed by the first adaptive filter. Secondary noise source signals refer to the reference signal after being processed by the second adaptive filter. They represent the active noise source signals generated by the system to reduce the burden, such as "adjusting the texture of distant buildings to half of the original accuracy" or "simplifying the model of background buildings to the outline". The embodiments of the present invention do not impose too many restrictions on this. The processed secondary noise source signal is the original secondary noise source signal after error correction. The error processor (i.e., the performance monitoring module) senses the correction effect of the secondary noise source signal. The processed secondary signal is the result of fine-tuning the secondary noise source signal based on the "residual stuttering effect" or "image quality loss effect" information fed back by the error processor.

[0025] The first rendering channel refers to the rendering channel that renders the main visual area data. It may include a first adaptive filter and a rendering module. The instruction set output by this channel retains a high degree of visual fidelity and is used to render the key areas that the user is focusing on. The second rendering channel refers to the rendering channel that renders the secondary visual area data. It may include a second adaptive filter and a rendering module. The output instruction set performs lightweight operations, such as greatly simplifying geometry, reducing texture, or replacing complex models with simple color blocks. The embodiments of the present invention do not impose too many restrictions on lightweight operations.

[0026] Specifically, in this embodiment of the invention, the oblique photography model, BIM model, GIS data and laser point cloud data are first fused and aligned. Then, based on the current viewpoint (camera position and direction), the feature weights of all data within the field of view are calculated in real time. The feature weights are combined into a continuous numerical sequence according to the time sequence of viewpoint movement, which is the reference signal.

[0027] Furthermore, the reference signal is processed separately by two rendering strategy generators, producing two instruction streams with different tendencies. The first adaptive filter (main channel) generates the main noise source signal, which basically preserves the outline of the original image; the second adaptive filter (secondary channel) analyzes the reference signal and generates instructions to simplify rendering of region data.

[0028] After the secondary noise source signal is executed as an optimization instruction, the execution effect is immediately observed by the performance monitoring module corresponding to the error processor. The two main indicators observed are: whether the frame rate meets the preset standard and whether the image quality degrades beyond the limit. If the image quality degrades significantly and it is detected that the area is not being watched, the error processor will maintain the original instruction. If the image quality degrade happens to occur near the user's gaze point, the error processor will immediately weaken the secondary signal in that area and restore the image quality. Through this repeated execution-observation-correction cycle, the processed secondary noise source signal is obtained.

[0029] Based on the gaze point information of the current frame (which can be simulated using the default screen center), corresponding weight coefficients are assigned to each rendering region on the screen. At the screen center or near the user's gaze point, the primary noise source signal is assigned a high weight (e.g., 0.9), while the processed secondary noise source signal is assigned a low weight (e.g., 0.1). The resulting fused signal is the first rendering channel data, requiring high-precision rendering. At the screen edges or in the distant background, the process is reversed: the processed secondary noise source signal is assigned a high weight (e.g., 0.9), while the primary noise source signal is assigned a low weight (e.g., 0.1). The resulting fused signal is the second rendering channel data. Ultimately, these two data streams drive different parts of the GPU, jointly rendering a frame that ensures visual quality in the core areas while controlling overall rendering overhead, achieving dynamic and intelligent load balancing.

[0030] Step 103: Extract the rendering multi-level parameters of the rendering data; In this embodiment of the invention, extracting the rendering multi-level parameters of the rendering data includes: Extract the geometric layer parameters, texture layer parameters, and shader layer parameters from the rendering data; combine the geometric layer parameters, texture layer parameters, and shader layer parameters into a multi-level rendering parameter set.

[0031] In practical applications, rendering multi-level parameters can be further extracted from the rendering data. Specifically, these rendering multi-level parameters can include geometric layer parameters, texture layer parameters, and shader layer parameters. Specifically, geometric layer parameters can include geometric error thresholds and edge crease angles; texture layer parameters can include texture spectral energy and anisotropic filtering levels; and shader layer parameters can include shader instruction complexity and material roughness. Geometric error threshold refers to the error tolerance of mesh simplification algorithms in digital twin scenarios; edge crease angle refers to the smoothness of changes in the surface normal of the model; texture spectral energy refers to the energy proportion of high-frequency details (such as edges and textures) analyzed by performing Fourier transform on the texture; anisotropic filtering level refers to the level parameter of the sharpness of textures from distant oblique views; shader instruction complexity refers to the amount of computation that affects the GPU overhead shader program; material roughness refers to the number of samples and the amount of computation that affect ray tracing or screen space reflections; Specifically, the first step is to acquire the geometric mesh information contained in the rendering data. For each model to be rendered, whether it's a continuous triangular mesh generated by oblique photogrammetry or a component in the BIM model, its mesh structure is analyzed. The geometric error threshold is extracted by evaluating the deviation between the current mesh and the original high-precision mesh. During the pre-calculation stage, different degrees of simplification are simulated using various preset mesh simplification algorithms, such as edge collapse algorithms, and the geometric error (usually measured by Hausdorff distance) between the simplified mesh and the original mesh is recorded. This geometric error serves as a candidate value for the geometric error threshold. Based on the accuracy requirements of the current channel, the corresponding geometric error threshold is selected from the preset candidate values.

[0032] Furthermore, crease information is obtained by calculating the angle between the normals of adjacent faces in the mesh. For each vertex or each edge, if the angle between the normals of adjacent faces exceeds a certain angle (e.g., 60 degrees), it is considered that there is an edge crease angle. The distribution of crease angles in the entire scene can be statistically analyzed, and edge crease angles that can characterize the level of detail retention of the current model can be extracted.

[0033] Furthermore, extracting texture spectral energy requires frequency domain analysis. The system performs a Fast Fourier Transform on the texture image, converting it from the spatial domain to the frequency domain. In the frequency domain, high-frequency components correspond to texture edges, noise, and fine patterns, while low-frequency components correspond to smooth color transitions. The system calculates the energy proportion of the high-frequency components, which is the texture spectral energy. The anisotropic filtering level can be directly read from the texture sampling configuration of the rendering data.

[0034] Extracting shader instruction complexity requires static analysis of the shader code associated with the rendering data. First, the intermediate representations or source code of rendering programs such as vertex shaders and fragment shaders are parsed, and the number of arithmetic operation instructions (such as addition, multiplication, and trigonometric functions), texture sampling instructions, branch judgment instructions, and loop instructions are counted. Then, the weighted sum is calculated according to the characteristics of the GPU architecture to obtain the shader instruction complexity.

[0035] Material roughness can be directly extracted from the material property data. Each material in the rendering pipeline contains a roughness parameter, with a value typically ranging from 0 to 1.

[0036] Step 104: Establish a visual rendering response surface model for the multi-level rendering parameters to obtain the rendering response value; In this embodiment of the invention, the step of establishing a visual rendering response surface model for the multi-level rendering parameters to obtain rendering response values ​​includes: A visual rendering response surface model is established based on the Kriging approximation model; Combined sampling is performed on the rendering multi-level parameters, and the initial response value under each set of parameters is obtained in the visual rendering response surface model; Sensitivity analysis is performed using the initial response value to obtain the rendering response value.

[0037] In this embodiment of the invention, the visual rendering response surface model uses a small amount of real rendering data as training samples to train a model of the mapping relationship between rendering parameters (such as geometric error threshold, texture spectrum energy, etc.) and output response (such as frame time, visual quality score). The initial response value refers to the basic data actually measured by a real rendering engine before the response surface model is built. Specifically, based on a preset combination of parameters (e.g., 100 sets of parameters selected by optimizing the Latin hypercube design), the scene is rendered in a real rendering environment, and the rendering performance indicators and visual quality indicators corresponding to each set of parameters, as well as the initial response value predicted by the visual rendering response surface model, are recorded.

[0038] The initial response values ​​include rendering overhead metrics and visual quality metrics. Rendering overhead metrics include frame rendering time (milliseconds), GPU utilization (%), and video memory bandwidth (GB / s). Visual quality metrics include peak signal-to-noise ratio (reflecting the degree of image distortion), structural similarity index (reflecting the structural similarity of images), and learned perceptual image patch similarity.

[0039] By optimizing the Latin hypercube design, several combinations of multi-level rendering parameters were generated, and the initial response values ​​corresponding to each set of parameters were obtained through a realistic rendering engine. These data constitute the known sample points.

[0040] Then, the difference in response values ​​between any two sample points is related to their distance in the parameter space; that is, the closer the points are, the more similar their response values ​​should be.

[0041] Next, the model will solve for a set of weight coefficients such that for each known sample point, the model's predicted value is equal to the actual measured value of that point; The range of values ​​for each rendering multi-level parameter (such as geometric error threshold, texture spectrum energy, shader instruction complexity, etc.) is divided into several equal parts, and then these equal parts are randomly combined to ensure that each value range of each parameter is selected throughout the sampling process.

[0042] In this embodiment of the invention, a sensitivity coefficient can be calculated between each rendering multi-level parameter and each initial response value. The sensitivity coefficient reflects how much the response value changes when a certain parameter changes by a certain percentage. For example, if the geometric error threshold increases by 10% and the frame rendering time decreases by 15%, then the sensitivity coefficient is -1.5; if the peak signal-to-noise ratio decreases by only 0.5% at the same time, then the sensitivity coefficient is -0.05. Several indicators with high sensitivity coefficients are selected from the initial response values ​​as the final rendering response values.

[0043] Specifically, the vector representation of the rendering multi-level parameters is as follows: ; in This refers to the number of parameters (e.g., geometric error threshold, texture spectral energy, shader instruction complexity, etc.). The expression for the visual rendering response surface model, i.e., the Kriging approximation model, is as follows: ; The model represents the combination of parameters The predicted response value.

[0044] This represents the regression part, where It is a vector of basis functions (usually taking the constant term 1, or a low-order polynomial). It is the corresponding regression coefficient vector; This represents the random process part, with a mean of 0 and a variance of . And any two points and The covariance between them is determined by the spatial correlation function.

[0045] Where b represents the additive coefficient corresponding to the processed secondary noise source signal, which is used to improve the prediction accuracy of the model.

[0046] In this embodiment of the invention, since directly calling the real rendering engine for parameter optimization requires a significant amount of time, this model, trained with a small amount of real sample data, can quickly predict rendering performance (such as frame time) and visual quality (such as structural similarity index) under any combination of parameters. This replaces expensive real-world computation and significantly improves optimization efficiency. The originally time-consuming real-world rendering optimization problem is transformed into a fast prediction and multi-objective optimization problem based on the Kriging model, achieving a scientific design for lightweight rendering of city-level digital twins, significantly improving optimization efficiency and ensuring the robustness of the final solution.

[0047] Step 105: Perform multi-objective optimization on the rendered response value to obtain the rendered relationship parameters; In this embodiment of the invention, the step of performing multi-objective optimization on the rendering response value to obtain rendering relationship parameters includes: Establish a multi-objective optimization function based on the rendering multi-level parameters and rendering response values; The NSGA-II genetic algorithm is used to solve the Pareto optimal solution set on the multi-objective optimization function. The corresponding optimal parameter combination is selected from the Pareto front and determined as the rendering relation parameters.

[0048] Specifically, in this embodiment of the invention, establishing a multi-objective optimization function based on rendering multi-level parameters and rendering response values ​​includes: establishing a first sub-objective optimization function that maximizes visual quality based on rendering multi-level parameters; establishing a second sub-objective optimization function that minimizes rendering overhead based on rendering multi-level parameters; establishing a third sub-objective optimization function that minimizes rendering time based on rendering response values; and establishing a multi-objective optimization function based on the first sub-objective optimization function, the second sub-objective optimization function, and the third sub-objective optimization function.

[0049] The multi-objective optimization function is expressed as follows: ; in, Let represent the first sub-objective optimization function that maximizes visual quality. This represents the second sub-objective optimization function that minimizes rendering overhead; The third sub-objective optimization function represents the minimized rendering time; X represents the parameters for rendering multiple levels. Rendering relation parameters refer to the optimal combination of multi-level rendering parameters determined after multi-objective optimization. It is the final output of the entire optimization process and includes the parameter configurations of geometry layer, texture layer, and shader layer that achieve the best balance between visual quality and rendering overhead for the current digital twin scene and hardware environment.

[0050] Specifically, the NSGA-II algorithm simulates the natural evolutionary process, iteratively approaching the Pareto front. The specific process is as follows: A large number of candidate parameter combinations are randomly generated, and a bunch of seeds are randomly scattered, with each seed representing a set of possible rendering parameters.

[0051] The Kriging model is used to score each candidate combination, and three metrics are calculated for the candidate combination: visual quality, rendering cost, and rendering time.

[0052] All candidate combinations are divided into different levels, forming the current Pareto front approximation; the second level consists of combinations dominated only by the first level, and so on.

[0053] Within the same level, the algorithm calculates the "crowding" of each combination—that is, how many other combinations it is surrounded by in the target space. Combinations with higher crowding indicate that the region where the combination is located is more sparse, and these combinations will be preferentially retained to maintain population diversity and prevent all candidates from crowding together.

[0054] Based on the non-dominated hierarchy and crowding, select excellent candidate combinations and pair them together to generate new combinations; By merging the parent and offspring generations, performing non-dominated sorting and crowding calculations again, and selecting the best combinations for the next generation, each generation of the population is closer to the true Pareto front than the previous generation.

[0055] Repeat the above steps for several generations (e.g., 200 generations). The individuals in the population gradually approach the true Pareto front and no longer change significantly. At this point, the Pareto optimal solution set is obtained.

[0056] The final rendering relationship parameters for different application scenarios are selected from the Pareto front. For example, if the digital twin system is to be deployed on a high-performance workstation and the user has high requirements for image quality, then the solution with the best image quality can be selected from the front. If it is deployed on a mobile phone or tablet and the smoothness requirement is higher, then the solution with the lowest overhead is selected. If both need to be considered, then the solution with a relatively balanced image quality and overhead is selected.

[0057] In practice, this preference can be quantified by assigning different weights to the three objectives. For example, assign a weight of 0.4 to visual quality, 0.4 to rendering overhead, and 0.2 to rendering time stability. Then, calculate a weighted score for each solution on the frontier, and the highest score is the final selected rendering relation parameter.

[0058] Combined with the Kriging model, NSGA-II only needs to be trained on a small number of real samples, and all subsequent iterations are performed on the fast prediction model, without repeatedly calling the real rendering engine. Compared with directly traversing the parameter space, this improves optimization efficiency; NSGA-II directly outputs the entire Pareto front, preserving all possible trade-offs, and can be flexibly selected according to different scenarios, achieving one-time optimization and multi-platform adaptation.

[0059] Step 106: When loading the model, automatically read the list corresponding to the rendering relationship parameters according to the rendering relationship parameters, and perform parameterized configuration of geometry, texture, and shaders to obtain a digital twin city model.

[0060] After obtaining the rendering relationship parameters, the list corresponding to the rendering relationship parameters is automatically read when loading the model, and the geometry, texture, and shader are parameterized to obtain the digital twin city model.

[0061] In this embodiment of the invention, the lightweight rendering method for city-level digital twins includes: acquiring multi-source heterogeneous data; calculating the rendering data corresponding to the first and second rendering channels of the multi-source heterogeneous data; extracting multi-level rendering parameters of the rendering data; establishing a visual rendering response surface model for the multi-level rendering parameters to obtain rendering response values; performing multi-objective optimization on the rendering response values ​​to obtain rendering relationship parameters; and, based on the rendering relationship parameters, reading the list corresponding to the rendering relationship parameters when loading the model, parameterizing the geometry, texture, and shaders to obtain a digital twin city model. This model, trained with a small amount of real sample data, can quickly predict rendering performance (such as frame time) and visual quality (such as structural similarity index) under any combination of parameters, thereby replacing expensive real computations and significantly improving optimization efficiency. This transforms the originally time-consuming real rendering optimization problem into a fast prediction and multi-objective optimization problem based on the Kriging model, realizing a scientific design for lightweight rendering of city-level digital twins, significantly improving optimization efficiency, and ensuring the robustness of the final solution.

[0062] It should be noted that, for the sake of simplicity, the method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments of the present invention are not limited to the described order of actions, because according to the embodiments of the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions involved are not necessarily essential to the embodiments of the present invention.

[0063] Reference Figure 2 The diagram illustrates a structural block diagram of an embodiment of a city-level digital twin lightweight rendering system according to the present invention, which may specifically include the following modules: The first acquisition module 301 is used to acquire multi-source heterogeneous data; Calculation module 302 is used to calculate the rendering data corresponding to the first rendering channel and the second rendering channel of the multi-source heterogeneous data; Extraction module 303 is used to extract the rendering multi-level parameters of the rendering data; The rendering response value module 304 is used to establish a visual rendering response surface model for the rendering multi-level parameters and obtain the rendering response value. The optimization module 305 is used to perform multi-objective optimization on the rendering response value to obtain rendering relationship parameters. The digital twin city model module 306 is used to automatically read the list corresponding to the rendering relationship parameters when loading the model, and to perform parameterized configuration of geometry, texture, and shaders to obtain the digital twin city model.

[0064] Preferably, the multi-source heterogeneous data includes oblique photogrammetry model data, building information model data, geographic information system data, and laser point cloud data, and the computing module includes: The conversion submodule is used to convert the oblique photogrammetry model data, building information model data, geographic information system data, and laser point cloud data into reference signals; The input submodule is used to input the reference signal to the first adaptive filter and the second adaptive filter to obtain the main noise source signal and the secondary noise source signal; The error processing submodule is used to process the secondary noise source signal to obtain the processed secondary noise source signal. The weighted filtering submodule is used to perform adaptive weighted filtering on the main noise source signal and the processed secondary noise source signal to obtain the rendering data corresponding to the first rendering channel and the second rendering channel.

[0065] Preferably, the extraction module includes: The extraction submodule is used to extract the geometric layer parameters, texture layer parameters, and shader layer parameters of the rendering data. The component submodule is used to combine the geometry layer parameters, texture layer parameters, and shader layer parameters into multi-level rendering parameters.

[0066] Preferably, the rendering response value module includes: The first submodule is used to build a visual rendering response surface model based on the Kriging approximation model. The sampling submodule is used to perform combined sampling of multi-level rendering parameters and obtain the initial response value for each set of parameters in the visual rendering response surface model. The sensitivity analysis submodule is used to perform sensitivity analysis based on the initial response value to obtain the rendered response value.

[0067] Preferably, the optimization module includes: The second submodule is used to establish a multi-objective optimization function based on the rendering multi-level parameters and rendering response values. The optimal parameter combination submodule is used to solve the Pareto optimal solution set on a multi-objective optimization function using the NSGA-II genetic algorithm, select the corresponding optimal parameter combination from the Pareto front, and determine the optimal parameter combination as the rendering relation parameters.

[0068] Preferably, the second establishment submodule includes: The first establishment unit is used to establish a first sub-objective optimization function that maximizes visual quality based on the rendering multi-level parameters; The second establishment unit is used to establish a second sub-objective optimization function that minimizes the rendering overhead based on the parameters of the multi-level rendering layers; The third establishment unit is used to establish a third sub-objective optimization function that minimizes the rendering time based on the rendering response value; The multi-objective optimization function establishment unit is used to establish a multi-objective optimization function based on the first sub-objective optimization function, the second sub-objective optimization function, and the third sub-objective optimization function.

[0069] As the device embodiment is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.

[0070] Specific limitations regarding the city-level digital twin lightweight rendering system can be found in the limitations of the city-level digital twin lightweight rendering method described above, and will not be repeated here. Each module in the aforementioned city-level digital twin lightweight rendering system can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0071] The city-level digital twin lightweight rendering system provided above can be used to execute the city-level digital twin lightweight rendering method provided in any of the above embodiments, and has corresponding functions and beneficial effects.

[0072] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 3 As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements a lightweight rendering method for city-level digital twins. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.

[0073] Those skilled in the art will understand that Figure 3The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0074] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps: Acquire multi-source heterogeneous data; Calculate the rendering data corresponding to the first and second rendering channels of the multi-source heterogeneous data; Extract the rendering multi-level parameters from the rendering data; A visual rendering response surface model is established for the aforementioned multi-level rendering parameters to obtain rendering response values; Multi-objective optimization is performed on the rendered response value to obtain the rendering relationship parameters; When loading the model, the system automatically reads the list corresponding to the rendering relationship parameters and performs parameterized configuration of geometry, texture, and shaders to obtain a digital twin city model.

[0075] Preferably, the multi-source heterogeneous data includes oblique photogrammetry model data, building information model data, geographic information system data, and laser point cloud data. The step of calculating the rendering data corresponding to the first and second rendering channels of the multi-source heterogeneous data includes: The oblique photogrammetry model data, building information model data, geographic information system data, and laser point cloud data are converted into reference signals; The reference signal is input to the first adaptive filter and the second adaptive filter to obtain the main noise source signal and the secondary noise source signal; The secondary noise source signal is processed through error processing to obtain the processed secondary noise source signal; The primary noise source signal and the processed secondary noise source signal are subjected to adaptive weight filtering to obtain the rendering data corresponding to the first rendering channel and the second rendering channel.

[0076] Preferably, the step of extracting the rendering multi-level parameters of the rendering data includes: Extract the geometry layer parameters, texture layer parameters, and shader layer parameters from the rendering data; The geometry layer parameters, texture layer parameters, and shader layer parameters are combined to form a multi-level rendering parameter set.

[0077] Preferably, the step of establishing a visual rendering response surface model for the multi-level rendering parameters to obtain rendering response values ​​includes: A visual rendering response surface model is established based on the Kriging approximation model; Combined sampling is performed on the rendering multi-level parameters, and the initial response value under each set of parameters is obtained in the visual rendering response surface model; Sensitivity analysis is performed using the initial response value to obtain the rendering response value.

[0078] Preferably, the step of performing multi-objective optimization on the rendering response value to obtain rendering relationship parameters includes: Establish a multi-objective optimization function based on the rendering multi-level parameters and rendering response values; The NSGA-II genetic algorithm is used to solve the Pareto optimal solution set on the multi-objective optimization function. The corresponding optimal parameter combination is selected from the Pareto front and determined as the rendering relation parameters.

[0079] Preferably, the step of establishing a multi-objective optimization function based on multi-level rendering parameters and rendering response values ​​includes: A first sub-objective optimization function for maximizing visual quality is established based on multi-level rendering parameters; A second sub-objective optimization function that minimizes rendering overhead is established based on the parameters of multiple rendering levels; A third sub-objective optimization function is established based on the rendering response value to minimize the rendering time; A multi-objective optimization function is established based on the first sub-objective optimization function, the second sub-objective optimization function, and the third sub-objective optimization function.

[0080] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, performs the following steps: Acquire multi-source heterogeneous data; Calculate the rendering data corresponding to the first and second rendering channels of the multi-source heterogeneous data; Extract the rendering multi-level parameters from the rendering data; A visual rendering response surface model is established for the aforementioned multi-level rendering parameters to obtain rendering response values; Multi-objective optimization is performed on the rendered response value to obtain the rendering relationship parameters; When loading the model, the system automatically reads the list corresponding to the rendering relationship parameters and performs parameterized configuration of geometry, texture, and shaders to obtain a digital twin city model.

[0081] Preferably, the multi-source heterogeneous data includes oblique photogrammetry model data, building information model data, geographic information system data, and laser point cloud data. The step of calculating the rendering data corresponding to the first and second rendering channels of the multi-source heterogeneous data includes: The oblique photogrammetry model data, building information model data, geographic information system data, and laser point cloud data are converted into reference signals; The reference signal is input to the first adaptive filter and the second adaptive filter to obtain the main noise source signal and the secondary noise source signal; The secondary noise source signal is processed through error processing to obtain the processed secondary noise source signal; The primary noise source signal and the processed secondary noise source signal are subjected to adaptive weight filtering to obtain the rendering data corresponding to the first rendering channel and the second rendering channel.

[0082] Preferably, the step of extracting the rendering multi-level parameters of the rendering data includes: Extract the geometry layer parameters, texture layer parameters, and shader layer parameters from the rendering data; The geometry layer parameters, texture layer parameters, and shader layer parameters are combined to form a multi-level rendering parameter set.

[0083] Preferably, the step of establishing a visual rendering response surface model for the multi-level rendering parameters to obtain rendering response values ​​includes: A visual rendering response surface model is established based on the Kriging approximation model; Combined sampling is performed on the rendering multi-level parameters, and the initial response value under each set of parameters is obtained in the visual rendering response surface model; Sensitivity analysis is performed using the initial response value to obtain the rendering response value.

[0084] Preferably, the step of performing multi-objective optimization on the rendering response value to obtain rendering relationship parameters includes: Establish a multi-objective optimization function based on the rendering multi-level parameters and rendering response values; The NSGA-II genetic algorithm is used to solve the Pareto optimal solution set on the multi-objective optimization function. The corresponding optimal parameter combination is selected from the Pareto front and determined as the rendering relation parameters.

[0085] Preferably, the step of establishing a multi-objective optimization function based on multi-level rendering parameters and rendering response values ​​includes: A first sub-objective optimization function for maximizing visual quality is established based on multi-level rendering parameters; A second sub-objective optimization function that minimizes rendering overhead is established based on the parameters of multiple rendering levels; A third sub-objective optimization function is established based on the rendering response value to minimize the rendering time; A multi-objective optimization function is established based on the first sub-objective optimization function, the second sub-objective optimization function, and the third sub-objective optimization function.

[0086] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0087] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0088] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, apparatus, or computer program products. Therefore, embodiments of the present invention can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of the present invention can take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0089] Embodiments of the present invention are described with reference to flowchart illustrations and / or block diagrams of apparatus, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0090] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction methods implemented in a process. Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0091] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0092] Although preferred embodiments of the present invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present invention.

[0093] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, apparatus, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, apparatus, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, apparatus, article, or terminal device that includes said element.

[0094] The present invention has provided a detailed description of a city-level digital twin lightweight rendering method, a city-level digital twin lightweight rendering system, a computer device, and a storage medium. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A lightweight rendering method for city-level digital twins, characterized in that, include: Acquire multi-source heterogeneous data; Calculate the rendering data corresponding to the first and second rendering channels of the multi-source heterogeneous data; Extract the rendering multi-level parameters from the rendering data; A visual rendering response surface model is established for the aforementioned multi-level rendering parameters to obtain rendering response values; Multi-objective optimization is performed on the rendered response value to obtain the rendering relationship parameters; When loading the model, the system automatically reads the list corresponding to the rendering relationship parameters and performs parameterized configuration of geometry, texture, and shaders to obtain a digital twin city model.

2. The method according to claim 1, characterized in that, in, The multi-source heterogeneous data includes oblique photogrammetry model data, building information model data, geographic information system data, and laser point cloud data. The calculation of the rendering data corresponding to the first and second rendering channels of the multi-source heterogeneous data includes: The oblique photogrammetry model data, building information model data, geographic information system data, and laser point cloud data are converted into reference signals; The reference signal is input to the first adaptive filter and the second adaptive filter to obtain the main noise source signal and the secondary noise source signal; The secondary noise source signal is processed through error processing to obtain the processed secondary noise source signal; The primary noise source signal and the processed secondary noise source signal are subjected to adaptive weight filtering to obtain the rendering data corresponding to the first rendering channel and the second rendering channel.

3. The method according to claim 1, characterized in that, The extraction of rendering multi-level parameters from the rendering data includes: Extract the geometry layer parameters, texture layer parameters, and shader layer parameters from the rendering data; The geometry layer parameters, texture layer parameters, and shader layer parameters are combined to form a multi-level rendering parameter set.

4. The method according to claim 1, characterized in that, The step of establishing a visual rendering response surface model for the multi-level rendering parameters to obtain rendering response values ​​includes: A visual rendering response surface model is established based on the Kriging approximation model; Combined sampling is performed on the rendering multi-level parameters, and the initial response value under each set of parameters is obtained in the visual rendering response surface model; Sensitivity analysis is performed using the initial response value to obtain the rendering response value.

5. The method according to claim 1, characterized in that, The multi-objective optimization of the rendering response value to obtain rendering relationship parameters includes: Establish a multi-objective optimization function based on the rendering multi-level parameters and rendering response values; The NSGA-II genetic algorithm is used to solve the Pareto optimal solution set on the multi-objective optimization function. The corresponding optimal parameter combination is selected from the Pareto front and determined as the rendering relation parameters.

6. The method according to claim 5, characterized in that, The step of establishing a multi-objective optimization function based on multi-level rendering parameters and rendering response values ​​includes: A first sub-objective optimization function for maximizing visual quality is established based on multi-level rendering parameters; A second sub-objective optimization function that minimizes rendering overhead is established based on the parameters of multiple rendering levels; A third sub-objective optimization function is established based on the rendering response value to minimize the rendering time; A multi-objective optimization function is established based on the first sub-objective optimization function, the second sub-objective optimization function, and the third sub-objective optimization function.

7. A city-level digital twin lightweight rendering system, characterized in that, include: The first acquisition module is used to acquire multi-source heterogeneous data; The calculation module is used to calculate the rendering data corresponding to the first rendering channel and the second rendering channel of the multi-source heterogeneous data; The extraction module is used to extract the rendering multi-level parameters of the rendering data; The rendering response value module is used to establish a visual rendering response surface model for the rendering multi-level parameters and obtain the rendering response value. The optimization module is used to perform multi-objective optimization on the rendering response value to obtain rendering relationship parameters. The digital twin city model module is used to automatically read the list corresponding to the rendering relationship parameters when loading the model, and to perform parameterized configuration of geometry, texture, and shaders to obtain the digital twin city model.

8. The system according to claim 7, characterized in that, in, The multi-source heterogeneous data includes oblique photogrammetry model data, building information model data, geographic information system data, and laser point cloud data. The computing module includes: The conversion submodule is used to convert the oblique photogrammetry model data, building information model data, geographic information system data, and laser point cloud data into reference signals; The input submodule is used to input the reference signal to the first adaptive filter and the second adaptive filter to obtain the main noise source signal and the secondary noise source signal; The error processing submodule is used to process the secondary noise source signal to obtain the processed secondary noise source signal. The weighted filtering submodule is used to perform adaptive weighted filtering on the main noise source signal and the processed secondary noise source signal to obtain the rendering data corresponding to the first rendering channel and the second rendering channel.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the city-level digital twin lightweight rendering method according to any one of claims 1 to 6.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the city-level digital twin lightweight rendering method according to any one of claims 1 to 6.

Images