Efficient real-time rendering method and system for three-dimensional terrain based on deep learning

Abstract

The present invention relates to the field of three-dimensional simulation technology, and discloses a method for efficient real-time rendering of three-dimensional terrain based on deep learning, the method comprising: collecting an image data set of terrain to be rendered, analyzing terrain attributes in the image data set; extracting local image features of the image data set according to the terrain attributes to identify mixed terrain in the image data set, and constructing a three-dimensional terrain model of the mixed terrain; analyzing the illumination direction, illumination intensity, shadow distribution area, and ground roughness of the three-dimensional terrain model; according to the illumination intensity, shadow distribution area, and ground roughness, eliminating irrelevant geometric elements of the three-dimensional terrain model under the simulation effect during the rendering process, and setting the detail rendering level of the three-dimensional terrain model; combining the three-dimensional terrain model and the detail rendering level, rendering the three-dimensional terrain in real time. The present invention can eliminate irrelevant geometric elements of the three-dimensional terrain model during the rendering process, and improve the real-time rendering efficiency of the three-dimensional terrain.

Description

Technical Field

[0001] The present invention relates to the field of three-dimensional simulation technology, and in particular to a method and system for efficient real-time rendering of three-dimensional terrain based on deep learning. Background Art

[0002] Efficient real-time rendering of 3D terrain refers to the process of converting a 3D terrain model into a 2D image with smooth picture update capability that can be seen by users through rapid processing and rendering of 3D terrain data within a limited time. Its core lies in the ability to quickly process and render a large amount of terrain data while maintaining high frame rates and images, providing powerful 3D terrain visualization capabilities for various applications. With the development of satellite remote sensing technology, the accuracy requirements of digital terrain data that can be obtained are getting higher and higher, which has led to an increase in the demand for large-scale terrain scenes. The ultimate pursuit of picture smoothness and detail realism has led to the need for further optimization of terrain processing rendering technology.

[0003] The existing real-time rendering method of 3D terrain mainly relies on the detail level technology, which dynamically adjusts the accuracy of the terrain grid according to the distance between the observation point and the terrain to achieve effective rendering of the terrain at different detail levels. However, this method requires manual design of models with different detail levels, which may generate many irrelevant geometric elements, such as the detailed texture structure of the ground in the telephoto lens, which increases the rendering workload and thus affects the efficiency of real-time rendering of 3D terrain. Summary of the invention

[0004] In order to solve the above problems, the present invention provides a method and system for efficient real-time rendering of three-dimensional terrain based on deep learning, which can eliminate irrelevant geometric elements of the three-dimensional terrain model during the rendering process and improve the real-time rendering efficiency of the three-dimensional terrain.

[0005] In a first aspect, the present invention provides a method for efficient real-time rendering of three-dimensional terrain based on deep learning, comprising:

[0006] Collecting an image data set of a terrain to be rendered, normalizing the image data set to obtain a target image data set, and analyzing terrain attributes in the target image data set;

[0007] Based on the terrain attributes, extracting local features of the image corresponding to the terrain to be rendered, analyzing the degree of deviation of the local features of the image, constructing a feature vector of the local features of the image based on the degree of deviation, identifying the mixed terrain in the image data set according to the feature vector, and performing three-dimensional reconstruction on the mixed terrain to obtain a three-dimensional terrain model;

[0008] Analyzing the illumination direction of the terrain to be rendered in the three-dimensional terrain model, calculating the illumination intensity of the three-dimensional terrain model based on the illumination direction, identifying the shadow distribution area of ​​the three-dimensional terrain model, partitioning the three-dimensional terrain model into blocks to obtain partitioned blocks, extracting dense point cloud data corresponding to the partitioned blocks, and calculating the terrain roughness of the terrain to be rendered based on the dense point cloud data;

[0009] Analyzing the simulation effect of the three-dimensional terrain model according to the illumination intensity, the shadow distribution area and the roughness of the terrain, identifying irrelevant geometric elements of the three-dimensional terrain model in the rendering process according to the simulation effect, performing view culling processing on the irrelevant geometric elements to obtain a target rendering object, and setting a detail rendering level of the target rendering object;

[0010] The three-dimensional terrain model and the detail rendering level are combined to perform real-time rendering on the three-dimensional terrain to obtain a real-time rendering result.

[0011] In a possible implementation manner of the first aspect, analyzing the terrain attributes in the target image dataset includes:

[0012] Performing image segmentation on the target image data set to obtain a segmented image;

[0013] identifying terrain features in the segmented image;

[0014] Analyzing specific objects in the segmented image according to the terrain features;

[0015] Based on the specific object, terrain attributes in the target image dataset are analyzed.

[0016] In a possible implementation manner of the first aspect, extracting local image features corresponding to the terrain to be rendered based on the terrain attributes includes:

[0017] Based on the terrain attributes, classifying the real terrain of the terrain to be rendered to obtain a real classified terrain;

[0018] Analyzing key feature lines corresponding to the real classified terrain;

[0019] Analyzing the image feature representation of the key feature line according to the real classified terrain;

[0020] Based on the image feature representation, local image features corresponding to the terrain to be rendered are extracted.

[0021] In a possible implementation manner of the first aspect, constructing a feature vector of the local feature of the image based on the degree of deviation includes:

[0022] Based on the degree of deviation, identifying structural features of the terrain to be rendered corresponding to the local features of the image;

[0023] Extracting specific objects from local features of the image;

[0024] analyzing the spatial relationship between the specific objects;

[0025] A feature vector of the local feature of the image is constructed according to the structural feature and the spatial relationship.

[0026] In a possible implementation of the first aspect, the three-dimensionally reconstructing the mixed terrain to obtain a three-dimensional terrain model includes:

[0027] Collecting terrain surface data corresponding to the mixed terrain, and performing data optimization processing on the terrain surface data to obtain target terrain data;

[0028] Based on the target terrain data, construct a triangular mesh of the mixed terrain surface;

[0029] Extracting elevation data from the target terrain data, and constructing a terrain surface model of the mixed terrain based on the elevation data;

[0030] Performing texture mapping on the terrain surface model to obtain a model texture surface;

[0031] The mixed terrain is three-dimensionally reconstructed according to the triangular mesh, the terrain surface model and the model texture surface to obtain a three-dimensional terrain model.

[0032] In a possible implementation manner of the first aspect, constructing a triangular mesh of the mixed terrain surface based on the target terrain data includes:

[0033] Matching image feature points in the target terrain data, and querying terrain features corresponding to the image feature points;

[0034] According to the terrain features, setting the position layout of the image feature points;

[0035] Collecting a three-dimensional point set of feature points of the image;

[0036] According to the position layout, a triangular mesh of the hybrid terrain surface is constructed within the three-dimensional point set.

[0037] In a possible implementation manner of the first aspect, analyzing the illumination direction of the terrain to be rendered in the three-dimensional terrain model includes:

[0038] Collecting elevation data corresponding to the terrain to be rendered;

[0039] Analyzing the morphological features of the terrain to be rendered in the elevation data;

[0040] Setting the simulation time type of the terrain to be rendered in the three-dimensional terrain model;

[0041] According to the simulation time type and the morphological characteristics, setting the azimuth and elevation of the three-dimensional terrain model;

[0042] Based on the azimuth angle and the elevation angle, the illumination direction of the terrain to be rendered in the three-dimensional terrain model is analyzed.

[0043] In a possible implementation manner of the first aspect, calculating the illumination intensity of the three-dimensional terrain model based on the illumination direction includes:

[0044] Collecting elevation data corresponding to the three-dimensional terrain model;

[0045] Extracting unit grid data from the elevation data;

[0046] The normal vector of the unit grid data is calculated using the following formula:

[0047]

[0048] in, Represents the normal vector of the unit grid data at the coordinate (x, y), h right (x+d,y) represents the elevation value of the adjacent cell on the right in the current cell grid data, h left (xd,y) represents the elevation value of the left adjacent cell in the current cell grid data, h down (x,y+d) represents the elevation value of the lower adjacent cell in the current cell grid data, h up (x, yd) represents the elevation value of the upper adjacent cell in the current cell grid data, and d represents the height of the cell grid in the cell grid data;

[0049] The illumination intensity of the three-dimensional terrain model is calculated according to the normal vector and the illumination direction.

[0050] In a possible implementation manner of the first aspect, identifying, according to the simulation effect, irrelevant geometric elements of the three-dimensional terrain model during rendering, includes:

[0051] According to the simulation effect, identifying the influence level of the scene elements in the three-dimensional terrain model;

[0052] Analyze the visualization effect corresponding to the impact level;

[0053] The contribution of the visualization effect to the overall effect is calculated using the following formula:

[0054]

[0055] Among them, G represents the contribution of visualization effect to the overall effect, N represents the total number of overall effect values ​​corresponding to the visualization effect value, and A g represents the value of the g-th visualization effect, A f represents the value of the fth overall effect, represents the sum of the overall effect values, Indicates the ratio of the g-th visualization effect value to the total effect value;

[0056] Irrelevant geometric elements of the three-dimensional terrain model during rendering are identified according to the contribution.

[0057] In a second aspect, the present invention provides a high-efficiency real-time rendering system for three-dimensional terrain based on deep learning, characterized in that the system comprises:

[0058] A data acquisition module is used to acquire an image data set of a terrain to be rendered, normalize the image data set to obtain a target image data set, and analyze the terrain attributes in the target image data set;

[0059] A model generation module, for extracting local features of the image corresponding to the terrain to be rendered based on the terrain attributes, analyzing the degree of deviation of the local features of the image, constructing a feature vector of the local features of the image based on the degree of deviation, identifying the mixed terrain in the image data set according to the feature vector, and performing three-dimensional reconstruction on the mixed terrain to obtain a three-dimensional terrain model;

[0060] a detail analysis module, configured to analyze the illumination direction of the terrain to be rendered in the three-dimensional terrain model, calculate the illumination intensity of the three-dimensional terrain model based on the illumination direction, identify the shadow distribution area of ​​the three-dimensional terrain model, partition the three-dimensional terrain model into blocks, obtain partitioned blocks, extract dense point cloud data corresponding to the partitioned blocks, and calculate the terrain roughness of the terrain to be rendered based on the dense point cloud data;

[0061] a rendering detail optimization module, for analyzing the simulation effect of the three-dimensional terrain model according to the illumination intensity, the shadow distribution area and the terrain roughness, identifying irrelevant geometric elements of the three-dimensional terrain model in the rendering process according to the simulation effect, performing view culling processing on the irrelevant geometric elements, obtaining a target rendering object, and setting a detail rendering level of the target rendering object;

[0062] The model rendering module is used to combine the three-dimensional terrain model and the detail rendering level to perform real-time rendering on the three-dimensional terrain to obtain a real-time rendering result.

[0063] Compared with the prior art, the technical principle and beneficial effects of this solution are:

[0064] The embodiment of the present invention can obtain image data of different terrain features by collecting an image data set of the terrain to be rendered, thereby providing diversified data support for training the deep learning model, improving the generalization ability of the model, and realistically simulating the visual effect of the terrain by analyzing different types of terrain features, thereby enhancing the realism of the scene; further, the embodiment of the present invention can identify different terrain features by extracting local image features corresponding to the terrain to be rendered based on the terrain attributes, thereby facilitating subsequent image analysis and three-dimensional terrain modeling, and inputting the extracted key terrain features into the deep model, thereby improving the accuracy and generalization ability of the model; the embodiment of the present invention can obtain more refined classified terrain features by identifying mixed terrain in the image data set according to the feature vector, thereby providing an accurate and vivid three-dimensional terrain model for later rendering work, thereby improving the rendering quality; further, the embodiment of the present invention can analyze the three-dimensional terrain model The lighting direction, lighting intensity, shadow distribution area and ground roughness of the terrain to be rendered can simulate the lighting conditions and texture mapping effects of different time periods, assist in identifying and highlighting the key features of the terrain, and make the terrain model look more realistic; the embodiment of the present invention can reduce the amount of calculation and resource consumption as much as possible by identifying the irrelevant geometric elements of the three-dimensional terrain model in the rendering process according to the simulation effect, thereby increasing the number of rendering frames per second, significantly reducing the number of objects to be rendered, thereby reducing the amount of calculation in the rendering process and accelerating the rendering efficiency; further, the embodiment of the present invention combines the three-dimensional terrain model and the detail rendering level to perform real-time rendering of the three-dimensional terrain to obtain a real-time rendering result, which can significantly reduce the computing load, especially when processing large scenes, it can improve the rendering speed and reduce jamming. At the same time, the detail level of the model can be dynamically adjusted according to the distance between the observer and the model, optimize resource allocation, and improve rendering efficiency. Therefore, the efficient real-time rendering method of three-dimensional terrain based on deep learning proposed in the embodiment of the present invention can eliminate irrelevant geometric elements of the three-dimensional terrain model in the rendering process and improve the real-time rendering efficiency of the three-dimensional terrain. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0066] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0067] Figure 1 A schematic diagram of a process flow of a method for efficient real-time rendering of three-dimensional terrain based on deep learning provided by an embodiment of the present invention;

[0068] Figure 2 A schematic diagram of a module of a deep learning-based efficient real-time rendering system for three-dimensional terrain provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0069] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0070] The embodiment of the present invention provides a method for efficient real-time rendering of three-dimensional terrain based on deep learning, and the execution subject of the method for efficient real-time rendering of three-dimensional terrain based on deep learning includes but is not limited to at least one of the electronic devices such as a server and a terminal that can be configured to execute the method provided by the embodiment of the present invention. In other words, the method for efficient real-time rendering of three-dimensional terrain based on deep learning can be executed by software or hardware installed on a terminal device or a server device, and the software can be a blockchain platform. The server includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, etc. The server can be an independent server, or it can be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), and big data and artificial intelligence platforms.

[0071] See also Figure 1 FIG. 1 is a flow chart of a method for efficient real-time rendering of three-dimensional terrain based on deep learning provided by an embodiment of the present invention. Figure 1 The deep learning-based efficient real-time rendering method for 3D terrain described in the paper includes:

[0072] S1. Collect an image data set of a terrain to be rendered, perform normalization processing on the image data set to obtain a target image data set, and analyze terrain attributes in the target image data set.

[0073] The embodiment of the present invention can obtain image data of different terrain features by collecting an image data set of a terrain to be rendered. The terrain to be rendered refers to a terrain that needs to be three-dimensionally modeled and rendered, and the image data set refers to a collection of images under different viewing angles and different lighting conditions containing the terrain to be rendered.

[0074] Optionally, the image dataset of the terrain to be rendered can be acquired through satellite tools, such as Google Earth acquisition images.

[0075] Furthermore, the embodiment of the present invention can ensure the consistency of image data, improve the rendering accuracy of three-dimensional scenes, and ensure that the terrain model can maintain high resolution and detail expression under different viewpoints by normalizing the image dataset to obtain a target image dataset. The target image dataset refers to a set of image data that is easy to analyze after the terrain image has been pre-processed by normalization, cropping, and other operations.

[0076] Optionally, the normalization processing of the image dataset can be implemented using ArcGIS Pro.

[0077] By analyzing the terrain attributes in the target image data set, the embodiment of the present invention can understand different types of terrain features, such as slope, roughness, etc., so as to truly simulate the visual effect of the terrain and enhance the realism of the scene. The terrain attributes refer to the characteristic attributes in the terrain data used to describe the shape, characteristics and relationship of the surface with other natural or man-made elements.

[0078] As an embodiment of the present invention, the analysis of terrain attributes in the target image data set includes: performing image segmentation on the target image data set to obtain a segmented image; identifying terrain features in the segmented image; analyzing specific objects in the segmented image based on the terrain features; and analyzing terrain attributes in the target image data set based on the specific objects.

[0079] Among them, the segmented image refers to dividing the target image data set into multiple regional images, the terrain features refer to various features that describe the corresponding form of the rendered terrain, such as landform and altitude, etc., and the specific object refers to a unit object with the terrain features, such as mountains and rivers.

[0080] Optionally, the image segmentation of the target image data set can be obtained by an image segmentation technology based on a deep convolutional neural network, such as DeepLab image segmentation technology, the terrain feature recognition in the segmented image can be implemented using U-Net, and the specific object analysis in the segmented image based on the terrain features can be obtained by a target detection model, such as Faster R-CNN.

[0081] S2. Based on the terrain attributes, extract the local features of the image corresponding to the terrain to be rendered, analyze the degree of deviation of the local features of the image, and construct a feature vector of the local features of the image based on the degree of deviation.

[0082] The embodiment of the present invention extracts the local image features corresponding to the terrain to be rendered based on the terrain attributes, so as to facilitate subsequent image analysis and three-dimensional terrain modeling. The local image features refer to the features exhibited by different terrain attributes in the local area of ​​the image, as illustrated by the example.

[0083] As an embodiment of the present invention, the extracting of local image features corresponding to the terrain to be rendered based on the terrain attributes includes: classifying the real terrain of the terrain to be rendered based on the terrain attributes to obtain real classified terrain; analyzing key feature lines corresponding to the real classified terrain; analyzing image feature representations of the key feature lines based on the real classified terrain; and extracting local image features corresponding to the terrain to be rendered based on the image feature representations.

[0084] Among them, the real terrain refers to the real terrain corresponding to the image terrain data, the real classified terrain refers to the real terrain classified according to different categories of terrain, the key feature lines refer to lines with significant features in the terrain, such as ridge lines and valley lines, and the image feature representation refers to converting the line information representing the terrain features into image data form.

[0085] For example, a threshold is used to convert the water flow accumulation map into a binary image, and then the ridge feature lines are extracted according to the double threshold algorithm to obtain a double threshold binary image. Morphological operations are then used to denoise the image, and finally the skeleton algorithm can be used to extract the key feature lines of the ridge.

[0086] Optionally, the image feature representation of the key feature line according to the real classified terrain can be achieved by contour analysis method.

[0087] Furthermore, the embodiments of the present invention can evaluate the error size of the local features of the terrain image and improve the accuracy of the deep learning model by analyzing the degree of deviation of the local features of the image. The degree of deviation refers to the degree of difference between the local features of the terrain image and the standard features.

[0088] Optionally, the deviation degree analysis of the local features of the image may be obtained through an error function, such as a Gaussian error function.

[0089] The embodiment of the present invention can further optimize the local features of the terrain image by constructing the feature vector of the local features of the image based on the degree of deviation, thereby extracting key terrain features and inputting them into the deep model, thereby improving the accuracy and generalization ability of the model. The feature vector refers to a group of numerical vectors extracted from the terrain image that can represent the local features of the terrain.

[0090] As an embodiment of the present invention, constructing the feature vector of the local feature of the image based on the degree of deviation includes: identifying the structural features of the terrain to be rendered corresponding to the local feature of the image based on the degree of deviation; extracting specific objects from the local feature of the image; analyzing the spatial relationship between the specific objects; and constructing the feature vector of the local feature of the image based on the structural features and the spatial relationship.

[0091] The structural features refer to the shape and texture features of the terrain to be rendered in the local features of the image, and the spatial relationship refers to the relationship between specific objects distributed at different positions in terms of distance and direction, such as the directional relationship between rivers and valleys.

[0092] Optionally, based on the degree of deviation, the structural feature identification of the terrain to be rendered corresponding to the local feature of the image can be obtained by an edge detection algorithm, such as Canny, and the spatial relationship analysis between the specific objects can be achieved using a cross-relationship method.

[0093] S3. According to the feature vector, identify the mixed terrain in the image data set, perform three-dimensional reconstruction on the mixed terrain to obtain a three-dimensional terrain model, and perform cyclic adversarial training on the image data set and the three-dimensional terrain model to obtain a target three-dimensional terrain model.

[0094] The embodiment of the present invention can obtain more refined classified terrain features and improve the simulation effect of the subsequent three-dimensional terrain simulation model by identifying the mixed terrain in the image data set according to the feature vector. The mixed terrain refers to a terrain with multiple features formed by two or more different types of terrains intertwined with each other.

[0095] Optionally, the mixed terrain identification in the image data set based on the feature vector can be obtained by a clustering analysis algorithm, such as a K-mean algorithm.

[0096] Furthermore, the embodiment of the present invention obtains a three-dimensional terrain model by performing three-dimensional reconstruction on the mixed terrain, which can provide an accurate and vivid three-dimensional terrain model for subsequent rendering work and improve the rendering quality. The three-dimensional terrain model refers to a digital model that is generated using two-dimensional terrain image data and has three-dimensional spatial characteristics and can be realistically displayed.

[0097] As an embodiment of the present invention, the three-dimensional reconstruction of the mixed terrain to obtain a three-dimensional terrain model includes: collecting terrain surface data corresponding to the mixed terrain, and performing data optimization processing on the terrain surface data to obtain target terrain data; constructing a triangular mesh of the mixed terrain surface based on the target terrain data; extracting elevation data from the target terrain data, and constructing a terrain surface model of the mixed terrain based on the elevation data; performing texture mapping on the terrain surface model to obtain a model texture surface; and performing three-dimensional reconstruction of the mixed terrain based on the triangular mesh, the terrain surface model and the model texture surface to obtain a three-dimensional terrain model.

[0098] Among them, the terrain surface data refers to the original terrain data collected from multiple data sources, such as lidar data source; the target terrain data refers to the data obtained after denoising, smoothing, interpolation, data fusion and other processing of the collected terrain surface data; the triangular mesh refers to a mesh composed of vertices, edges and faces representing the three-dimensional terrain surface; the elevation data refers to data composed of the vertical distance of each point on the terrain surface relative to a certain reference plane; the terrain surface model refers to a model that can reflect the terrain undulations and is constructed based on the elevation data; the model texture surface refers to the model surface formed after mapping the terrain texture features to the terrain surface model.

[0099] Optionally, the terrain surface data corresponding to the mixed terrain can be acquired through satellite remote sensing equipment, the data optimization processing of the terrain surface data can be implemented using SciPy, the construction of the terrain surface model of the mixed terrain based on the elevation data can be acquired through surface fitting technology, and the texture mapping of the terrain surface model can be implemented using a texture mapping algorithm, such as UV mapping.

[0100] As another embodiment of the present invention, constructing the triangular mesh of the mixed terrain surface based on the target terrain data includes: matching image feature points in the target terrain data, and querying terrain features corresponding to the image feature points; setting the position layout of the image feature points according to the terrain features; collecting a three-dimensional point set of the image feature points; and constructing the triangular mesh of the mixed terrain surface within the three-dimensional point set according to the position layout.

[0101] Among them, the image feature points refer to edge points with significant features in the image, the position layout refers to the distribution of terrain feature points in the image, and the three-dimensional point set refers to a set of points with specific coordinates in three-dimensional space.

[0102] Optionally, the image feature point matching in the target terrain data may be obtained through ORB (Oriented FAST and Rotated BRIEF), and the three-dimensional point set collection of the image feature points may be implemented using a RANSAC algorithm.

[0103] S3. Analyze the illumination direction of the terrain to be rendered in the three-dimensional terrain model, calculate the illumination intensity of the three-dimensional terrain model based on the illumination direction, identify the shadow distribution area of ​​the three-dimensional terrain model, partition the three-dimensional terrain model into plots to obtain partitioned plots, extract dense point cloud data corresponding to the partitioned plots, and calculate the terrain roughness of the terrain to be rendered based on the dense point cloud data.

[0104] By analyzing the lighting direction of the terrain to be rendered in the three-dimensional terrain model, the embodiment of the present invention can simulate the lighting conditions in different time periods to enhance the realism of the terrain. It can also achieve dynamic light and shadow effects of the terrain scene by dynamically adjusting the lighting direction, where the lighting direction refers to the position and direction of the light source relative to the terrain.

[0105] As an embodiment of the present invention, the analysis of the lighting direction of the terrain to be rendered in the three-dimensional terrain model includes: collecting elevation data corresponding to the terrain to be rendered; analyzing the morphological features of the terrain to be rendered in the elevation data; setting the simulation time type of the terrain to be rendered in the three-dimensional terrain model; setting the azimuth and elevation of the three-dimensional terrain model according to the simulation time type and the morphological features; and analyzing the lighting direction of the terrain to be rendered in the three-dimensional terrain model based on the azimuth and the elevation.

[0106] Among them, the elevation data refers to a data set of height representing a specific point on the terrain surface, the morphological feature refers to the physical appearance and structure of the terrain, including but not limited to slope, aspect, height difference, terrain undulation, etc., the simulation time type refers to the time period type of the model simulating illumination, the azimuth refers to the angle measured clockwise from due north to a specific direction, and the elevation refers to the angle from the horizontal plane to the illumination direction.

[0107] Optionally, the elevation data collection corresponding to the terrain to be rendered can be acquired through drone aerial photography, and the morphological feature analysis of the terrain to be rendered in the elevation data can be achieved by calculating the normal vector of each grid cell using DEM data. The azimuth and elevation of the three-dimensional terrain model can be achieved by analyzing the changing pattern of the sun throughout the day. For example, if it is necessary to simulate the morning lighting effect, an azimuth close to 90° (east direction) can be selected.

[0108] Furthermore, the embodiments of the present invention can assist in identifying and highlighting key features of the terrain by calculating the illumination intensity of the three-dimensional terrain model based on the illumination direction, making the terrain model look more realistic, thereby improving the quality of post-rendering. The illumination intensity refers to the amount of light received by each point on the terrain surface.

[0109] As an embodiment of the present invention, the calculation of the illumination intensity of the three-dimensional terrain model based on the illumination direction includes: collecting elevation data corresponding to the three-dimensional terrain model; extracting unit grid data in the elevation data; calculating the normal vector of the unit grid data; and calculating the illumination intensity of the three-dimensional terrain model according to the normal vector and the illumination direction.

[0110] The unit grid data refers to the elevation data after the terrain is divided into a plurality of grid areas, and the normal vector refers to a vector perpendicular to a specific point on the terrain surface.

[0111] In an optional embodiment of the present invention, the normal vector of the unit grid data is calculated using the following formula:

[0112]

[0113] in, Represents the normal vector of the unit grid data at the coordinate (x, y), h righ t (x+d,y) represents the elevation value of the adjacent cell on the right in the current cell grid data, h left (xd,y) represents the elevation value of the left adjacent cell in the current cell grid data, h down (x,y+d) represents the elevation value of the lower adjacent cell in the current cell grid data, h up (x, yd) represents the elevation value of the upper adjacent cell in the current cell grid data, and d represents the height of the cell grid in the cell grid data.

[0114] The embodiment of the present invention can pre-calculate shadow information by identifying the shadow distribution area of ​​the three-dimensional terrain model based on the lighting direction, thereby reducing the calculation burden during real-time rendering, and at the same time highlighting the characteristics of the terrain and enhancing the realism of the three-dimensional terrain model. The shadow distribution area refers to the area where the terrain surface is obscured.

[0115] Optionally, the identification of the shadow distribution area of ​​the three-dimensional terrain model based on the illumination direction may be implemented using shadow mapping technology.

[0116] Furthermore, the embodiment of the present invention can help identify and remove abnormal points or noise in specific types of plots by partitioning the three-dimensional terrain model to obtain partitioned plots, thereby improving the processing efficiency and accuracy of dense point cloud data. The partitioned plots refer to dividing a complete terrain into several smaller sub-plots with certain coherence and similarity according to certain standards.

[0117] Optionally, the land parcel partitions of the three-dimensional terrain model may be obtained by a classification algorithm, such as a decision tree algorithm.

[0118] The embodiment of the present invention can provide high-density spatial data points of the partitioned plots by extracting dense point cloud data corresponding to the partitioned plots, which enables the terrain surface to be represented finely and helps to improve the realism of rendering. The dense point cloud data refers to data distributed in three-dimensional space and composed of three-dimensional points representing the real terrain.

[0119] Optionally, the dense point cloud data corresponding to the partitioned plots may be extracted and obtained through point cloud processing software, such as CloudCompare software.

[0120] Furthermore, the embodiment of the present invention can adjust the texture mapping effect by calculating the terrain roughness of the terrain to be rendered based on the dense point cloud data, so that the rendering result is more realistic. The terrain roughness refers to the unevenness of the terrain surface.

[0121] As an embodiment of the present invention, the terrain roughness of the terrain to be rendered is calculated based on the dense point cloud data, including: collecting three-dimensional point cloud coordinates corresponding to the dense point cloud data; setting the plot boundaries of the partitioned plots corresponding to the terrain to be rendered; within the plot boundaries, calculating the average elevation difference between the three-dimensional point cloud coordinates; based on the average elevation difference, calculating the terrain roughness of the terrain to be rendered.

[0122] Among them, the three-dimensional point cloud coordinates refer to the position representation of each point in the three-dimensional space, the plot boundary refers to the boundary of each area divided in the three-dimensional terrain model, and the average elevation difference refers to the average height difference between each point in the point cloud data in a given area.

[0123] Optionally, the three-dimensional point cloud coordinates corresponding to the dense point cloud data can be acquired by a three-dimensional scanner, such as a handheld three-dimensional scanner, and the plot boundary setting of the partitioned plots corresponding to the terrain to be rendered can be achieved by using a drawing tool of QGIS.

[0124] In an optional embodiment of the present invention, the average elevation difference between the three-dimensional point cloud coordinates is calculated using the following formula:

[0125]

[0126] Wherein, R represents the average elevation difference between the three-dimensional point cloud coordinates, n represents the number of three-dimensional point clouds in the region, i represents the i-th coordinate point in the three-dimensional point cloud, k represents the next point after the i-th coordinate point in the three-dimensional point cloud, |z i -z k | represents the absolute value of the elevation difference between the i-th coordinate point and the k-th coordinate point in the three-dimensional point cloud.

[0127] S4. Analyze the simulation effect of the three-dimensional terrain model according to the illumination intensity, the shadow distribution area, and the roughness of the terrain; identify irrelevant geometric elements of the three-dimensional terrain model in the rendering process according to the simulation effect; perform view culling processing on the irrelevant geometric elements to obtain a target rendering object; and set a detail rendering level of the target rendering object.

[0128] The embodiment of the present invention can create a three-dimensional terrain model with a strong sense of reality and visual impact by analyzing the simulation effect of the three-dimensional terrain model according to the light intensity, the shadow distribution area and the roughness of the terrain, thereby providing users with a richer visual experience and a higher degree of simulation. The simulation effect refers to the visual and functional restoration and realistic effect of the three-dimensional terrain model on the real terrain environment.

[0129] Optionally, the simulation effect analysis of the three-dimensional terrain model in combination with the light intensity, the shadow distribution area and the terrain roughness can be obtained through simulation software, such as Simulink simulation software.

[0130] Furthermore, the embodiment of the present invention can reduce the amount of calculation and resource consumption as much as possible, thereby improving the number of rendering frames per second by identifying irrelevant geometric elements of the three-dimensional terrain model in the rendering process according to the simulation effect. The irrelevant geometric elements refer to detail elements that contribute little or almost no contribution to the overall effect but still occupy computing resources and memory space during the rendering process. For example, in a scene, the mountains in the distance may only exist as a background, and their level of detail has little impact on the overall effect.

[0131] As an embodiment of the present invention, identifying irrelevant geometric elements of the three-dimensional terrain model during the rendering process based on the simulation effect includes: identifying the influence level of scene elements in the three-dimensional terrain model based on the simulation effect; analyzing the visualization effect corresponding to the influence level; calculating the contribution of the visualization effect to the overall effect; and identifying irrelevant geometric elements of the three-dimensional terrain model during the rendering process based on the contribution.

[0132] Among them, the scene elements refer to the various parts or objects that constitute the three-dimensional terrain model, such as trees and buildings. The impact level refers to the degree of influence of different scene elements on the overall simulation effect. The visualization effect refers to the visual performance of the scene elements in the three-dimensional terrain model. The overall effect refers to the visual perception effect experienced by the audience when viewing the three-dimensional terrain model or scene. The contribution degree refers to the importance of the scene elements to the overall effect.

[0133] Optionally, according to the simulation effect, the impact level identification of the scene elements in the three-dimensional terrain model can be obtained by assigning an indicator weight to each scene element, and the visualization effect analysis corresponding to the impact level can be implemented using data visualization software, such as Tableau software.

[0134] In an optional embodiment of the present invention, the contribution of the visualization effect to the overall effect is calculated using the following formula:

[0135]

[0136] Among them, G represents the contribution of visualization effect to the overall effect, N represents the total number of overall effect values ​​corresponding to the visualization effect value, and A g represents the value of the g-th visualization effect, A f represents the value of the fth overall effect, represents the sum of the overall effect values, Indicates the ratio of the g-th visualization effect value to the total effect value.

[0137] The embodiment of the present invention performs view culling processing on the irrelevant geometric elements to obtain a target rendering object, which can significantly reduce the number of objects that need to be rendered, thereby reducing the amount of calculation in the rendering process and improving the rendering efficiency. The target rendering object refers to the object to be rendered after culling elements that do not contribute much to the rendering effect.

[0138] Optionally, the view area culling process of irrelevant geometric elements may be implemented using Unreal Engine.

[0139] Furthermore, the embodiment of the present invention can maintain the details of the model by setting the detail rendering level of the target rendering object, making the visual effect of the three-dimensional terrain model more realistic, improving the rendering speed, reducing the jamming, and providing a smoother visual effect. The detail rendering level refers to dynamically adjusting the detail level of the model in three-dimensional rendering in order to improve rendering efficiency and maintain visual quality.

[0140] Optionally, the detail rendering level setting of the target rendering object may be obtained through 3D modeling software, such as Blender.

[0141] S5. Combining the three-dimensional terrain model and the detail rendering level, the three-dimensional terrain is rendered in real time to obtain a real-time rendering result.

[0142] The embodiment of the present invention combines the three-dimensional terrain model and the detail rendering level to perform real-time rendering of the three-dimensional terrain to obtain a real-time rendering result, which can significantly reduce the computing load, especially when processing large scenes, it can improve the rendering speed and reduce jamming. At the same time, the detail level of the model can be dynamically adjusted according to the distance between the observer and the model, optimize resource allocation, and improve rendering efficiency. The three-dimensional terrain refers to the surface morphology represented in three-dimensional space, and the real-time rendering result refers to the final image or video output obtained after real-time rendering of the three-dimensional terrain model in computer graphics.

[0143] It can be seen that the embodiment of the present invention can obtain image data of different terrain features by collecting an image data set of the terrain to be rendered, thereby providing diversified data support for training the deep learning model, improving the generalization ability of the model, and truly simulating the visual effect of the terrain by analyzing different types of terrain features, thereby enhancing the realism of the scene; further, the embodiment of the present invention can identify different terrain features by extracting local image features corresponding to the terrain to be rendered based on the terrain attributes, thereby facilitating subsequent image analysis and three-dimensional terrain modeling, and inputting the extracted key terrain features into the deep model, thereby improving the accuracy and generalization ability of the model; the embodiment of the present invention can obtain more refined classified terrain features by identifying mixed terrain in the image data set according to the feature vector, thereby providing an accurate and vivid three-dimensional terrain model for later rendering work and improving the rendering quality; further, the embodiment of the present invention can analyze the three-dimensional terrain model to obtain the mixed terrain in the image data set according to the feature vector, thereby improving the rendering quality; The illumination direction, illumination intensity, shadow distribution area and ground roughness of the terrain to be rendered can simulate the illumination conditions and texture mapping effects of different time periods, assist in identifying and highlighting the key features of the terrain, and make the terrain model look more realistic; the embodiment of the present invention can reduce the amount of calculation and resource consumption as much as possible by identifying the irrelevant geometric elements of the three-dimensional terrain model in the rendering process according to the simulation effect, thereby increasing the number of rendering frames per second, significantly reducing the number of objects to be rendered, thereby reducing the amount of calculation in the rendering process and accelerating the rendering efficiency; further, the embodiment of the present invention combines the three-dimensional terrain model and the detail rendering level to perform real-time rendering of the three-dimensional terrain to obtain a real-time rendering result, which can significantly reduce the computing load, especially when processing large scenes, it can improve the rendering speed and reduce jamming. At the same time, the detail level of the model can be dynamically adjusted according to the distance between the observer and the model, optimize resource allocation, and improve rendering efficiency. Therefore, the efficient real-time rendering method of three-dimensional terrain based on deep learning proposed in the embodiment of the present invention can eliminate irrelevant geometric elements of the three-dimensional terrain model in the rendering process and improve the real-time rendering efficiency of the three-dimensional terrain.

[0144] like Figure 2 As shown, it is a system functional module diagram of efficient real-time rendering of three-dimensional terrain based on deep learning of the present invention.

[0145] The high-efficiency real-time rendering system 200 of three-dimensional terrain based on deep learning of the present invention can be installed in an electronic device. According to the functions implemented, the high-efficiency real-time rendering system of three-dimensional terrain based on deep learning can include a model monitoring module 201, a behavior detection module 202, an index analysis module 203, a model output module 204, and a quality assessment module 205. The module of the present invention can also be called a unit, which refers to a series of computer program segments that can be executed by an electronic device processor and can complete fixed functions, which are stored in the memory of the electronic device.

[0146] In the embodiment of the present invention, the functions of each module / unit are as follows:

[0147] The data acquisition module 201 is used to acquire an image data set of a terrain to be rendered, normalize the image data set to obtain a target image data set, and analyze the terrain attributes in the target image data set;

[0148] The model generation module 202 is used to extract the local features of the image corresponding to the terrain to be rendered based on the terrain attributes, analyze the deviation degree of the local features of the image, construct a feature vector of the local features of the image based on the deviation degree, identify the mixed terrain in the image data set according to the feature vector, and perform three-dimensional reconstruction on the mixed terrain to obtain a three-dimensional terrain model;

[0149] The detail analysis module 203 is used to analyze the illumination direction of the terrain to be rendered in the three-dimensional terrain model, calculate the illumination intensity of the three-dimensional terrain model based on the illumination direction, identify the shadow distribution area of ​​the three-dimensional terrain model, partition the three-dimensional terrain model into blocks, obtain partitioned blocks, extract dense point cloud data corresponding to the partitioned blocks, and calculate the terrain roughness of the terrain to be rendered based on the dense point cloud data;

[0150] A rendering detail optimization module 204 is used to analyze the simulation effect of the three-dimensional terrain model according to the illumination intensity, the shadow distribution area and the terrain roughness, identify irrelevant geometric elements of the three-dimensional terrain model in the rendering process according to the simulation effect, perform view culling processing on the irrelevant geometric elements, obtain a target rendering object, and set the detail rendering level of the target rendering object;

[0151] The model rendering module 205 is used to combine the three-dimensional terrain model and the detail rendering level to perform real-time rendering on the three-dimensional terrain to obtain a real-time rendering result.

[0152] In detail, each module in the high-efficiency real-time rendering system 200 of three-dimensional terrain based on deep learning in the embodiment of the present invention is used in the same manner as described above. Figure 1The method for efficient real-time rendering of three-dimensional terrain based on deep learning described in the text is the same technical means and can produce the same technical effects, so I will not go into details here.

[0153] In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit, each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional modules.

[0154] It is obvious to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0155] Therefore, no matter from which point of view, the embodiments should be regarded as illustrative and non-restrictive, and the scope of the present invention is limited by the appended claims rather than the above description, so it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the present invention. Any attached figure mark in the claims should not be regarded as limiting the claims involved.

[0156] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0157] The above description is only a specific embodiment of the present invention, so that those skilled in the art can understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but should conform to the widest scope consistent with the principles and novel features invented herein.

Claims

1. An efficient real-time rendering method for three-dimensional terrain based on deep learning, characterized in that: The method comprises: Collecting an image data set of a terrain to be rendered, normalizing the image data set to obtain a target image data set, and analyzing terrain attributes in the target image data set; Based on the terrain attributes, extracting local features of the image corresponding to the terrain to be rendered, analyzing the degree of deviation of the local features of the image, constructing a feature vector of the local features of the image based on the degree of deviation, identifying the mixed terrain in the image data set according to the feature vector, and performing three-dimensional reconstruction on the mixed terrain to obtain a three-dimensional terrain model; Analyzing the illumination direction of the terrain to be rendered in the three-dimensional terrain model, calculating the illumination intensity of the three-dimensional terrain model based on the illumination direction, identifying the shadow distribution area of ​​the three-dimensional terrain model, partitioning the three-dimensional terrain model into blocks to obtain partitioned blocks, extracting dense point cloud data corresponding to the partitioned blocks, and calculating the terrain roughness of the terrain to be rendered based on the dense point cloud data; Analyzing the simulation effect of the three-dimensional terrain model according to the illumination intensity, the shadow distribution area and the roughness of the terrain, identifying irrelevant geometric elements of the three-dimensional terrain model in the rendering process according to the simulation effect, performing view culling processing on the irrelevant geometric elements to obtain a target rendering object, and setting a detail rendering level of the target rendering object; The three-dimensional terrain model and the detail rendering level are combined to perform real-time rendering on the three-dimensional terrain to obtain a real-time rendering result.

2. The method according to claim 1, characterized in that The analyzing the terrain attributes in the target image data set includes: Performing image segmentation on the target image data set to obtain a segmented image; identifying terrain features in the segmented image; Analyzing specific objects in the segmented image according to the terrain features; Based on the specific object, terrain attributes in the target image dataset are analyzed.

3. The method according to claim 1, characterized in that The extracting, based on the terrain attributes, the local features of the image corresponding to the terrain to be rendered includes: Based on the terrain attributes, classifying the real terrain of the terrain to be rendered to obtain a real classified terrain; Analyzing key feature lines corresponding to the real classified terrain; Analyzing the image feature representation of the key feature line according to the real classified terrain; Based on the image feature representation, local image features corresponding to the terrain to be rendered are extracted.

4. The method according to claim 1, characterized in that The step of constructing a feature vector of the local feature of the image based on the degree of deviation includes: Based on the degree of deviation, identifying structural features of the terrain to be rendered corresponding to the local features of the image; Extracting specific objects from local features of the image; analyzing the spatial relationship between the specific objects; A feature vector of the local feature of the image is constructed according to the structural feature and the spatial relationship.

5. The method according to claim 1, characterized in that The three-dimensional reconstruction of the mixed terrain to obtain a three-dimensional terrain model includes: Collecting terrain surface data corresponding to the mixed terrain, and performing data optimization processing on the terrain surface data to obtain target terrain data; Based on the target terrain data, construct a triangular mesh of the mixed terrain surface; Extracting elevation data from the target terrain data, and constructing a terrain surface model of the mixed terrain based on the elevation data; Performing texture mapping on the terrain surface model to obtain a model texture surface; The mixed terrain is three-dimensionally reconstructed according to the triangular mesh, the terrain surface model and the model texture surface to obtain a three-dimensional terrain model.

6. The method according to claim 5, characterized in that The step of constructing a triangular mesh of the mixed terrain surface based on the target terrain data includes: Matching image feature points in the target terrain data, and querying terrain features corresponding to the image feature points; According to the terrain features, setting the position layout of the image feature points; Collecting a three-dimensional point set of feature points of the image; According to the position layout, a triangular mesh of the hybrid terrain surface is constructed within the three-dimensional point set.

7. The method according to claim 1, characterized in that The analyzing the illumination direction of the terrain to be rendered in the three-dimensional terrain model includes: Collecting elevation data corresponding to the terrain to be rendered; Analyzing the morphological features of the terrain to be rendered in the elevation data; Setting the simulation time type of the terrain to be rendered in the three-dimensional terrain model; According to the simulation time type and the morphological characteristics, setting the azimuth and elevation of the three-dimensional terrain model; Based on the azimuth angle and the elevation angle, the illumination direction of the terrain to be rendered in the three-dimensional terrain model is analyzed.

8. The method according to claim 1, characterized in that The calculating the illumination intensity of the three-dimensional terrain model based on the illumination direction includes: Collecting elevation data corresponding to the three-dimensional terrain model; Extracting unit grid data from the elevation data; The normal vector of the unit grid data is calculated using the following formula: in, Represents the normal vector of the unit grid data at the coordinate (x, y), h right (x+d,y) represents the elevation value of the adjacent cell on the right in the current cell grid data, h left (xd,y) represents the elevation value of the left adjacent cell in the current cell grid data, h down (x,y+d) represents the elevation value of the lower adjacent cell in the current cell grid data, h up (x, yd) represents the elevation value of the upper adjacent cell in the current cell grid data, and d represents the height of the cell grid in the cell grid data; The illumination intensity of the three-dimensional terrain model is calculated according to the normal vector and the illumination direction.

9. The method according to claim 1, characterized in that: The step of identifying irrelevant geometric elements of the three-dimensional terrain model during the rendering process according to the simulation effect includes: According to the simulation effect, identifying the influence level of the scene elements in the three-dimensional terrain model; Analyze the visualization effect corresponding to the impact level; The contribution of the visualization effect to the overall effect is calculated using the following formula: Among them, G represents the contribution of visualization effect to the overall effect, N represents the total number of overall effect values ​​corresponding to the visualization effect value, and A g represents the value of the g-th visualization effect, A f represents the value of the fth overall effect, represents the sum of the overall effect values, Indicates the ratio of the g-th visualization effect value to the total effect value; Irrelevant geometric elements of the three-dimensional terrain model during rendering are identified according to the contribution.

10. An efficient real-time rendering system for three-dimensional terrain based on deep learning, characterized in that: The system comprises: A data acquisition module is used to acquire an image data set of a terrain to be rendered, normalize the image data set to obtain a target image data set, and analyze the terrain attributes in the target image data set; A model generation module, for extracting local features of the image corresponding to the terrain to be rendered based on the terrain attributes, analyzing the degree of deviation of the local features of the image, constructing a feature vector of the local features of the image based on the degree of deviation, identifying the mixed terrain in the image data set according to the feature vector, and performing three-dimensional reconstruction on the mixed terrain to obtain a three-dimensional terrain model; a detail analysis module, configured to analyze the illumination direction of the terrain to be rendered in the three-dimensional terrain model, calculate the illumination intensity of the three-dimensional terrain model based on the illumination direction, identify the shadow distribution area of ​​the three-dimensional terrain model, partition the three-dimensional terrain model into blocks, obtain partitioned blocks, extract dense point cloud data corresponding to the partitioned blocks, and calculate the terrain roughness of the terrain to be rendered based on the dense point cloud data; a rendering detail optimization module, for analyzing the simulation effect of the three-dimensional terrain model according to the illumination intensity, the shadow distribution area and the terrain roughness, identifying irrelevant geometric elements of the three-dimensional terrain model in the rendering process according to the simulation effect, performing view culling processing on the irrelevant geometric elements, obtaining a target rendering object, and setting a detail rendering level of the target rendering object; The model rendering module is used to combine the three-dimensional terrain model and the detail rendering level to perform real-time rendering on the three-dimensional terrain to obtain a real-time rendering result.

Images