Simulator visual automatic modeling method based on convolutional neural network

Abstract

The invention discloses a simulator visual automatic modeling method based on a convolutional neural network, and relates to the technical field of simulator visual modeling and computer vision. The problems that in the prior art, simulator visual modeling is low in efficiency, poor in precision, insufficient in adaptability and low in building modeling automation degree are solved. The method comprises the steps of visual data preprocessing, visual network initialization, structure adaptive optimization, visual model training, visual model optimization output and optional visual model adaptation. Wherein the scene network is initialized to construct a special convolutional neural network framework containing a building structure extraction sub-module, and building area segmentation and structure parameter extraction are realized; the structure self-adaptive optimization adopts a reinforcement learning agent model to dynamically adjust the network structure; an improved AdamW algorithm is adopted in the training process, and an early stop mechanism is set to avoid overfitting; the optimization output guarantees the performance of the model through grid simplification, texture enhancement and illumination calibration. Synchronous automatic modeling of the terrain and the building is achieved, the modeling efficiency is improved by 80% or above, the rendering frame rate of the final output model is not lower than 60 fps, the geometric precision error of the terrain is smaller than or equal to 5%, the dimensional precision error of the building is smaller than or equal to 3%, and the method can directly adapt to a flight simulator and is especially suitable for low-altitude flight training visual construction of a mountain-building mixed scene. The practical value and the popularization prospect are extremely high.

Description

Technical Field

[0001] This invention relates to the fields of simulator visual modeling and computer vision technology. Specifically, it relates to an automated simulator visual modeling method based on convolutional neural networks, which supports the fully automated construction of high-fidelity visual models based on multi-source visual data. Background Technology

[0002] Visual simulation systems are a core component of flight simulators, mission simulators, and other simulation training equipment. They are responsible for transforming flight parameters, scenario data, and environmental information required for simulation training into visually dynamic images. High-fidelity visual presentation allows trainees to receive visual feedback highly consistent with the real flight environment, enhancing the immersion and effectiveness of training. Accurate airport and scenario models are the core of this visual presentation.

[0003] Traditional simulator-based visual modeling methods primarily rely on manual modeling, that is, professional modelers using 3D modeling software (such as 3ds Max). Manually building terrain and feature models using software like Max and Maya, followed by texture mapping and lighting settings, has several significant drawbacks: 1. Low modeling efficiency: Manual modeling requires meticulous depiction of terrain undulations and feature details, especially for large and complex scenes, often resulting in models taking weeks or even months, making it difficult to meet the demands of rapid iteration. 2. Poor accuracy and consistency: Visual modeling heavily relies on the experience and skills of the operators. Models of the same type of feature built by different modelers can easily differ in geometric dimensions and texture fit, leading to insufficient overall consistency in the visual model. Furthermore, the matching error between manual terrain elevation data and image data is significant, making it difficult to guarantee the geometric accuracy of the model. 3. Insufficient adaptability: Traditional modeling methods produce visual models with fixed parameters. When adapting to different simulation training scenarios, parameters such as mesh precision and texture resolution need to be manually readjusted, making dynamic matching of model performance impossible. 4. High cost: The training period for professional modelers is long and labor costs are high. Continuous investment in manpower is also required for later model optimization and maintenance, resulting in persistently high overall modeling costs.

[0004] In recent years, with the rapid development of convolutional neural networks in computer vision, some studies have attempted to apply them to 3D modeling. However, existing technologies mostly focus on modeling single objects and lack specific designs for scenarios like simulator scenes that integrate multiple elements such as terrain, features, and lighting. This results in problems such as difficulty in balancing visual realism and real-time rendering performance, as well as weak model generalization ability, making them unsuitable for direct application in simulator scene modeling. Therefore, there is an urgent need for a method that can automate, accurately, and efficiently construct simulator scene models. Summary of the Invention

[0005] The purpose of this invention is to address the problems of low efficiency, poor accuracy, insufficient adaptability, and high cost in existing simulator visual modeling methods. It provides an automated simulator visual modeling method based on convolutional neural networks, achieving fully automated construction of a high-fidelity visual model from multi-source visual data, while balancing visual effects and real-time rendering performance. The technical solution of this invention is as follows.

[0006] An automated modeling method for simulator visual models based on convolutional neural networks mainly includes the following steps:

[0007] Step 1: Obtain the raw simulation scene data. This raw data is a multi-source dataset supporting the construction of the scene model, specifically including scene imagery data (such as satellite remote sensing imagery and UAV aerial imagery), terrain elevation data (such as DEM digital elevation model data), ground feature texture data (such as real-shot texture maps of ground feature surfaces), and lighting environment parameters (such as solar altitude angle and ambient light intensity at different times). The raw data is then processed to eliminate noise, standardize the data format, and achieve spatial alignment.

[0008] Step 2: Initialize the visual network and build a dedicated convolutional neural network framework for simulator visual modeling. This framework is modularly designed to address the characteristics of the visual model's "terrain-feature-lighting" fusion.

[0009] Step 3: Introduce a reinforcement learning agent model to dynamically optimize the constructed convolutional neural network framework in order to solve the problem of network structure adaptation under different visual scenarios;

[0010] Step 4: Divide the standardized visual dataset into training, validation, and test sets in a 7:2:1 ratio. The training set is used for iterative updates of network parameters, the validation set is used to monitor model performance to avoid overfitting, and the test set is used for final model performance evaluation.

[0011] Step 5: Lighten and enhance the initial visual model to meet the real-time rendering requirements of the simulator;

[0012] Preferably, step one involves targeted processing of the original data to eliminate data noise, unify the data format, and achieve data spatial alignment. The specific processing steps include:

[0013] Outlier removal: The Mahalanobis distance method is used to analyze the terrain elevation data, identify and remove abrupt elevation changes caused by measurement errors, and ensure the continuity of the terrain data. The Mahalanobis distance calculation formula is as follows:

[0014] (1)

[0015] in, This is a sample of terrain elevation data. The average elevation data. Let be the inverse of the covariance matrix of the elevation data, when If the value is out of the question, it is considered an outlier and removed.

[0016] Data registration: Feature points (such as terrain inflection points and feature outline corners) are extracted from scene images and terrain elevation data using the SIFT (Scale Invariant Feature Transform) algorithm. Feature point matching is then used to align the spatial coordinates of the image data and terrain data, avoiding the "texture offset" problem in subsequent modeling. Euclidean distance is used to determine the similarity of feature point matching, and the calculation formula is as follows:

[0017] (2)

[0018] in Let n be the descriptor vector of two feature points, where n is the descriptor dimension. , This is a highly suitable match and is therefore considered a valid match.

[0019] Resolution unification: Unify the scaling of raw data from different sources to a preset resolution (e.g., image data is unified to 2048×2048 pixels, terrain data is unified to 1m×1m grid precision, and building textures are unified to 4096×4096 pixels).

[0020] Normalization: To eliminate the impact of differences in data units on network training, elevation data is linearly mapped to the interval [0, 1000]. The mapping formula is as follows:

[0021] (3)

[0022] in, This is the original elevation data. , These are the minimum and maximum values ​​of the elevation data, respectively. This is the normalized elevation data. The texture data (RGB channels) is mapped to the [0,255] interval to obtain the final standardized visual dataset.

[0023] Preferably, the convolutional neural network framework in step two specifically includes a data input layer, a terrain feature extraction module, a building structure extraction sub-module, a ground feature recognition module, a lighting rendering fusion module, and a visual model output layer. The functions and collaborative relationships of each module are as follows:

[0024] Data input layer: adopts a multi-channel input structure to receive pre-processed terrain elevation data, scene image data, ground texture data (including building textures) and lighting environment parameters, respectively, to realize the synchronous input of multi-source data;

[0025] Terrain feature extraction module: It adopts a multi-scale convolutional structure and extracts features from terrain data through convolutional kernels of different sizes, capturing the macroscopic undulations and microscopic texture features of the terrain;

[0026] The building structure extraction submodule, based on an improved Mask R-CNN network, is specifically designed for building region segmentation and structural parameter extraction, including building outline, floor height, window position and size, etc. Floor height is calculated using the building shadow length and solar altitude angle in the image, using the following formula:

[0027] (4)

[0028] in, For building floor height, The length of the building's shadow in the image. This represents the solar altitude angle for the corresponding time period.

[0029] Ground feature recognition module: Based on the feature classification capabilities of convolutional neural networks, it automatically identifies non-building ground feature types (such as roads and vegetation) in images and extracts geometric features such as ground feature outlines and dimensions.

[0030] Lighting rendering blending module: Combining input lighting environment parameters with extracted terrain, building, and other feature features, it calculates lighting distribution and building shadow effects in different scenes, achieving a natural blending of terrain, buildings, other features, and lighting.

[0031] Visual model output layer: Maps the fused features into a 3D mesh model and texture information, and outputs the initial visual model data.

[0032] Based on the characteristics of the flight simulator, the initial parameters of the network are automatically configured. Since the flight simulator needs to present a large-scale terrain scene and the distribution of buildings in the area, the terrain grid precision is set to 5m×5m and the building grid precision is set to 0.5m×0.5m. An 11×11 large-size convolutional kernel is used to improve the macro feature extraction capability. The initial parameters specifically include the convolutional kernel size, the number of feature map channels, the terrain grid precision, the building grid precision, the texture sampling rate, and the illumination calculation factor.

[0033] Preferably, in step three, the visual realism of the visual model (evaluated by peak signal-to-noise ratio PSNR and structural similarity SSIM), real-time rendering frame rate, and geometric accuracy (including building structural accuracy, evaluated by the error between the actual building size and the model size) are used as the comprehensive reward signal. The reward function is calculated as follows:

[0034] (5)

[0035] in, Here are the weighting coefficients (all with a value of 0.25), and F is the rendering frame rate. These are the actual dimensions of the building. The building dimensions in the model guide the reinforcement learning agent model to adjust the network structure through an "exploration-exploitation" strategy.

[0036] Specific adjustments include: optimizing the number of convolutional layers in the terrain feature extraction module based on terrain complexity; adjusting the kernel size and number of convolutional layers in the building structure extraction submodule based on architectural style, increasing the number of 3×3 convolutional kernels for buildings to improve detail extraction capabilities; and adjusting the embedding position of the attention mechanism in the lighting rendering fusion module based on lighting scene type, placing the attention mechanism forward in strong light scenes to highlight building shadow features, and placing it backward in low light scenes to enhance building brightness information.

[0037] Preferably, in step four, the improved AdamW algorithm is used to train the optimized network. This algorithm suppresses overfitting through a weight decay mechanism. The specific parameters are configured as follows: the initial learning rate is dynamically set according to the size of the scene data: 2e-3 when the data size is greater than 50GB, 1e-3 when the data size is 20GB≤50GB, and 5e-4 when the data size is less than 20GB; the weight decay coefficient is fixed at 5e-5, and the batch size is dynamically adjusted according to the GPU memory to ensure that the single training data includes complete data of at least 20 terrain units and 10 building units to ensure the stability of training.

[0038] During training, the visual model performance is evaluated using a validation set after every 50 iterations. If the visual realism score improves by less than 0.3 points and the rendering frame rate fluctuates by more than 5fps for eight consecutive iterations, or if the PSNR improvement on the validation set is less than 0.5dB, the SSIM improvement is less than 0.01, and the difference in rendering time between the training and validation sets exceeds 10ms, the model is considered overfitted, and an early stopping mechanism is immediately triggered. The optimal model parameters before the early stopping are saved, resulting in an initial visual model containing the 3D building structure. The PSNR calculation formula is as follows:

[0039] (6)

[0040] (7)

[0041] in, Mean square error, For the height and width of the image, These are the actual image pixel values. The SSIM formula is used to calculate the pixel values ​​of the rendered image for the model as follows:

[0042] (8)

[0043] in, These are the average values ​​of the real image and the rendered image, respectively. , These are the standard deviations of the real image and the rendered image, respectively. Let the covariance of the two be , , is a constant, taken as 6.5025 and 58.5225 respectively.

[0044] Preferably, step five involves lightweighting and enhancing the initial visual model, with a focus on optimizing the mesh quality of the building model, primarily including:

[0045] Mesh simplification: An edge folding algorithm based on geometric features is adopted. The importance of terrain key points and building structure key points is calculated by curvature. The facets corresponding to the top 30% of key points in importance score are retained, and redundant facets are removed. Building structure key points include building corners, window edges, etc., to ensure the integrity of building structure. The number of triangular facets in the simplified model is reduced by 40%-60%, which reduces the amount of computation while ensuring that the core features of terrain and buildings are not lost.

[0046] Texture Enhancement: ESRGAN (Enhanced Super-Resolution Generative Adversarial Network) is used for super-resolution reconstruction of texture data, focusing on improving the resolution of building facade and roof textures to 4K level. Simultaneously, a texture error calibration algorithm is used to control the matching error between textures and 3D meshes to within 2%. The texture error calculation formula is as follows:

[0047] (9)

[0048] in, These are the pixel values ​​of the actual building texture. These are the pixel values ​​of the building texture in the model.

[0049] Lighting calibration: Combining the physics engine to simulate real lighting patterns, the light intensity and building shadow projection angle of the model are calibrated. The focus is on calibrating the light transmission effect of building windows and the reflection effect of walls to ensure that the lighting effect at different times and under different weather conditions conforms to the laws of physics.

[0050] The final output is a high-fidelity visual model with a rendering frame rate of no less than 60fps, containing high-precision building models. This model can be directly adapted to flight simulator devices. The specific format specifications are as follows: 1. Core model file format: adopts the FBX format commonly used in the flight simulator industry, containing integrated 3D mesh data of terrain and buildings. Mesh vertices are stored with float32 precision, and the coordinate system is the WGS-84 geographic coordinate system (adapting to flight positioning requirements); 2. Texture file format: building facades, roofs, and terrain textures are in PNG 24-bit lossless format. The super-resolution reconstructed 4K textures (4096×4096 pixels) are stored separately and associated with the FBX model through a texture mapping table, supporting dynamic switching of texture compression formats (ETC2, BC7); 3. Lighting data format: lighting parameters are stored in JSON format, including configurations such as solar altitude angle, light intensity, and shadow projection parameters at different times, which can be called and loaded in real time by visual simulation software.

[0051] The beneficial effects of this invention are as follows:

[0052] 1) Significantly improved modeling efficiency: Through automated preprocessing of multi-source data, end-to-end modeling of convolutional neural networks, and a dedicated building structure extraction submodule, synchronous automated modeling of terrain and buildings is achieved, reducing the traditional manual modeling work of several weeks to several hours, improving modeling efficiency by more than 80%, which is especially suitable for large-scale and complex flight simulator visual modeling needs that include buildings.

[0053] 2) Improved model accuracy and consistency: SIFT feature matching is used to achieve accurate data registration. Combined with multi-scale convolutional structure and improved Mask R-CNN to extract fine building features, the terrain geometry accuracy error is controlled within 5% and the building size accuracy error is controlled within 3%. The automated modeling process avoids the subjectivity of manual operation and ensures the consistency of the same type of visual model and building model.

[0054] 3) Balancing visual effects and real-time performance: By using reinforcement learning to achieve adaptive optimization of the network structure, combined with targeted mesh simplification and texture enhancement techniques, the model can maintain 4K texture accuracy and architectural detail integrity while keeping the rendering frame rate stable at over 60fps, meeting the real-time interactive requirements of flight simulators.

[0055] 4) Strong generalization ability: Through the exploration mechanism of data augmentation and reinforcement learning, the model can adapt to the visual modeling needs of different terrains (mountains, plains, water), different architectural styles (modern, traditional), and different weather (rain, snow, fog), and its generalization ability is significantly better than that of traditional methods. Attached Figure Description

[0056] Figure 1This is an overall flowchart of the simulator visual automation modeling method based on convolutional neural networks according to the present invention.

[0057] Figure 2 This is a schematic diagram of the modular structure of the visual convolutional neural network framework of the present invention.

[0058] Figure 3 This is a comparison diagram of the modeling effects of the mountain-building hybrid scene visual model in an embodiment of the present invention.

[0059] Figure 4 This is a schematic diagram of the reinforcement learning interaction for adaptive optimization of the structure of the present invention.

[0060] Figure 5 This is a schematic diagram of the building structure extraction effect in an embodiment of the present invention. Detailed Implementation

[0061] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments.

[0062] This embodiment takes the visual modeling of a mountainous-rural building mixed scene of a flight simulator as an example to explain in detail the implementation process of the present invention. The target scene is a mountainous rural area at 30° North latitude, which includes 20 traditional Chinese sloping roof buildings and 5 modern bungalow buildings, and is used for the visual construction of low-altitude flight training of the flight simulator.

[0063] Step 1: Visual data preprocessing.

[0064] Acquire multi-source raw data for the target area: ① Satellite remote sensing imagery, resolution 0.5m, containing complete mountain terrain and building distribution; ② DEM elevation data, accuracy 1m, covering an area of ​​5km×5km; ③ Texture data of ground features, including brick patterns on the facades of traditional Chinese buildings, tile patterns on pitched roofs, and textures on the cement walls of modern bungalows, all with a resolution of 2048×2048 pixels; ④ Illumination environmental parameters, hourly solar altitude angle data from 7:00 to 18:00 in summer at 30°N latitude, ranging from 25° to 82°.

[0065] Data preprocessing operations: ① Outlier removal: Calculate the mean (mu=850m) and standard deviation (sigma=32m) of the DEM elevation data. Four elevation abrupt change points (1020m, 680m, 1050m, and 650m respectively) with D_M(x) > 3 sigma are selected using the Mahalanobis distance formula and removed. ② Data registration: Extract 1500 feature points (including 800 building outline corner points) from the image and DEM data using the SIFT algorithm. Euclidean distance is used for matching, resulting in 1380 effective matching points, achieving a 92% success rate. This ensures precise spatial alignment between the image and terrain data, with a building outline / terrain position deviation ≤0.3m. ③ Resolution Unification: Satellite remote sensing images were scaled to 2048×2048 pixels, DEM data was resampled to a 5m×5m grid precision, and building textures were uniformly scaled to 4096×4096 pixels; ④ Normalization: DEM data was mapped to the [0,1000] interval (original elevation range 600m-1200m) using an elevation normalization formula, and texture data was mapped to the [0,255] interval, finally obtaining a standardized visual dataset containing the features of 25 buildings, with a data volume of 68GB.

[0066] Step 2: Visual Network Initialization

[0067] A dedicated convolutional neural network framework was constructed, with the following module composition and parameter configuration: ① Data Input Layer: A 5-channel input structure was adopted, accepting normalized DEM data, satellite remote sensing imagery, building texture data, non-building texture data, and illumination parameters; ② Terrain Feature Extraction Module: Three parallel convolutional branches were set up, using 11×11, 5×5, and 3×3 convolutional kernels respectively, with 64, 128, and 256 kernels respectively, a stride of 1, and Same padding method; ③ Building Structure Extraction Submodule: Based on an improved Mask R-CNN network, the backbone adopted ResNet-50, and a new unit for calculating building floor height was added, using the formula:

[0068] (10)

[0069] Calculate building height (e.g., at 7:00 AM, the solar altitude angle is 25°, and the shadow length of a Chinese-style building is 12m, the calculated building height is H = 12 × tan25° ≈ 5.6m, corresponding to a 2-story building); ④ Ground feature recognition module: Based on the ResNet-18 network, it identifies three types of non-building ground features: vegetation, roads, and rocks; ⑤ Lighting rendering fusion module: Embeds channel attention mechanism and spatial attention mechanism, and sets the lighting calculation factor to 0.75 (adapting to the lighting reflection characteristics of mountain vegetation and buildings); ⑥ Output layer: Outputs a 1024×1024 grid 3D model, where the grid precision of the building area is 0.5m × 0.5m, and the grid precision of the terrain area is 5m × 5m.

[0070] Step 3: Adaptive structural optimization.

[0071] The DDPG (Deep Deterministic Policy Gradient) reinforcement learning algorithm was used as the surrogate model. The parameters were set as follows: the initial exploration noise standard deviation was 0.2, which was reduced by 0.01 every 100 iterations until it was reduced to 0.01. The total number of exploration iterations was 800.

[0072] Reward function weight coefficients The initial reward value R = 0.62;

[0073] Optimization Process and Results: Through an exploration-utilization strategy, the network structure underwent three rounds of core adjustments: ① Terrain Feature Extraction Module: Due to the mountainous terrain with significant undulations in the target scene, two 11×11 convolutional layers were added to enhance the extraction capability of macroscopic undulation features; ② Building Structure Extraction Submodule: Addressing the complex roof outlines of traditional Chinese buildings, two 5×5 convolutional kernels were added, increasing the total number of kernels to 192, thus enhancing the accuracy of roof outline extraction; For the flat walls of modern bungalows, the parameters of the 3×3 convolutional kernels were optimized, and the stride was adjusted to 2; ③ Lighting Rendering Blending Module: In summer scenes with strong light, the attention mechanism was moved forward to focus on strengthening the extraction of building shadow features. After optimization, the reward value R increased to 0.89, with PSNR=38.2dB, SSIM=0.96, rendering frame rate=68fps, and building size accuracy error=2.1%.

[0074] Step 4: Visual model training.

[0075] Dataset partitioning: The 68GB standardized visual dataset was divided into a training set (47.6GB), a validation set (13.6GB), and a test set (6.8GB) in a ratio of 7:2:1. The training set contains data from 17 buildings, the validation set contains data from 5 buildings, and the test set contains data from 3 buildings.

[0076] Training parameter configuration: The improved AdamW algorithm is adopted, the data volume is 68GB > 50GB, the initial learning rate is set to 2e-3; the weight decay coefficient is 5e-5; the GPU used is NVIDIA RTX5090 (24GB VRAM), the batch size is set to 16, and each training session contains complete data of 20 terrain units (each unit is 250m×250m) and 10 building units; the total number of training iterations is set to 5000, and a validation set evaluation is performed every 50 iterations.

[0077] Early stopping mechanism triggering process: During training up to the 3500th iteration, performance monitoring of the validation set begins. From iterations 3500 to 3570 (8 consecutive iterations), the visual realism score improvement on the validation set is 0.21, 0.18, 0.25, 0.15, 0.23, 0.19, 0.22, and 0.17 respectively, all below 0.3. The rendering frame rate fluctuations are 6.2fps, 5.8fps, 7.1fps, 6.5fps, 5.9fps, 6.8fps, 7.3fps, and 6.4fps, all exceeding 5fps. Simultaneously, the PSNR improvement is 0.32dB... The values ​​were 0.28dB, 0.35dB, 0.25dB, 0.31dB, 0.29dB, 0.33dB, and 0.27dB, all below 0.5dB; the SSIM improvements were 0.008, 0.006, 0.007, 0.005, 0.009, 0.006, 0.007, and 0.005, all below 0.01; the training set rendering time was 28ms, the validation set rendering time was 42ms, and the difference was 14ms > 10ms, which met the early stopping mechanism trigger condition. Training was immediately stopped, the optimal model parameters of 3499 iterations were saved, and an initial visual model containing the 3D structure of 25 buildings was obtained.

[0078] Step 5: Optimize the visual model output

[0079] Model optimization details: ① Mesh simplification: Using an edge-folding algorithm, the curvature of terrain and building key points is calculated, retaining the faces corresponding to the top 30% of highly important key points; the number of terrain faces is reduced from 12 million to 4.8 million, a reduction of 60%; in the building area, key points such as corners and window edges are retained, and the number of faces is reduced from 5 million to 2.2 million, a reduction of 56%, achieving a 95% retention rate of building structural integrity; ② Texture enhancement: Using an ESRGAN network for super-resolution reconstruction of building textures, the resolution of traditional Chinese building tile and brick patterns and modern bungalow wall textures is increased to 4K level. Texture error is calculated using the texture error formula. E_{tex}=3.2\), controlled within 2%; ③ Lighting calibration: Combine the physics engine (UnityPhysics) to calibrate the lighting effect, focusing on adjusting the building shadow projection angle and length at different times from 7:00 to 18:00. For example, at 12:00, the sun altitude angle is 82°, and the calculated value of the shadow length of a 3-story building (height 12m) is 12×tan(90°-82°)=12×0.1405≈1.69m. After calibration, the model renders a shadow length of 1.71m, with an error of ≤1.2%; Adjust the light transmittance of the building windows to 85%, and the wall reflectance coefficient to 0.3 (cement wall) and 0.25 (brick wall).

[0080] Final output: A high-fidelity flight simulator visual model, including complete mountain terrain and high-precision buildings. The model renders at a frame rate of 75fps, meeting the real-time rendering requirements of flight simulators (≥60fps). Visual realism is achieved with a PSNR of 39.5dB and an SSIM of 0.97. The terrain geometry accuracy error is 2.8%, and the building dimension accuracy error is 1.9%. It can be directly used for low-altitude flight training scenarios in flight simulators. The output file package consists of: ① The main model file (model.fbx format, WGS-84 coordinate system, containing 17 million triangles); ② The texture file directory (textures folder, containing 4K resolution PNG textures); ③ The lighting configuration file (light_config.json).

Claims

1. A method for automated modeling of simulator visuals based on convolutional neural networks, characterized in that, Includes the following steps: Step 1: Visual data preprocessing. Obtain raw data including scene image data, terrain elevation data, ground texture data and lighting environment parameters. After outlier removal, data registration, resolution unification and normalization, a standardized visual dataset is obtained. Step 2: Initialize the visual network and construct a convolutional neural network framework containing a building structure extraction submodule. The framework includes a data input layer, a terrain feature extraction module, a building structure extraction submodule, a ground feature recognition module, a lighting rendering and fusion module, and a visual model output layer. Configure the initial network parameters based on the characteristics of the flight simulator. Step 3: Adaptive structural optimization. A reinforcement learning agent model is introduced, and the network structure parameters are adjusted based on the visual realism of the scene model, the real-time rendering frame rate, and the geometric accuracy as a comprehensive reward signal. Step 4: Visual model training. The standardized visual dataset is divided into training, validation and test sets. The improved AdamW algorithm is used to train the network. An early stopping mechanism is set to avoid overfitting, and the initial visual model is obtained. Step 5: Optimize the visual model output. The initial visual model is processed through mesh simplification, texture enhancement and lighting calibration to output a high-fidelity visual model with a rendering frame rate of no less than 60fps.

2. The automated modeling method based on convolutional neural networks according to claim 1, characterized in that, In step one, outlier removal uses the Mahalanobis distance method, calculated as follows: （1） in, This is a sample of terrain elevation data. The average elevation data. Let be the inverse of the covariance matrix of the elevation data, when When an outlier occurs, it is identified as an outlier and removed. Furthermore, according to the method described in claim 1, the data registration in step one uses the SIFT algorithm to extract feature points, and the similarity of feature point matching is determined by Euclidean distance. The Euclidean distance calculation formula is: （2） in Let n be the descriptor vector of two feature points, where n is the descriptor dimension. , This is a highly suitable match and is therefore considered a valid match.

3. The automated modeling method based on convolutional neural networks according to claim 1, characterized in that, In step two, the building structure extraction submodule is based on an improved Mask R-CNN network for building region segmentation and structural parameter extraction. The building floor height is calculated using the following formula: （3） in, For building floor height, The length of the building's shadow in the image. This represents the solar altitude angle for the corresponding time period.

4. The method according to claim 1, characterized in that, In step two, the terrain feature extraction module uses three parallel convolutional branches with 11×11, 5×5, and 3×3 convolutional kernels, respectively. The number of convolutional kernels is 64, 128, and 256, respectively, with a stride of 1 and a padding method of Same.

5. The automated modeling method based on convolutional neural networks according to claim 1, characterized in that, In step three, the reinforcement learning agent model uses the DDPG algorithm. The initial exploration noise standard deviation is 0.2, which linearly decays to 0.01 with the number of iterations. The overall reward function is: （4） in, Here are the weighting coefficients (all with a value of 0.25), and F is the rendering frame rate. These are the actual dimensions of the building. The building dimensions in the model guide the reinforcement learning agent model to adjust the network structure through an "explore-exploit" strategy.

6. The automated modeling method based on convolutional neural networks according to claim 1, characterized in that, In step four, the parameters of the improved AdamW algorithm are configured as follows: the initial learning rate is dynamically set according to the size of the scene data, which is 2e-3 when the data size is >50GB, 1e-3 when the data size is 20GB≤50GB, and 5e-4 when the data size is <20GB; the weight decay coefficient is 5e-5; and the batch size is dynamically adjusted according to the GPU memory.

7. The method according to claim 1, characterized in that, The early stopping mechanism in step 4 is triggered when the PSNR improvement of the validation set is less than 0.5dB and the SSIM improvement is less than 0.01 for 8 consecutive iterations, and the difference in rendering time between the training set and the validation set exceeds 10ms.

8. The automated modeling method based on convolutional neural networks according to claim 1, characterized in that, In step five, mesh simplification uses an edge-folding algorithm based on geometric features to retain the faces corresponding to the top 30% of the importance scores of the terrain and building key points; texture enhancement uses the ESRGAN network to improve the texture resolution to 4K level, and the texture error is controlled within 2%.

9. The method according to claim 1, characterized in that, It also includes a visual model adaptation step: obtaining flight simulator hardware configuration information through a hardware detection unit, automatically adjusting model parameters according to the configuration level, enabling full-precision textures and real-time global illumination for high-configuration devices, and enabling texture compression and baked lighting for medium and low-configuration devices.

10. An automated modeling system for flight simulator visual models implementing the method of any one of claims 1-10, characterized in that, It includes a visual data preprocessing module, a visual network initialization module, a structure optimization module, a visual training module, a visual optimization module, and an adaptation module. The visual network initialization module includes a building structure extraction sub-unit, the structure optimization module includes a DDPG proxy unit and a performance evaluation unit, and the visual optimization module includes a mesh simplification unit, a texture enhancement unit, and a lighting calibration unit. Each module achieves data interaction through a high-speed data bus.

Images