Real scene three-dimensional automatic modeling method and system based on unmanned aerial vehicle

Abstract

The invention relates to the field of three-dimensional modeling, in particular to a live-action three-dimensional automatic modeling method and system based on an unmanned aerial vehicle. The method comprises the following steps: acquiring an aerial video of an unmanned aerial vehicle; motion blur convolution suppression processing and panoramic geometric stitching are performed on the unmanned aerial vehicle aerial video, so that a panoramic stitching reconstruction image is constructed; pixel-by-pixel deep semantic recognition is carried out on the panoramic splicing reconstruction image, so that an urban building area and a natural environment area are generated; performing dynamic light rendering evolution and building physical morphological structure analysis on the urban building area, and constructing a three-dimensional urban building rendering model; and carrying out vegetation type fine-grained identification on the natural environment region, carrying out three-dimensional panoramic fusion according to the three-dimensional city building rendering model, carrying out fuzzy region detection on the three-dimensional panoramic model, carrying out low-altitude secondary aerial photography, and extracting a low-altitude high-definition video of the fuzzy region. According to the invention, through clear and accurate three-dimensional modeling, the visual sense of reality and the spatial restoration degree of the live-action model are improved.

Description

Technical Field

[0001] The present invention relates to the field of three-dimensional modeling, and in particular to a real-scene three-dimensional automated modeling method and system based on an unmanned aerial vehicle. Background Art

[0002] With the continuous development of science and technology and the growing demand for industrial applications, 3D modeling technology has been widely used in various fields, especially in urban planning, architectural design, archaeological site reconstruction, natural landscape analysis, etc. Traditional 3D modeling methods usually rely on ground measurement, manual photography and complex computer-aided design software. These methods are not only cumbersome to operate, but also have poor adaptability to the environment. Especially in the modeling of large-scale complex scenes, there is a high time cost and personnel investment.

[0003] In recent years, with the rapid development of drone technology, drone aerial photography, as a new means of data acquisition, has been widely used in 3D modeling of various practical scenes. By using drones equipped with high-definition cameras or laser radar (LiDAR) sensors, large-scale, high-precision images and point cloud data can be quickly acquired, greatly improving the efficiency and accuracy of modeling. However, traditional drone-based 3D modeling methods still face some challenges, especially in complex environments. How to effectively process massive data, identify different scene elements, and perform accurate modeling is still a difficulty.

[0004] In addition, most existing drone 3D modeling technologies rely on human intervention and manual post-processing, and have problems such as low data processing efficiency, low accuracy, and insufficient automation. For some dynamic scenes or detailed areas that are difficult to obtain, traditional modeling methods often have difficulty providing accurate 3D data. In order to cope with these problems, modern industry and research fields are in urgent need of a more intelligent and efficient drone real-scene 3D automated modeling method. Summary of the invention

[0005] In order to solve the above technical problems, the present invention proposes a real-scene three-dimensional automatic modeling method and system based on unmanned aerial vehicle to solve at least one of the above technical problems.

[0006] To achieve the above object, the present invention provides a real-scene three-dimensional automatic modeling method based on a drone, comprising the following steps: Step S1: obtaining drone aerial video; performing motion blur convolution suppression processing and panoramic geometric stitching on the drone aerial video, thereby constructing a panoramic stitching reconstruction image; Step S2: performing pixel-by-pixel depth semantic recognition on the panoramic stitching and reconstruction image, thereby generating an urban building area and a natural environment area; Step S3: Perform dynamic light rendering evolution and building physical form structure analysis on the urban building area to construct a three-dimensional urban building rendering model; Step S4: performing fine-grained identification of vegetation types in the natural environment area, and performing three-dimensional panoramic fusion according to the three-dimensional urban building rendering model to construct a three-dimensional panoramic model; Step S5: Detect blurry areas of the three-dimensional panoramic model, perform secondary low-altitude aerial photography, and extract low-altitude high-definition video of the blurry areas; Step S6: Optimize the local details of the blurred area based on the low-altitude high-definition video of the blurred area, so as to construct a three-dimensional real scene optimization model.

[0007] The present invention uses motion blur convolution suppression processing, and the system can eliminate the negative effects of blur in real time, improve the clarity of the video, and enhance the accuracy of subsequent image analysis and modeling. Panoramic geometric stitching technology generates a seamless and complete panorama by aligning and synthesizing multiple aerial video frames. This not only avoids the problem of multi-perspective inconsistency in traditional shooting, but also ensures that data from different shooting angles can complement each other, providing broader and more accurate scene information, laying the foundation for subsequent semantic analysis and three-dimensional modeling. Through pixel-by-pixel deep semantic recognition, the system can accurately distinguish and mark urban buildings and natural environment areas in the image. Using a deep learning model for pixel-level semantic segmentation, different elements such as buildings, roads, and vegetation can be extracted in different scenes, thereby providing reliable semantic information for building a high-precision three-dimensional model. The identified urban building area and natural environment area provide clear boundary divisions for subsequent steps, so that the modeling and processing of different areas adopt the most suitable algorithms and technologies. Fine-grained modeling can be used for building areas, while terrain analysis and vegetation recognition are performed for natural environment areas. Through these precise regional divisions, the modeling system can efficiently perform local optimization and integration in subsequent stages. Changes in lighting and shadows have a profound impact on the visual effects of buildings. Dynamic light rendering evolution technology can simulate lighting effects in different time periods and weather conditions, generating realistic shadow projection and light reflection effects. Through this dynamic lighting change, the 3D city model can be more vivid and realistic, providing high-quality visual performance. By analyzing the physical structure of the building (such as wall thickness, roof shape, window position, etc.), the system extracts the geometric features of the building and accurately models it. This process not only ensures the authenticity of the building's appearance, but also accurately reflects the actual structure and spatial layout of the building in the virtual environment. Vegetation in natural environment areas is usually diverse and widely distributed. Through fine-grained vegetation recognition technology, the system can accurately identify different types of vegetation (such as trees, shrubs, lawns, etc.) and analyze their coverage area and distribution. This helps to build a more realistic natural environment model and provide data support for subsequent landscape design, ecological assessment and other applications. By integrating the 3D models of urban building areas with natural environment areas, the system can create a comprehensive and accurate 3D panoramic scene. The seamless integration of urban buildings and the natural environment not only improves the visual effect of the model, but also enhances the spatial sense and layering of the scene, truly realizing the natural presentation of the virtual world. Even high-resolution aerial videos can still have blurred areas in some cases, affecting the accuracy of 3D modeling. Through blurred area detection technology, the system can automatically identify and mark blurred parts in the image. This technology not only improves the accuracy of modeling, but also reduces the need for later manual intervention. For the detected blurred areas, the system uses a drone to fly at low altitude for a second shot to obtain high-definition local detail images.This process not only eliminates blur and improves detail clarity, but also ensures that every detail in the 3D model is accurately restored, especially in the complex facades of buildings or details in natural environments. Through the high-definition images captured by low-altitude video, the system can optimize the details of blurred areas. Using image enhancement techniques (such as super-resolution reconstruction, texture detail enhancement, etc.), local details are finely processed to make the texture of the 3D model clearer and more realistic. The optimized model effectively improves the user experience, especially in applications such as virtual reality (VR) or augmented reality (AR), where detail optimization is crucial. By combining the local details optimized by low-altitude video, the constructed 3D real-scene optimized model will be more accurate and can present high-quality visual effects in various scenes. This optimization makes the final 3D scene model not only geometrically accurate, but also more realistic in light and shadow effects and material surfaces, and can perfectly restore the actual environment.

[0008] In this specification, a real-scene 3D automatic modeling system based on a drone is provided, which is used to execute the real-scene 3D automatic modeling method based on a drone as described above, including: The geometric stitching module is used to obtain drone aerial videos; perform motion blur convolution suppression and panoramic geometric stitching on drone aerial videos to construct a panoramic stitching reconstruction image; The deep semantic recognition module is used to perform pixel-by-pixel deep semantic recognition on the panoramic stitching and reconstruction image, thereby generating urban building areas and natural environment areas; Light rendering evolution module, used to perform dynamic light rendering evolution and building physical form structure analysis on urban building areas, and build a three-dimensional urban building rendering model; The 3D panoramic fusion module is used to identify the vegetation types in the natural environment area in fine granularity and perform 3D panoramic fusion based on the 3D urban building rendering model to construct a 3D panoramic model. The blur area detection module is used to detect blur areas in the 3D panoramic model and perform secondary low-altitude aerial photography to extract low-altitude high-definition video of the blur area. The local detail optimization module is used to optimize the local details of the blurred area based on the low-altitude high-definition video of the blurred area, so as to construct a three-dimensional real-scene optimization model.

[0009] The present invention can effectively remove the blur effect, restore image details, and ensure that each frame of the image has a high-quality visual effect by using a motion blur convolution suppression algorithm (such as a blind deblurring algorithm). This is particularly important for subsequent three-dimensional modeling, because any degradation in image quality will affect feature extraction and model accuracy. Through panoramic geometric stitching technology, aerial images from multiple perspectives can be accurately stitched into a complete panorama. Geometric stitching ensures that the perspective distortion between different images is minimized and effectively synthesizes a seamless large-scale view, thereby providing a high-precision image basis for subsequent deep semantic recognition and three-dimensional reconstruction. Through deep learning networks (such as convolutional neural networks, Transformers, etc.) for pixel-by-pixel deep semantic recognition, each element in the image can be classified and labeled at the pixel level. This not only helps to accurately identify elements such as buildings, roads, and vegetation, but also distinguishes the boundaries between natural environment areas and urban construction areas based on environmental complexity, thereby improving the accuracy of subsequent modeling. Use physical lighting and dynamic rendering algorithms (such as ray tracing or physically based rendering PBR technology) to simulate light changes at different times and angles. Accurately reproduce the effects of light on buildings, such as light reflection, shadow changes, etc., which provides a sense of reality for the visual presentation of buildings under different lighting conditions. By simulating the evolution of light, the final building model is more vivid and natural in the real environment. Analyzing the physical structure of the building (such as wall thickness, window frames, roof structure, etc.) can build a more accurate 3D building model, avoiding the limitations of models based solely on plane drawings. This not only improves the modeling accuracy, but also enhances the realism of the building, providing high-quality support for urban planning and virtual reality applications. Fine-grained classification and identification of vegetation types in the natural environment through deep learning algorithms (such as FCN, MaskR-CNN, etc.). This process can accurately identify different types of plants, trees, etc., and even identify the growth status of plants (such as healthy, withered, etc.), providing strong data support for accurate modeling of the natural environment. Combining the modeling results of the urban building area and the natural environment area, the two are integrated into a whole through 3D panoramic fusion technology (such as seamless fusion of lighting, materials and geometric models). In this way, the final constructed 3D panoramic model not only shows the architectural appearance of the city, but also accurately reproduces the details of the surrounding natural environment, enhancing the realism and immersion of the entire scene. Through image processing technology, it is possible to identify blurred areas caused by distance, movement and other reasons during the aerial photography process. This detection step is very important because blurred areas will affect the accuracy and integrity of modeling. Timely discovery and marking of these areas provide guidance for subsequent optimization. Once the blurred areas are found, low-altitude secondary aerial photography can be used for high-definition acquisition at a closer distance and in a more stable flight state.This can obtain clearer and higher-resolution video data, make up for the loss of details in the first aerial photography, and provide a high-quality data source for subsequent 3D modeling detail repair. Through deep learning detail recognition of low-altitude videos, important information such as texture details and structural features in the blurred area are extracted and applied to the optimization of the 3D model. This not only improves the accuracy of the blurred area, but also increases the delicacy and realism of the entire 3D model. Based on the optimized detail information, a high-precision, high-quality 3D real-scene optimization model is constructed. The 3D model with detailed repair has higher visual realism and spatial restoration, and can be applied to multiple fields, such as urban planning, virtual reality, architectural design, etc. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 A schematic diagram of the steps of a method for automatic three-dimensional modeling of a real scene based on a drone according to the present invention; Figure 2 Detailed implementation flow chart of step S1; Figure 3 Detailed implementation flow chart of step S2. DETAILED DESCRIPTION

[0011] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0012] The present application example provides a method and system for automated three-dimensional modeling of real scenes based on drones. The execution subjects of the method and system for automated three-dimensional modeling of real scenes based on drones include but are not limited to: mechanical equipment, data processing platform, cloud server nodes, network upload equipment, etc. equipped with the system can be regarded as general computing nodes of the present application, and the data processing platform includes but is not limited to: at least one of an audio and image management system, an information management system, and a cloud data management system.

[0013] See also Figures 1 to 3 The present invention provides a real-scene 3D automated modeling method based on an unmanned aerial vehicle, and the real-scene 3D automated modeling method based on an unmanned aerial vehicle comprises the following steps: Step S1: obtaining drone aerial video; performing motion blur convolution suppression processing and panoramic geometric stitching on the drone aerial video, thereby constructing a panoramic stitching reconstruction image; Step S2: performing pixel-by-pixel depth semantic recognition on the panoramic stitching and reconstruction image, thereby generating an urban building area and a natural environment area; Step S3: Perform dynamic light rendering evolution and building physical form structure analysis on the urban building area to construct a three-dimensional urban building rendering model; Step S4: performing fine-grained identification of vegetation types in the natural environment area, and performing three-dimensional panoramic fusion according to the three-dimensional urban building rendering model to construct a three-dimensional panoramic model; Step S5: Detect blurry areas of the three-dimensional panoramic model, perform secondary low-altitude aerial photography, and extract low-altitude high-definition video of the blurry areas; Step S6: Optimize the local details of the blurred area based on the low-altitude high-definition video of the blurred area, so as to construct a three-dimensional real scene optimization model.

[0014] The present invention uses motion blur convolution suppression processing, and the system can eliminate the negative effects of blur in real time, improve the clarity of the video, and enhance the accuracy of subsequent image analysis and modeling. Panoramic geometric stitching technology generates a seamless and complete panorama by aligning and synthesizing multiple aerial video frames. This not only avoids the problem of multi-perspective inconsistency in traditional shooting, but also ensures that data from different shooting angles can complement each other, providing broader and more accurate scene information, laying the foundation for subsequent semantic analysis and three-dimensional modeling. Through pixel-by-pixel deep semantic recognition, the system can accurately distinguish and mark urban buildings and natural environment areas in the image. Using a deep learning model for pixel-level semantic segmentation, different elements such as buildings, roads, and vegetation can be extracted in different scenes, thereby providing reliable semantic information for building a high-precision three-dimensional model. The identified urban building area and natural environment area provide clear boundary divisions for subsequent steps, so that the modeling and processing of different areas adopt the most suitable algorithms and technologies. Fine-grained modeling can be used for building areas, while terrain analysis and vegetation recognition are performed for natural environment areas. Through these precise regional divisions, the modeling system can efficiently perform local optimization and integration in subsequent stages. Changes in lighting and shadows have a profound impact on the visual effects of buildings. Dynamic light rendering evolution technology can simulate lighting effects in different time periods and weather conditions, generating realistic shadow projection and light reflection effects. Through this dynamic lighting change, the 3D city model can be more vivid and realistic, providing high-quality visual performance. By analyzing the physical structure of the building (such as wall thickness, roof shape, window position, etc.), the system extracts the geometric features of the building and accurately models it. This process not only ensures the authenticity of the building's appearance, but also accurately reflects the actual structure and spatial layout of the building in the virtual environment. Vegetation in natural environment areas is usually diverse and widely distributed. Through fine-grained vegetation recognition technology, the system can accurately identify different types of vegetation (such as trees, shrubs, lawns, etc.) and analyze their coverage area and distribution. This helps to build a more realistic natural environment model and provide data support for subsequent landscape design, ecological assessment and other applications. By integrating the 3D models of urban building areas with natural environment areas, the system can create a comprehensive and accurate 3D panoramic scene. The seamless integration of urban buildings and the natural environment not only improves the visual effect of the model, but also enhances the spatial sense and layering of the scene, truly realizing the natural presentation of the virtual world. Even high-resolution aerial videos can still have blurred areas in some cases, affecting the accuracy of 3D modeling. Through blurred area detection technology, the system can automatically identify and mark blurred parts in the image. This technology not only improves the accuracy of modeling, but also reduces the need for later manual intervention. For the detected blurred areas, the system uses a drone to fly at low altitude for a second shot to obtain high-definition local detail images.This process not only eliminates blur and improves detail clarity, but also ensures that every detail in the 3D model is accurately restored, especially in the complex facades of buildings or details in natural environments. Through the high-definition images captured by low-altitude video, the system can optimize the details of blurred areas. Using image enhancement techniques (such as super-resolution reconstruction, texture detail enhancement, etc.), local details are finely processed to make the texture of the 3D model clearer and more realistic. The optimized model effectively improves the user experience, especially in applications such as virtual reality (VR) or augmented reality (AR), where detail optimization is crucial. By combining the local details optimized by low-altitude video, the constructed 3D real-scene optimized model will be more accurate and can present high-quality visual effects in various scenes. This optimization makes the final 3D scene model not only geometrically accurate, but also more realistic in light and shadow effects and material surfaces, and can perfectly restore the actual environment.

[0015] In the embodiment of the present invention, refer to Figure 1 , is a schematic diagram of the steps of a method for automatic modeling of real-scene 3D based on drones of the present invention. In this example, the steps of the method include: Step S1: obtaining drone aerial video; performing motion blur convolution suppression processing and panoramic geometric stitching on the drone aerial video, thereby constructing a panoramic stitching reconstruction image; In this embodiment, a drone equipped with a high-definition camera is used to take aerial photos of the target area. The scope of the aerial photography area should be determined according to the subsequent modeling requirements. The flight route of the drone should ensure that it can cover all angles of the required area to obtain sufficient image data. During the flight, the drone is controlled to fly stably to ensure that the lens is always pointed at the target area, and rapid movement is avoided as much as possible to reduce the generation of motion blur. A deconvolution method based on image restoration is used to restore the image by calculating the blur kernel in the video frame. The specific method is to use frequency domain filtering or convolution inversion technology in the spatial domain to estimate the blur kernel (such as the direction and degree of motion blur) and apply deconvolution processing to restore the details of the image. This method can effectively eliminate motion blur from the image, so that the image clarity is significantly improved. For each frame of video, the blur is first analyzed in the frequency domain by an image processing algorithm (such as Fourier transform), and then a clear image is restored by a deconvolution algorithm. For motion blur, L2 regularization is used to optimize the deconvolution result and reduce noise and artifacts. Use a feature matching-based stitching algorithm (such as SIFT, SURF or ORB algorithm) to extract key feature points in the image for matching and transformation correction. First, extract feature points for each frame of the image, then use the RANSAC (random sampling consistency) algorithm to remove incorrect matching points and calculate the transformation matrix. Apply these transformation matrices to the image to achieve the geometric transformation of the image. Finally, stitching is performed through image fusion technology to ensure that the transition between images is natural and there are no obvious seams. The stitched images often have different exposure differences, which will cause color differences at the seams. To solve this problem, multi-exposure fusion (MEF) or image blending technology (such as multi-resolution fusion technology) is used to smooth the brightness and color differences between transition images to generate smooth and natural panoramas.

[0016] Step S2: performing pixel-by-pixel depth semantic recognition on the panoramic stitching and reconstruction image, thereby generating an urban building area and a natural environment area; In this embodiment, U-Net or DeepLabv3+ is selected as the algorithm for deep semantic segmentation. These models perform well in pixel-level classification and are suitable for processing urban buildings and natural environment areas in complex scenes. DeepLabv3+ is particularly good at capturing multi-scale features and improves the receptive field by introducing dilated convolution. Use high-quality semantic segmentation datasets, such as Cityscapes or ADE20K, to pre-train the model. These datasets contain rich urban and natural scene annotations and are suitable for model learning. Fine-tune on the selected dataset, use the Adam optimizer, set the learning rate to 0.0001, the batch size to 4, and the training cycle to 100 epochs. During the training process, monitor the loss and accuracy on the validation set to prevent overfitting. Preprocess the panoramic stitching reconstruction, including adjusting the image size (such as 512x512) to meet the input requirements of the model. Use the cv2.resize() function in OpenCV to scale the image. Normalize and scale the pixel values ​​to between [0, 1] to increase the convergence speed of the model. The preprocessed panoramic image is input into the trained deep learning model, and forward inference is performed to generate the class label for each pixel. The model outputs a probability map of the same size as the input image, with each pixel corresponding to the probability of a different category. Threshold the probability map, set the threshold to 0.5, and mark the pixels with a probability greater than this value as the corresponding category, such as buildings, roads, vegetation, etc. Visualize the recognition results, using different colors for each category. Use OpenCV's cv2.applyColorMap() function to map the category labels to a visual image. Save the processing results, generate a labeled image and output it as a file (such as PNG format) for subsequent analysis. Count the number of pixels in each category (urban building area and natural environment area) in the panoramic mosaic to quantify the proportion of different areas. Use NumPy's np.unique() function to achieve this. Record the analysis results and generate a report containing information such as the pixel proportion of each area and the area of ​​the area.

[0017] Step S3: Perform dynamic light rendering evolution and building physical morphology structure analysis on the urban building area to construct a three-dimensional urban building rendering model; In this embodiment, Unity or Unreal Engine is selected as a three-dimensional rendering engine. These engines support high-quality dynamic lighting and real-time rendering, and are suitable for building complex urban building scenes. Unreal Engine's Lumen lighting system can achieve global illumination and dynamic reflection, which is suitable for analyzing the changes in ambient lighting of buildings. Set multiple light sources in the scene, including directional light (simulating sunlight), point light sources, and spotlights, and adjust the intensity (such as Intensity = 3.0) and color of the light source to simulate the lighting effects at different time periods. Set the attenuation range of the light source to ensure that the lighting covers the entire building area and simulate the lighting changes in the real world. Through drone aerial video, collect lighting data of urban building areas at different time periods, and calculate the reflection and shadow effects of each building under different lighting conditions. Use image processing technology to analyze the impact of light intensity and direction on the surface of the building, and generate time series data of lighting changes. Create a three-dimensional model of the building in the rendering engine, and apply the materials and textures derived from the lighting analysis. Use a dynamic lighting system to adjust the angle and intensity of the light source in real time to simulate the evolution of lighting. Record the rendering results at each time point to generate a lighting animation. Record the dynamic light rendering process as a video file and compress it using a codec such as H.264 to ensure a balance between picture quality and file size. Output and visualize the lighting effects of different time periods to facilitate observation of the performance of the building under changing lighting. Use Blender or Maya to analyze the physical form of the building. These tools support accurate measurement and analysis of 3D models and are suitable for extracting the structural features of the building. In the analysis tool, measure various parts of the building, including wall thickness, opening size, top structure, etc. Set the measurement accuracy to 0.01 meters to ensure data accuracy. Analyze the building layer by layer, and extract structural parameters such as wall thickness, window height, and number of floors from the bottom to the top. Record the data value of each parameter. Use automated scripts (such as Python scripts) to batch extract data in Blender to improve efficiency. Combine the dynamic light rendering results with the physical form structure data of the building to generate a complete 3D urban building rendering model. Use Unity or Unreal Engine to integrate all elements together. Make final lighting and material adjustments in the rendering engine to ensure that the generated model is as close to the real world as possible. Export complete 3D city building rendering models to shareable 3D file formats such as FBX or OBJ for use and presentation on other platforms.

[0018] Step S4: performing fine-grained identification of vegetation types in the natural environment area, and performing three-dimensional panoramic fusion according to the three-dimensional urban building rendering model to construct a three-dimensional panoramic model; In this embodiment, ResNet50 or EfficientNet is selected as a model for fine-grained vegetation type recognition. These models perform well in image classification tasks and are suitable for processing complex natural environment images. ResNet50 can effectively learn deep features by introducing residual connections, which are used to identify different types of vegetation, such as trees, shrubs, lawns, etc. The model is pre-trained using public datasets containing multiple vegetation types, such as PlantNet or Oxford Pets. These datasets contain a large number of labeled samples and are suitable for fine-grained classification. Fine-tune on the selected dataset, use the Adam optimizer, set the learning rate to 0.0001, the batch size to 16, and the training cycle to 50 epochs. During the training process, monitor the loss and accuracy on the validation set to ensure the generalization ability of the model. Preprocess the natural environment area images, including adjusting the image size (such as 224x224) to meet the input requirements of the model. Use OpenCV's cv2.resize() function for image scaling. Normalize the image and scale the pixel values ​​to [0, 1] to improve the convergence speed of the model. Input the preprocessed image into the trained model, perform forward reasoning, and generate the vegetation type label for each pixel. The model outputs a probability map of the same size as the input image, with each pixel corresponding to the probability of a different vegetation type. Threshold the probability map, set the threshold to 0.6, and mark the pixels with a probability greater than this value as the corresponding vegetation type. Visualize the recognition results, and mark each vegetation type with a different color. Use OpenCV's cv2.applyColorMap() function to map the category label to a visualization image. Save the processing results, generate a labeled image, and output it as a file (such as PNG format) for subsequent analysis. Select point cloud fusion and voxel fusion technology to fuse the fine-grained recognized vegetation information with the 3D urban building rendering model to generate a complete 3D panoramic model. Use Open3D or PCL (Point Cloud Library) for point cloud processing, which supports efficient point cloud data manipulation and visualization. Integrate the identified vegetation area with the data of the 3D urban building rendering model to ensure that the coordinate systems of the two are consistent and avoid positional misalignment. Generate a 3D model of the vegetation area and use relevant algorithms (such as Delaunay Triangulation) to convert the planar vegetation information into a 3D model. Use the create_from_point_cloud function in Open3D to convert the point cloud data of urban buildings and vegetation into a 3D model. Set the normal calculation parameters of the point cloud to improve the accuracy of the model details. Downsample and filter the point cloud data, use the voxel grid method (Voxel Grid) for noise reduction, and set the voxel size to 0.05 meters to ensure the smoothness of the point cloud.In a rendering engine (such as Unity or Unreal Engine), the fused 3D model is rendered in real time to ensure the realism of lighting and materials. The final 3D panoramic model is exported to a shareable 3D file format (such as FBX or OBJ) for use and display on other platforms.

[0019] Step S5: Detect blurry areas of the three-dimensional panoramic model, perform secondary low-altitude aerial photography, and extract low-altitude high-definition video of the blurry areas; In this embodiment, a detection algorithm based on image sharpness is selected. Commonly used detection indicators include Laplace transform and Sobel operator, which can effectively evaluate the clarity of the image. Laplace transform calculates the second-order derivative of the image, and the transformation result of the blurred image is usually small, so the blurriness of the image is judged by setting a threshold. During the detection process, the blur threshold is set to 100 (the specific threshold can be adjusted according to the actual situation). If the result of the Laplace transform is lower than this value, the area is considered to be blurred. Load the image data generated by the three-dimensional panoramic model, and use the cv2.imread() function of OpenCV to read the image. Grayscale the image and use the cv2.cvtColor() function to convert the image into a grayscale image. Apply the Laplace transform and use the cv2.Laplacian() function to calculate the sharpness of the image. Compare the result with the set blur threshold to identify the blurred area. Mark the detected blurred area with a border or color, and use the cv2.rectangle() function to draw a rectangular box on the original image to mark the position of the blurred area. Save the marked image for subsequent analysis and reference. Determine the aerial photography altitude and flight path, and set the flight altitude to 30 meters to ensure that the details of the blurred area can be clearly captured. Plan the aerial photography route to ensure that all blurred areas are covered, and use a grid flight method so that each blurred area can be fully photographed. Choose a drone equipped with a high-resolution camera, such as the DJI Mavic 3, which can shoot 4K (3840x2160 pixels) high-definition video to ensure video quality. Set the camera parameters and choose an appropriate shutter speed (such as 1 / 1000 seconds) and ISO value (such as 100) to adapt to different lighting conditions and ensure image clarity. According to the planned aerial photography path, start the drone for low-altitude aerial photography, monitor the flight status and shooting effect in real time, and ensure that the drone flies at a safe altitude. Take multiple shots above each blurred area to ensure that high-definition video data from multiple angles and perspectives is obtained. Save the high-definition video data generated during the aerial photography process to the drone's memory card to ensure data backup to prevent loss. Ensure that the shooting time of each blurred area is not less than 10 seconds for subsequent analysis and processing. Import the aerial video for editing using video processing software (such as Adobe Premiere Pro or Final Cut Pro). Cut out the video clips of the blurred area, ensuring that each clip has the same duration for subsequent comparison and analysis. Export the processed low-altitude HD video and select an appropriate encoding format (such as H.264) to ensure that the video quality and file size are moderate. Save it in a standard format (such as MP4) for subsequent display and analysis.

[0020] Step S6: Optimize the local details of the blurred area based on the low-altitude high-definition video of the blurred area, so as to construct a three-dimensional real scene optimization model.

[0021] In this embodiment, a detail optimization method based on super-resolution reconstruction technology is selected, such as SRCNN (Super-Resolution Convolutional Neural Network) or GAN (Generative Adversarial Network), which can effectively improve the details and clarity of blurred videos. SRCNN can restore more details in blurred areas by learning the mapping relationship between low-resolution images and high-resolution images. Use a public dataset containing rich architectural and vegetation details (such as Set5 or DIV2K) to pre-train the model, set the learning rate to 0.0001, the batch size to 32, and the training cycle to 100 epochs. Set the super-resolution output size to twice the original video to increase the clarity of the details. Use OpenCV or FFmpeg to load low-altitude high-definition video in the blurred area to ensure frame-by-frame processing of the video frames. Extract the video frames as image sequences, use the cv2.VideoCapture() function to read the video, and call cv2.imwrite() to save each frame as an image file. Preprocess each extracted image frame, including resizing it to meet the input requirements of the super-resolution model (e.g., 33x33) and normalizing it to ensure that the pixel values ​​are between [0, 1]. Input each preprocessed image frame into the trained super-resolution model, perform forward inference, and generate a high-resolution output image. Record the super-resolution image of each frame and save the processed image using the cv2.imwrite() function. Synthesize all high-resolution images into a video using FFmpeg or OpenCV's cv2.VideoWriter() function for video synthesis, setting the output frame rate to 30FPS to ensure smooth video. Select MP4 as the output video format to ensure compatibility and quality. Generate point cloud data using the optimized high-resolution image and select the Open3D library for point cloud processing. Use the create_from_rgbd_image() function to combine the high-resolution image with the depth information to generate a point cloud. Ensure that the density of the point cloud is high enough to capture the details of buildings and vegetation, and set the voxel size of the point cloud to 0.01 meters to improve the fineness of the model. Mesh the point cloud and generate a 3D mesh model using the Poisson Surface Reconstruction method. Set the reconstruction depth to 9 to ensure the detail and smoothness of the mesh. Optimize the details of the generated 3D model and apply texture mapping to improve the visual effect of the model and ensure the realism of the buildings and vegetation. Export the optimized 3D real scene model to a common 3D file format (such as OBJ or FBX) for use and display on other platforms. Make sure to include texture and material information when exporting to maintain the integrity of the model.Use 3D visualization software (such as Blender or Sketchfab) to load the exported 3D model for display and interaction, and ensure that the model performs well from different perspectives. Record the visualization effect and generate a video or screenshots for subsequent presentations or reports.

[0022] In this embodiment, refer to Figure 2 , is a flowchart of detailed implementation steps of step S1. In this embodiment, the detailed implementation steps of step S1 include: Step S11: obtaining a drone aerial video; performing motion blur convolution suppression processing on the drone aerial video, thereby constructing a motion blur optimized video; Step S12: performing visual recognition of scene feature points frame by frame on the motion blur optimized video, marking multiple scene feature points in each frame; Step S13: performing dynamic inter-frame correlation matching based on multiple scene feature points in each frame to generate a geometric transformation relationship of the feature points; Step S14: mining the temporal continuity constraints of the geometric transformation relationship of the feature points to generate temporal continuity constraints between frames; Step S15: Perform panoramic geometric stitching on the motion blur optimized video based on the temporal continuity constraint between frames, and fuse the segmentation lines at the stitching points, so as to construct a panoramic stitching reconstruction image.

[0023] In this embodiment, a high-performance drone is selected with a 4K resolution camera that can capture high-quality images. When selecting a drone, consider its flight stability and wind resistance to reduce jitter and blur during shooting. Configure the drone's flight mode to "route flight" to ensure that the drone flies smoothly on the preset route and reduce blur caused by irregular flight. Choose the best shooting time, such as morning or dusk, to obtain good natural lighting conditions and avoid the impact of strong sunlight or shadows on image quality. Set the video frame rate to 30fps and the resolution to 3840x2160 (4K) to ensure that sufficient details and smooth pictures are captured. During the recording process, ensure that the drone flies at a reasonable speed (such as 2-3 meters per second) to avoid motion blur caused by too fast. Use Fourier transform to analyze the video frame and identify the motion blur therein. By converting the image to the frequency domain and observing the degree of loss of the high-frequency component, the direction and intensity of the blur are judged. Set the blur threshold. If the energy of the high-frequency component is lower than the set value (such as 0.1), it is determined that there is blur. Design the inverse motion blur convolution kernel and generate a suitable convolution kernel (such as a 5x5 or 7x7 box filter) based on the degree of blur. For complex blur situations, use a Gaussian filter and adjust the kernel parameters according to the direction and degree of blur. Apply the convolution kernel to each frame of the video and use the filter2D function in OpenCV to perform blur suppression. Ensure that the clarity of the image is significantly improved after the convolution operation. Record image quality indicators (such as PSNR and SSIM) before and after processing to ensure that the optimization effect is significant. Expect a PSNR value of at least 10dB improvement and an SSIM value close to 1 to indicate a significant improvement in image quality. Use the (Oriented FAST and Rotated BRIEF) algorithm for feature point detection. This algorithm is fast and rotationally invariant, and is suitable for processing dynamic scenes in the video. Configure the parameters of the ORB algorithm, set the number of feature points to 500, and the number of scale pyramid levels to 8 to ensure that enough feature points are detected at different scales. At the same time, set Harris corner detection as an auxiliary method to capture key locations. For each frame of the motion blur optimized video, execute the ORB feature point detection algorithm. In each frame, record the coordinates of the detected feature points and their descriptors. Set a minimum brightness threshold to ensure that the detected feature points have sufficient contrast, for example, set it to 10% brightness change. Draw the detected feature points on each frame image using OpenCV's drawKeypoints function. Feature points are marked as red circles for easy visualization and subsequent analysis. Generate a sequence of marked frames, record the number of marked feature points in each frame, and ensure that there are at least 10 valid feature points in each frame to ensure the reliability of subsequent matching. When choosing ORB, make sure that the algorithm can effectively handle feature points under different lighting and viewing conditions.Select FLANN (Fast Library for Approximate Nearest Neighbors) for feature point matching. FLANN is suitable for processing a large number of feature points and can quickly find the best match. Set the KNN (K-Nearest Neighbors) algorithm and set the k value to 2 to obtain the two nearest neighbors of each feature point for subsequent matching accuracy evaluation. Configure the parameters of the FLANN algorithm, set the distance metric (such as Euclidean distance), and set the matching distance threshold to 30 to ensure that only high-quality matching results are retained. Match the feature points of adjacent frames, use FLANN or KNN algorithm to compare the feature point descriptors of each frame, and record the matched feature point pairs. Set the matching distance threshold to filter out matches that do not meet the conditions to ensure the accuracy of the match. Based on the matched feature point pairs, use RANSAC (random sampling consensus algorithm) to calculate the geometric transformation relationship (such as homography matrix or basic matrix) between each pair of frames. Set the number of iterations to 1000 and the inlier threshold to 3 pixels to improve the robustness of the geometric transformation relationship calculation and ensure that the interference of the outlier pair calculation is eliminated. The Kalman filter is selected as the temporal constraint model. The Kalman filter is suitable for linear systems and can effectively predict the motion state of an object and establish a dynamic model. The Kalman filter can be used to smooth the motion trajectory of the feature points and improve the continuity between frames. The state transfer matrix and observation matrix are set for the Kalman filter, the initial state estimate is the initial position of the feature point, and the covariance matrix is ​​set to a relatively small value (such as 1.0) to increase the trust in the initial estimate. According to the geometric transformation relationship between the previous and next frames, the Kalman filter is used to calculate the motion trajectory of the feature points and extract the temporal continuity constraint. The position change of each feature point in the time series is recorded for subsequent analysis. The temporal data of each feature point is iteratively updated to ensure the validity of the continuity constraint. The extracted temporal continuity constraint is recorded in the data structure to ensure that each constraint record includes the feature point ID, timestamp and motion trajectory information. A threshold is set. If the constraint change exceeds the set range (such as 10 pixels), it is marked as an abnormality for subsequent processing. A multi-view geometric stitching algorithm is selected, which can integrate images from multiple perspectives to generate a panoramic image based on the extracted geometric transformation relationship. Ensure that the temporal continuity constraints of feature points are taken into account during the stitching process to enhance the accuracy of the stitching. Set the overlap area of ​​the stitching to 30% to ensure that enough feature points can be matched during the stitching process to generate a high-quality panorama. Perform panoramic geometric stitching on the motion blur optimized video based on the extracted temporal continuity constraints and geometric transformation relationships. Use OpenCV or a specific stitching library (such as OpenMVG) for processing. During the stitching process, ensure that the overlapping area of ​​each frame with the previous frame can transition smoothly to avoid obvious breaks at the stitching.The segmentation lines generated during the stitching process are fused, and image processing techniques (such as gradient fusion or multiple exposure fusion) are used to ensure the consistency of lighting and color at the stitching. The size of the fusion area (such as 5 pixels) is set to ensure smooth transition of the edges and reduce visible traces at the stitching.

[0024] In this embodiment, the specific steps of step S11 are: Adaptively optimize the histogram of drone aerial videos and construct brightness-optimized aerial videos; Identify abnormal noise points in brightness-optimized aerial videos and mark them; Perform high-frequency filtering on abnormal noise points to obtain noise-filtered video; Decompose the noise-filtered video into time-series frames and extract multiple time-series frame images; Estimating the optical flow between adjacent frames of multiple time-series frame images and identifying the relative jitter data between frames; Performing inter-frame jitter smoothing on a plurality of time-series frame images according to inter-frame relative jitter data to construct a smooth and optimized time-series frame sequence; Perform pixel-by-pixel optical flow motion calculation on the smooth optimized time sequence frame sequence and extract the pixel motion vector of each frame image; Calculate the video motion blur kernel function according to the pixel motion vector of each frame image; The noise filtered video is subjected to motion blur convolution suppression processing according to the picture video motion blur kernel function, thereby constructing a motion blur optimized video.

[0025] In this embodiment, the CLAHE (Contrast Limited Adaptive Histogram Equalization) method is selected for brightness optimization. Compared with the traditional histogram equalization method, CLAHE can effectively avoid noise amplification caused by over-enhancement. This method divides the image into small areas (called "tiles") and performs equalization on each area to enhance the local contrast. The image is divided into 8x8 tiles, and the contrast limit is set to 2.0 to ensure that the influence of noise is suppressed while enhancing the contrast. For each frame of the drone aerial video, the CLAHE algorithm is applied for histogram equalization. Use the createCLAHE function of OpenCV to implement it. Record the image brightness mean and contrast before and after optimization to ensure that the brightness and clarity of the processed image are improved, and the brightness mean should be between 100 and 200. Reassemble each frame after histogram optimization into a video file, and use a video encoding format such as H.264 to ensure the quality and compression rationality of the output video. High-pass filtering combined with the threshold method is used to identify abnormal noise in the image. High-pass filtering can highlight the edges and details in the image, facilitating the identification of noise. Use Laplacian filter and set the convolution kernel size to 3x3 to enhance the high-frequency information of the image. Apply Laplacian filter to each frame of the image to obtain a high-frequency image. Use OpenCV's Laplacian function for processing. Set a threshold (such as 20) to binarize the high-frequency image and mark the abnormal noise area. Mark the identified abnormal noise on the original image with a red box or circle for subsequent processing and analysis. Use Gaussian filtering to smooth the image and reduce the impact of abnormal noise. Gaussian filter can effectively remove high-frequency noise in the image while retaining edge information. Set the convolution kernel size of the Gaussian filter to 5x5 and the standard deviation to 1.0 to ensure a moderate smoothing effect. Apply Gaussian filtering to each frame of the image and use OpenCV's GaussianBlur function to implement it. Reduce the impact of abnormal noise by processing frame by frame. Reassemble each frame of the filtered image into a video file to ensure the quality and clarity of the output video. Use OpenCV's VideoCapture function to extract each frame in the video for subsequent processing. Set the extraction to occur every other frame to ensure that sufficient timing information is captured. Use a loop to traverse the video, read and save each frame. Each frame image can be saved in PNG or JPEG format. Save the extracted frame images in sequence to the specified folder, with the naming format of "frame_001.png" for subsequent processing and identification. Use the Lucas-Kanade optical flow method, which can effectively estimate the motion information between adjacent frames and is suitable for processing small displacement images.The window size is set to 5x5 and the minimum number of feature points is set to 100 to ensure accurate estimation of optical flow. Optical flow is calculated for adjacent frames using the calcOpticalFlowFarneback function of OpenCV. The motion vector of each feature point is recorded. The optical flow information of adjacent frames is extracted and the motion vector of each frame is stored for subsequent analysis and processing. The motion vector is smoothed using the weighted average method to reduce the motion discontinuity caused by jitter. The weight coefficients are set to 0.7 (current frame) and 0.3 (previous frame) to ensure the smoothing effect for the current frame. The motion vector of each frame is smoothed using the weighted average method to eliminate the error caused by jitter between adjacent frames. The smoothed frame images are recombined to form a new time-series video to ensure the smoothness of the video. The Horn-Schunck optical flow method is used, which is suitable for estimating the motion vector of each pixel and providing a smooth optical flow field. The smoothing parameter is set to 0.1 to ensure the smoothness and continuity of the optical flow field. Perform optical flow calculation on the smoothed optimized time series frame sequence, and use OpenCV's calc Optical FlowFarneback function to calculate the motion vector of each pixel. Save the pixel motion vector of each frame as an array for subsequent analysis and processing. Calculate the motion blur kernel based on the pixel motion vector, and use a linear blur model that is suitable for describing the direction and speed of motion. Calculate the size and direction of the blur kernel based on the motion vector, set the kernel size to 5x5, and the blur radius is determined by the amplitude of the motion vector. Generate the corresponding motion blur kernel function based on the pixel motion vector of each frame. Use a two-dimensional Gaussian function to describe the shape and strength of the blur kernel. Save the generated motion blur kernel function for subsequent application. Use convolution operation to suppress motion blur on the video, combined with the previously generated motion blur kernel function. Ensure that the boundary treatment mode of the convolution operation is set to "edge filling" to avoid loss of information at the boundary. Apply the motion blur kernel function to each frame of the noise filtered video for convolution. Use OpenCV's filter2D function to implement it. Reassemble each frame of the video after convolution into a video file. Ensure that the output video has a significant improvement in image quality and clarity.

[0026] In this embodiment, refer to Figure 3 , is a flowchart of detailed implementation steps of step S2. In this embodiment, the detailed implementation steps of step S2 include: Step S21: performing pixel-by-pixel depth semantic recognition on the panoramic stitching and reconstruction image, marking each scene element; Step S22: analyzing the temporal displacement change of each scene element to generate a temporal displacement trajectory of each element; Step S23: classifying dynamic and static elements according to the temporal displacement trajectory of each element, thereby obtaining dynamic scene elements and static scene elements; Step S24: performing full-image spatial distribution analysis on dynamic scene elements and static scene elements to obtain dynamic and static element spatial distribution data; Step S25: dividing the panoramic stitching reconstruction image into image regions according to the spatial distribution data of dynamic and static elements, and performing precise boundary segmentation, thereby generating urban building areas and natural environment areas. In this embodiment, DeepLabv3+ or Mask R-CNN is selected as the algorithm for deep semantic segmentation. These models perform well in pixel-level classification of complex scenes and can handle the recognition of multiple scene elements. DeepLabv3+ uses dilated convolution to expand the receptive field, which is suitable for processing details in high-resolution images. The model is pre-trained on large data sets (such as Cityscapes or Pascal VOC) to ensure that it can recognize various elements in urban and natural scenes. The model is fine-tuned and trained according to the data set of the panoramic stitching reconstruction image, with the learning rate set to 0.0001, the batch size set to 8, and the training cycle set to 50 epochs. For each frame of the panoramic stitching reconstruction image, the trained model is used for pixel-by-pixel deep semantic recognition. Generate a class label for each pixel to identify elements such as buildings, roads, and vegetation in the scene. Set a threshold to ensure that the confidence of each pixel is higher than 0.5 to ensure the accuracy of the labeling. Generate a labeled image based on the recognition results, using different colors to represent different scene elements. Buildings are red, roads are blue, and vegetation is green. Save the marking information as an image file for subsequent analysis and processing. Extract the location information of each scene element from the marked image obtained in step S21, and generate the initial coordinates and category of the element (such as buildings, roads, and vegetation). Set a unique identifier for each element for tracking in subsequent analysis. Integrate the time-series frame data of the panoramic stitching reconstruction image into a list, and record the timestamp and corresponding marking information of each frame. Ensure that the element information of each frame can correspond to the time series. For each scene element, calculate its displacement change between consecutive frames. Record the displacement information of each element in each frame to generate a time-series displacement trajectory. Based on the calculated displacement information, generate a time-series displacement trajectory diagram for each element. Use different colors or line types to represent different types of scene elements to facilitate observation of the movement state of the elements. Save the trajectory data as a CSV file for subsequent analysis. Set the classification criteria for dynamic and static elements: if the displacement of an element in the video is greater than 5 pixels, it is judged as a dynamic element; otherwise, it is judged as a static element. Set a time window, such as observing the displacement of an element within 3 frames, to reduce misjudgment. Traverse the temporal displacement trajectory of each scene element and classify it according to the set standards. If the displacement exceeds the threshold, it is marked as dynamic; otherwise it is marked as static. Record the classification results, including element ID, category (dynamic or static), and displacement information. Use two-dimensional histogram or heat map method to analyze the spatial distribution of dynamic and static elements. Generate the corresponding heat map according to the coordinates of the elements to intuitively observe the distribution of the elements. Set the resolution of the heat map, for example, divide the entire image into 50x50 grids. Count the position information of each dynamic and static scene element and calculate the number of elements in each grid.Use Python's Matplotlib library to generate heat maps, using different colors to represent different numbers of elements to ensure that the heat map can clearly reflect the distribution of elements. Use image segmentation-based region partitioning methods, such as GrabCut or K-means clustering, to segment the image into urban building areas and natural environment areas. Set the initial parameters for region partitioning based on the spatial distribution data obtained in the previous analysis. Set the initial masks of the foreground and background in GrabCut, and set the number of iterations to 5 to ensure the accuracy of segmentation. Apply the selected region partitioning method to the panoramic stitching reconstruction image to generate mask maps of urban building areas and natural environment areas. Record the number of pixels and area of ​​each area for subsequent analysis. For the segmented areas, use morphological operations (such as closing operations) to smooth the boundaries to ensure the accuracy and consistency of the region boundaries. Draw the segmentation boundaries on the segmented image to ensure that each area is clearly visible.

[0027] In this embodiment, step S3 includes the following steps: Step S31: Calculate the scene light intensity based on the drone aerial video to obtain the scene light intensity; Step S32: performing ambient light reflection evolution on the urban building area based on the scene light source intensity to generate building ambient light reflection evolution characteristics; Step S33: Detect the illumination and shadow projection range of the urban building area and extract the shadow projection range; Step S34: performing dynamic light rendering evolution based on the building environment illumination reflection evolution characteristics and shadow projection range to generate dynamic light rendering evolution characteristics; Step S35: Calculating the wall thickness of the urban building area to generate the wall thickness of each building; Step S36: identifying top structure details according to the urban building area and extracting top structure details; Step S37: Performing building physical morphology and structural analysis based on the wall thickness and top structural details of each building to generate physical morphology and structural data of each building; Step S38: Perform three-dimensional building point cloud modeling on the dynamic light rendering evolution characteristics and the physical morphological structure data of each building to construct a three-dimensional urban building rendering model.

[0028] In this embodiment, in order to calculate the light source intensity of the scene, it is first necessary to perform brightness analysis of the image. The scene brightness of each frame is obtained by graying the image and applying a pixel-level light intensity detection algorithm (such as a simple image brightness value calculation formula: brightness = 0.299×R + 0.587×G + 0.114×B). Based on the brightness value of each frame of the image, an image segmentation technique (such as threshold-based segmentation, K-means clustering or graph cut algorithm) is applied to extract the light source area from the environment. The identification of the light source area is further optimized by the local contrast and brightness distribution of the image, and the light source is classified according to the type of light source in the environment (such as sunlight, reflected light, etc.). In the detected light source area, a physical illumination model (such as a Phong reflection model or a Lambertian model) is further used to estimate the intensity, and the light source intensity of different areas in the scene is obtained by combining the reflection characteristics of the image. The direction and intensity of the light source are calculated pixel by pixel, and further mapped in 3D space. According to the light source intensity data obtained in step S31, a physically based illumination reflection model (such as a Cook-Torrance model) is applied to simulate the illumination reflection characteristics. This model is used to model each urban building surface in the scene (such as glass, metal, concrete and other different materials), taking into account the intensity and distribution of the light source reflected on the building surface. According to the scene light intensity and building material, the light reflection at different angles and positions is calculated. The reflection direction and intensity are calculated pixel by pixel to generate the reflection evolution data of each building surface. Specifically, when the light intensity changes, the reflection intensity and glossiness of the building surface will change, which will affect the visual presentation of the building. Considering the changes of the light source at different times and environmental conditions, the evolution of light reflection in the building area is dynamically analyzed, especially the reflection differences related to day and night changes and weather changes. By taking multiple shots under different lighting conditions and analyzing their changes, the reflection evolution characteristics are further refined. Light reflection model: Cook-Torrance model, the reflection coefficient is set to the typical value of the building surface material (such as glass reflection coefficient 0.85, concrete 0.25), dynamic light source change time period: 8:00, 12:00, and 18:00 every day, simulating the reflection evolution under different lighting conditions, and the light intensity range: 500–1500 lux. Use lighting and geometry information to detect the shadow projection range of buildings. First, calculate the surface normal direction of each building and simulate the projection path of light by combining the position and direction of the light source. In this way, the shadow area between the building surface and the ground is obtained. Use a lighting-based shadow projection algorithm (such as a ray tracing algorithm) to combine lighting and building geometry to simulate the shadow projection effect at different times. Through this algorithm, the shadow range of the building at different times is extracted.The shadow projection range of each building is extracted, and the shape of the shadow area is optimized through image post-processing (such as edge detection and contour extraction) to ensure that the edge of the shadow is smooth and natural. The shadow projection map is used to detect the shadow coverage of each building at a given moment. Assuming that the light source height is 50 meters and the light source angle is 30°–90°, shadow detection is performed at different lighting angles. Combining the building environment lighting reflection evolution characteristics generated in step S32 with the shadow projection range extracted in S33, a lighting rendering engine (such as UnrealEngine or Unity 3D) is applied for dynamic light rendering. The rendering process dynamically adjusts the appearance and lighting effects of the building based on light source changes, reflection evolution and shadow projection. Use a real-time ray tracing algorithm to perform fine light simulation on the building. By simulating the reflection and refraction of light on the building surface and shadow area, the appearance characteristics of the building are dynamically rendered, and rendering features related to lighting changes are generated. The rendering results are analyzed to optimize the reflection effect of the lighting and the shadow details, so that the building can present a natural visual effect under different lighting conditions and enhance the realism of the three-dimensional model. Rendering time: 60 frames per second are rendered to ensure real-time dynamic effects. The number of light tracing paths is set to 10 to ensure that the lighting details are realistically presented. The external contours and wall structures of the buildings are extracted through image processing algorithms (such as edge detection and depth map generation). Combined with the 3D point cloud data of the building, deep learning algorithms (such as Mask R-CNN) are applied to identify the edges of the building walls. The thickness of the wall is estimated based on the structural characteristics of the building (such as architectural drawings, typical wall thickness standards) and point cloud data. The thickness of different building materials (such as brick walls and reinforced concrete walls) is predicted using a deep learning regression model. The wall thickness estimation results of each building are verified by actual measurement data and standard values ​​of wall thickness to ensure that the wall thickness estimation results of each building are accurate. Wall thickness standard: Depending on the building type, the wall thickness is estimated to be between 300mm–700mm, and the point cloud density is 10 points / cm² to ensure accurate wall boundary extraction.

[0029] Combined with multi-angle images taken by drones, deep learning-based target detection algorithms (such as YOLO and Faster R-CNN) are used to analyze the top of the building, extract the shape of the roof, roof facilities (such as skylights, chimneys, etc.) and texture features of the roof. Semantic segmentation algorithms (such as U-Net) are used to accurately segment the roof area and extract specific structural details of each building top, such as skylights, sun panels, chimneys and other facilities. Through these detail recognition, the accuracy of the roof structure is improved to ensure the realism of subsequent modeling. The extracted top structural details are applied to 3D modeling software to form a detailed roof model. Ensure that these details can present high-precision effects from different perspectives. Through multiple shots (each with an interval of 10 degrees), information from all angles of the roof is obtained, and a 3D physical form model of the building is constructed based on the wall thickness and roof structure detail data. First, the building is modeled using computer-aided design (CAD) tools in combination with the wall thickness and top structure details. Through a detailed analysis of the physical form of the building (such as wall strength, roof load-bearing, etc.), the model is optimized as necessary to ensure that the physical structure of the building can meet the actual application requirements. The algorithm generates the physical form data of each building, including the building's dimensions, wall materials, roof facilities and other features. The physical form data of each building includes parameters such as height, width, depth, wall thickness, roof type, and building materials.

[0030] Based on the dynamic light rendering features in S34 and the physical form data in S37, point cloud generation algorithms (such as the Poisson reconstruction algorithm) are used to combine the geometric shape and light reflection characteristics of the building to generate a high-precision 3D building point cloud. In the 3D modeling software, the point cloud data and material textures are combined to render the building. In particular, the effects of light and shadows need to be considered to ensure that the building looks natural and realistic under different lighting conditions. All building volumes, details and lighting data are integrated to generate a complete 3D urban building rendering model, and performance is optimized to make it suitable for application scenarios such as virtual reality and augmented reality. Point cloud density: The point cloud density is 100,000 points / m², ensuring accurate presentation of building details. Rendering accuracy: The texture resolution is 4096x4096 to ensure high-quality rendering effects.

[0031] In this embodiment, step S4 includes the following steps: Step S41: performing fine-grained identification of vegetation types in natural environment areas to generate multi-region vegetation types; Step S42: Calculating the vegetation coverage area of ​​multiple regional vegetation types to obtain the coverage area of ​​each vegetation type; Step S43: performing stereoscopic visual analysis on the natural environment area to generate regional stereoscopic visual information; Step S44: analyzing the terrain undulation characteristics according to the regional stereoscopic vision information to obtain the terrain characteristics of the environmental area; Step S45: performing three-dimensional vegetation distribution fitting on the terrain characteristics of the environmental area according to the coverage area of ​​each vegetation type, and constructing a three-dimensional vegetation area model; Step S46: performing three-dimensional panoramic fusion on the three-dimensional urban building rendering model and the three-dimensional vegetation area model to construct a three-dimensional panoramic model.

[0032] In this embodiment, image data of the natural environment area is extracted from the drone aerial video or image data. Since the natural environment area usually includes various vegetation types, such as grass, shrubs, trees, etc., different types of vegetation have significant differences in color, texture, morphology, etc. A deep learning algorithm (such as a deep convolutional neural network CNN, or a network specially designed for image segmentation such as U-Net) is used to perform fine-grained classification of vegetation areas. During the training process, different categories of vegetation are used for annotation, such as grass, trees, shrubs, etc., and fine regional division is achieved with the help of a multi-level feature extraction model. This process relies on high-resolution images and sufficient training data sets to extract and identify the specific type of each vegetation unit in the image through a pre-trained model. According to the prediction results of the model, the natural environment area is subdivided into multiple vegetation type areas. Each area corresponds to a specific vegetation type, and the location information and boundaries of each area are generated. The spatial distribution of vegetation is further accurately obtained by combining geographic information system (GIS) technology. Using the spatial distribution data of each vegetation type in the image, the area occupied by each vegetation type is calculated based on the pixel-level segmentation results. By converting the size of each pixel into the actual geographic area, the coverage area of ​​each vegetation type in the natural environment area is obtained. To ensure the high accuracy of area calculation, the GPS information and image coordinate system in the remote sensing image are combined to perform spatial geometric transformation and geographic coordinate calibration. In particular, image registration technology (such as SURF algorithm) is used to align images at different image angles, different times or different geographical environments to ensure the consistency of area calculation. The coverage area of ​​each vegetation type is comprehensively calculated based on image data at multiple times and under different weather and lighting conditions. Through multiple aerial photography and image acquisition, the temporal and spatial accuracy of the data is ensured, and calibration is performed in a dynamically changing environment. Using the binocular camera or multi-view image data of the drone, a depth map of the area is generated through a stereo matching algorithm (such as Semi-Global Matching, SGM). The depth map contains the relative distance between each pixel and the camera, thereby constructing the stereoscopic visual information of the natural environment area. First, the image quality is improved through image preprocessing technology (such as illumination compensation and denoising), and then the binocular images are aligned to find different perspectives in the same scene through pixel matching. Using the principle of triangulation, the depth information of each point is calculated, and a stereo image with a depth value is generated. Multi-view images are used for depth optimization. In the reconstruction process, the smoothing algorithm and the depth correction model are combined to remove noise and mismatched areas to ensure that the generated stereoscopic visual information has higher accuracy. The depth information obtained through stereoscopic vision analysis is used to perform terrain analysis on the natural environment area using terrain analysis algorithms (such as slope analysis and undulation analysis). Slope analysis calculates the slope of the surface based on the elevation difference of each pixel point, and then determines the undulation of the terrain.Combined with the elevation data in the depth map, different features of the terrain are extracted, such as flat areas, hills, valleys, etc. Use geographic information system (GIS) tools to classify the terrain, automatically mark the terrain type of each area, and calculate the area, slope and other indicators of each terrain. Combined with different vegetation types and terrain characteristics, data fusion and modeling are performed to generate multi-dimensional terrain feature data. These data provide the basis for subsequent vegetation distribution modeling and three-dimensional landscape reconstruction. Based on the vegetation type coverage area data and terrain characteristics obtained in the previous steps, a three-dimensional modeling algorithm (such as a three-dimensional interpolation method based on particle swarm optimization) is used to fit each vegetation type on the terrain surface. Through this fitting method, the distribution of different vegetation types is accurately fitted to the terrain surface, and the three-dimensional spatial distribution of each vegetation type in the region is generated taking into account the influence of different terrains. In the process of three-dimensional modeling, it is necessary to fit the three-dimensional spatial distribution of vegetation according to the characteristics of each vegetation type (such as the height of trees, the distribution density of shrubs, etc.). The vegetation model is coupled with the terrain model to ensure that the plant distribution matches the undulating characteristics of the terrain. Combine the existing vegetation data and ground survey data to optimize the three-dimensional vegetation model to ensure the authenticity and accuracy of vegetation distribution. By comparing different vegetation distribution models, the optimal fitting scheme is selected. Vegetation distribution density: Set the distribution density according to the vegetation type. The tree distribution density is 1-3 trees / square meter, and the shrub is 5-10 trees / square meter. Use the particle swarm optimization algorithm for distribution fitting, and the error is controlled within ±0.2 meters. The three-dimensional urban building rendering model in step S38 is merged with the three-dimensional vegetation area model in step S45. First, ensure that the spatial coordinates of the two are consistent. Through coordinate transformation and alignment technology (such as ICP algorithm), the two models are merged into a unified three-dimensional panoramic model. In the process of model fusion, physical rendering technologies such as lighting, shadow, and reflection (such as ray tracing and ambient light occlusion) are used to enhance the interaction between buildings and vegetation, so that they show more natural light and shadow changes. At the same time, the visual effect of the panoramic model is optimized through the interaction between the growth state of vegetation and lighting. Finally, a complete three-dimensional panoramic model is output, which is suitable for application scenarios such as virtual reality, augmented reality or digital city. Ensure that the model can be rendered smoothly at different resolutions and ensure high-quality visual effects. Rendering accuracy: Use high-quality ray tracing rendering with a resolution of 3840×2160. Fusion algorithm: Use ICP (Iterative Closest Point) algorithm for model alignment, with an error control within 2cm. Rendering time: The rendering time of each scene is controlled within 30 seconds.

[0033] In this embodiment, the specific steps of step S5 are: Step S51: Detect blurry areas of the 3D panoramic model and mark blurry areas of the model; Step S52: positioning the fuzzy area of ​​the model in three-dimensional space and extracting the three-dimensional spatial coordinates of the fuzzy area; Step S53: Based on the three-dimensional spatial coordinates of the blurred area, a drone is used to perform a secondary low-altitude aerial photography to extract a low-altitude high-definition video of the blurred area.

[0034] In this embodiment, relevant data including geometric shape, texture, illumination and other information are extracted from the three-dimensional panoramic model. Since some areas in the three-dimensional model are blurred due to modeling accuracy, image quality or viewing angle problems, we need to detect these areas. In order to detect the blurred area, an image recognition method or image quality analysis technology based on deep learning is adopted. A deep learning model (such as a convolutional neural network CNN) is trained to identify blurred image areas. In this process, the model analyzes the clarity of each face or texture area in the three-dimensional model. By comparing the clarity of the texture and the complexity of the geometric details, the areas with blurred texture and unclear details are marked. Common image quality assessment indicators such as structural similarity index (SSIM) or peak signal-to-noise ratio (PSNR) are used to quantify the degree of blurriness of the image or model area. The lower the SSIM or the lower the PSNR, the higher the degree of blurriness of the area. The panoramic model is scanned using a blurred area detection algorithm to determine which areas do not meet the clarity standard. For these areas, marker information is added to the three-dimensional model, including position, area size and degree of blurriness. The SSIM value and PSNR value are used to evaluate the clarity of the area. The SSIM threshold is set to 0.7 and the PSNR threshold is set to 25dB. Model resolution: The resolution of the 3D panoramic model is 4096×4096, and the accuracy is controlled within 0.5 meters. Calculation method: Convolutional neural network (CNN) is used for blur detection. The training data set contains 10,000 model images of different clarity. Once the blur area is identified through image processing and deep learning model, the spatial position of the blur area needs to be determined by 3D geometric modeling technology. This process requires locating the blur area in a 3D coordinate system. Use a 3D reconstruction algorithm (such as least squares method, 3D reconstruction algorithm, or rigid transformation algorithm) to extract the spatial coordinates of the detected blur area. At this time, the location of the blur area will be converted from the 2D projection of the model to a 3D spatial coordinate system. Combined with the data structure of the 3D model, these blur areas are located by point cloud data or mesh model. Specifically, each marked blur area will calculate its precise 3D coordinates through interpolation technology. Use technologies such as triangular mesh or octree to ensure the accuracy of coordinate extraction. In order to improve positioning accuracy, the panoramic image is compared with the actual ground measurement data to verify the accuracy of the coordinates. If necessary, further geometric correction is used to reduce the error. After obtaining the three-dimensional spatial coordinates of the blurred area, the next step is to use a drone to perform low-altitude secondary aerial photography. First, plan the flight path of the drone based on the three-dimensional coordinates of the blurred area. Ensure that the camera can accurately cover the blurred area when the drone is flying. This process needs to be planned through flight planning software (such as Mission Planner) to control parameters such as flight altitude, shooting angle, and speed. Select high-resolution camera equipment (such as 4K HD camera or high-performance thermal imaging camera) to obtain clearer and more detailed video data.When flying, the drone needs to accurately reach and shoot the target area based on the real-time 3D coordinates and GPS information, especially in complex terrain or between buildings, to ensure that the blurred area can obtain sufficiently clear image data. During the aerial photography process, when extracting high-definition video data, high-speed camera technology is used to ensure smooth and clear video. Multi-angle shooting with multiple lenses is selected to further improve the video quality. In order to ensure that effective blurred area data is obtained, each frame in the video needs to be screened and optimized by the image processing algorithm to avoid interference from irrelevant areas.

[0035] The collected low-altitude high-definition video needs to be transmitted and stored in the data server in a timely manner. During the storage process, data compression and encryption technology are applied to ensure the integrity and security of the video data. The video data is then imported into the subsequent image processing system for analysis and optimization.

[0036] In this embodiment, the specific steps of step S6 are: Step S61: performing texture detail analysis on the low-altitude high-definition video of the blurred area to generate texture details of the blurred area; Step S62: performing geometric morphology recognition on the low-altitude high-definition video of the blurred area to extract geometric morphology features of the blurred area; Step S63: Optimizing the local details of the fuzzy area of ​​the three-dimensional panoramic model based on the texture details of the fuzzy area and the geometric features of the fuzzy area, thereby constructing a three-dimensional real scene optimization model.

[0037] In this embodiment, high-definition video data taken by a drone at low altitude is collected. In order to improve the accuracy of texture detail analysis, necessary video data preprocessing is performed, including denoising, illumination correction, image enhancement, etc. These operations ensure that the texture details in the video are not disturbed by noise and avoid false detection caused by illumination changes. Texture analysis techniques in computer vision (such as gray-level co-occurrence matrix (GLCM), local binary pattern (LBP) or deep learning methods such as convolutional neural network (CNN)) are used to extract texture information in video frames. By analyzing the local area of ​​the image, texture features (such as contrast, uniformity, roughness, etc.) are extracted, which are crucial for judging the degree of detail of the area. For the blurred area, a deep learning algorithm is used to supplement and enhance the texture details. Using an image enhancement model based on a generative adversarial network (GAN), by learning high-quality texture samples in the training data set, details close to the real texture are generated to make up for the missing parts of the original blurred area. The texture information extracted from each frame of the video image is integrated to generate high-definition texture details of the blurred area. These texture details will be marked and recorded and prepared for subsequent model optimization. Extract key frames of blurred areas from low-altitude high-definition videos and perform geometric morphology analysis. Through video frame processing, eliminate motion blur, noise and lighting differences in low-altitude shooting. In order to extract the geometric morphology features of blurred areas, traditional computer vision methods such as Hough transform, edge detection (such as Canny operator) and image segmentation technology are used to identify the shapes in blurred areas. Through edge extraction and contour analysis, the basic geometric features of the area, such as straight lines, curves, angles and surfaces, are obtained. For the identified geometric shapes, morphological analysis techniques (such as dilation, erosion, opening and closing operations) are used to further optimize the geometric structure and enhance the geometric features of the area. Use convolutional neural networks (CNNs) in deep learning for geometric morphology classification and recognition. Through training data sets, learn the features of different geometric forms and further extract the geometric details of complex structures. Through the above steps, the geometric feature information of the blurred area is extracted, including the basic shape, size, proportion, contour, etc. of the object. And further convert this information into three-dimensional spatial coordinates, ready for subsequent three-dimensional model optimization. In the three-dimensional panoramic model, the spatial coordinates and geometric morphology features of the blurred area are marked. First, the texture features obtained by texture detail analysis (step S61) are combined with the geometric structure information obtained by geometric morphology recognition (step S62) to identify the local area that needs to be optimized. For the blurred area, the optimized texture details are extracted from the low-altitude high-definition video through texture mapping technology and mapped to the blurred area of ​​the three-dimensional panoramic model. The original blurred texture is replaced by a high-resolution texture image to ensure that the texture details of the optimized area are richer and clearer. For the geometric morphology features, the geometric shape of the blurred area is refined through geometric reconstruction technology (such as point cloud-based reconstruction method or surface fitting technology) to make it more consistent with the structure of the real scene.Combined with the texture and geometry information of the blurred area, the 3D model fusion technology (such as B-spline interpolation or finite element method) is used to smoothly transition the local area to ensure that the optimized details are naturally integrated into the existing 3D panoramic model without abrupt changes. For dynamic features such as lighting, shadows and reflections, ray tracing technology is used to optimize the details of local lighting and reflections. After the local details are optimized, the consistency check of the entire 3D model is performed to ensure that the optimized texture and geometry are consistent with the style of the original panoramic model. Finally, the final 3D real scene optimization model is generated and applied to application scenarios such as virtual reality, augmented reality and digital cities.

[0038] In this embodiment, a real-scene 3D automated modeling system based on a drone is provided, which is used to execute the real-scene 3D automated modeling method based on a drone as described above, including: The geometric stitching module is used to obtain drone aerial videos; perform motion blur convolution suppression and panoramic geometric stitching on drone aerial videos to construct a panoramic stitching reconstruction image; The deep semantic recognition module is used to perform pixel-by-pixel deep semantic recognition on the panoramic stitching and reconstruction image, thereby generating urban building areas and natural environment areas; Light rendering evolution module, used to perform dynamic light rendering evolution and building physical form structure analysis on urban building areas, and build a three-dimensional urban building rendering model; The 3D panoramic fusion module is used to identify the vegetation types in the natural environment area in fine granularity and perform 3D panoramic fusion based on the 3D urban building rendering model to construct a 3D panoramic model. The blur area detection module is used to detect blur areas in the 3D panoramic model and perform secondary low-altitude aerial photography to extract low-altitude high-definition video of the blur area. The local detail optimization module is used to optimize the local details of the blurred area based on the low-altitude high-definition video of the blurred area, so as to construct a three-dimensional real-scene optimization model.

[0039] The present invention can effectively remove the blur effect, restore image details, and ensure that each frame of the image has a high-quality visual effect by using a motion blur convolution suppression algorithm (such as a blind deblurring algorithm). This is particularly important for subsequent three-dimensional modeling, because any degradation in image quality will affect feature extraction and model accuracy. Through panoramic geometric stitching technology, aerial images from multiple perspectives can be accurately stitched into a complete panorama. Geometric stitching ensures that the perspective distortion between different images is minimized and effectively synthesizes a seamless large-scale view, thereby providing a high-precision image basis for subsequent deep semantic recognition and three-dimensional reconstruction. Through deep learning networks (such as convolutional neural networks, Transformers, etc.) for pixel-by-pixel deep semantic recognition, each element in the image can be classified and labeled at the pixel level. This not only helps to accurately identify elements such as buildings, roads, and vegetation, but also distinguishes the boundaries between natural environment areas and urban construction areas based on environmental complexity, thereby improving the accuracy of subsequent modeling. Use physical lighting and dynamic rendering algorithms (such as ray tracing or physically based rendering PBR technology) to simulate light changes at different times and angles. Accurately reproduce the effects of light on buildings, such as light reflection, shadow changes, etc., which provides a sense of reality for the visual presentation of buildings under different lighting conditions. By simulating the evolution of light, the final building model is more vivid and natural in the real environment. Analyzing the physical structure of the building (such as wall thickness, window frames, roof structure, etc.) can build a more accurate 3D building model, avoiding the limitations of models based solely on plane drawings. This not only improves the modeling accuracy, but also enhances the realism of the building, providing high-quality support for urban planning and virtual reality applications. Fine-grained classification and identification of vegetation types in the natural environment through deep learning algorithms (such as FCN, MaskR-CNN, etc.). This process can accurately identify different types of plants, trees, etc., and even identify the growth status of plants (such as healthy, withered, etc.), providing strong data support for accurate modeling of the natural environment. Combining the modeling results of the urban building area and the natural environment area, the two are integrated into a whole through 3D panoramic fusion technology (such as seamless fusion of lighting, materials and geometric models). In this way, the final constructed 3D panoramic model not only shows the architectural appearance of the city, but also accurately reproduces the details of the surrounding natural environment, enhancing the realism and immersion of the entire scene. Through image processing technology, it is possible to identify blurred areas caused by distance, movement and other reasons during the aerial photography process. This detection step is very important because blurred areas will affect the accuracy and integrity of modeling. Timely discovery and marking of these areas provide guidance for subsequent optimization. Once the blurred areas are found, low-altitude secondary aerial photography can be used for high-definition acquisition at a closer distance and in a more stable flight state.This can obtain clearer and higher-resolution video data, make up for the loss of details in the first aerial photography, and provide a high-quality data source for subsequent 3D modeling detail repair. Through deep learning detail recognition of low-altitude videos, important information such as texture details and structural features in the blurred area are extracted and applied to the optimization of the 3D model. This not only improves the accuracy of the blurred area, but also increases the delicacy and realism of the entire 3D model. Based on the optimized detail information, a high-precision, high-quality 3D real-scene optimization model is constructed. The 3D model with detailed repair has higher visual realism and spatial restoration, and can be applied to multiple fields, such as urban planning, virtual reality, architectural design, etc.

[0040] Therefore, the embodiments should be regarded as illustrative and non-restrictive from all points, and the scope of the present invention is limited by the appended claims rather than the above description, and it is therefore intended that all changes falling within the meaning and range of equivalent elements of the application documents are included in the present invention.

[0041] The above is only a specific embodiment of the present invention, so that those skilled in the art can understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but should conform to the widest scope consistent with the principles and novel features invented herein.

Claims

1. A real-scene 3D automated modeling method based on drone, characterized in that: The following steps are involved: Step S1: obtaining drone aerial video; performing motion blur convolution suppression processing and panoramic geometric stitching on the drone aerial video, thereby constructing a panoramic stitching reconstruction image; Step S2: performing pixel-by-pixel depth semantic recognition on the panoramic stitching and reconstruction image, thereby generating an urban building area and a natural environment area; Step S3: Perform dynamic light rendering evolution and building physical morphology structure analysis on the urban building area to construct a three-dimensional urban building rendering model; Step S4: performing fine-grained identification of vegetation types in the natural environment area, and performing three-dimensional panoramic fusion according to the three-dimensional urban building rendering model to construct a three-dimensional panoramic model; Step S5: Detect blurry areas of the three-dimensional panoramic model, perform secondary low-altitude aerial photography, and extract low-altitude high-definition video of the blurry areas; Step S6: Optimize the local details of the blurred area based on the low-altitude high-definition video of the blurred area, so as to construct a three-dimensional real scene optimization model.

2. The method for automatic three-dimensional modeling of real scenes based on drones according to claim 1, characterized in that: The specific steps of step S1 are: Step S11: obtaining a drone aerial video; performing motion blur convolution suppression processing on the drone aerial video, thereby constructing a motion blur optimized video; Step S12: performing visual recognition of scene feature points frame by frame on the motion blur optimized video, marking multiple scene feature points in each frame; Step S13: performing dynamic inter-frame correlation matching based on multiple scene feature points in each frame to generate a geometric transformation relationship of the feature points; Step S14: mining the temporal continuity constraints of the geometric transformation relationship of the feature points to generate temporal continuity constraints between frames; Step S15: Perform panoramic geometric stitching on the motion blur optimized video based on the temporal continuity constraint between frames, and fuse the segmentation lines at the stitching points, so as to construct a panoramic stitching reconstruction image.

3. The method for automatic three-dimensional modeling of real scenes based on drones according to claim 2, characterized in that: The specific steps of step S11 are: Adaptively optimize the histogram of drone aerial videos and construct brightness-optimized aerial videos; Identify abnormal noise points in brightness-optimized aerial videos and mark them; Perform high-frequency filtering on abnormal noise points to obtain noise-filtered video; Decompose the noise-filtered video into time-series frames and extract multiple time-series frame images; Estimating the optical flow between adjacent frames of multiple time-series frame images and identifying the relative jitter data between frames; Performing inter-frame jitter smoothing on a plurality of time-series frame images according to inter-frame relative jitter data to construct a smooth and optimized time-series frame sequence; Perform pixel-by-pixel optical flow motion calculation on the smooth optimized time sequence frame sequence and extract the pixel motion vector of each frame image; Calculate the video motion blur kernel function according to the pixel motion vector of each frame image; The noise filtered video is subjected to motion blur convolution suppression processing according to the picture video motion blur kernel function, thereby constructing a motion blur optimized video.

4. The method for automatic real-scene 3D modeling based on drone according to claim 1, characterized in that: The specific steps of step S2 are: Step S21: performing pixel-by-pixel depth semantic recognition on the panoramic stitching and reconstruction image, marking each scene element; Step S22: analyzing the temporal displacement change of each scene element to generate a temporal displacement trajectory of each element; Step S23: classifying dynamic and static elements according to the temporal displacement trajectory of each element, thereby obtaining dynamic scene elements and static scene elements; Step S24: performing full-image spatial distribution analysis on dynamic scene elements and static scene elements to obtain dynamic and static element spatial distribution data; Step S25: dividing the panoramic stitching reconstruction image into image regions according to the spatial distribution data of dynamic and static elements, and performing precise boundary segmentation, thereby generating urban building areas and natural environment areas.

5. The method according to claim 1, characterized in that The specific steps of step S3 are: Step S31: Calculate the scene light intensity based on the drone aerial video to obtain the scene light intensity; Step S32: performing ambient light reflection evolution on the urban building area based on the scene light source intensity to generate building ambient light reflection evolution characteristics; Step S33: Detect the illumination and shadow projection range of the urban building area and extract the shadow projection range; Step S34: performing dynamic light rendering evolution based on the building environment illumination reflection evolution characteristics and shadow projection range to generate dynamic light rendering evolution characteristics; Step S35: Calculating the wall thickness of the urban building area to generate the wall thickness of each building; Step S36: identifying top structure details according to the urban building area and extracting top structure details; Step S37: Performing building physical morphology and structural analysis based on the wall thickness and top structural details of each building to generate physical morphology and structural data of each building; Step S38: Perform three-dimensional building point cloud modeling on the dynamic light rendering evolution characteristics and the physical morphological structure data of each building to construct a three-dimensional urban building rendering model.

6. The method for automatic real-scene 3D modeling based on drone according to claim 1, characterized in that: The specific steps of step S4 are: Step S41: performing fine-grained identification of vegetation types in natural environment areas to generate multi-region vegetation types; Step S42: Calculating the vegetation coverage area of ​​multiple regional vegetation types to obtain the coverage area of ​​each vegetation type; Step S43: performing stereoscopic visual analysis on the natural environment area to generate regional stereoscopic visual information; Step S44: analyzing the terrain undulation characteristics according to the regional stereoscopic vision information to obtain the terrain characteristics of the environmental area; Step S45: performing three-dimensional vegetation distribution fitting on the terrain characteristics of the environmental area according to the coverage area of ​​each vegetation type, and constructing a three-dimensional vegetation area model; Step S46: performing three-dimensional panoramic fusion on the three-dimensional urban building rendering model and the three-dimensional vegetation area model to construct a three-dimensional panoramic model.

7. The method for automatic real-scene 3D modeling based on drone according to claim 1, characterized in that: The specific steps of step S5 are: Step S51: Detect blurry areas of the 3D panoramic model and mark blurry areas of the model; Step S52: positioning the fuzzy area of ​​the model in three-dimensional space and extracting the three-dimensional spatial coordinates of the fuzzy area; Step S53: Based on the three-dimensional spatial coordinates of the blurred area, a drone is used to perform a secondary low-altitude aerial photography to extract a low-altitude high-definition video of the blurred area.

8. The method for automatic real-scene 3D modeling based on drone according to claim 1, characterized in that: The specific steps of step S6 are: Step S61: performing texture detail analysis on the low-altitude high-definition video of the blurred area to generate texture details of the blurred area; Step S62: performing geometric morphology recognition on the low-altitude high-definition video of the blurred area to extract geometric morphology features of the blurred area; Step S63: Optimizing the local details of the fuzzy area of ​​the three-dimensional panoramic model based on the texture details of the fuzzy area and the geometric features of the fuzzy area, thereby constructing a three-dimensional real scene optimization model.

9. A real-scene 3D automated modeling system based on drones, characterized in that: The method for executing the real-scene 3D automated modeling method based on a drone as claimed in claim 1 comprises: The geometric stitching module is used to obtain drone aerial videos; perform motion blur convolution suppression and panoramic geometric stitching on drone aerial videos to construct a panoramic stitching reconstruction image; The deep semantic recognition module is used to perform pixel-by-pixel deep semantic recognition on the panoramic stitching and reconstruction image, thereby generating urban building areas and natural environment areas; Light rendering evolution module, used to perform dynamic light rendering evolution and building physical form structure analysis on urban building areas, and build a three-dimensional urban building rendering model; The 3D panoramic fusion module is used to identify the vegetation types in the natural environment area in a fine-grained manner and to perform 3D panoramic fusion based on the 3D urban building rendering model to construct a 3D panoramic model. The blur area detection module is used to detect blur areas in the 3D panoramic model and perform secondary low-altitude aerial photography to extract low-altitude high-definition video of the blur area. The local detail optimization module is used to optimize the local details of the blurred area based on the low-altitude high-definition video of the blurred area, so as to construct a three-dimensional real-scene optimization model.

Images