Visual imaging method and system combined with unmanned aerial vehicle multi-temporal automatic comparison

Abstract

The invention discloses a visual imaging method and system combined with unmanned aerial vehicle multi-temporal automatic comparison, and the system comprises an unmanned aerial vehicle aerial survey module which is used for carrying out the repeated aerial survey of the same target region at different time points, and obtaining multi-period image data; the data transmission and storage module is used for transmitting the multi-period image data acquired by the unmanned aerial vehicle aerial survey module in real time; the method has the beneficial effects that the aerial survey of the unmanned aerial vehicle supports dynamic route adjustment, multi-sensor cooperative acquisition is realized, the overlap ratio of multiple aerial survey paths is high, the image data integrity and consistency are high, and the manual intervention and data supplementary acquisition cost is greatly reduced; multi-stage image automatic registration, feature extraction and change detection are realized through deep learning, the processing precision is high, a large amount of manual operation is not needed, and the efficiency is remarkably improved; three-dimensional modeling and multi-mode comparison are supported, dynamic changes of ground objects can be clearly displayed, interaction functions are rich, and the requirements for single-person detail checking and multi-person collaborative rechecking can be met.

Description

Technical Field

[0001] This invention belongs to the field of visualization imaging technology, specifically relating to a visualization imaging method and system that combines multi-temporal automatic comparison of UAVs. Background Technology

[0002] In modern engineering construction, environmental protection, disaster monitoring and other fields, the demand for dynamic monitoring and precise analysis of target areas is becoming increasingly urgent. UAV aerial surveying technology, with its advantages of high flexibility, wide coverage and high data acquisition efficiency, has become an important means of acquiring regional image data. UAV multi-temporal automatic comparison technology, by repeatedly conducting aerial surveys of the same area at different time points, and combining computer vision, deep learning and other technologies to process and analyze multiple images, can intuitively present changes in regional features and provide key data support for decision-making. Its core technical links cover multi-phase image acquisition, automated data processing and visualization output.

[0003] Currently, there are already relevant patents involving UAV image management and visualization technology. For example, patent publication number CN105260389B proposes a method for managing and visualizing UAV reconnaissance image data. This method proposes to establish a data table based on metadata, store image data using a hybrid management approach of files and relational databases, and achieve image geographic information visualization through secondary development of Google Earth. Although this patent solves the problems of UAV image storage management and basic visualization, it is limited to the simple storage and display of single or multiple images. It does not involve core processing steps such as automated registration and change detection of multi-period images, and cannot achieve accurate identification and quantitative analysis of ground feature changes. Moreover, the visualization form is singular and cannot meet the dynamic monitoring needs in complex scenarios.

[0004] A building information and fire information visualization system and method, with patent publication number CN117152592B, collects fire data by using a drone equipped with a camera and thermal imager, and combines it with a 3D building model to determine and visualize the location of the fire. This patent focuses on a specific fire monitoring scenario, only collecting and analyzing flame data, and its application scenarios are limited. At the same time, its data processing is limited to a simple merging of fire information and building model, without forming an automatic comparison mechanism for multi-period data, failing to capture the dynamic changes in the development of the fire, and lacking quantitative evaluation and decision support functions for the changes.

[0005] Furthermore, existing technologies have several shortcomings in multi-temporal applications of UAVs: First, in the image acquisition stage, flight path planning is mostly fixed and difficult to dynamically adjust according to real-time environmental changes such as wind speed, visibility, and terrain, resulting in poor consistency of aerial survey paths across multiple phases and significant image offset; Second, in the data processing stage, traditional registration algorithms rely on manual intervention, are poorly adaptable to terrain undulations and differences in shooting angles, have low change detection accuracy, and lack the ability to fuse and process multiple types of images; Third, in terms of visualization, it mainly relies on two-dimensional image comparison, with insufficient accuracy in three-dimensional modeling, limited interactive functions, and an inability to support multi-person collaborative review and detailed viewing; Fourth, the data storage and security system is incomplete, making it difficult to meet the needs of efficient storage, backup, and security protection of massive multi-phase image data, and lacking traceability management throughout the entire data lifecycle.

[0006] Therefore, there is an urgent need for a visualization imaging system and method that can achieve automated and accurate processing of multi-phase images, multi-dimensional visualization, full-scene decision support, and complete data storage and security, combined with automatic comparison of multi-temporal images from UAVs, in order to overcome the shortcomings of existing technologies and meet the high-precision and high-efficiency requirements for dynamic change monitoring in fields such as engineering construction, disaster monitoring, and environmental protection. Summary of the Invention

[0007] The purpose of this invention is to provide a visualization imaging method and system that combines automatic comparison of multiple time phases of UAVs, which solves the problems of low automation in multi-phase image processing, insufficient change detection accuracy, single visualization form, lack of data storage security and lack of decision support function in the prior art. It realizes the intelligentization of the whole process from acquisition, processing and storage of multi-phase images to visualization and decision support, and meets the dynamic monitoring needs in complex scenarios.

[0008] To achieve the above objectives, the present invention provides the following technical solution: a visualization imaging system combining multi-temporal automatic comparison of unmanned aerial vehicles (UAVs), including... The UAV aerial survey module is used to repeatedly conduct aerial surveys of the same target area at different times to acquire multiple periods of image data. The data transmission and storage module is used to transmit multi-phase image data acquired by the UAV aerial survey module in real time, and to store the image data and related metadata in a hierarchical storage structure. A multi-temporal data processing module is used to perform automated registration, feature extraction, change detection, and difference enhancement processing on multi-phase image data. The visualization generation and interaction module is used to generate visualization content from the change results obtained by the multi-temporal data processing module in a variety of intuitive forms, and provide manual review and interaction. The decision support module provides users with decision suggestions based on visualized content and change analysis results.

[0009] As a preferred technical solution of the present invention, the UAV aerial survey module includes a UAV body, a sensor unit, a route planning submodule, and a flight control submodule; the sensor unit is mounted on the UAV body and includes a high-definition optical camera, a thermal imaging camera, and a lidar, used to collect various types of image data of the target area; the route planning submodule is used to generate a preset route or dynamically adjust the route based on the terrain information of the target area, the aerial survey accuracy requirements, and historical aerial survey routes; the flight control submodule is used to control the flight attitude, flight speed, and shooting parameters of the UAV body according to the route generated by the route planning submodule.

[0010] As a preferred technical solution of the present invention, the data transmission and storage module includes a wireless transmission submodule, an edge storage submodule, and a cloud storage submodule; the wireless transmission submodule adopts a 5G combined with satellite dual-mode transmission mode to realize real-time transmission of image data; the edge storage submodule is set on the UAV body or ground control station for temporary storage of image data; the cloud storage submodule adopts a distributed storage architecture, stores image data in layers according to acquisition time, image type, and target area, and establishes metadata index, the metadata including acquisition time, geographical location, sensor parameters, flight attitude, and image resolution.

[0011] As a preferred technical solution of the present invention, the cloud storage submodule is further provided with a data backup and recovery submodule, which adopts a combination of off-site backup and incremental backup to back up the stored image data and metadata. When the data is damaged or lost, the data can be quickly recovered through the data recovery submodule.

[0012] As a preferred technical solution of the present invention, the multi-temporal data processing module includes an image registration submodule, a feature extraction submodule, a change detection submodule, and a difference enhancement submodule. The image registration submodule uses a feature point matching algorithm based on deep learning, combined with the geographic coordinate information of the target area, to automatically register multi-phase image data, eliminating image shifts caused by changes in UAV flight attitude, differences in shooting angles, and terrain undulations. The feature extraction submodule extracts key features from multi-phase image data through a convolutional neural network, including ground feature outlines, texture features, and grayscale features. The change detection submodule, based on the feature extraction results, uses a combination of the difference method and machine learning classification algorithm to detect changes in ground features in the target area at different time points, determining the changed areas and change types. The difference enhancement submodule highlights the differences between changed and unchanged areas through contrast enhancement, color mapping, and edge sharpening processing.

[0013] As a preferred technical solution of the present invention, the image registration submodule preprocesses the image data before performing automated registration, including denoising, distortion correction and radiometric correction, to remove the influence of noise interference, optical distortion and illumination differences in the image on the registration accuracy; the change detection submodule also verifies the accuracy of the detected change results by comparing them with manually measured data or high-resolution satellite image data, and calculates the accuracy, recall and F1 value of change detection.

[0014] As a preferred technical solution of the present invention, the visualization generation interaction module includes a 3D modeling submodule, a dynamic rendering submodule, a comparison display submodule, and an interactive operation submodule; the 3D modeling submodule constructs a multi-temporal 3D model of the target area based on multi-period image data and terrain data collected by LiDAR; the dynamic rendering submodule uses real-time rendering technology to perform color rendering, lighting simulation, and texture mapping on the multi-temporal 3D model to restore the real scene of the target area; the comparison display submodule intuitively displays the change process of the target area at different points in time.

[0015] As a preferred technical solution of the present invention, the decision support module includes a change analysis submodule, a risk assessment submodule, and a suggestion generation submodule; the change analysis submodule performs quantitative analysis on the area, rate of change, and trend of change of the changed area and generates a change analysis report; the risk assessment submodule, combined with the application scenario of the target area, establishes a risk assessment model, assesses the risks that the changed area may cause, and determines the risk level; the suggestion generation submodule, based on the change analysis results and risk assessment results, combined with a preset decision rule library, provides users with targeted decision suggestions.

[0016] This invention also discloses a visualization imaging method combining multi-temporal automatic comparison of UAVs, comprising the following steps: S1. Aerial Survey Preparation: Based on the terrain information of the target area, the accuracy requirements of aerial survey, and the application scenario, determine the time interval, shooting parameters, and flight path planning scheme for UAV aerial survey, select UAVs equipped with high-definition optical cameras, thermal imaging cameras, and lidar, and debug the UAV's flight control system, sensor units, and data transmission system. S2. Multi-temporal image acquisition: According to the flight path planning scheme, control the UAV to repeatedly conduct aerial surveys of the same target area at different time points, collect multi-type image data of the target area through the sensor unit, and record the metadata of each aerial survey; S3. Data transmission and storage: Through 5G combined with satellite dual-mode transmission, the collected multi-stage image data and metadata are transmitted in real time to the edge storage device for temporary storage. At the same time, the data is synchronously transmitted to the cloud storage system. A distributed hierarchical storage architecture is adopted, and the data is classified and stored according to the collection time, image type and target area, and a metadata index is established. S4. Multi-temporal data processing: S41. Image preprocessing: Denoising, distortion correction and radiometric correction are performed on multi-phase image data to remove noise interference, optical distortion and illumination differences in the images. S42. Automated registration: A feature point matching algorithm based on deep learning is used to register the preprocessed multi-period image data in combination with the geographic coordinate information of the target area to eliminate image offset; S43. Feature Extraction: Extract key features from the registered image data using a convolutional neural network, including ground feature outlines, texture features, and grayscale features; S44. Change Detection: Using a combination of interpolation and machine learning classification algorithms, changes in ground features in the target area are detected based on feature extraction results. The changed areas and types of changes are determined, and the accuracy is verified by comparing with manually measured data. S45. Differential Enhancement: Perform contrast enhancement, color mapping, and edge sharpening on the change detection results to highlight the differences in the changed areas; S5. Visualization Generation: Based on the processed multi-phase image data and laser point cloud data, a multi-temporal 3D model of the target area is constructed. Real-time rendering technology is used for color rendering, lighting simulation and texture mapping. It supports three comparison modes: dual-screen comparison, roll-up comparison and timeline playback, and generates visual content. S6. Interactive Review and Decision Support: Provides interactive operation functions such as model scaling, rotation, translation, and region selection. It supports multiple people to collaboratively review the visualized content manually and mark suspected changed areas. It performs quantitative analysis on changed areas, assesses the risk level, generates decision suggestions in combination with the decision rule base, and exports the visualization results and decision reports.

[0017] Compared with the prior art, the beneficial effects of the present invention are: UAV aerial surveying supports dynamic route adjustment and multi-sensor collaborative data acquisition. The overlap of multiple aerial survey routes is high, and the integrity and consistency of image data are strong, which greatly reduces the cost of manual intervention and data supplementation. By using deep learning, we can achieve automatic registration, feature extraction and change detection of multi-phase images. The processing accuracy is high, and there is no need for a lot of manual operation, which significantly improves efficiency. It supports 3D modeling and multi-mode comparison, clearly displays dynamic changes in ground features, and has rich interactive functions to meet the needs of single-person detailed viewing and multi-person collaborative review. Multi-dimensional quantitative analysis of changing data, combined with scenario-based risk assessment models for graded early warning, and matching targeted decision-making suggestions, provide direct decision support for engineering, disaster, environmental protection and other scenarios; Ensure the security and traceability of massive amounts of data across multiple periods, and avoid the risk of data loss or leakage. Attached Figure Description

[0018] Figure 1 This is a system configuration diagram of the present invention; Figure 2 This is a flowchart of the visualization imaging method combining multi-temporal automatic comparison of UAVs according to the present invention. Detailed Implementation

[0019] Please see Figure 1 and Figure 2 This invention provides a visualization imaging system that combines multi-temporal automatic comparison of UAVs, including... UAV aerial survey module The UAV aerial survey module is fundamental to acquiring high-quality imagery data across multiple periods. Its core function is to ensure the consistency of flight paths and the integrity of data at different points in time. Specifically, it includes the UAV itself, sensor units, flight path planning submodule, and flight control submodule. The UAV itself is a multi-rotor or fixed-wing UAV with long endurance and high stability, equipped with a redundant flight control system. It can fly stably in complex terrain (such as mountains and water areas) and in adverse weather conditions (such as light wind and low visibility). The endurance of a single aerial survey is no less than 50 minutes, and the operating radius can reach 10km, meeting the multi-temporal aerial survey needs of medium and large areas. Sensor Unit: Integrates three types of sensors: a high-definition optical camera, a thermal imaging camera, and a lidar, forming a multi-dimensional data acquisition capability. Among them, the high-definition optical camera has a resolution of no less than 20 million pixels, supports multispectral imaging, and can capture the texture and color details of ground objects; the thermal imaging camera has a temperature measurement range of -20℃ to 500℃ and a resolution of no less than 640×512, which can identify temperature changes of ground objects and is suitable for scenarios such as fire monitoring and abnormal equipment heating detection; the lidar has a ranging accuracy of ±5cm and a point cloud density of no less than 200 points / ㎡, which can accurately acquire the terrain elevation data of the target area and provide a foundation for 3D modeling. The flight path planning submodule employs a dual-mode flight path generation strategy of "preset + dynamic adjustment". Before aerial surveying, an initial preset flight path is generated by combining DEM (Digital Elevation Model) data of the target area, aerial survey accuracy requirements (e.g., ground sampling distance GSD not greater than 5cm), and historical aerial survey flight paths to ensure that the flight path covers the target area without omission. During the aerial survey, real-time UAV flight status data (position, speed, attitude) and environmental data (wind speed, wind direction, visibility) are received. The flight path is dynamically corrected using a PID control algorithm. For example, when the wind speed exceeds 5m / s, the flight speed and altitude are automatically adjusted to avoid flight path deviation. Simultaneously, a historical flight path database is established to record the flight path coordinates and flight parameters of each aerial survey, providing path references for subsequent repeated aerial surveys and ensuring that the overlap of multiple aerial survey paths is not less than 95%. Flight control submodule: Based on the flight path planning results, it precisely controls the UAV's flight attitude (roll, pitch, and yaw angle errors not exceeding ±1°), flight speed (adjustable within the range of 3~15m / s), and shooting parameters (such as shutter speed, ISO, and shooting interval). During shooting, it automatically matches the optimal parameters according to the sensor type. For example, optical cameras automatically reduce ISO to 100 in strong light environments, and thermal imaging cameras automatically increase gain at night. At the same time, it achieves centimeter-level positioning accuracy for the UAV through RTK (real-time dynamic positioning) technology, ensuring the consistency of shooting positions across multiple image phases and reducing the difficulty of image registration. Data transmission and storage module The data transmission and storage module is responsible for the real-time transmission, efficient storage, and security of multi-phase image data, addressing the risks of transmission interruptions, low storage efficiency, and security vulnerabilities associated with massive amounts of multi-phase data. It includes a wireless transmission submodule, an edge storage submodule, and a cloud storage submodule. Wireless transmission submodule: It adopts a dual-mode transmission method of "5G combined with satellite" to achieve real-time and stable transmission of image data. In areas with 5G signal coverage, it achieves high-speed transmission with peak rates of over 1Gbps through the 5G network to meet the real-time backhaul requirements of high-definition images and laser point cloud data. In remote mountainous areas and other areas with weak 5G signals, it automatically switches to satellite transmission mode with a transmission rate of up to 10Mbps to ensure no data loss. At the same time, it supports the function of resuming interrupted data transmission. When the connection is restored after the transmission is interrupted, it can continue the transmission from the point of interruption to avoid bandwidth waste caused by repeated transmission. Edge storage submodule: Located on the drone fuselage (with a built-in 1TB SSD) and the ground control station (equipped with an 8TB NAS storage device), it is used for temporary storage of image data. During data transmission, the data is first stored on the edge device, and the local temporary files are deleted after the transmission is complete, effectively avoiding data loss due to network interruption. At the same time, the edge storage device has data preprocessing capabilities, which can perform preliminary compression (using H.265 encoding format, with a compression ratio of 10:1) and format conversion on the image data, reducing the pressure on subsequent cloud processing. The cloud storage submodule adopts a distributed storage architecture, building a storage cluster based on the Hadoop Distributed File System (HDFS) to support petabyte-level massive data storage. It stores image data according to a hierarchical logic of "acquisition time-image type-target area" and establishes a metadata index. The metadata covers information such as acquisition time, geographical location (latitude and longitude range), sensor parameters (camera model, lidar frequency), flight attitude (roll, pitch angle), image resolution, and data size, and supports fast data retrieval and location by metadata. Data Security and Backup Submodule: To ensure data security and integrity, the cloud storage submodule employs multiple protection mechanisms. Firstly, data encryption: AES-256 encryption is used to encrypt stored data, and TLS 1.3 encryption is used during transmission to prevent data theft. Secondly, access control: Based on the RBAC (Role-Based Access Control) model, different permissions are assigned to different users (such as administrators, operators, and viewers), ensuring only authorized users can operate on the corresponding data. Thirdly, data backup: A combination of off-site backup (establishing backup nodes in different cities) and incremental backup (backing up only newly added or modified data) is used. Data is automatically backed up daily at midnight, and backup data is retained for 30 days. In case of data corruption or loss, data recovery can be completed within one hour through the data recovery submodule. Fourthly, operation logs: Record all user operations (login time, operation content, data access path), and retain logs for six months for subsequent auditing and traceability. Multi-temporal data processing module The multi-temporal data processing module is the core of achieving automated and accurate analysis of multi-phase images. It eliminates image interference and accurately identifies ground feature changes through a series of processing steps, including an image registration submodule, a feature extraction submodule, a change detection submodule, and a difference enhancement submodule. Image preprocessing submodule: Before registration, comprehensive preprocessing is performed on multi-phase image data to remove various interference factors. First, noise reduction: an algorithm combining Gaussian filtering and median filtering is used to remove salt-and-pepper noise and Gaussian noise in optical images, and thermal noise in thermal images. Second, distortion correction: based on camera intrinsic parameters (focal length, principal point coordinates) and distortion coefficients, radial and tangential distortion correction is performed on optical images to ensure image geometric accuracy. Third, radiometric correction: through dark current correction and atmospheric correction (based on the MODTRAN atmospheric radiative transfer model), the influence of light intensity changes and atmospheric scattering on image grayscale values ​​is eliminated, ensuring that the radiometric characteristics of multi-phase images remain consistent, laying the foundation for subsequent registration and change detection. The image registration submodule employs a deep learning-based feature point matching algorithm (such as the SuperPoint algorithm) combined with the geographic coordinate information of the target area to achieve automated, high-precision registration of multiple images. First, feature points with scale invariance and rotation invariance are extracted from the preprocessed images, and these feature points are matched using feature descriptors (such as the SIFT descriptor). Second, combined with terrain elevation data acquired by LiDAR and GPS positioning information, geographic coordinate constraints are applied to the matched feature points, and incorrect matching points are eliminated (using the RANSAC algorithm). Finally, spatial alignment of multiple images is achieved through homography matrix calculation and image resampling, with registration errors controlled within 2 pixels, far exceeding the accuracy of traditional manual registration. Feature Extraction Submodule: A feature extraction model is built based on a convolutional neural network (CNN) to extract key features from registered multi-period images. The model uses ResNet-50 as the backbone network and extracts multi-level features of ground objects through multi-layer convolution and pooling operations: shallow networks extract low-level features such as edges and textures, while deep networks extract high-level features such as outlines and shapes. At the same time, dedicated feature extraction branches are designed for different types of images (optical, thermal imaging, and laser point clouds). For example, a temperature feature extraction layer is added to thermal imaging images, and an elevation feature extraction layer is added to laser point clouds. Finally, multiple types of features are fused to form a comprehensive set of ground object features, providing rich data support for change detection. The change detection submodule employs a fusion detection strategy combining "difference method + machine learning classification algorithm" to accurately identify ground feature changes. First, it calculates the grayscale difference between corresponding pixels from multiple image periods to initially identify suspected change areas. Second, it inputs the ground feature features obtained from the feature extraction submodule into a random forest classification model to classify suspected change areas and determine the change type. Finally, it verifies the accuracy by comparing the changes with manually measured data (such as RTK-measured ground feature coordinates) or high-resolution satellite imagery (resolution not less than 0.5m) to calculate the accuracy (not less than 95%), recall (not less than 92%), and F1 score (not less than 93%), ensuring the reliability of the change detection results. If the accuracy does not meet the standards, it returns to the feature extraction stage, adjusts the model parameters, and reprocesses. The difference enhancement submodule highlights the differences between changed and unchanged areas through various image processing techniques, improving the recognizability of the changes. Firstly, contrast enhancement uses a histogram equalization algorithm to adjust the grayscale contrast of changed areas, making subtle changes more noticeable. Secondly, color mapping assigns unique colors to different types of changed areas (e.g., blue for construction changes, yellow for vegetation reduction, and red for fires), creating intuitive color markers. Thirdly, edge sharpening uses the Laplacian operator to sharpen the edges of changed areas, enhancing the clarity of terrain features. Fourthly, 3D difference enhancement combines laser point cloud data to render elevation changes in changed areas, such as using warm colors for areas with increased elevation and cool colors for areas with decreased elevation, visually presenting changes in terrain undulations.

[0020] Visualization and interactive module The visualization and interactive generation module is responsible for presenting the processed changes in a multi-dimensional and highly interactive format, supporting manual review and detailed viewing. It includes a 3D modeling submodule, a dynamic rendering submodule, a comparison display submodule, and an interactive operation submodule. The 3D modeling submodule constructs a multi-temporal, high-precision 3D model of the target area based on multi-period imagery and terrain data acquired by LiDAR. First, the LiDAR point cloud data undergoes denoising (removing ground vegetation and building obstructions), filtering (separating ground points from non-ground points), and interpolation to generate a DSM (Digital Surface Model) and a DTM (Digital Terrain Model). Second, the texture information of the optical images is mapped onto the DSM surface, achieving precise image-model alignment through texture mapping algorithms (such as perspective texture mapping). Finally, 3D models are constructed for data from different time points, forming a multi-temporal model sequence with centimeter-level accuracy, clearly showcasing subtle changes in terrain features (such as increased building height and changes in road smoothness). The dynamic rendering submodule employs a real-time rendering engine (such as Unity or Unreal Engine) to perform high-quality rendering of multi-temporal 3D models, restoring the realistic scene of the target area. Firstly, color rendering: based on the real color information of the image, and combined with ambient lighting conditions (such as sunny days, cloudy days, and nighttime), the model's colors are adjusted to ensure the rendering effect is consistent with the actual scene. Secondly, lighting simulation: simulating the solar altitude angle and light intensity at different times (such as morning, noon, and evening) generates dynamic lighting effects, showcasing the appearance changes of ground features under different lighting conditions. Thirdly, texture mapping: applying high-resolution texture mapping (resolution no less than 2K) to the model surface to restore the detailed textures of ground features (such as building wall materials and road markings). Fourthly, special effects rendering: adding special effects such as raindrops and fog for special scenes (such as rainy days and foggy days) to enhance the realism and immersion of the model. The comparison display submodule offers three flexible comparison modes to meet the needs of viewing changes in different scenarios. First, the dual-screen comparison mode displays 3D models or 2D images from different time points on the left and right screens respectively. Users can simultaneously operate the models on both screens (zoom, rotate) to intuitively compare changes in terrain features from the same perspective. Second, the roll-up comparison mode overlays images or models from two time points on the same screen by sliding the roll-up. The transition effect of changing areas can be observed in real time during the roll-up, facilitating the location of subtle changes. Third, the timeline playback mode arranges multiple models in chronological order on a timeline. Users can dynamically view the process of terrain feature changes (such as the entire process of a building from excavation to completion) using playback, pause, fast forward, and slow motion operations. The playback speed can be adjusted between 0.5x and 2x. Interactive Operation Submodule: Provides rich interactive functions, supporting users to deeply view and review the visualization results; First, basic operations: Supports model scaling (scaling ratio 1:10~1:1000), rotation (360° free rotation), and translation (movement in any direction), which users can operate via mouse, touch screen, or gamepad; Second, region selection: Supports three region selection methods: rectangle, circle, and polygon. Users can select areas of interest (such as a construction section or a vegetation area), and the system automatically zooms in on the area and displays detailed change information (such as changed area, change type, and change time); Third, marking and annotation: Users can mark suspected changed areas on the model and add text annotations (such as "This is where the road..."). The system supports multiple users simultaneously logging in. Through account permission control, multiple users can operate and view the same model and communicate through real-time message chat and voice call functions to jointly complete change review and decision discussion. The system also supports exporting results in various formats, including images (JPG, PNG, resolution up to 8K), videos (MP4, AVI, frame rate 30fps), 3D model files (OBJ, FBX), and PDF change reports. The exported files can be used directly for reporting, archiving, or further analysis.

[0021] Decision support module The decision support module, based on visualized results and change analysis data, provides users with quantitative analysis, risk assessment, and targeted decision-making suggestions, realizing the transformation from data to decision. It includes a change analysis submodule, a risk assessment submodule, and a suggestion generation submodule. The Change Analysis submodule performs multi-dimensional quantitative analysis of the changed area, generating a detailed change analysis report. Firstly, it analyzes the area and scale of the changed region, calculating the projected area and actual occupied area (combined with terrain slope correction), and statistically analyzing the proportion of different change types (e.g., building construction accounts for 60%, road construction for 30%, and greening for 10%). Secondly, it analyzes the rate of change, calculating the average rate of change of the changed area based on multiple aerial survey intervals (e.g., average monthly building height and average monthly vegetation reduction area), and predicting future change trends (based on linear regression or exponential smoothing models). Thirdly, it analyzes specific land cover attributes, such as changes in building height (calculated using lidar elevation data), road smoothness (calculated using DSM to determine the elevation standard deviation), vegetation coverage (calculated using the NDVI index from optical imagery), and temperature (calculated using the average temperature from thermal imaging), forming a multi-dimensional change data matrix. Risk Assessment Submodule: This module establishes a multi-factor risk assessment model based on the application scenarios of the target area, classifying and assessing the risks that may arise from changes in the area. First, risk assessment indicators are determined, varying across different scenarios: In engineering construction scenarios, indicators include construction progress deviation (the difference between actual and planned progress), quality defects (such as excessive road settlement and building verticality deviation), and safety hazards (such as lack of guardrails and improper stacking of construction materials). In disaster monitoring scenarios, indicators include disaster spread rate (such as fire spread area / hour and flood inundation rate), impact range (the population and number of buildings threatened by the disaster), and severity (such as building damage rate and vegetation mortality rate). In environmental protection scenarios, indicators include vegetation damage degree, water pollution concentration, and soil erosion. Second, the Analytic Hierarchy Process (AHP) is used to determine the weights of each indicator, and a fuzzy comprehensive evaluation method is used to calculate risk scores, classifying risk levels into low risk (0-30 points), medium risk (31-60 points), and high risk (61-100 points). A risk heatmap is then generated to visually mark high-risk areas. The system recommends generating a sub-module that, based on change analysis and risk assessment results and combined with a pre-defined decision rule base, provides users with targeted and actionable decision suggestions. The decision rule base is built by collecting industry standards, expert experience, and historical cases. For example, in engineering construction scenarios, if construction progress is delayed by more than 10%, the rule base triggers a suggestion to "increase the input of construction personnel and equipment, and optimize construction procedures"; if road settlement exceeds the standard (more than 5cm), it triggers a suggestion to "suspend traffic on this section of road and repair it using grouting reinforcement technology"; in disaster monitoring scenarios, if the fire risk level is high, it triggers a suggestion to "deploy fire-fighting equipment and personnel to the scene, evacuate surrounding residents, and set up firebreaks"; in environmental protection scenarios, if the vegetation damage rate exceeds 20%, it triggers a suggestion to "stop destructive behavior, carry out artificial replanting, and establish a vegetation monitoring ledger". Simultaneously, the system supports user-defined decision rules, allowing users to add or modify rules according to specific project needs, ensuring the applicability and flexibility of the suggestions.

[0022] The visualization imaging method combining multi-temporal automatic comparison of UAVs includes the following steps: S1. Aerial Survey Preparation: Based on the terrain information of the target area, the accuracy requirements of aerial survey, and the application scenario, determine the time interval, shooting parameters, and flight path planning scheme for UAV aerial survey, select UAVs equipped with high-definition optical cameras, thermal imaging cameras, and lidar, and debug the UAV's flight control system, sensor units, and data transmission system. S2. Multi-temporal image acquisition: According to the flight path planning scheme, control the UAV to repeatedly conduct aerial surveys of the same target area at different time points, collect multi-type image data of the target area through the sensor unit, and record the metadata of each aerial survey; S3. Data transmission and storage: Through 5G combined with satellite dual-mode transmission, the collected multi-stage image data and metadata are transmitted in real time to the edge storage device for temporary storage. At the same time, the data is synchronously transmitted to the cloud storage system. A distributed hierarchical storage architecture is adopted, and the data is classified and stored according to the collection time, image type and target area, and a metadata index is established. S4. Multi-temporal data processing: S41. Image preprocessing: Denoising, distortion correction and radiometric correction are performed on multi-phase image data to remove noise interference, optical distortion and illumination differences in the images. S42. Automated registration: A feature point matching algorithm based on deep learning is used to register the preprocessed multi-period image data in combination with the geographic coordinate information of the target area to eliminate image offset; S43. Feature Extraction: Extract key features from the registered image data using a convolutional neural network, including ground feature outlines, texture features, and grayscale features; S44. Change Detection: Using a combination of interpolation and machine learning classification algorithms, changes in ground features in the target area are detected based on feature extraction results. The changed areas and types of changes are determined, and the accuracy is verified by comparing with manually measured data. S45. Differential Enhancement: Perform contrast enhancement, color mapping, and edge sharpening on the change detection results to highlight the differences in the changed areas; S5. Visualization Generation: Based on the processed multi-phase image data and laser point cloud data, a multi-temporal 3D model of the target area is constructed. Real-time rendering technology is used for color rendering, lighting simulation and texture mapping. It supports three comparison modes: dual-screen comparison, roll-up comparison and timeline playback, and generates visual content. S6. Interactive Review and Decision Support: Provides interactive operation functions such as model scaling, rotation, translation, and region selection. It supports multiple people to collaboratively review the visualized content manually and mark suspected changed areas. It performs quantitative analysis on changed areas, assesses the risk level, generates decision suggestions in combination with the decision rule base, and exports the visualization results and decision reports.

Claims

1. A visualization imaging system combining multi-temporal automatic comparison of UAVs, characterized in that: include The UAV aerial survey module is used to repeatedly conduct aerial surveys of the same target area at different times to acquire multiple periods of image data. The data transmission and storage module is used to transmit multi-phase image data acquired by the UAV aerial survey module in real time, and to store the image data and related metadata in a hierarchical storage structure. A multi-temporal data processing module is used to perform automated registration, feature extraction, change detection, and difference enhancement processing on multi-phase image data. The visualization generation and interaction module is used to generate visualization content from the change results obtained by the multi-temporal data processing module in a variety of intuitive forms, and provide manual review and interaction. The decision support module provides users with decision suggestions based on visualized content and change analysis results.

2. The visualization imaging system combining multi-temporal automatic comparison of UAVs according to claim 1, characterized in that: The UAV aerial survey module includes the UAV body, sensor units, a flight path planning submodule, and a flight control submodule. The sensor units, mounted on the UAV body, include a high-definition optical camera, a thermal imaging camera, and a lidar, used to collect various types of image data of the target area. The flight path planning submodule is used to generate a preset flight path or dynamically adjust the flight path based on the terrain information of the target area, the aerial survey accuracy requirements, and historical aerial survey routes. The flight control submodule is used to control the flight attitude, flight speed, and shooting parameters of the UAV body according to the flight path generated by the flight path planning submodule.

3. The visualization imaging system combining multi-temporal automatic comparison of UAVs according to claim 1, characterized in that: The data transmission and storage module includes a wireless transmission submodule, an edge storage submodule, and a cloud storage submodule. The wireless transmission submodule uses a 5G-satellite dual-mode transmission method to achieve real-time transmission of image data. The edge storage submodule is set up on the UAV itself or a ground control station for temporary storage of image data. The cloud storage submodule adopts a distributed storage architecture, storing image data in layers according to acquisition time, image type, and target area, and establishing metadata indexes. The metadata includes acquisition time, geographical location, sensor parameters, flight attitude, and image resolution.

4. The visualization imaging system combining multi-temporal automatic comparison of UAVs according to claim 3, characterized in that: The cloud storage submodule also includes a data backup and recovery submodule, which uses a combination of off-site backup and incremental backup to back up the stored image data and metadata. When data is damaged or lost, it can be quickly recovered through the data recovery submodule.

5. The visualization imaging system combining multi-temporal automatic comparison of UAVs according to claim 1, characterized in that: The multi-temporal data processing module includes an image registration submodule, a feature extraction submodule, a change detection submodule, and a difference enhancement submodule. The image registration submodule uses a deep learning-based feature point matching algorithm, combined with the geographic coordinate information of the target area, to automatically register multi-phase image data, eliminating image shifts caused by changes in UAV flight attitude, differences in shooting angles, and terrain undulations. The feature extraction submodule extracts key features from multi-phase image data through a convolutional neural network, including ground feature outlines, texture features, and grayscale features. The change detection submodule, based on feature extraction results, uses a combination of difference method and machine learning classification algorithm to detect changes in ground features in the target area at different time points, and determines the changed areas and change types; the difference enhancement submodule highlights the differences between changed and unchanged areas through contrast enhancement, color mapping and edge sharpening processing.

6. The visualization imaging system combining multi-temporal automatic comparison of UAVs according to claim 5, characterized in that: Before performing automated registration, the image registration submodule preprocesses the image data, including denoising, distortion correction, and radiometric correction, to remove noise interference, optical distortion, and illumination differences from the image that affect the registration accuracy. The change detection submodule also verifies the accuracy of the detected changes by comparing them with manually measured data or high-resolution satellite image data to calculate the accuracy, recall, and F1 score of the change detection.

7. The visualization imaging system combining multi-temporal automatic comparison of UAVs according to claim 1, characterized in that: The visualization generation and interaction module includes a 3D modeling submodule, a dynamic rendering submodule, a comparison display submodule, and an interactive operation submodule; the 3D modeling submodule constructs a multi-temporal 3D model of the target area based on multi-period image data and terrain data collected by LiDAR. The dynamic rendering submodule uses real-time rendering technology to perform color rendering, lighting simulation and texture mapping on multi-temporal 3D models to restore the real scene of the target area. The comparison display submodule visually shows the changes in the target area at different points in time.

8. The visualization imaging system combining multi-temporal automatic comparison of UAVs according to claim 1, characterized in that: The decision support module includes a change analysis submodule, a risk assessment submodule, and a suggestion generation submodule; the change analysis submodule performs quantitative analysis on the area, rate of change, and trend of change of the changed area and generates a change analysis report. The risk assessment submodule, in conjunction with the application scenarios of the target area, establishes a risk assessment model to assess the risks that may be caused by changes in the area and determine the risk level. The suggestion generation submodule provides users with targeted decision suggestions based on change analysis results and risk assessment results, combined with a preset decision rule library.

9. A visualization imaging method combining multi-temporal automatic comparison of UAVs, characterized by: The visualization imaging system combining multi-temporal automatic comparison of UAVs as described in any one of claims 1-8 includes the following steps: S1. Aerial Survey Preparation: Based on the terrain information of the target area, the accuracy requirements of aerial survey, and the application scenario, determine the time interval, shooting parameters, and flight path planning scheme for UAV aerial survey, select UAVs equipped with high-definition optical cameras, thermal imaging cameras, and lidar, and debug the UAV's flight control system, sensor units, and data transmission system. S2. Multi-temporal image acquisition: According to the flight path planning scheme, control the UAV to repeatedly conduct aerial surveys of the same target area at different time points, collect multi-type image data of the target area through the sensor unit, and record the metadata of each aerial survey; S3. Data transmission and storage: Through 5G combined with satellite dual-mode transmission, the collected multi-stage image data and metadata are transmitted in real time to the edge storage device for temporary storage. At the same time, the data is synchronously transmitted to the cloud storage system. A distributed hierarchical storage architecture is adopted, and the data is classified and stored according to the collection time, image type and target area, and a metadata index is established. S4. Multi-temporal data processing: S41. Image preprocessing: Denoising, distortion correction and radiometric correction are performed on multi-phase image data to remove noise interference, optical distortion and illumination differences in the images. S42. Automated registration: A feature point matching algorithm based on deep learning is used to register the preprocessed multi-period image data in combination with the geographic coordinate information of the target area to eliminate image offset; S43. Feature Extraction: Extract key features from the registered image data using a convolutional neural network, including ground feature outlines, texture features, and grayscale features; S44. Change Detection: Using a combination of interpolation and machine learning classification algorithms, changes in ground features in the target area are detected based on feature extraction results. The changed areas and types of changes are determined, and the accuracy is verified by comparing with manually measured data. S45. Differential Enhancement: Perform contrast enhancement, color mapping, and edge sharpening on the change detection results to highlight the differences in the changed areas; S5. Visualization Generation: Based on the processed multi-phase image data and laser point cloud data, a multi-temporal 3D model of the target area is constructed. Real-time rendering technology is used for color rendering, lighting simulation and texture mapping. It supports three comparison modes: dual-screen comparison, roll-up comparison and timeline playback, and generates visual content. S6. Interactive Review and Decision Support: Provides interactive operation functions such as model scaling, rotation, translation and region selection, supports multiple people to collaboratively review the visualized content manually, and marks suspected areas of change; Quantitative analysis of the changing areas is conducted to assess the risk level. Decision recommendations are generated by combining the decision rule base, and visualization results and decision reports are exported.

Images