Airborne aerial survey real-time processing system and mapping method

Abstract

The invention discloses an airborne aerial survey real-time processing system and a mapping method. According to the method, an integrated load integration module, an aerial survey controller, a power supply module, a communication module and an onboard processing module are included; the integrated load integration module is used for synchronously acquiring optical images, radar images and high-precision POS data; the aerial survey controller is used for receiving and analyzing the high-precision POS data in real time; the power supply module is used for supplying power; the communication module is used for realizing bidirectional data mutual transmission between the aerial survey controller and the onboard processing module and the ground control station; and the onboard processing module is used for receiving the optical image, the radar image and the high-precision POS data in real time, and sequentially performing real-time compression, sensor correction, geometric correction, image enhancement and splicing on the optical image to obtain a splicing-seam-free orthoimage. According to the method, minute-level acquisition of the first map of the disaster situation can be realized, and severe weather adaptability, high map forming precision and system safety and convenience are considered.

Description

Technical Field

[0001] This invention belongs to the field of unmanned aerial vehicle (UAV) technology, and more specifically, relates to an airborne aerial survey real-time processing system and mapping method. Background Technology

[0002] Unmanned aerial vehicle (UAV) surveying technology has been widely used in emergency mapping support for sudden natural disasters such as earthquakes, floods, and landslides. After a disaster occurs, quickly acquiring and transmitting the first on-site image of the disaster area (the "first image") is crucial for the command center to assess the disaster situation, evaluate losses, plan rescue routes, and allocate resources, and is key to improving emergency response efficiency.

[0003] However, current mainstream disaster emergency aerial survey systems and operational modes have two inherent flaws that severely restrict the timeliness of obtaining the "first map":

[0004] 1. Dispersed payloads, poor integration and synchronization: Traditional airborne aerial survey systems typically install or simply stack mission payloads such as optical cameras, synthetic aperture radar (SAR), and position and attitude (POS) systems separately. This non-integrated design leads to significant physical reference deviations and time synchronization errors between the sensors, making it difficult to strictly align the multi-source data (optical, radar, position and attitude) in time and space. Subsequent data fusion and comprehensive analysis require complex post-registration and processing, which is not only cumbersome but also introduces additional accuracy losses.

[0005] 2. Delayed processing and low timeliness of mapping: Existing systems generally adopt a "flying-onboard acquisition, on-the-ground processing, and ground mapping" model. After completing their flight missions, UAVs need to bring back massive amounts of raw image data to the ground, where high-performance ground workstations are used for data download, distortion correction, aerial triangulation, digital elevation model generation, orthorectification, and stitching, among other processing steps. The entire process takes several hours or even longer, failing to meet the urgent need for minute-level information response within the "golden 72 hours" of disaster relief. Especially under adverse conditions such as fog, rain, and nighttime, the imaging quality of optical cameras drops sharply, and SAR data with penetrating capabilities cannot be quickly coordinated with optical data, further delaying the acquisition of critical information.

[0006] In summary, existing technologies cannot achieve multi-source synchronous acquisition and real-time onboard processing of geographic information at disaster sites, making it difficult to support the minute-level rapid acquisition of the "first map" of the disaster situation for emergency command. Therefore, there is an urgent need for a highly integrated real-time aerial survey system capable of processing and transmitting data while in flight, in order to overcome the current timeliness bottleneck in emergency mapping support. Summary of the Invention

[0007] The purpose of this invention is to propose an airborne aerial survey real-time processing system and mapping method to solve the problems of poor data synchronization caused by the dispersed payload of traditional aerial survey systems and low timeliness of mapping due to reliance on ground processing; to achieve minute-level acquisition of the "first map" of disaster situations, taking into account adaptability to severe weather, high mapping accuracy and system safety and convenience, and to support emergency rescue decision-making and basic surveying and mapping applications.

[0008] To achieve the above objectives, in a first aspect, the present invention proposes an airborne aerial survey real-time processing system, applied to medium and large-sized unmanned aerial vehicle (UAV) flight platforms, the system comprising:

[0009] Integrated payload module, aerial survey controller, power supply module, communication module and onboard processing module;

[0010] The integrated payload module is rigidly connected to the bottom plate inside the payload compartment of the medium and large UAV flight platform, and is used to simultaneously acquire optical images, radar images and high-precision POS data.

[0011] The aerial survey controller is communicatively connected to the integrated payload integration module and the onboard processing module, respectively, and is used to receive and parse the high-precision POS data in real time, and to send the parsed high-precision POS data to the integrated payload integration module and the onboard processing module.

[0012] The power supply module is centrally powered by the UAV flight platform and is used to power the integrated payload module, the aerial survey controller, the communication module, and the onboard processing module.

[0013] The communication module is connected to the aerial survey controller, the onboard processing module and the ground control station respectively, and is used to realize bidirectional data exchange between the aerial survey controller and the onboard processing module and the ground control station.

[0014] The onboard processing module is communicatively connected to the integrated payload module and is used to receive the optical image, radar image and high-precision POS data in real time. It performs real-time compression, sensor correction, geometric correction, image enhancement and stitching on the optical image in sequence to obtain a seamless orthophoto image. Geometric correction is performed using the high-precision POS data and image enhancement is performed using the radar image.

[0015] Optionally, the integrated load integration module includes:

[0016] The system includes an aerial survey camera, a synthetic aperture radar, a combined inertial navigation system, and a stabilization platform, wherein the stabilization platform integrates slip rings to form a reference side and a load side with relative motion.

[0017] The solver of the synthetic aperture radar is fixed to the reference side above the slip ring. The solver is used to perform real-time motion compensation and geocoding processing on the radar image based on the parsed high-precision POS data.

[0018] The inertial measurement unit of the combined inertial navigation system is fixed to the reference side above the slip ring;

[0019] The lens of the aerial survey camera and the transceiver antenna of the synthetic aperture radar are rigidly connected side by side to the load side below the slip ring, and the lens and transceiver antenna are exposed outside the load compartment to achieve unobstructed observation;

[0020] The triaxial installation error angle between the aerial survey camera, transceiver antenna and inertial measurement unit shall not exceed 2° to ensure geometric consistency between data.

[0021] The stabilization platform is fixed to the bottom plate inside the payload compartment in a seated installation manner. It can automatically adjust the attitude of the payload side based on the parsed high-precision POS data to compensate for disturbances during the flight of the medium and large UAV flight platform.

[0022] Optionally, the communication module integrates a line-of-sight link and a satellite communication link, and can switch between the two according to instructions or link status.

[0023] Optionally, the aerial survey controller is provided with two asynchronous 422 interfaces, which are used to receive uplink remote control commands from the ground control station and send status information to the ground control station via the line-of-sight link, respectively; it is also provided with a synchronous 422 interface, which is used to send real-time data to the ground control station via the line-of-sight link; and it is also provided with a gigabit network port, which is used to send real-time data to the ground control station via the satellite communication link.

[0024] Optionally, the onboard processing module includes an IIQ lossless compression unit for real-time lossless compression of the optical image.

[0025] Optionally, the image enhancement process includes:

[0026] The radar image is used to perform one or more of the following processing on the optical image: haze removal, color cast adjustment, and dark enhancement.

[0027] Optionally, the onboard processing module employs multi-threaded parallel processing technology to perform pipelined parallel processing on the compression, correction, and stitching tasks of multiple optical images.

[0028] Optionally, the aerial survey camera is a dual-lens camera, the optical image is a dual-lens image, and the stitching uses fully automatic texture matching to seamlessly stitch the dual-lens images to generate the seamless orthophoto image.

[0029] Optionally, the geometric correction is a direct geolocation correction based on the high-precision POS data;

[0030] The sensor calibration includes distortion correction based on calibration parameters and dynamic black level correction based on sensor temperature.

[0031] Secondly, the present invention proposes a mapping method based on the airborne aerial survey real-time processing system described in any one of the first aspects, the method comprising:

[0032] Based on the mission area, a flight path is planned, and the medium and large-sized UAVs equipped with the integrated payload module are controlled to fly to the mission area.

[0033] Simultaneously acquire optical images, radar images, and high-precision POS data, and analyze the high-precision POS data;

[0034] The flight attitude of the medium and large UAV is actively compensated in real time based on the parsed high-precision POS data.

[0035] The optical image is sequentially compressed in real time, sensor calibrated, geometrically calibrated, image enhanced and stitched to obtain a seamless orthophoto image. Geometric calibration is performed using the high-precision POS data and image enhancement is performed using the radar image.

[0036] The seamless orthophoto image is then transmitted to the ground control station.

[0037] The beneficial effects of this invention are as follows: By rigidly connecting the integrated payload module to the bottom plate inside the payload bay of a medium-to-large UAV, synchronous acquisition of optical images, radar images, and high-precision POS data is achieved, effectively avoiding problems such as data asynchrony and poor geometric consistency caused by the dispersed payloads in traditional aerial survey systems. This provides high-quality, highly collaborative raw data support for subsequent image processing. The aerial survey controller receives and parses high-precision POS data in real time and transmits it back to the integrated payload module, establishing an efficient collaborative link between various payload components. This ensures precise linkage between data acquisition and attitude control, reducing errors caused by the disconnect between data transmission and parsing. Furthermore, the centralized power supply module of the UAV flight platform provides unified power to all modules of the system, simplifying the power supply architecture, improving the stability and reliability of power supply, and reducing the risk of failure that may be caused by independent power supply to multiple modules. The system establishes a two-way data transmission channel between the aerial survey controller, the onboard processing module, and the ground control station through a real-time transmission module. This ensures the rapid issuance of ground control commands and the efficient return of processed image data, meeting the stringent real-time requirements of emergency aerial surveys and other scenarios. The onboard processing module sequentially performs real-time compression, sensor correction, geometric correction, radar image enhancement, and seamless stitching on optical images. This not only efficiently reduces data volume, corrects image distortion and positional deviations, and optimizes image quality, but also rapidly generates seamless orthophotos. This breaks the limitations of traditional aerial surveys that rely on ground-based post-processing, significantly improving mapping efficiency. Furthermore, leveraging the enhancement advantages of radar imagery, the system's adaptability to complex weather and severe lighting conditions is enhanced, ensuring high accuracy and high availability of aerial survey results. This provides efficient and reliable technical support for emergency rescue, basic surveying, and other fields.

[0038] The system of the present invention has other features and advantages that will be apparent from or will be set forth in detail in the accompanying drawings and following detailed description, which together serve to explain the particular principles of the invention. Attached Figure Description

[0039] The above and other objects, features and advantages of the present invention will become more apparent from the accompanying drawings, in which like reference numerals generally denote like parts.

[0040] Figure 1 A schematic diagram of an airborne aerial survey real-time processing system according to Embodiment 1 of the present invention is shown.

[0041] Figure 2 A schematic diagram of an integrated load module according to Embodiment 1 of the present invention is shown.

[0042] Figure 3 A schematic diagram of an integrated load module according to Embodiment 1 of the present invention is shown.

[0043] Figure 4 A schematic diagram of an integrated load module according to Embodiment 1 of the present invention is shown.

[0044] Figure 5 A schematic diagram of a communication interface according to Embodiment 1 of the present invention is shown. Detailed Implementation

[0045] The invention will now be described in more detail with reference to the accompanying drawings. While preferred embodiments of the invention are shown in the drawings, it should be understood that the invention can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that the invention will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

[0046] Example 1

[0047] like Figure 1 As shown, this embodiment provides an airborne aerial survey real-time processing system applied to medium and large-sized unmanned aerial vehicle (UAV) flight platforms. The system includes:

[0048] Integrated payload module, aerial survey controller, power supply module, communication module and onboard processing module;

[0049] The integrated payload module is rigidly connected to the bottom plate inside the payload bay of the medium and large UAV flight platform, and is used to simultaneously acquire optical images, radar images and high-precision POS data;

[0050] The aerial survey controller is communicatively connected to the integrated payload integration module and the onboard processing module, respectively, to receive and parse high-precision POS data in real time, and to send the parsed high-precision POS data to the integrated payload integration module and the onboard processing module.

[0051] The power supply module is centrally powered by the UAV flight platform and is used to power the integrated payload module, aerial survey controller, communication module and onboard processing module;

[0052] The communication module is connected to the aerial survey controller, the onboard processing module and the ground control station respectively, and is used to realize bidirectional data exchange between the aerial survey controller and the onboard processing module and the ground control station.

[0053] The onboard processing module is communicatively connected to the integrated payload module to receive optical images, radar images, and high-precision POS data in real time. It then performs real-time compression, sensor correction, geometric correction, image enhancement, and stitching on the optical images to obtain a seamless orthophoto. Geometric correction is performed using high-precision POS data, and image enhancement is performed using radar images.

[0054] Specifically, this embodiment presents a highly integrated airborne aerial survey real-time processing system. Using a medium-to-large-sized UAV as its flight platform, it aims to achieve rapid acquisition, real-time processing, and reliable transmission of geographic information in disaster emergency scenarios. The system consists of five closely coordinated functional modules: an integrated payload module, serving as the system's sensing front end, is rigidly fixed to the UAV's payload bay floor. This module integrates a visible light aerial survey camera, synthetic aperture radar, and a combined navigation system, enabling simultaneous acquisition of high-resolution optical images with spatiotemporal consistency, all-weather radar images, and crucial high-precision position and attitude data. This provides a unified and reliable multi-source data foundation for subsequent processing. The aerial survey controller, acting as the intelligent control hub of the entire system, establishes communication connections with both the integrated payload module and the onboard processing module. It receives and parses raw POS data from the combined navigation system in real time, then synchronously distributes the parsed high-precision pose information to the actuators within the integrated payload module and the onboard processing module, thereby providing a unified spatiotemporal reference for flight attitude stabilization and image geometry processing. The power supply module adopts a centralized design, with power provided uniformly by the UAV flight platform. It also possesses necessary circuit protection functions to ensure the continuous and stable operation of all airborne electronic equipment in complex flight environments. The communication module serves as a data bridge between the system and the ground command center, interconnecting with the aerial survey controller, onboard processing module, and ground control station to establish a reliable two-way data transmission link. This link not only enables real-time transmission of processing results and system status but also receives and forwards control commands from the ground, achieving closed-loop interaction between the airborne platform and ground command. The onboard processing module is the core of the entire system's real-time processing capabilities. It receives real-time data streams from the integrated module, including optical images, radar images, and synchronized high-precision POS data, and executes a highly optimized onboard processing pipeline: First, massive amounts of optical images are compressed in real time to alleviate transmission and storage pressure; then, sensor calibration is performed to eliminate the equipment's own system errors; next, geometric correction is completed using high-precision POS data to assign accurate geographic coordinates to the images; finally, a large-scale seamless orthorectified image map is generated through an efficient stitching algorithm. During this process, the system innovatively introduces synchronously acquired radar image data, utilizing its penetration characteristics to enhance the optical images, effectively improving image quality and information integrity under obscured weather conditions such as fog, haze, and clouds. Through the deep coupling and pipeline operation of the above modules, the entire system achieves full automation from synchronous data acquisition and real-time processing to instant result download, completely changing the traditional operation mode that relies on ground post-processing, and providing powerful technical equipment support for application scenarios requiring minute-level response speeds, such as emergency rescue and disaster assessment.

[0055] like Figure 2 , Figure 3 and Figure 4As shown, in this embodiment, the integrated load integration module 1 includes:

[0056] The system includes an aerial survey camera 2, a synthetic aperture radar, a combined inertial navigation system, and a stabilization platform 3. The stabilization platform 3 integrates a slip ring 4 to form a reference side and a load side with relative motion.

[0057] The solver 5 of the synthetic aperture radar is fixed to the reference side above the slip ring 4. The solver 5 is used to perform real-time motion compensation and geocoding processing on radar images based on the parsed high-precision POS data.

[0058] The inertial measurement unit 6 of the integrated inertial navigation system is fixed to the reference side above the slip ring 4;

[0059] The lens 7 of the aerial survey camera and the transceiver antenna 8 of the synthetic aperture radar are rigidly connected side by side to the load side below the slip ring 4, and the lens 7 and the transceiver antenna 8 are exposed outside the load compartment 9 to achieve unobstructed observation.

[0060] The triaxial installation error angle between the aerial survey camera 2, the transceiver antenna 8, and the inertial measurement unit 6 shall not exceed 2° to ensure geometric consistency between data.

[0061] The stabilization platform 3 is fixed to the bottom plate inside the payload compartment 9 in a seat-mounted manner. It can automatically adjust the attitude of the payload side based on the analyzed high-precision POS data to compensate for disturbances during the flight of medium and large UAV flight platforms.

[0062] Specifically, the integrated payload module 1 constitutes the core sensing and stabilization unit of the entire system, employing a highly precise mechanical and electrical integrated design. This module integrates an aerial survey camera 2, a synthetic aperture radar, a combined inertial navigation system, and a three-axis stabilization platform 3. The stabilization platform 3 internally integrates the crucial slip ring 4 component, which is securely mounted to the platform frame via its flange 11 and features a handle 10 for easy operation. The presence of the slip ring 4 mechanically divides the entire module into a reference side and a load side, forming relative motion. During mechanical integration, it is crucial to ensure that the measurement coordinate system (XYZ axes) of the inertial measurement unit 6, fixed to the reference side above the slip ring 4, is strictly aligned with the mechanical control coordinate system of the stabilization platform 3 itself. Specifically, the two axes must be parallel and perpendicular to each other, corresponding to the front, horizontal right, and vertical upward axes, respectively, with an installation error of less than 2°. This forms the physical basis for subsequent high-precision attitude closed-loop control. On this reference, a synthetic aperture radar (SAR) solver 5 is securely mounted on the reference side above slip ring 4. This solver is responsible for real-time motion compensation and geocoding of the radar imagery based on the resolved high-precision POS data. On the payload side below slip ring 4, the lens 7 of the aerial survey camera 2, the SAR transceiver antenna 8, and the aerial survey controller are rigidly connected side-by-side as a single unit. The lens 7 and transceiver antenna 8 are exposed outside the UAV payload bay 9 to achieve unobstructed observation, and the radiating portion of the transceiver antenna 8 is located in the external space of slip ring 4. To ensure the uniformity of the spatial reference for multi-source data, the triaxial installation error angle between the optical axis of the aerial survey camera 2, the beam center of the SAR transceiver antenna 8, and the measurement axis of the inertial measurement unit 6 is precisely calibrated to no more than 2°. The entire stabilization platform 3 is fixed to the internal base plate of the payload bay 9 in a seat-mounted manner. Based on the resolved high-precision POS data, it can automatically drive the payload side to perform precise attitude adjustments relative to the reference side, effectively compensating for attitude disturbances during UAV flight and providing an ultra-stable platform for the imaging payload.

[0063] In this embodiment, the communication module integrates a line-of-sight link and a satellite communication link, and can switch between the two according to instructions or link status.

[0064] Specifically, the communication module is designed as a highly intelligent dual-link redundant communication system. Its hardware and software deeply integrate a line-of-sight (LAS) wireless link and a satellite communication link, and it has the ability to adaptively or controllably switch between the two links based on preset strategies or real-time conditions. The LAS link is typically based on a high-speed wireless data radio, operating in a specific frequency band. It has the advantages of large transmission bandwidth, low communication latency, and relatively low cost, and is the primary channel for high-speed data exchange between the UAV and the ground control station within visual range. The satellite communication link relies on a satellite communication terminal integrated on the UAV and a space-based satellite network. Its greatest advantage is that it is not limited by geographical line-of-sight, enabling ultra-long-distance, all-weather global coverage communication. This ensures that communication can be maintained even in mountainous areas, offshore areas, or beyond-line-of-sight missions, making it an important backup and emergency communication method. The module's intelligent switching function manifests in two main modes: first, command-based controlled switching, where ground operators can proactively send commands to force the system to activate a designated link based on mission planning, area restrictions, or real-time conditions; second, link-state-based adaptive switching, where the module continuously monitors the signal strength, communication quality, and connection stability of the primary link (usually a line-of-sight link). When signal interruption, severe bandwidth degradation, or a bit error rate exceeding a threshold is detected, the control logic automatically and seamlessly switches the data stream to the satellite communication link to prioritize the continuity of control commands and critical data (especially the real-time generated "first image"). The entire switching process strives to be transparent to upper-layer applications, minimizing data transmission interruption time. This dual-link integration and intelligent switching design completely solves the reliability bottleneck of a single communication method in complex mission environments, significantly improving the communication survivability and mission success rate of the entire airborne aerial survey system when performing critical tasks such as disaster emergency response and long-range maritime patrols.

[0065] like Figure 5 As shown, in this embodiment, the aerial survey controller has two asynchronous 422 interfaces, which are used to receive uplink remote control commands from the ground control station and send status information to the ground control station via line-of-sight link, respectively; it also has one synchronous 422 interface, which is used to send real-time data to the ground control station via line-of-sight link; and it also has one gigabit network port, which is used to send real-time data to the ground control station via satellite communication link (satellite communication link in the figure).

[0066] Specifically, the hardware interface configuration of the aerial survey controller has been specifically optimized for a dual-link communication architecture, clearly defining the correspondence between each interface and the transmission link, as well as the data flow. The controller has two independent asynchronous 422 interfaces: one dedicated to reliably receiving all uplink remote control commands from the ground control station via a line-of-sight link, such as mission commands and parameter modifications; the other asynchronous 422 interface is used to periodically send system status information, health diagnostic reports, and other feedback data to the ground control station via the same line-of-sight link, achieving bidirectional asynchronous communication of the command channel. In addition, the controller is equipped with a synchronous 422 interface, which, with its strict timing characteristics, is specifically designed for synchronously and rapidly transmitting high-priority real-time data streams to the ground station via a line-of-sight link, such as parsed high-frequency POS data or emergency event identifiers. Simultaneously, the controller integrates a gigabit Ethernet interface. In this embodiment, this high-speed interface is defined as the core channel for sending large volumes of real-time data (such as compressed and processed orthophoto data) to the ground control station via a satellite communication link, fully utilizing the wide-area coverage advantage of the satellite link and the high throughput capability of the Ethernet interface to ensure reliable transmission of critical mission data beyond line-of-sight or in emergency situations. This interface allocation scheme clearly defines the transmission paths and bearer links for different data types, allowing line-of-sight links to focus on low-latency command interaction and critical status synchronization, while satellite links undertake the remote relay of high-bandwidth data. The two achieve collaboration and backup through intelligent scheduling within the controller, jointly constructing an efficient, reliable, and clearly defined air-ground integrated data communication system.

[0067] In this embodiment, the on-board processing module includes an IIQ lossless compression unit for real-time lossless compression of optical images.

[0068] Specifically, the core component of the onboard processing module includes an IIQ lossless compression unit optimized for aerial surveying. This unit is specifically responsible for real-time, high-fidelity lossless compression of raw optical images (typically in 16-bit RAW format) acquired by aerial survey cameras. Its technical necessity lies in the fact that the data volume of a single raw image generated by a high-resolution aerial survey camera is extremely large (e.g., up to hundreds of MB). Direct storage, transmission, or subsequent processing would quickly exhaust onboard storage space, consume limited data link bandwidth, and place an unbearable load on the real-time processing pipeline, thus completely undermining the system's design goal of "minute-level" response. IIQ (Intelligent Image Quality) is an advanced lossless or visually lossless compression format that significantly reduces data size (e.g., compressing raw data to about one-tenth of its original size) while maintaining the highest image fidelity through efficient algorithms. Its key advantage lies in the fact that the compression process does not lose any pixel or radiometric information, fully preserving the dynamic range, color depth, and subtle textures of the image. This makes subsequent processing such as precise sensor calibration, high-precision geometric positioning, and pixel-level image matching and stitching completely equivalent in data basis to using the uncompressed raw data, thus ensuring that the geometric accuracy and information integrity of the final orthophoto product are not compromised in any way. This compression unit is deeply integrated into the on-machine multi-threaded parallel processing architecture, enabling real-time compression of continuously incoming image streams with extremely high throughput. It is a key technical link in ensuring the smooth and efficient operation of the entire real-time processing chain from data acquisition to result download.

[0069] In this embodiment, the image enhancement processing includes:

[0070] Utilize radar imagery to perform one or more of the following processing on optical images: haze removal, color cast adjustment, and dark enhancement.

[0071] Specifically, image enhancement processing is an intelligent optimization process based on multi-source data fusion. Its core lies in utilizing the unique physical information of Synthetic Aperture Radar (SAR) imagery to specifically improve the quality of optical images. This processing includes one or more coordinated operations using radar imagery to remove haze, adjust color cast, and enhance dark areas in optical images. The underlying principle is that radar microwaves have strong penetrating power through particles such as clouds, smoke, and aerosols. Therefore, SAR imagery can acquire the underlying structure and contour information of the earth's surface even under adverse weather conditions where optical cameras cannot clearly image it. In haze removal, the system uses the real surface structure revealed by the radar imagery as a reference, and uses algorithms to identify and separate the haze layer caused by atmospheric scattering in the optical imagery, thereby effectively restoring the details and contrast of obscured ground features. In color cast adjustment, the scene consistency information provided by radar can be used as a calibration benchmark to help correct color distortion in optical images caused by poor lighting conditions or atmospheric color cast, restoring their true color balance. In dark image enhancement processing, by leveraging radar's sensitivity to terrain undulations and surface roughness, it can intelligently brighten under-lit shadowed areas in optical images (such as deep valleys and the shaded sides of buildings). Without causing overexposure in bright areas, it balances the dynamic range of brightness across the entire image, revealing more hidden details. This series of enhancement processes is not simply image filtering, but a data-driven restoration based on physical perception. It significantly improves the visual quality of the orthophoto images output by the system. More importantly, it fundamentally ensures the amount of effective information contained in the images, which can be used for disaster assessment, even under adverse weather conditions such as fog, haze, and smoke. This greatly enhances the reliability and practicality of the entire system in complex emergency environments.

[0072] In this embodiment, the on-board processing module adopts multi-threaded parallel processing technology to perform pipelined parallel processing of the compression, correction and stitching tasks of multiple optical images.

[0073] Specifically, to achieve the core performance of real-time image generation within minutes, the onboard processing module employs multi-threaded parallel processing technology in its software architecture. This technology doesn't simply distribute tasks across multiple processor cores; instead, it meticulously designs a pipelined parallel processing model tailored to the data flow characteristics of aerial survey image processing. Specifically, the system decouples the three main processing stages—compression, correction (including sensor and geometric correction), and stitching—into relatively independent task units, managed by different thread pools. When the UAV continuously acquires optical images, multiple images enter the processing queue in a pipeline manner: thread A is responsible for real-time compression of the latest acquired image N; simultaneously, thread B performs sensor and geometric correction on the slightly earlier compressed image N-1; and thread C uses the even earlier corrected images N-2 and N-3 for real-time matching and stitching. This architecture allows the processing tasks of a single image to be executed serially to ensure data dependency, while the processing flows between multiple images completely overlap and run in parallel in time, thereby maximizing the utilization of the multi-core processor's computing resources and transforming the heavy processing load that originally had to be executed sequentially into a highly efficient pipeline operation. This technology completely eliminates the large amount of idle waiting time present in traditional serial processing, enabling an order-of-magnitude increase in the overall processing throughput from data reception to result generation. It is a core software innovation that ensures that massive aerial survey images can be processed "while flying" on an airborne embedded computing platform, directly supporting the ultimate goal of generating the "first map" of disaster situations in minutes.

[0074] In this embodiment, the aerial survey camera is a dual-lens camera, the optical image is a dual-lens image, and the stitching adopts fully automatic texture matching to seamlessly stitch the dual-lens images to generate a seamless orthophoto image.

[0075] Specifically, the aerial survey camera adopts a dual-lens integrated design, meaning that two independent imaging lenses and sensors are precisely calibrated and integrated within the same camera body, forming a synchronously exposed, common-reference acquisition unit. During aerial survey missions, this dual-lens camera can simultaneously acquire two optical images with a certain degree of overlap, i.e., dual-lens images. These two images instantly form a wider effective baseline and multi-view observation conditions in space. Subsequent stitching is a crucial step in generating high-quality orthophoto products. The system employs fully automatic texture matching technology to efficiently and accurately fuse the dual-lens images. This technology automatically detects and extracts a large number of significant and stable feature points (such as corner points, edges, and texture patterns) in the overlapping area of ​​the two images through algorithms, and performs rapid matching based on descriptors, thereby calculating the geometric transformation relationship (such as homography matrix) between the two images with sub-pixel accuracy. Subsequently, based on this transformation relationship, the system resamples and fuses the two images into a unified high-precision geographic coordinate framework. During this process, advanced feathering and color equalization algorithms are applied to the seam areas to eliminate abrupt changes in brightness and color caused by differences in lighting or lens vignetting, ultimately generating visually continuous and geometrically accurate seamless orthophotos. This fully automated process not only significantly improves stitching efficiency and eliminates the tediousness and subjectivity of manual intervention, but also, thanks to the extremely short time interval and fixed relative orientation between the two-lens images, achieves a matching success rate and accuracy far exceeding that of stitching images from different flight zones or different voyages. This ensures the high quality and reliability of the final results, meeting the stringent requirements of emergency mapping for both timeliness and accuracy.

[0076] In this embodiment, geometric correction is direct geolocation correction based on high-precision POS data;

[0077] Sensor calibration includes distortion correction based on calibration parameters and dynamic black level correction based on sensor temperature.

[0078] Specifically, geometric correction refers to "direct geolocation correction based on high-precision POS data." This technology abandons the indirect methods of traditional aerial surveying that rely on ground control points and complex aerial triangulation, instead directly utilizing high-precision position (longitude, latitude, elevation) and attitude (pitch, roll, heading) data provided in real time by the integrated navigation system and strictly synchronized with the exposure time of each image frame. Through rigorous collinearity equations or sensor models, the system can instantly solve each pixel on the image into a unified geodetic coordinate system, thus completing the projection transformation from image coordinates to geographic coordinates in one step. The core advantage of this method lies in its real-time and autonomous nature. It allows the UAV to assign a precise geographic location to each image while in flight, making it a key enabling technology for real-time generation of orthophoto maps on board, fully meeting the extreme speed requirements of emergency mapping.

[0079] Sensor calibration is a set of internal calibrations targeting the camera's own physical characteristics, comprising two indispensable parts: first, lens distortion correction based on laboratory calibration parameters, which aims to use pre-measured lens distortion parameters (such as radial and tangential distortion) and mathematical models to reverse-correct image geometric distortions (such as edge curvature) caused by lens optical defects, restoring the true geometry of the scene; second, dynamic black level correction based on sensor temperature, a real-time processing technique to cope with changes in the airborne environment. Because the dark current of image sensors (CMOS / CCD) increases significantly with their operating temperature, it causes background (black level) drift, creating fixed noise. This system monitors sensor temperature in real time and dynamically adjusts calibration parameters, accurately subtracting the temperature-dependent dark current signal, thereby ensuring that the image's radiometric reference remains stable under different flight phases and external environments, maximizing the preservation of the original signal-to-noise ratio and dynamic range. Geometric correction solves the external spatial positioning problem of "where is the image," while sensor calibration solves the internal quality optimization problem of "whether the image itself is accurate." The combination of these two elements provides geometrically accurate and radiometrically pure standardized image input for subsequent stitching and fusion processing, which is the fundamental guarantee that the output of the entire system can meet professional surveying and mapping standards (GB / T 12341-2022).

[0080] In one embodiment, the CH-4 medium-to-large UAV platform is selected, the aerial survey camera is an iXU-RS1900 dual-lens camera, the SAR (synthetic aperture radar) is a lightweight and compact fully polarized module, and the IMU (inertial measurement unit) is a high-precision fiber optic inertial navigation system.

[0081] Secondly, this invention proposes a mapping method based on the airborne aerial survey real-time processing system according to any one of the first aspects, the method comprising:

[0082] Based on the mission area, the flight path is planned and controlled to fly the medium and large UAVs carrying integrated payload modules to the mission area.

[0083] Simultaneously acquire optical images, radar images, and high-precision POS data, and analyze the high-precision POS data;

[0084] Real-time active compensation for the flight attitude of medium and large UAVs is performed based on the parsed high-precision POS data.

[0085] The optical images are sequentially compressed in real time, sensor calibrated, geometrically calibrated, image enhanced and stitched to obtain a seamless orthophoto. Geometric calibration is performed using high-precision POS data and image enhancement is performed using radar images.

[0086] Seamless orthophotos are transmitted to the ground control station.

[0087] The various embodiments of the present invention have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments.

Claims

1. An airborne aerial survey real-time processing system, characterized in that, The system, applied to medium and large-sized unmanned aerial vehicle (UAV) flight platforms, includes: Integrated payload module, aerial survey controller, power supply module, communication module and onboard processing module; The integrated payload module is rigidly connected to the bottom plate inside the payload compartment of the medium and large UAV flight platform, and is used to simultaneously acquire optical images, radar images and high-precision POS data. The aerial survey controller is communicatively connected to the integrated payload integration module and the onboard processing module, respectively, and is used to receive and parse the high-precision POS data in real time, and to send the parsed high-precision POS data to the integrated payload integration module and the onboard processing module. The power supply module is centrally powered by the UAV flight platform and is used to power the integrated payload module, the aerial survey controller, the communication module, and the onboard processing module. The communication module is connected to the aerial survey controller, the onboard processing module and the ground control station respectively, and is used to realize bidirectional data exchange between the aerial survey controller and the onboard processing module and the ground control station. The onboard processing module is communicatively connected to the integrated payload module and is used to receive the optical image, radar image and high-precision POS data in real time. It performs real-time compression, sensor correction, geometric correction, image enhancement and stitching on the optical image in sequence to obtain a seamless orthophoto image. Geometric correction is performed using the high-precision POS data and image enhancement is performed using the radar image.

2. The airborne aerial survey real-time processing system according to claim 1, characterized in that, The integrated load integration module includes: The system includes an aerial survey camera, a synthetic aperture radar, a combined inertial navigation system, and a stabilization platform, wherein the stabilization platform integrates slip rings to form a reference side and a load side with relative motion. The solver of the synthetic aperture radar is fixed to the reference side above the slip ring. The solver is used to perform real-time motion compensation and geocoding processing on the radar image based on the parsed high-precision POS data. The inertial measurement unit of the combined inertial navigation system is fixed to the reference side above the slip ring; The lens of the aerial survey camera and the transceiver antenna of the synthetic aperture radar are rigidly connected side by side to the load side below the slip ring, and the lens and transceiver antenna are exposed outside the load compartment to achieve unobstructed observation; The triaxial installation error angle between the aerial survey camera, transceiver antenna and inertial measurement unit shall not exceed 2° to ensure geometric consistency between data. The stabilization platform is fixed to the bottom plate inside the payload compartment in a seated installation manner. It can automatically adjust the attitude of the payload side based on the parsed high-precision POS data to compensate for disturbances during the flight of the medium and large UAV flight platform.

3. The airborne aerial survey real-time processing system according to claim 1, characterized in that, The communication module integrates a line-of-sight link and a satellite communication link, and can switch between the two according to instructions or link status.

4. The airborne aerial survey real-time processing system according to claim 3, characterized in that, The aerial survey controller is equipped with two asynchronous 422 interfaces, which are used to receive uplink remote control commands from the ground control station and send status information to the ground control station via the line-of-sight link, respectively; it also has a synchronous 422 interface, which is used to send real-time data to the ground control station via the line-of-sight link; and it also has a gigabit network port, which is used to send real-time data to the ground control station via the satellite communication link.

5. The airborne aerial survey real-time processing system according to claim 1, characterized in that, The onboard processing module includes an IIQ lossless compression unit for real-time lossless compression of the optical image.

6. The airborne aerial survey real-time processing system according to claim 1, characterized in that, The image enhancement process includes: The radar image is used to perform one or more of the following processing on the optical image: haze removal, color cast adjustment, and dark enhancement.

7. The airborne aerial survey real-time processing system according to claim 1, characterized in that, The onboard processing module employs multi-threaded parallel processing technology to perform pipelined parallel processing of the compression, correction, and stitching tasks of multiple optical images.

8. The airborne aerial survey real-time processing system according to claim 2, characterized in that, The aerial survey camera is a dual-lens camera, the optical image is a dual-lens image, and the stitching uses fully automatic texture matching to seamlessly stitch the dual-lens images to generate the seamless orthophoto image.

9. The airborne aerial survey real-time processing system according to claim 1, characterized in that, The geometric correction is a direct geolocation correction based on the high-precision POS data; The sensor calibration includes distortion correction based on calibration parameters and dynamic black level correction based on sensor temperature.

10. A mapping method, characterized in that, Based on the airborne aerial survey real-time processing system according to any one of claims 1-9, the method includes: Based on the mission area, a flight path is planned, and the medium and large-sized UAVs equipped with the integrated payload module are controlled to fly to the mission area. Simultaneously acquire optical images, radar images, and high-precision POS data, and analyze the high-precision POS data; The flight attitude of the medium and large UAV is actively compensated in real time based on the parsed high-precision POS data. The optical image is sequentially compressed in real time, sensor calibrated, geometrically calibrated, image enhanced and stitched to obtain a seamless orthophoto image. Geometric calibration is performed using the high-precision POS data and image enhancement is performed using the radar image. The seamless orthophoto image is then transmitted to the ground control station.

Images