Billimeter-pixel unmanned aerial vehicle (UAV)-carried multi-scale camera system and method
By acquiring scene depth information through a depth sensor, and combining integral control and hardware-level color calibration, the camera baseline distance is dynamically adjusted, solving the problems of timing synchronization and parallax control in UAV-borne camera systems, and achieving real-time output of high-definition megapixel video and highly robust imaging.
Patent Information
- Application Number
- CN202511687424.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-18
- Publication Date
- 2026-02-17
AI Technical Summary
Traditional UAV-borne camera systems suffer from problems such as difficulty in timing synchronization, parallax control, and poor sensor consistency in high-speed flight and complex vibration environments, resulting in decreased image quality and failing to meet the real-time and high-precision requirements of megapixel-level imaging.
The system uses a depth sensor to acquire scene depth information, dynamically adjusts the camera baseline distance through integral control, and combines hardware-level color calibration to achieve synchronization and color consistency between cameras. It also utilizes power line carrier synchronization technology to achieve nanosecond-level synchronization, reducing system complexity and latency.
It achieves high-definition, 100-megapixel video output in the high-speed movement environment of drones, reduces stitching error rate, improves system robustness and imaging quality, and reduces system weight and power consumption.
Smart Images
Figure CN121547565A_ABST
Abstract
Description
Technical Field
[0001] This application belongs to the field of drone photography, and in particular relates to a multi-scale camera system and method for a 100-megapixel drone. Background Technology
[0002] With the rapid advancement of drone technology, its applications in various fields such as aerial photography, environmental monitoring, and geographic information collection are becoming increasingly widespread, placing increasingly stringent demands on imaging systems. Especially in scenarios requiring ultra-high resolution imaging at the megapixel level, traditional camera systems face many insurmountable technical bottlenecks, among which issues such as timing synchronization, parallax control, and sensor consistency are particularly prominent.
[0003] Traditional UAV-borne camera systems often employ a multi-camera array layout with a fixed baseline distance, which can achieve good imaging results in static or low-dynamic environments.
[0004] However, the limitations of fixed baseline distance become apparent in high-speed drone flight and complex vibration environments. Due to the severe vibrations and rapid position changes introduced by drone motion, temporal synchronization between camera arrays becomes extremely difficult. Traditional wireless synchronization methods are susceptible to interference and suffer from transmission delays, leading to misalignment between video frames and severely impacting image quality. Simultaneously, parallax, an inherent challenge in multi-camera systems, is exacerbated in dynamic scenes by the continuous changes in scene depth. Fixed baseline distance cannot adapt to these changes in real time, resulting in geometrical distortion in imaging and reducing data accuracy and usability.
[0005] Furthermore, the inherent differences in color response, sensitivity, and other characteristics among different camera sensors further exacerbate the inconsistency in imaging results. In traditional methods, color calibration often relies on post-processing software. This approach not only increases the complexity of the data processing workflow but may also introduce additional errors due to the limitations of software algorithms, failing to correct for differences between sensors in real time and accurately.
[0006] Crucially, most existing technologies are designed for fixed ground arrays and lack specific optimizations for mobile platforms such as drones.
[0007] In dynamic environments, relying solely on software post-processing methods is insufficient to address hardware-level issues in real time, leading to decreased video quality, low processing efficiency, and an inability to meet the stringent requirements of UAV-borne aerial imaging for real-time performance, high precision, and high resolution. Summary of the Invention
[0008] The purpose of this application is to overcome the deficiencies in the prior art and provide a 100-megapixel drone-borne multi-scale camera system and method.
[0009] This application provides a 100-megapixel drone-borne multi-scale camera system, including:
[0010] The acquisition module acquires scene depth information from a depth sensor, wherein the scene depth information is generated by measuring the depth of the target scene in real time through the depth sensor;
[0011] The calculation module calculates the depth error based on the scene depth information and a preset reference depth, wherein the reference depth is set based on the expected minimum disparity threshold.
[0012] An integration module generates a baseline adjustment amount based on the integration of the depth error. The integration is used to achieve smooth adjustment and avoid abrupt changes.
[0013] The adjustment module adjusts the physical distance between the cameras via a piezoelectric actuator platform based on the baseline adjustment amount to minimize parallax.
[0014] Optionally, the integration module generates a baseline adjustment amount based on the integral of the depth error, including:
[0015] The integration is performed using a fixed gain coefficient, which is determined by the hardware characteristics of the piezoelectric actuator platform.
[0016] Optionally, the acquisition module acquires scene depth information from a depth sensor, including:
[0017] The scene depth information is collected in real time by a time-of-flight sensor, and the sampling rate of the time-of-flight sensor is matched with the integral control cycle.
[0018] Optionally, the calculation module calculates the depth error based on the scene depth information and a preset reference depth, including:
[0019] The reference depth is set to a fixed value based on a disparity threshold, which is set for medium-range scenes.
[0020] Optionally, the adjustment module adjusts the physical distance of the camera via a piezoelectric actuator platform based on the baseline adjustment amount, including:
[0021] The baseline adjustment amount is directly driven by an analog signal to drive the piezoelectric actuator platform, without the need for digital signal conversion.
[0022] Optionally, the integration module generates a baseline adjustment amount based on the integral of the depth error, including:
[0023] Integration is performed using a preset integration time constant, which is set according to the typical flight speed of the UAV.
[0024] Optionally, the integration module generates a baseline adjustment amount based on the integral of the depth error, including:
[0025] The depth error is integrated using an analog integrator circuit, which then directly outputs the baseline adjustment amount.
[0026] This application also provides a method for using a multi-scale camera mounted on a 100-megapixel drone, including:
[0027] Scene depth information is obtained from a depth sensor, wherein the scene depth information is generated by measuring the depth of the target scene in real time using the depth sensor;
[0028] The depth error is calculated based on the scene depth information and the preset reference depth, wherein the reference depth is set based on the expected minimum disparity threshold.
[0029] Based on the integral of the depth error, a baseline adjustment amount is generated, which is used to achieve smooth adjustment and avoid abrupt changes;
[0030] Based on the baseline adjustment, the physical distance between the cameras is adjusted via a piezoelectric actuator platform to minimize parallax.
[0031] This application also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, it implements the system as described above.
[0032] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to execute the system described above.
[0033] The beneficial effects of this application are:
[0034] Invention Point 1: Dynamic Baseline Adjustment Mechanism Based on Integral Control
[0035] Invention Point 2: Nanosecond-level synchronization technology without additional wiring based on power line carrier
[0036] Invention Point 3: Hardware-level pre-optical color calibration mechanism
[0037] This application provides a 100-megapixel UAV-borne multi-scale camera system, comprising: an acquisition module for acquiring scene depth information from a depth sensor, wherein the scene depth information is generated by real-time measurement of the target scene depth using the depth sensor; a calculation module for calculating a depth error based on the scene depth information and a preset reference depth, wherein the reference depth is set based on a desired minimum parallax threshold; an integration module for generating a baseline adjustment amount based on the integral of the depth error, wherein the integral is used to achieve smooth adjustment and avoid abrupt changes; and an adjustment module for adjusting the physical spacing of the cameras via a piezoelectric actuator platform based on the baseline adjustment amount to minimize parallax. This application, by acquiring scene depth in real-time and calculating the error, integrating to generate a smooth baseline adjustment amount, dynamically adjusting the camera spacing to minimize parallax, and combining hardware-level color calibration, significantly reduces the stitching error rate, achieves real-time output of high-definition 100-megapixel video, and improves system robustness and imaging quality. Attached Figure Description
[0038] Figure 1 This is a schematic diagram of the 100-megapixel UAV-borne multi-scale camera system in this application;
[0039] Figure 2 This is a schematic diagram of the control process for a multi-scale camera on a 100-megapixel unmanned aerial vehicle in this application. Detailed Implementation
[0040] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it is to be understood that various forms of implementation of the present disclosure are intended and should not be limited to the embodiments set forth herein. Rather, the embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
[0041] Please refer to Figure 1 As shown, this application provides a baseline adjustment method for a 100-megapixel UAV-borne multi-scale camera system, applied in the field of UAV navigation, to solve the parallax minimization problem of UAV-borne camera systems. The method includes:
[0042] The acquisition module 101 acquires scene depth information from the depth sensor, wherein the scene depth information is generated by measuring the depth of the target scene in real time through the depth sensor.
[0043] The system uses depth sensors, such as time-of-flight sensors, to collect scene depth data in real time.
[0044] The depth sensor samples scene depth at a high frequency, with its sampling rate matched to the integral control cycle to ensure the timeliness and accuracy of depth information and avoid delays or errors caused by UAV movement. For example, during UAV flight, the time-of-flight sensor calculates distance by emitting light pulses and measuring the return time, generating real-time depth information d(t) to provide a basis for subsequent calculations.
[0045] It utilizes the rapid sampling capability of the depth sensor and synchronizes it with integral control to achieve smooth adjustment.
[0046] Furthermore, the synchronization timing is provided by the power line carrier synchronization module. This module utilizes the UAV's power infrastructure to distribute the synchronization signal through the power line. The main controller generates high-frequency pulses, such as 10MHz, and modulates them onto the power line. The built-in demodulation circuits of each camera extract the signal based on the phase-locked loop principle and adjust the local clock to achieve nanosecond-level synchronization. This avoids additional wiring, reduces weight and complexity, and thus ensures accurate synchronization of depth information.
[0047] The calculation module 102 calculates the depth error based on the scene depth information and the preset reference depth, wherein the reference depth is set based on the expected minimum disparity threshold.
[0048] The system compares the real-time scene depth information d(t) with the preset reference depth dref and calculates the depth error, i.e., d(t)-dref.
[0049] The reference depth dref is a fixed value set based on the desired minimum disparity threshold. For example, for a medium-range scene, dref is set to 10 meters, which means that the disparity is minimized when the object is 10 meters away.
[0050] Depth error reflects the difference between the current scene depth and the ideal depth, and is used to drive baseline adjustment. This calculation process ensures that baseline adjustment is targeted, avoids unnecessary adjustments, and improves system efficiency.
[0051] The integration module 103 generates a baseline adjustment amount based on the integration of the depth error. The integration is used to achieve smooth adjustment and avoid abrupt changes.
[0052] The system uses an integral control mechanism to handle depth error and generates a baseline adjustment amount, as shown in the following formula:
[0053]
[0054] Where b(t) represents the baseline distance at time t, in meters, which is the physical distance between the cameras; d(t) is the initial baseline distance in meters, based on system design and focal length preset; d(t) is the scene depth measured by the depth sensor at time t, in meters, reflecting the distance between the object and the camera. This is a reference depth, in meters, set as the depth corresponding to the desired minimum parallax threshold, typically based on application requirements. =10 meters is used for medium-range scenarios; k is the gain coefficient, in meters per second·meter, which controls the adjustment rate and is optimized according to the system response time. k is fixed and determined by the hardware characteristics of the piezoelectric actuator platform and cannot be adjusted; the integral symbol represents the accumulation over time to achieve smooth adjustment and avoid abrupt changes.
[0055] Integral control is commonly used in industrial control systems, but in this application it is used for the dynamic adjustment of the camera baseline, representing a cross-disciplinary innovation. While integral control has a relatively slow response, its combination with the system's fast-sampling depth sensor enables smooth and stable adjustments, reducing parallax oscillations caused by vibration and improving stitching stability.
[0056] Furthermore, the integration module performs integration calculations using a preset integration time constant. This integration time constant is set to match the typical UAV flight speed; for example, if the UAV's flight speed is 10 m / s, the integration time constant is set to 0.1 seconds to balance response speed and stability. The integration module also performs depth error integration calculations using an analog integration circuit. This analog integration circuit, composed of operational amplifiers and other components, directly outputs the baseline adjustment amount without digital conversion, reducing latency.
[0057] Furthermore, since it is typically used in industrial control systems such as PID controllers and has never been applied to the dynamic adjustment of camera baselines, the baseline in optical arrays is usually fixed or based on static proportional adjustment. However, this application uses the integral of the depth error for real-time baseline control, which essentially changes the function of the baseline from a static geometric parameter to a dynamic control variable, achieving smooth and stable adjustment and improving stitching stability.
[0058] The adjustment module 104 adjusts the physical distance of the camera via the piezoelectric actuator platform according to the baseline adjustment amount to minimize parallax.
[0059] The system converts the generated baseline adjustment amount into a control signal, which is then used to adjust the physical distance between the cameras via a piezoelectric actuator platform. The baseline adjustment amount directly drives the piezoelectric actuator platform via analog signals, eliminating the need for digital signal conversion. This reduces processing latency and complexity, enabling rapid response.
[0060] The piezoelectric actuator platform responds to electrical signals to change the camera's position, thereby dynamically adjusting the baseline distance b(t) to minimize parallax and improve image stitching quality. For example, when the depth error integral indicates that the baseline needs to be increased, the piezoelectric actuator pushes the camera apart, and vice versa, ensuring that parallax is always kept to a minimum.
[0061] Among them, "minimizing parallax" refers to aligning the pixel positions of the same object as much as possible in the images of each camera through the above dynamic adjustment. Its core purpose is to reduce geometric misalignment and distortion during subsequent image stitching, thereby improving the accuracy and quality of megapixel-level image stitching.
[0062] Furthermore, color consistency must be ensured after adjustment. This is achieved through an optically coupled color calibration module. This module distributes uniform light signals to each camera via a central reference sensor, such as a spectrophotometer, through an optical fiber bundle. Each camera receives a calibration pulse before capturing the signal and adjusts its analog gain and offset accordingly to achieve hardware-level color consistency. The consistent optical fiber path length ensures synchronous signal transmission.
[0063] The formula for color calibration is as follows:
[0064]
[0065] Where Gain_i represents the analog gain adjustment value of the i-th camera, Gain_ref represents the reference gain base value, I_ref represents the light intensity measured by the reference sensor, and I_i represents the light intensity received by the i-th camera.
[0066]
[0067] Where Offset_i represents the offset adjustment value of the i-th camera, Offset_ref represents the reference offset base value, and k_offset represents the offset adjustment coefficient. The difference between standard light intensity and light intensity. The perceptual differences between each individual camera and the reference standard were quantified.
[0068] Traditional methods rely on image processing algorithms to perform color matching after image acquisition, which increases computational burden and cannot completely eliminate inherent differences between sensors. The formula in this application, however, directly adjusts the analog gain and offset parameters of each camera before image capture, ensuring color consistency across all cameras at the source of data acquisition. This pre-calibration method not only significantly reduces the computational complexity of post-processing but, more importantly, fundamentally eliminates systematic errors caused by differences in sensor characteristics. It ensures high consistency in color information output by all cameras, even during high-speed acquisition, laying a solid foundation for subsequent high-quality image stitching.
[0069] Traditional systems require additional dedicated synchronization lines, which not only increases system weight and complexity but also introduces additional points of failure. This application achieves nanosecond-level synchronization accuracy by modulating the synchronization signal onto the power line and extracting the synchronization signal using phase-locked loop (PLL) technology. This design cleverly utilizes the power network inherent in the UAV, avoiding the need for additional wiring, significantly reducing system weight, and improving system reliability. Particularly noteworthy is that this synchronization method provides a unified time reference for the entire system, ensuring strict synchronization between various stages such as depth information acquisition, baseline adjustment, and image capture—a crucial foundation for achieving high-quality, megapixel imaging.
[0070] The above formula ensures color consistency at the hardware level for each camera, and consistent fiber optic path length ensures synchronous signal transmission.
[0071] The core formula for power line carrier synchronization is as follows:
[0072]
[0073] Where f_sync represents the synchronization frequency in Hertz, and T_pll represents the locking time of the phase-locked loop in seconds, this formula describes the mechanism for achieving nanosecond-level synchronization through the principle of phase-locked loop.
[0074] The aforementioned color calibration formula, along with the power line carrier synchronization formula and the original baseline adjustment formula, constitute a complete technical system with a close synergistic relationship. The power synchronization formula provides a time reference that ensures the synchronization of depth information acquisition and baseline adjustment; the color calibration formula ensures color consistency across all cameras after baseline adjustment; and the baseline adjustment formula is responsible for optimizing geometric accuracy. This multi-faceted synergy produces unexpected technical effects: the system not only obtains images with high geometric accuracy but also guarantees color consistency, while effectively controlling the weight and power consumption of the entire system. This comprehensive advantage cannot be achieved by any single technical improvement, enabling the system to provide high-quality, megapixel imaging capabilities in applications such as UAVs, where weight, power consumption, and reliability are strictly controlled, without relying on complex post-processing.
[0075] In this application, the above modules work together in a mutually dependent manner. The power line carrier synchronization module provides a stable timing basis for baseline adjustment, and the optical coupling color calibration module ensures color consistency after parallax correction. The overall system produces technical effects in the UAV motion environment, showing excellent robustness, and the overall performance far exceeds the sum of the independent actions of each module.
[0076] This system features a lightweight design, adapts to drone payload limitations, has low power consumption, and is suitable for real-time operation. It solves the fundamental problems of drone-borne camera systems. The timing synchronization module reduces motion blur, the dynamic baseline adjustment module minimizes geometric distortion, and the optical color calibration module eliminates sensor differences. The synergistic effect significantly reduces the video stitching error rate, while the increase in power consumption is negligible. The system is suitable for high-speed mobile scenarios such as drone aerial photography, providing high-definition megapixel video without post-processing. Its advantages include lightweight design, improved real-time performance, and strong adaptability.
[0077] Please refer to Figure 2 As shown, this application also provides a method for using a multi-scale camera mounted on a 100-megapixel drone, comprising:
[0078] S201. Obtain scene depth information from a depth sensor, wherein the scene depth information is generated by measuring the depth of the target scene in real time using the depth sensor;
[0079] S202. Calculate the depth error based on the scene depth information and the preset reference depth, wherein the reference depth is set based on the expected minimum disparity threshold.
[0080] S203. Based on the integration of the depth error, a baseline adjustment amount is generated, wherein the integration is used to achieve smooth adjustment and avoid abrupt changes;
[0081] S204. Based on the baseline adjustment amount, adjust the physical distance of the camera via the piezoelectric actuator platform to minimize parallax.
[0082] This application also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, it implements the system as described above.
[0083] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to execute the system described above.
[0084] The above description of the embodiments is provided to enable those skilled in the art to understand and apply this application. Those skilled in the art will readily make various modifications to the above embodiments and apply the general principles described herein to other embodiments without inventive effort. Therefore, this application is not limited to the above embodiments, and any improvements and modifications made to this application based on the disclosure thereof should be within the scope of protection of this application.
Claims
1. A multi-scale camera system for unmanned aerial vehicles with a resolution of 100 megapixels, characterized in that, include: The acquisition module acquires scene depth information from a depth sensor, wherein the scene depth information is generated by measuring the depth of the target scene in real time through the depth sensor; The calculation module calculates the depth error based on the scene depth information and a preset reference depth, wherein the reference depth is set based on the expected minimum disparity threshold. An integration module generates a baseline adjustment amount based on the integration of the depth error. The integration is used to achieve smooth adjustment and avoid abrupt changes. The adjustment module adjusts the physical distance between the cameras via a piezoelectric actuator platform based on the baseline adjustment amount to minimize parallax.
2. The system according to claim 1, characterized in that, The integration module generates a baseline adjustment amount based on the integral of the depth error, including: The integration is performed using a fixed gain coefficient, which is determined by the hardware characteristics of the piezoelectric actuator platform.
3. The method according to claim 1, characterized in that, The acquisition module acquires scene depth information from the depth sensor, including: The scene depth information is collected in real time by a time-of-flight sensor, and the sampling rate of the time-of-flight sensor is matched with the integral control cycle.
4. The method according to claim 1, characterized in that, The calculation module calculates the depth error based on the scene depth information and a preset reference depth, including: The reference depth is set to a fixed value based on a disparity threshold, which is set for medium-range scenes.
5. The method according to claim 1, characterized in that, The adjustment module adjusts the physical distance of the camera via a piezoelectric actuator platform according to the baseline adjustment amount, including: The baseline adjustment amount is directly driven by an analog signal to drive the piezoelectric actuator platform, without the need for digital signal conversion.
6. The method according to claim 1, characterized in that, The integration module generates a baseline adjustment amount based on the integral of the depth error, including: Integration is performed using a preset integration time constant, which is set according to the typical flight speed of the UAV.
7. The method according to claim 1, characterized in that, The integration module generates a baseline adjustment amount based on the integral of the depth error, including: The depth error is integrated using an analog integrator circuit, which then directly outputs the baseline adjustment amount.
8. A method for using a multi-scale camera mounted on a 100-megapixel drone, characterized in that, include: Scene depth information is obtained from a depth sensor, wherein the scene depth information is generated by measuring the depth of the target scene in real time using the depth sensor; The depth error is calculated based on the scene depth information and the preset reference depth, wherein the reference depth is set based on the expected minimum disparity threshold. Based on the integral of the depth error, a baseline adjustment amount is generated, which is used to achieve smooth adjustment and avoid abrupt changes; Based on the baseline adjustment, the physical distance between the cameras is adjusted via a piezoelectric actuator platform to minimize parallax.
9. An electronic device, characterized in that, It includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the method as described in claim 8.
10. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed in a computer, causes the computer to perform the method of claim 8.