High-fidelity splicing method and system for unmanned aerial vehicle image

Abstract

The invention discloses a high-fidelity splicing method and system for an unmanned aerial vehicle image, and the method employs a dual-stage processing strategy, achieves precise positioning through coarse-fine two-stage alignment, calculates an overall transformation parameter through global estimation to complete coarse alignment, refines non-rigid offset through local correction, and achieves high-fidelity splicing of the unmanned aerial vehicle image. The feature weight is dynamically adjusted in combination with self-adaptive enhancement, and geometric smooth constraint is used for limiting regional displacement; smooth weights with continuous boundaries and the weight sum being 1 are generated based on the shape of the overlapping region, images are decomposed to different frequency layers for layer-by-layer fusion, and the system comprises an image acquisition unit, a dual-stage processing unit and an output display unit and can be deployed on the end side of an unmanned aerial vehicle or a ground station. The method is high in alignment precision, can process large parallax and non-rigid deformation, is traceless in fusion, is high in structural fidelity, and meets the real-time processing requirements of airborne or edge equipment.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) technology, specifically to a high-fidelity image stitching method and system for UAVs. Background Technology

[0002] A drone is an unmanned aerial vehicle that does not require a pilot to operate and can be controlled via radio remote control equipment or its own program control device. It integrates technologies from multiple fields such as aerodynamics, electronic communication, and automatic control. According to the application scenario, it can be divided into three categories: consumer-grade, industrial-grade, and military-grade. Consumer-grade drones are mostly used for aerial photography and entertainment, while industrial-grade drones can undertake tasks such as agricultural plant protection, power line inspection, surveying and exploration, and logistics delivery. Military-grade drones can perform military operations such as reconnaissance, surveillance, and strike. They are characterized by their mobility, low operating costs, and ability to adapt to complex and dangerous environments, and have been widely used in many fields such as people's livelihood, industry, and national defense.

[0003] In practical use, existing drones need to capture images. However, drones are easily affected by airflow during flight, resulting in unstable shooting angles. In addition, the shooting scene has a complex three-dimensional structure, which causes visual differences, distortion, and irregular overlap between the captured images. This makes it difficult for stitching technology to align them accurately, resulting in alignment errors and affecting the realism of the image stitching.

[0004] Therefore, we propose a high-fidelity stitching method and system for UAV images. Summary of the Invention

[0005] The purpose of this invention is to provide a high-fidelity image stitching method and system for unmanned aerial vehicles (UAVs), which solves the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a high-fidelity image stitching method for UAVs, comprising the following steps:

[0007] Step 1: Alignment stage. First, coarse alignment is achieved by calculating the overall transformation parameters between the two images. Then, local non-rigid offsets are estimated to refine the alignment results. During feature extraction, feature weights are dynamically adjusted according to the importance of the regions. At the same time, smoothing constraints are applied to the local offsets to avoid structural tearing.

[0008] Step 2: Fusion stage. Based on the shape of the overlapping area of ​​the aligned image, calculate the pixel-level smoothing weight. The pixel-level smoothing weight satisfies the condition that the boundaries are continuous and the weights at the same position sum to 1. Decompose the image into different frequency layers and fuse them layer by layer according to the pixel-level smoothing weight to reconstruct the stitched image.

[0009] In a preferred embodiment of the present invention, the alignment stage achieves precise image positioning through a two-stage "coarse-fine" processing method, which is used to handle problems of large parallax and non-rigid deformation.

[0010] In a preferred embodiment of the present invention, during the adaptive adjustment of feature weights, key feature regions are highlighted and interference from irrelevant background regions is suppressed.

[0011] As a preferred embodiment of the present invention, the smoothing restriction applied to the local offset is used to maintain the structural naturalness of rigid targets such as buildings and avoid distortion and deformation.

[0012] In a preferred embodiment of the present invention, the generation of the pixel-level weighted mask does not require seam line detection and can adapt to overlapping areas of any shape.

[0013] In a preferred embodiment of the present invention, the multi-band fusion is used to eliminate splicing seams and ghosting phenomena, thereby achieving a natural image transition.

[0014] This invention also relates to a high-fidelity image stitching system for unmanned aerial vehicles (UAVs), comprising an image acquisition unit, a two-stage processing unit, and an output display unit: the image acquisition unit is used to acquire images of the target area during the flight of the UAV and transmit the acquired image data to the two-stage processing unit; the two-stage processing unit includes an alignment module and a fusion module, the alignment module is responsible for implementing global estimation, local correction, adaptive enhancement, and geometric smoothing constraint functions, and the fusion module is responsible for implementing weight mask generation and multi-band fusion functions; the output display unit is used to receive the stitched image output by the two-stage processing unit and display or store it.

[0015] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0016] This invention combines coarse-fine two-level alignment and adaptive enhancement mechanism with noise adaptive suppression processing, which can not only effectively handle the large parallax and non-rigid deformation problems between UAV images, but also resist noise interference caused by flight vibration, sudden changes in illumination, etc., thereby improving image alignment accuracy and stability.

[0017] Pixel-level smooth weighted masks are generated based on the shape of overlapping regions, and combined with multi-band fusion technology, so that overlapping regions of any shape can achieve natural transition, which helps to eliminate splicing seams and ghosting phenomena.

[0018] By applying geometric smoothing constraints, the relative displacement of adjacent areas is limited, ensuring that rigid targets such as buildings do not twist or deform during the splicing process. At the same time, noise suppression processing avoids damage to the edge structure and maintains the naturalness and authenticity of the target structure.

[0019] The algorithm flow for alignment, fusion and noise suppression has been optimized, and a hierarchical processing strategy has been adopted to reduce computational complexity. It can meet the real-time processing requirements of airborne or edge devices and is suitable for real-time image stitching scenarios during UAV flight.

[0020] By employing a noise adaptive suppression mechanism to specifically optimize the dynamic noise characteristics of UAV images, the applicable environment of the system is broadened, which is conducive to improving the stable stitching effect under complex flight conditions. Attached Figure Description

[0021] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0022] Figure 1 This is a flowchart of a high-fidelity image stitching method for UAVs according to the present invention. Detailed Implementation

[0023] To make the technical means, creative features, objectives and effects of this invention easier to understand, the invention will be further described below in conjunction with specific embodiments.

[0024] like Figure 1 As shown, a high-fidelity image stitching system for UAVs includes:

[0025] Image acquisition unit

[0026] It uses a high-definition camera mounted on the drone, which has image acquisition and data transmission capabilities. It can collect image data of the target area in real time and transmit the image data synchronously to the dual-stage processing unit through the drone's built-in data transmission module.

[0027] Two-stage processing unit

[0028] Employing an embedded processor, integrating alignment and fusion modules, it possesses hardware acceleration processing capabilities, enabling efficient completion of operations such as image feature extraction, transformation parameter calculation, alignment optimization, and fusion reconstruction, ensuring high-speed data processing and transmission.

[0029] Output display unit

[0030] It includes a display component integrated on the UAV and a ground station terminal. The UAV display component is used to preview the stitched image thumbnail in real time; the ground station terminal receives the complete stitched image via wireless communication and supports image display, storage and export functions to meet the needs of subsequent analysis applications.

[0031] A high-fidelity image stitching method for UAV images is implemented as follows:

[0032] Image acquisition and data input

[0033] During the drone's flight, the image acquisition unit acquires two adjacent images of the target area in real time. After acquisition, the image data is transmitted to the dual-stage processing unit in real time through the data transmission module. The dual-stage processing unit performs format parsing and preprocessing on the received image data to ensure data integrity and lay the foundation for subsequent stitching processing.

[0034] Alignment stage processing

[0035] Global estimation (coarse alignment): The alignment module of the two-stage processing unit first preprocesses the two input images, extracts global feature points of the images and performs matching. After removing mismatched points, it calculates the overall transformation parameters between the two images. Based on these transformation parameters, it performs coordinate transformation on one of the images to achieve coarse alignment between the two images and initially eliminates the overall positional offset between the images.

[0036] Local Correction (Fine Alignment): Based on coarse alignment, the alignment module uses an adaptive algorithm to estimate the local non-rigid offset between two images, divides the image into several image blocks, calculates the local offset vector for each image block, adjusts the offset vector for areas with obvious local deformation, and corrects the coarse alignment result block by block based on the optimized local non-rigid offset, further improving the accuracy of image alignment and achieving accurate matching of detailed areas.

[0037] Adaptive Enhancement: During feature extraction, the alignment module analyzes features such as grayscale gradient, texture complexity, and information entropy to determine the importance of image regions. Regions with rich texture and high information content are identified as key feature regions, while regions with simple texture and low information content are identified as irrelevant background regions. By dynamically adjusting the feature weight coefficients, the feature response of key feature regions is strengthened, feature interference from irrelevant background regions is suppressed, and the accuracy of feature matching is improved.

[0038] Geometric smoothing constraint: The alignment module applies a smoothing constraint to the local offsets obtained during the local correction process, constructs an offset correlation model between adjacent image blocks, limits the offset differences between adjacent image blocks, avoids problems such as tearing and breaking of the image structure due to excessive local offsets, and ensures the structural integrity and naturalness of rigid targets.

[0039] Fusion stage processing

[0040] Weighted mask generation:

[0041] The fusion module of the dual-stage processing unit first determines the overlapping area and contour shape of the two aligned images by comparing image pixels and mapping coordinates. Based on the shape of the overlapping area, an adaptive algorithm is used to solve the pixel-level smoothing weights, so that the weight of each pixel in the overlapping area smoothly transitions from the center of the overlapping area to the edge. The weight of pixels in the non-overlapping area is set to 1, ensuring that the weight of two pixels at the same position in the overlapping area is 1, and the weight is continuous without abrupt changes at the boundary of the overlapping area, so there is no need to perform seam line detection.

[0042] Multi-band fusion:

[0043] The fusion module employs an image decomposition algorithm to break down the two aligned images into different frequency layers, covering frequency levels that reflect overall image features, detailed textures, and edge sharpness. Based on the characteristics of different frequency layers, each frequency layer image is weighted and fused according to the generated pixel-level weighted mask, balancing image brightness consistency, detail preservation, and edge clarity during the fusion process. After fusion, an image reconstruction algorithm integrates the fusion results of each frequency layer to obtain the final stitched image, effectively eliminating stitching seams and ghosting phenomena.

[0044] Output and display:

[0045] After the fusion phase is completed, the dual-stage processing unit transmits the final stitched image data to the output display unit. The display component on the UAV displays a thumbnail of the stitched image in real time, allowing operators to quickly view the stitching effect. At the same time, the complete stitched image data is transmitted to the ground station terminal via the wireless communication module. After receiving the data, the ground station terminal decodes and displays the image. Operators can view, store, or export the stitched image for subsequent scene analysis applications.

[0046] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or basic characteristics. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, it is intended that all variations falling within the meaning and scope of equivalents of the claims be included within the present invention.

[0047] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A high-fidelity image stitching method for UAV images, characterized in that: The methods and steps include the following: Step 1: Alignment stage. First, coarse alignment is achieved by calculating the overall transformation parameters between the two images. Then, local non-rigid offsets are estimated to refine the alignment results. During feature extraction, feature weights are dynamically adjusted according to the importance of the regions. At the same time, smoothing constraints are applied to the local offsets to avoid structural tearing. Step 2: Fusion stage. Based on the shape of the overlapping area of ​​the aligned image, calculate the pixel-level smoothing weight. The pixel-level smoothing weight satisfies the condition that the boundaries are continuous and the weights at the same position sum to 1. Decompose the image into different frequency layers and fuse them layer by layer according to the pixel-level smoothing weight to reconstruct the stitched image.

2. The high-fidelity image stitching method for UAVs according to claim 1, characterized in that: The alignment stage achieves precise image positioning through two-stage "coarse-fine" processing, which is used to handle problems with large parallax and non-rigid deformation.

3. The high-fidelity stitching method and system for UAV images according to claim 1, characterized in that: In the process of adaptively adjusting feature weights, key feature regions are highlighted while interference from irrelevant background regions is suppressed.

4. The high-fidelity image stitching method for UAVs according to claim 1, characterized in that: The smoothing constraint applied to the local offset is used to maintain the structural naturalness of rigid targets such as buildings and avoid distortion and deformation.

5. The high-fidelity stitching method for UAV images according to claim 1, characterized in that: The generation of the pixel-level weighted mask does not require seam line detection and can adapt to overlapping areas of any shape.

6. The high-fidelity stitching method for UAV images according to claim 1, characterized in that: The multi-band fusion is used to eliminate stitching seams and ghosting, achieving a natural image transition.

7. A high-fidelity stitching system for UAV images, applicable to the high-fidelity stitching method for UAV images as described in any one of claims 1-6, characterized in that: It includes an image acquisition unit, a two-stage processing unit, and an output display unit: the image acquisition unit is used to acquire images of the target area during the flight of the UAV and transmit the acquired image data to the two-stage processing unit; The dual-stage processing unit includes an alignment module and a fusion module. The alignment module is responsible for global estimation, local correction, adaptive enhancement, and geometric smoothing constraint functions, while the fusion module is responsible for weight mask generation and multi-band fusion functions. The output display unit is used to receive the stitched image output by the dual-stage processing unit and display or store it.

Images