Unmanned aerial vehicle aerial multi-image splicing algorithm based on opencv

By using OpenCV CUDA parallel processing technology and efficient memory management mechanism on GPU, aerial images are preprocessed and spliced, solving the problems of limited computing resources and low memory management efficiency in the existing technology, and achieving efficient and accurate image splicing Effect.

CN119444561AActive Publication Date: 2025-02-14GUANGXI NORMAL UNIV

Patent Information

Application Number
CN202411474678.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-10-22
Publication Date
2025-02-14
Estimated Expiration
2044-10-22

AI Technical Summary

Technical Problem

When processing large-scale and high-resolution aerial images, the computing resources are limited and the processing speed is slow, which affects the real-time and efficiency of image processing. In addition, dynamic memory allocation and release are frequent, resulting in memory fragmentation and increasing the number of system calls. , reduce performance.

Method used

Adopting OpenCV CUDA parallel processing technology, aerial images are preprocessed on the GPU, and an efficient memory management mechanism is established to dynamically calculate the memory size requests of image information, optimize memory allocation, and reduce memory fragmentation by predefined memory blocks of different sizes. and use smooth transition algorithm and weighted smooth transition mechanism during image stitching to reduce the sense of discontinuity in splicing traces and visual discontinuity.

Benefits of technology

It significantly improves the speed and efficiency of image processing, optimizes memory management, reduces the number of system calls, reduces the risk of memory fragmentation, improves the accuracy and nature of stitching, and ensures the quality and continuity of stitching images.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119444561A_ABST
    Figure CN119444561A_ABST
Patent Text Reader

Abstract

The invention discloses an unmanned aerial vehicle aerial multi-image splicing algorithm based on opencv, and the algorithm comprises the following steps: S1, obtaining an aerial image data set, employing an OpenCV CUDA parallel processing technology, and carrying out the preprocessing of the image information of the aerial image data set on a GPU; s2, establishing a memory management mechanism, dynamically calculating a memory size request of the image information by combining and utilizing the parallel computing capability of CUDA in the preprocessing process, pre-defining memory blocks with different sizes in the memory management mechanism, and optimizing memory allocation; s3, grouping the preprocessed image information according to the feature information of the image information, and carrying out parallelization processing on image splicing tasks in each group by using CUDA (Compute Unified Device Architecture); by adopting the OpenCV CUDA parallel processing technology, the aerial image is preprocessed on the GPU, the image processing speed and efficiency are remarkably improved, the memory size request of the image information is dynamically calculated by utilizing the parallel computing capability of the CUDA, and the memory allocation is optimized, so that the overall performance is further improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an OpenCV-based unmanned aerial vehicle (UAV) aerial photography multi-image stitching algorithm. Background Art

[0002] Image stitching technology, also known as image mosaicking, is an important technology in the field of computer vision. It can stitch multiple images into a complete and continuous image. In practical applications, image stitching technology is widely used in panoramic image generation, virtual reality, image restoration and other fields.

[0003] The application of image stitching technology in the field of drone aerial photography is crucial. It can stitch multiple images taken by drones into a panoramic image of a large scene, providing more comprehensive and intuitive information.

[0004] However, when processing large-scale, high-resolution aerial images, existing technologies often face the problems of limited computing resources and slow processing speed, which affects the real-time and efficiency of image processing. In addition, during the image processing process, dynamic memory allocation and release are frequent, which can easily lead to memory fragmentation, thereby increasing the number of system calls and reducing performance.

[0005] Therefore, there is an urgent need for an opencv-based drone aerial photography multi-image stitching algorithm to solve the technical problems existing in the above-mentioned prior art. Summary of the invention

[0006] The present invention overcomes the shortcomings of the prior art and provides an OpenCV-based UAV aerial photography multi-image stitching algorithm.

[0007] To achieve the above object, the technical solution adopted by the present invention is: an opencv-based drone aerial photography multi-image stitching algorithm, comprising the following steps:

[0008] S1, obtaining an aerial image dataset, and using OpenCV CUDA parallel processing technology to pre-process the image information of the aerial image dataset on the GPU;

[0009] S2. Establish a memory management mechanism. In the preprocessing process, combine and utilize the parallel computing capability of CUDA to dynamically calculate the memory size request of image information, and predefine memory blocks of different sizes in the memory management mechanism to optimize memory allocation;

[0010] S3. According to the feature information of the image information, the pre-processed image information is grouped, and CUDA is used to parallelize the image stitching tasks in each group.

[0011] In a preferred embodiment of the present invention, the memory allocation of the memory management mechanism includes the following steps:

[0012] S201, creating multiple levels of free memory block linked lists according to predefined memory block sizes, each level corresponding to a memory block of a fixed size;

[0013] S202, when requesting memory allocation, dynamically calculate the minimum and appropriate memory block size and the corresponding level according to the size of the request, and take out the memory block from the free list of the corresponding level; if there is no available memory block in the free list of the level, apply for a new memory block from the operating system and return it to the user;

[0014] S203: When the user releases memory, the header information of the memory block is extracted to determine the level to which it belongs, and the memory block is inserted into the free list of the corresponding level.

[0015] In a preferred embodiment of the present invention, a thread-safe mechanism is adopted in the memory management mechanism, and the thread-safe mechanism is organized into thread blocks and grids, and the image stitching task is decomposed into multiple subtasks that can be processed in parallel.

[0016] In a preferred embodiment of the present invention, the stitching process of the image stitching task includes the following steps:

[0017] S301, automatically extracting and grouping image features using machine learning or deep learning technology, wherein the basis for grouping includes overlapping areas of image information, shooting angles, timestamps, image similarities, and changes in lighting conditions;

[0018] 302. Dynamically adjust the allocation of memory blocks according to the size and complexity of image information features, and use transformation models and fusion algorithms based on image information features to reduce splicing traces;

[0019] A smooth transition algorithm is used at the seams of image stitching to reduce the visual discontinuity. At the same time, the stitching result is automatically verified by comparing the overlapping area of ​​the stitched image with the original image and checking the smoothness of the stitching seams.

[0020] In a preferred embodiment of the present invention, the stitching process of the image stitching task includes feature point detection, feature point description, feature point matching, feature point alignment, image fusion and image optimization, wherein the feature point alignment is used to calculate the relative displacement and angle between images, and the least squares method is used to fit the process expression of image stitching:

[0021] M=arg min M ∑ x ||x′ i -Mx i ||2

[0022] Where M is the transformation matrix, which is used to represent the feature points after image splicing; x i is the feature point in the original image; x′ i is the splicing process with x i The corresponding feature points.

[0023] In a preferred embodiment of the present invention, in the image stitching process, a smooth transition algorithm is adopted, using a weighted smooth transition mechanism, wherein the weight varies with the distance from the stitching edge, and the formula expression is:

[0024] I out (x,y)=αI1(x,y)+(1-α)I2(x,y)

[0025] In the formula, I out is the output image; I1 and I2 are the two images to be stitched; α is the weight coefficient, which changes with the position.

[0026] In a preferred embodiment of the present invention, the automatic verification is achieved by comparing the overlapping area of ​​the spliced ​​image with the original image and checking the smoothness of the spliced ​​seam, and the formula expression is:

[0027]

[0028] In the formula, μ x and μ y are the average values ​​of images x and y respectively; and is the variance; σ xy is the covariance; C1 and C2 are constants used to stabilize the denominator; the value of A ranges from 0 to 1, and the closer the value is to 1, the more similar the images are.

[0029] In a preferred embodiment of the present invention, an opencv-based drone aerial photography multi-image stitching system is applied to the above-mentioned opencv-based drone aerial photography multi-image stitching algorithm, including a GPU image preprocessing module, an image grouping and parallel stitching module and an efficient memory pool management module; the GPU image preprocessing module is used to use opencvCUDA to crop and histogram equalize the aerial images; the image grouping and parallel stitching module is used to group the preprocessed images and process the image stitching tasks in each group in parallel on the GPU; the efficient memory pool management module is used to dynamically calculate and allocate memory blocks, optimize memory management, and improve the efficiency of memory allocation and release.

[0030] In a preferred embodiment of the present invention, the efficient memory pool management module also includes: an initialization submodule, a memory allocation submodule and a memory release submodule; the initialization submodule is used to create multiple levels of free memory block linked lists according to predefined memory block sizes; the memory allocation submodule is used to dynamically calculate and allocate memory blocks according to the size requested by the user; the memory release submodule is used to insert the released memory blocks into the free linked lists of the corresponding levels.

[0031] In a preferred embodiment of the present invention, a computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned method and system when executing the computer program.

[0032] The present invention solves the defects existing in the background technology and has the following beneficial effects:

[0033] (1) The present invention adopts OpenCV CUDA parallel processing technology to pre-process aerial images on the GPU, which significantly improves the speed and efficiency of image processing. It also uses CUDA's parallel computing capability to dynamically calculate the memory size request of image information and optimize memory allocation, thereby further improving the overall performance.

[0034] (2) The present invention establishes an efficient memory management mechanism and pre-manages memory blocks of different sizes to optimize the efficiency of memory allocation and release, reduce frequent system calls, and reduce the risk of memory fragmentation. The memory pool adopts a thread-safe mechanism to ensure stable operation in a multi-threaded environment, thereby further improving the reliability and stability of the system.

[0035] (3) The present invention groups the pre-processed image information according to the feature information of the image information, and uses CUDA to parallelize the image stitching tasks in each group, thereby improving the accuracy and naturalness of stitching. At the same time, in the image stitching process, a smooth transition algorithm and a weighted smooth transition mechanism are used to reduce the stitching traces and visual discontinuity. The stitching results are automatically checked and optimized through an automatic verification mechanism, thereby ensuring the quality and continuity of the stitched image.

[0036] (4) The present invention combines FAST feature point detection and BRIEF feature description and uses an improved version of rotation invariance to improve detection speed and robustness, so that the algorithm can better adapt to changes in different lighting conditions and shooting angles. In addition, in image fusion, multiple factors such as image quality, frequency components, and regional characteristics are comprehensively considered, and appropriate weight coefficients are selected to achieve a better fusion effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art are briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work.

[0038] Figure 1 is an image stitching flow chart of a preferred embodiment of the present invention;

[0039] Figure 2 is a flowchart of a memory management mechanism of a preferred embodiment of the present invention;

[0040] Figure 3 It is a comparison diagram of the stitching results of multiple input images using the drone aerial photography multi-image stitching algorithm of the present invention. DETAILED DESCRIPTION

[0041] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0042] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited to the specific embodiments disclosed below.

[0043] like Figure 1 As shown in the figure, a multi-image stitching algorithm for drone aerial photography based on OpenCV includes the following steps:

[0044] S1, obtaining an aerial image dataset, and using OpenCV CUDA parallel processing technology to pre-process the image information of the aerial image dataset on the GPU;

[0045] Furthermore, the input aerial image is grayed to reduce the computational complexity, and then Gaussian blur processing is performed to reduce the impact of noise on feature extraction. By constructing a Gaussian difference pyramid (DoG), extreme points are found in the scale space as feature points, and the determinant of the Hessian matrix is ​​used to detect points of interest in the image. The calculation is accelerated by the integral image, combined with FAST feature point detection and BRIEF feature description, and an improved version of rotation invariance is used to improve the detection speed and robustness.

[0046] Calculate the gradient direction and size around the feature point to generate a 64-dimensional or 128-dimensional description vector. Then use the binary descriptor to generate a description vector by comparing the brightness difference of the pixels around the feature point. Then use the Euclidean distance, Hamming distance (for binary descriptors) or other distance metrics to calculate the similarity of the feature point descriptors in the two images.

[0047] like Figure 2 As shown, S2, a memory management mechanism is established. During the preprocessing process, the parallel computing capability of CUDA is combined to dynamically calculate the memory size request of the image information, and memory blocks of different sizes are predefined in the memory management mechanism to optimize memory allocation;

[0048] It should be further explained that the memory pool optimizes the efficiency of memory allocation and release by pre-managing memory blocks of different sizes. Its workflow is as follows: During initialization, the memory pool creates multiple levels of free memory block lists based on the predefined memory block size. Each level corresponds to a fixed-size memory block. When a user requests memory allocation, the memory pool calculates the minimum and appropriate memory block size and the corresponding level based on the requested size. If there is an available memory block in the free list of that level, it is directly taken out and returned to the user; if not, a new memory block is requested from the operating system and returned. When the user releases memory, the memory pool extracts the header information of the memory block, determines the level it belongs to, and then inserts the memory block into the free list of the corresponding level for subsequent use. In this way, the memory pool realizes the rapid allocation and recovery of memory blocks, reduces frequent system calls, reduces the risk of memory fragmentation, and improves overall performance. The memory pool also adopts a thread-safe mechanism to ensure stable operation in a multi-threaded environment.

[0049] Furthermore, the memory allocation of the memory management mechanism includes the following steps:

[0050] S201, creating multiple levels of free memory block linked lists according to predefined memory block sizes, each level corresponding to a memory block of a fixed size;

[0051] S202, when requesting memory allocation, dynamically calculate the minimum and appropriate memory block size and the corresponding level according to the size of the request, and take out the memory block from the free list of the corresponding level; if there is no available memory block in the free list of the level, apply for a new memory block from the operating system and return it to the user;

[0052] S203: When the user releases memory, the header information of the memory block is extracted to determine the level to which it belongs, and the memory block is inserted into the free list of the corresponding level.

[0053] Furthermore, a thread-safe mechanism is adopted in the memory management mechanism, and the thread-safe mechanism is organized into thread blocks and grids, which decomposes the image stitching task into multiple subtasks that can be processed in parallel.

[0054] S3. According to the feature information of the image information, the pre-processed image information is grouped, and CUDA is used to parallelize the image stitching tasks in each group.

[0055] Furthermore, the stitching process of the image stitching task includes the following steps:

[0056] S301, automatically extracting and grouping image features using machine learning or deep learning technology, wherein the basis for grouping includes overlapping areas of image information, shooting angles, timestamps, image similarities, and changes in lighting conditions;

[0057] 303. According to the size and complexity of image information features, dynamically adjust the allocation of memory blocks, and based on image information features, use transformation models and fusion algorithms to reduce splicing traces;

[0058] A smooth transition algorithm is used at the seams of image stitching to reduce the visual discontinuity. At the same time, the stitching result is automatically verified by comparing the overlapping area of ​​the stitched image with the original image and checking the smoothness of the stitching seams.

[0059] The stitching process of the image stitching task includes feature point detection, feature point description, feature point matching, feature point alignment, image fusion and image optimization. Feature point alignment is used to calculate the relative displacement and angle between images. The least squares method is used to fit the process expression of image stitching:

[0060] M=argmin M ∑ x ||x′ i -Mx i || 2

[0061] Where M is the transformation matrix, which is used to represent the feature points after image splicing; x i is the feature point in the original image; x′ i is the splicing process with x i The corresponding feature points.

[0062] Furthermore, in the image stitching process, a smooth transition algorithm is adopted, using a weighted smooth transition mechanism, in which the weight varies with the distance from the stitching edge, and its formula expression is:

[0063] I out (x,y)=αI1(x,y)+(1-α)I2(x,y)

[0064] In the formula, I out is the output image; I1 and I2 are the two images to be stitched; α is the weight coefficient, which changes with the position.

[0065] In image fusion, choosing a suitable weight coefficient α is crucial because it directly affects the quality and visual effect of the fused image. The weight coefficient α is determined based on the following factors:

[0066] Image quality: The weight is determined based on the image’s clarity, contrast and other quality indicators. Images with higher quality can be given larger weights to ensure that more details and clarity are retained in the fused image. For example, image quality can be measured using indicators such as local variance and gradient, and weights can be assigned accordingly.

[0067] Spatial frequency: The weight is determined according to the energy distribution of different frequency components of the image. High-frequency components usually contain detailed information of the image, while low-frequency components contain the overall information of the image. The spectral characteristics of the image can be analyzed by performing wavelet transform or Fourier transform on the image, and weights can be assigned according to the spectral characteristics.

[0068] Regional features: weights are determined based on the features of different regions in the image (such as edges, textures, etc.). For example, a larger weight can be given to the edge region to retain the detailed information of the image.

[0069] Pixel intensity and global gradient: Combining pixel intensity, local gradient and global gradient information to construct an adaptive weight function to effectively fuse multi-exposure images;

[0070] Experimental verification: A large number of experiments are conducted to verify the effects of different weight settings and fusion algorithms, and to find the solution that best suits the current task. For example, in multi-exposure image fusion, the optimal weights of different exposure images can be determined through experiments to improve the dynamic range and detail retention of the fused image.

[0071] Adaptive weight function: In some advanced fusion algorithms, an adaptive weight function is constructed, which can dynamically adjust the weight coefficient according to the image content. This function usually requires the use of machine learning or deep learning technology to achieve this;

[0072] Weighted average method: In the simple weighted average method, the weight coefficients α and (1-α) (for another image) usually sum to 1, that is, α+(1-α) = 1. This method is simple and easy to implement, but the values ​​of α and β need to be adjusted according to the specific situation to obtain the best fusion effect;

[0073] Deep learning technology: In the image fusion method based on deep learning, the optimal weight coefficient can be learned through the training process, thereby achieving a better fusion effect.

[0074] In summary, the selection of an appropriate weight coefficient α requires comprehensive consideration of image quality, frequency components, regional characteristics, and specific application scenarios. In practical applications, the optimal weight coefficient needs to be determined through experiments and adjustments.

[0075] In a preferred embodiment, a multi-band fusion algorithm, such as Laplacian pyramid fusion or gradient domain fusion, is used to reduce stitching traces and improve the naturalness and continuity of the image.

[0076] Automatic verification is achieved by comparing the overlapping area of ​​the stitched image with the original image and checking the smoothness of the stitching. The formula is:

[0077]

[0078] In the formula, μ x and μ y are the average values ​​of images x and y respectively; and is the variance; σ xy is the covariance; C1 and C2 are constants used to stabilize the denominator; the value of A ranges from 0 to 1, and the closer the value is to 1, the more similar the images are.

[0079] In a preferred embodiment, the calculation of A can be implemented based on a local window, that is, each calculation takes a window of size N×N from the image, calculates the A index based on the window, and then takes the average of the values ​​of all windows after traversing the entire image as the A index of the entire image, thereby better simulating the human eye's perception of the image, because the human eye is more sensitive to local areas of the image.

[0080] Furthermore, a UAV aerial photography multi-image stitching system based on OpenCV is provided, which is applied to the above-mentioned UAV aerial photography multi-image stitching algorithm based on OpenCV, including a GPU image preprocessing module, an image grouping and parallel stitching module and an efficient memory pool management module; the GPU image preprocessing module is used to use OpenCV CUDA to crop and histogram equalize the aerial images; the image grouping and parallel stitching module is used to group the preprocessed images, and process the image stitching tasks in each group in parallel on the GPU; the efficient memory pool management module is used to dynamically calculate and allocate memory blocks, optimize memory management, and improve the efficiency of memory allocation and release.

[0081] Among them, the efficient memory pool management module also includes: an initialization submodule, a memory allocation submodule and a memory release submodule; the initialization submodule is used to create multiple levels of free memory block linked lists according to predefined memory block sizes; the memory allocation submodule is used to dynamically calculate and allocate memory blocks according to the size requested by the user; the memory release submodule is used to insert the released memory blocks into the free linked lists of the corresponding levels.

[0082] like Figure 3 As shown, the left side is the image stitching result using the opendrone platform, and the right side is the image stitching result using the present invention. Under the same memory, for the same number of data sets, the opendrone platform is used to compare with the present invention, and it can be seen that under the same memory, the result of the present invention is clearer.

[0083] Comparison in running time:

[0084] Table 1

[0085]

[0086] From Table 1, it can be seen that the stitching method of the present invention is significantly better than the Opendrone platform in terms of running time. Whether on a small-scale or large-scale image dataset, the present invention can complete the stitching task faster. Therefore, from the perspective of running time, the present invention is a more efficient drone aerial multi-image stitching method.

[0087] When the present invention is used, it comprises the following steps:

[0088] GPU image preprocessing module: obtain aerial image datasets, and use OpenCV CUDA parallel processing technology to preprocess images on the GPU, grayscale the input aerial images to reduce computational complexity, perform Gaussian blur processing to reduce the impact of noise on feature extraction, and construct a Gaussian difference pyramid (DoG). Find extreme points as feature points in the scale space, and use the determinant of the Hessian matrix to detect points of interest in the image. By combining FAST feature point detection and BRIEF feature description, an improved version of rotation invariance is used to improve detection speed and robustness. After that, the gradient direction and size around the feature points are calculated to generate a description vector, and a binary descriptor is used to accelerate the calculation. The similarity of the feature point descriptors in the two images is calculated using Euclidean distance, Hamming distance or other distance measurement methods.

[0089] Efficient memory pool management module: establish a memory management mechanism, combine CUDA's parallel computing capabilities, dynamically calculate the memory size request of image information, predefine memory blocks of different sizes in the memory management mechanism, and optimize the efficiency of memory allocation and release. The memory pool manages memory blocks of different sizes in advance to achieve fast allocation and recovery of memory blocks, reduce frequent system calls, and reduce the risk of memory fragmentation. The memory management mechanism adopts a thread-safe mechanism to ensure stable operation in a multi-threaded environment. Finally, the memory management mechanism is decomposed into an initialization sub-module, a memory allocation sub-module, and a memory release sub-module, which are responsible for creating a free memory block list, dynamically calculating and allocating memory blocks, and inserting released memory blocks into the free list of the corresponding level.

[0090] Image grouping and parallel stitching module: According to the feature information of the image information, the pre-processed image information is grouped, and CUDA is used to parallelize the image stitching tasks in each group to improve the stitching efficiency. The image stitching process includes feature point detection, feature point description, feature point matching, feature point alignment, image fusion and image optimization. A smooth transition algorithm is used at the seams of image stitching to reduce the visual discontinuity. The stitching result is automatically verified by comparing the overlapping area of ​​the stitched image with the original image and checking the smoothness of the stitching seams.

[0091] The above is based on the ideal embodiment of the present invention. Through the above description, relevant personnel can make various changes and modifications without departing from the technical concept of the present invention. The technical scope of the present invention is not limited to the content in the specification, and the technical scope must be determined according to the scope of the claims.

Claims

1. A multi-image stitching algorithm for drone aerial photography based on OpenCV, characterized in that: The following steps are involved: S1, obtaining an aerial image dataset, and using OpenCV CUDA parallel processing technology to pre-process the image information of the aerial image dataset on a GPU; S2. Establish a memory management mechanism. In the preprocessing process, combine and utilize the parallel computing capability of CUDA to dynamically calculate the memory size request of image information, and predefine memory blocks of different sizes in the memory management mechanism to optimize memory allocation; S3. According to the feature information of the image information, the pre-processed image information is grouped, and CUDA is used to parallelize the image stitching tasks in each group.

2. The OpenCV-based UAV aerial photography multi-image stitching algorithm according to claim 1 is characterized by: The memory allocation of the memory management mechanism includes the following steps: S201, creating multiple levels of free memory block linked lists according to predefined memory block sizes, each level corresponding to a memory block of a fixed size; S202, when requesting memory allocation, dynamically calculate the minimum and appropriate memory block size and the corresponding level according to the size of the request, and take out the memory block from the free list of the corresponding level; if there is no available memory block in the free list of the level, apply for a new memory block from the operating system and return it to the user; S203: When the user releases memory, the header information of the memory block is extracted to determine the level to which it belongs, and the memory block is inserted into the free list of the corresponding level.

3. The OpenCV-based UAV aerial photography multi-image stitching algorithm according to claims 1 and 2 is characterized in that: The memory management mechanism adopts a thread-safe mechanism, and organizes the thread-safe mechanism into thread blocks and grids, decomposing the image stitching task into multiple subtasks that can be processed in parallel.

4. The OpenCV-based UAV aerial photography multi-image stitching algorithm according to claims 1 and 2 is characterized in that: The stitching process of the image stitching task includes the following steps: S301, automatically extracting and grouping image features using machine learning or deep learning technology, wherein the basis for grouping includes overlapping areas of image information, shooting angles, timestamps, image similarities, and changes in lighting conditions; 302. Dynamically adjust the allocation of memory blocks according to the size and complexity of image information features, and use transformation models and fusion algorithms based on image information features to reduce splicing traces; 303. A smooth transition algorithm is used at the seams of image stitching to reduce the visual discontinuity. At the same time, the stitching result is automatically verified by comparing the overlapping area of ​​the stitching image with the original image and checking the smoothness of the stitching seams.

5. The OpenCV-based UAV aerial photography multi-image stitching algorithm according to claim 1 is characterized by: The stitching process of the image stitching task includes feature point detection, feature point description, feature point matching, feature point alignment, image fusion and image optimization, wherein feature point alignment is used to calculate the relative displacement and angle between images, and the least squares method is used to fit the process expression of image stitching: M=arg min M S x ‖x′ i -Mx i || 2 Where M is the transformation matrix, which is used to represent the feature points after image splicing; x i is the feature point in the original image; x i ′ is the splicing process with x i The corresponding feature points.

6. The OpenCV-based UAV aerial photography multi-image stitching algorithm according to claims 4 and 5 is characterized in that: In the process of image stitching, a smooth transition algorithm is adopted, using a weighted smooth transition mechanism, in which the weight changes with the distance from the stitching edge. The formula is: I out (x,y)=αI1(x,y)+(1-α)I2(x,y) In the formula, I out is the output image; I1 and I2 are the two images to be stitched; α is the weight coefficient, which changes with the position.

7. The OpenCV-based UAV aerial photography multi-image stitching algorithm according to claims 1 and 4 is characterized by: The automatic verification is achieved by comparing the overlapping area of ​​the spliced ​​image with the original image and checking the smoothness of the spliced ​​seam. The formula is: In the formula, μ x and μ y are the average values ​​of images x and y respectively; and is the variance; σ xy is the covariance; C1 and C2 are constants used to stabilize the denominator; the value of A ranges from 0 to 1, and the closer the value is to 1, the more similar the images are.

8. An OpenCV-based drone aerial photography multi-image stitching system, based on the OpenCV-based drone aerial photography multi-image stitching algorithm described in any one of claims 1 to 7, characterized in that: It includes a GPU image preprocessing module, an image grouping and parallel stitching module, and an efficient memory pool management module; the GPU image preprocessing module is used to use opencv CUDA to crop and histogram equalize the aerial images; the image grouping and parallel stitching module is used to group the preprocessed images and process the image stitching tasks in each group in parallel on the GPU; The efficient memory pool management module is used to dynamically calculate and allocate memory blocks, optimize memory management, and improve the efficiency of memory allocation and release.

9. The OpenCV-based drone aerial photography multi-image stitching system according to claim 8, characterized in that: The efficient memory pool management module also includes: an initialization submodule, a memory allocation submodule and a memory release submodule; the initialization submodule is used to create multiple levels of free memory block linked lists according to predefined memory block sizes; the memory allocation submodule is used to dynamically calculate and allocate memory blocks according to the size requested by the user; the memory release submodule is used to insert the released memory blocks into the free linked lists of the corresponding levels.

10. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method and system according to any one of claims 1 to 9 are implemented.

Citation Information

Patent Citations

  • Quantified evaluation method of image mosaic algorithms

    CN102169576A

  • Video stitching method and system

    CN105915804A

  • A video mosaic method based on large scene and multi-GPU mode

    CN109040701A

  • An image fusion and mosaic method based on parallel computing algorithm

    CN109461121A

  • Unmanned aerial vehicle aerial image splicing method for enhancing robustness

    CN111080529A

Cited By

  • High-definition mapping method based on low-altitude unmanned aerial vehicle live video

    CN120091111A

  • High-resolution image generation method based on live video from low-altitude drones

    CN120091111B