An image stitching method and device, electronic equipment and storage medium
By acquiring the boundary point coordinates of UAV images for image stitching, the problem of long image stitching time and high cost in existing technologies is solved, and efficient image fusion is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- QINGDAO CLOUD CENTURY INFORMATION TECH CO LTD
- Filing Date
- 2022-09-28
- Publication Date
- 2026-05-12
AI Technical Summary
Current technologies for drone image stitching require specialized software and powerful server computing capabilities, resulting in long processing times and high costs.
By acquiring multiple images and their parameter information captured by a drone, the coordinates of the boundary points of the images are determined, and these coordinates are matched with the pixels in the resulting image to achieve direct image fusion and stitching.
It reduces the cost of image stitching, improves stitching efficiency, and avoids dependence on professional software and high-performance servers.
Smart Images
Figure CN115439328B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of image processing technology, and more specifically, to an image stitching method, apparatus, electronic device, and storage medium. Background Technology
[0002] Currently, when stitching images captured by drones, it is necessary to extract feature points from each captured image and stitch together images with the same feature points. This process generally requires professional data processing engineers to use multiple specialized software programs to perform special processing on the images and interim results. During this special processing, a professional data processing server is needed to support the process and obtain the final stitched image.
[0003] The above image stitching methods have a long processing time and require servers with powerful computing capabilities, which increases the cost of image stitching. Summary of the Invention
[0004] The purpose of this application is to provide an image stitching method, apparatus, electronic device, and storage medium that can improve the efficiency of image stitching.
[0005] To achieve the above objectives, the technical solutions adopted in the embodiments of this application are as follows:
[0006] In a first aspect, embodiments of this application provide an image stitching method, the method comprising:
[0007] Acquire multiple images to be stitched together, captured by a drone, as well as a result image representing the drone's shooting range;
[0008] Obtain the coordinates of each pixel in the resulting image;
[0009] Determine the parameter information of each of the images to be stitched together;
[0010] For each image to be stitched, the boundary point coordinate information of the image to be stitched is determined based on the parameter information of the image to be stitched. The boundary point coordinate information represents the coordinate information of the top left vertex, bottom left vertex, top right vertex and bottom right vertex of the image to be stitched.
[0011] The boundary point coordinates of each image to be stitched are matched with the coordinates of each pixel in the result image to obtain the region range of each image to be stitched in the result image.
[0012] Each of the images to be stitched is fused with the corresponding region of the resulting image to obtain the stitched target image.
[0013] In an optional implementation, the step of obtaining the coordinate positions of each pixel in the resulting image includes:
[0014] Determine the operational range of the drone and the drone camera parameters;
[0015] Acquire images captured by the drone;
[0016] Determine the raster information of the captured image;
[0017] Based on the grid information and the UAV camera parameters, an affine transformation is performed on the spatial operation range to obtain the coordinate positions of each pixel in the resulting image.
[0018] In an optional implementation, the parameter information includes: sensor information of the UAV that captured the images to be stitched, focal length, yaw angle, pitch angle, roll angle, shooting altitude, and center point coordinates of the images to be stitched. The step of determining the boundary point coordinates of each image to be stitched based on its parameter information includes:
[0019] Determine the width and height information of the sensor data;
[0020] For each image to be stitched, the actual width and actual height of the image are determined based on the width and height information of the sensor and the focal length.
[0021] Based on the actual width information, pitch angle, roll angle, shooting height, and actual height information, the vertical distance from the center point of each image to be stitched to the image edge is determined, wherein the image edge includes the top edge, bottom edge, left edge, and right edge;
[0022] Based on the center point coordinates, the vertical distance from the center point to the image edge, and the yaw angle, the boundary point coordinate information of each image to be stitched is determined.
[0023] In an optional implementation, the actual width and actual height information of each of the images to be stitched together satisfy the following formula:
[0024] Actual width information = 2.0 * tan(x / f / 2);
[0025] Actual height information = 2.0 * tan(y / f / 2);
[0026] Where x represents the width of the sensor, y represents the height of the sensor, and f represents the focal length.
[0027] In an optional implementation, the vertical distance from the center point of each image to be stitched to the image edge satisfies the following formula:
[0028] bottom=H*tan(Pitch-0.5*yView);
[0029] top=H*tan(Pitch+0.5*yView);
[0030] left=H*tan(Roll-0.5*xView);
[0031] right=H*tan(Roll+0.5*xView);
[0032] Wherein, bottom is the vertical distance from the center point to the bottom edge, top is the vertical distance from the center point to the top edge, left is the vertical distance from the center point to the left edge, right is the vertical distance from the center point to the right edge, H is the shooting height, Pitch is the pitch angle, yView is the actual height information, xView is the actual width information, and Roll is the roll angle.
[0033] In an optional implementation, the method further includes: feathering the edges of each of the images to be stitched together;
[0034] The step of fusing each of the images to be stitched with the corresponding regions in the resulting image to obtain the stitched target image includes:
[0035] Each feathered image to be stitched is fused with the corresponding region in the resulting image to obtain the stitched target image.
[0036] In an optional implementation, the step of fusing each of the images to be stitched with the corresponding region of the resulting image to obtain the stitched target image includes:
[0037] For each image to be stitched, the image to be stitched and its corresponding region are divided according to a preset size to obtain multiple sub-images to be stitched and multiple sub-regions.
[0038] Each sub-image to be stitched is merged into its corresponding sub-region to obtain the stitched target image.
[0039] Secondly, embodiments of this application provide an image stitching device, the device comprising:
[0040] The first acquisition module is used to acquire multiple images to be stitched together and a result image representing the shooting range of the drone, captured by the drone.
[0041] The second acquisition module is used to determine the coordinate position of each pixel in the resulting image;
[0042] The first determining module is used to determine the parameter information of each of the images to be stitched together;
[0043] The second determining module is used to determine the boundary point coordinate information of each image to be stitched based on the parameter information of the image to be stitched, wherein the boundary point coordinate information represents the coordinate information of the upper left vertex, lower left vertex, upper right vertex and lower right vertex of the image to be stitched.
[0044] The matching module is used to match the boundary point coordinate information of each image to be stitched with the coordinate position of each pixel in the result image to obtain the region range corresponding to each image to be stitched in the result image.
[0045] The fusion module is used to fuse each of the images to be stitched with the corresponding regions in the result image to obtain the stitched target image.
[0046] Thirdly, embodiments of this application provide an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the image stitching method.
[0047] Fourthly, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the image stitching method.
[0048] This application has the following beneficial effects:
[0049] This application acquires multiple images to be stitched by a drone and a result image representing the drone's shooting range. It determines the coordinates of each pixel in the result image and the parameter information of each image to be stitched. For each image to be stitched, based on its parameter information, it determines the coordinates of its boundary points. Image stitching is then performed based on these boundary point coordinates. The boundary point coordinates of each image to be stitched are matched with the coordinates of each pixel in the result image to obtain the corresponding region of each image to be stitched in the result image. Each image to be stitched is then fused with its corresponding region in the result image to obtain the stitched target image. This method eliminates the need for multiple specialized software programs to obtain the feature points of each image to be stitched; instead, it stitches images with the same feature points together. Image stitching can be achieved simply by determining the boundary point coordinates of the images to be stitched, thereby reducing the cost and improving the efficiency of image stitching. Attached Figure Description
[0050] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.
[0051] Figure 1 A block diagram illustrating an electronic device provided in an embodiment of this application;
[0052] Figure 2 One of the flowcharts for an image stitching method provided in this application embodiment;
[0053] Figure 3 A second flowchart illustrating an image stitching method provided in this application embodiment;
[0054] Figure 4 A flowchart of an image stitching method provided in this application embodiment is shown as the third one.
[0055] Figure 5 A flowchart of an image stitching method provided in this application embodiment;
[0056] Figure 6 This is a schematic diagram of the image to be fused provided in an embodiment of this application;
[0057] Figure 7 This is a structural block diagram of an image stitching device provided in an embodiment of this application. Detailed Implementation
[0058] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.
[0059] Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.
[0060] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.
[0061] In the description of this application, it should be noted that if terms such as "upper," "lower," "inner," or "outer" are used to indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, or the orientation or positional relationship in which the product of the invention is usually placed during use, they are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application.
[0062] Furthermore, the terms "first" and "second" are used only to distinguish descriptions and should not be interpreted as indicating or implying relative importance.
[0063] Through extensive research, the inventors discovered that current image stitching methods generally require extracting feature points from captured images and stitching together images with the same feature points. This stitching method is time-consuming and requires servers with powerful computing capabilities, increasing the cost of image stitching.
[0064] In view of the above-mentioned problems, this embodiment provides an image stitching method, apparatus, electronic device, and storage medium that can achieve image stitching by determining the boundary point coordinate information of the images to be stitched, without the need to obtain the feature points of each image to be stitched through multiple professional software programs. By stitching together images with the same feature points, the cost of image stitching can be reduced and the efficiency of image stitching can be improved. The solution provided in this embodiment will be described in detail below.
[0065] This embodiment provides an electronic device capable of stitching images together. In one possible implementation, the electronic device can be a user terminal, such as, but not limited to, a server, smartphone, personal computer (PC), tablet computer, personal digital assistant (PDA), mobile internet device (MID), etc.
[0066] Please refer to Figure 1 , Figure 1 This is a schematic diagram of the structure of the electronic device 100 provided in the embodiments of this application. The electronic device 100 may further include... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown. Figure 1 The components shown can be implemented using hardware, software, or a combination thereof.
[0067] The electronic device 100 includes an image stitching device 110, a memory 120, and a processor 130.
[0068] The components of the memory 120 and processor 130 are electrically connected directly or indirectly to achieve data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines. The image stitching device 110 includes at least one software function module that can be stored in the memory 120 in the form of software or firmware or embedded in the operating system (OS) of the electronic device 100. The processor 130 is used to execute the executable modules stored in the memory 120, such as the software function modules and computer programs included in the image stitching device 110.
[0069] The memory 120 may be, but is not limited to, Random Access Memory (RAM), Read Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), etc. The memory 120 is used to store programs, and the processor 130 executes the programs after receiving execution instructions.
[0070] Please refer to Figure 2 , Figure 2 For application Figure 1 The flowchart of an image stitching method for an electronic device 100 is shown below, and the method includes each step in detail.
[0071] Step 201: Acquire multiple images to be stitched together and a result image representing the shooting range of the drone, taken by the drone.
[0072] Step 202: Obtain the coordinates of each pixel in the resulting image.
[0073] Step 203: Determine the parameter information of each image to be stitched.
[0074] Step 204: For each image to be stitched, determine the boundary point coordinates of the image based on its parameter information.
[0075] Step 205: Match the boundary point coordinates of each image to be stitched with the coordinates of each pixel in the result image to obtain the corresponding region of each image to be stitched in the result image.
[0076] Step 206: Merge each image to be stitched with the corresponding region in the result image to obtain the stitched target image.
[0077] A drone is used to capture images to be stitched together. The stitched images are of a size smaller than or equal to the size of the image captured by the drone.
[0078] For example, the pixel size of the resulting image can be 2048*1536, and the pixel size of a single image to be stitched together is less than 2048*1536.
[0079] To facilitate subsequent image stitching, it is necessary to obtain the coordinate positions of each pixel in the final image. For example, when the pixel size of the final image is 2048*1536, that is, when the final image contains 2048*1536 pixels, the coordinate position of each pixel needs to be obtained. The coordinate position of each pixel can be latitude and longitude coordinates or planar coordinates.
[0080] When using a drone to capture each image to be stitched, each image has different parameter information. The parameter information of each image represents the drone's attitude when capturing the image. The coordinate information of the boundary points of the images to be stitched is calculated based on the drone's attitude.
[0081] It should be noted that the boundary point coordinate information represents the coordinates of the top left, bottom left, top right, and bottom right vertices of the image to be stitched together.
[0082] To stitch together the images to be stitched, the corresponding region is determined from the resulting image based on the boundary point coordinates of each image, and the images to be stitched are then fused with the corresponding region.
[0083] For example: The images to be stitched include a first image, a second image, and a third image. By determining the parameter information of the first, second, and third images, the coordinate information of the first boundary point of the first image, the second boundary point of the second image, and the third boundary point of the third image are determined respectively. Based on the coordinate information of the first, second, and third boundary points, region A corresponding to the first boundary point coordinate information, region B corresponding to the second boundary point coordinate information, and region C corresponding to the third boundary point coordinate information are determined from the resulting image. The first image to be stitched is merged with region A, the second image to be stitched is merged with region B, and the third image to be stitched is merged with region C, finally obtaining a merged target image containing the first, second, and third images, thus completing the stitching of multiple images.
[0084] There are multiple ways to obtain the coordinates of each pixel in the resulting image. In one example, such as... Figure 3 As shown, the specific steps include:
[0085] Step 202-1: Determine the spatial operating range of the UAV and the UAV camera parameters.
[0086] Step 202-2: Acquire images captured by the drone.
[0087] Step 202-3: Determine the raster information of the captured image.
[0088] Step 202-4: Perform an affine transformation on the spatial operation range based on the grid information and UAV camera parameters to obtain the coordinate positions of each pixel in the resulting image.
[0089] Affine transformation refers to the process of performing various operations on an image, such as translation and rotation, through a series of geometric transformations. This transformation preserves the flatness and parallelism of the image. Flatness means that after an affine transformation, straight lines remain straight lines; parallelism means that after an affine transformation, parallel lines remain parallel lines.
[0090] It should be noted that raster information can include pixel information, ratio information, and other basic information that characterizes the captured image.
[0091] By performing an affine transformation on the spatial operation range based on grid information and UAV camera parameters, the UAV can be transformed from a three-dimensional spatial operation range to a two-dimensional planar range for UAV imaging, resulting in an image and the coordinate position of each pixel in the image.
[0092] It should be noted that the coordinates of each pixel in the resulting image include the coordinates of the four vertices of the resulting image. Based on the coordinates of the four vertices of the resulting image, the position of the resulting image in the actual geographical location can be fixed. In the actual geographical location, the position information of each pixel in the resulting image is the corresponding actual geographical location.
[0093] The coordinates of the four vertices of the resulting image include the coordinates of the top left vertex, the bottom left vertex, the top right vertex, and the bottom right vertex.
[0094] Determining the coordinates of boundary points in images to be stitched can be achieved in several ways, such as... Figure 4 As shown, the specific steps include the following: In one example, the specific steps include the following:
[0095] It should be noted that the parameter information may include the following: sensor information of the drone that took the image to be stitched, focal length, yaw angle, pitch angle, roll angle, shooting altitude, and the coordinates of the center point of the image to be stitched.
[0096] Step 204-1: Determine the width and height information of the sensor data.
[0097] Step 204-2: For each image to be stitched, determine the actual width and actual height information of the image to be stitched based on the width information, height information, and focal length of the sensor.
[0098] Step 204-3: Based on the actual width information, pitch angle, roll angle, shooting height and actual height information, determine the vertical distance from the center point of each image to be stitched to the image edge.
[0099] The image edges include the top edge, bottom edge, left edge, and right edge.
[0100] Step 2041-4: Based on the center point coordinates, the vertical distance from the center point to the image edge, and the yaw angle, determine the boundary point coordinate information of each image to be stitched.
[0101] For example, based on the sensor's width, height, and focal length, the actual width and height of the image to be stitched can be determined using the following formula:
[0102] Actual width information = 2.0 * tan(x / f / 2);
[0103] Actual height information = 2.0 * tan(y / f / 2);
[0104] Where x represents the width of the sensor, y represents the height of the sensor, and f represents the focal length.
[0105] Based on the actual width information, pitch angle, roll angle, shooting height, and actual height information, the vertical distance from the center point of each image to be stitched to the image edge is determined using the following formula:
[0106] bottom=H*tan(Pitch-0.5*yView);
[0107] top=H*tan(Pitch+0.5*yView);
[0108] left=H*tan(Roll-0.5*xView);
[0109] right=H*tan(Roll+0.5*xView);
[0110] Wherein, bottom is the vertical distance from the center point to the bottom edge, top is the vertical distance from the center point to the top edge, left is the vertical distance from the center point to the left edge, right is the vertical distance from the center point to the right edge, H is the shooting height, Pitch is the pitch angle, yView is the actual height information, xView is the actual width information, and Roll is the roll angle.
[0111] Based on the calculated vertical distance from the center point to the image edge, the coordinates of the center point of the images to be stitched, and the yaw angle of the drone that captured the images, the boundary point coordinates of each image to be stitched are determined, for example:
[0112] Assuming the center point coordinates of the images to be stitched are (PX, PY), then the coordinates of any vertex are (PX', PY'), which can be calculated through translation and rotation. Taking the top-left vertex as an example, the center point coordinates (PX, PY) are translated by top and left distances, as shown in the following formula:
[0113] PX`=cos(-Yaw)*left-sin(-Yaw)*tpo+PX;
[0114] PY`=sin(-Yaw)*left+cos(-Yaw)*top+PY.
[0115] Using a similar method, the coordinates of the lower left, upper right, and lower right vertices of each image to be stitched are calculated separately.
[0116] Finally, the top-left, top-right, bottom-left, and bottom-right vertices of the images to be stitched are used as boundary point coordinates. These boundary point coordinates are then converted to the same coordinate system as the pixels in the final image. In other words, when the coordinates of each pixel in the final image are latitude and longitude coordinates, the boundary point coordinates of the four vertices of each image to be stitched are converted to latitude and longitude coordinates. This facilitates the subsequent matching of the corresponding region range of each image to be stitched in the final image.
[0117] To make the edges of adjacent images blend more naturally, the edges of each image to be stitched are feathered, and each feathered image to be stitched is then merged with the corresponding region in the result image to obtain the stitched target image.
[0118] It should be noted that the edges of the 20 pixels to be stitched can be progressively blurred, which can avoid the problem of the edges being too obvious during stitching.
[0119] To further improve the efficiency of image stitching, such as Figure 5 As shown, in one example, the specific steps include:
[0120] Step 206-1: For each image to be stitched, divide the image to be stitched and its corresponding region according to a preset size to obtain multiple sub-images to be stitched and multiple sub-regions.
[0121] Step 206-2: Merge each sub-image to be stitched into the corresponding sub-region range to obtain the stitched target image.
[0122] For example, each image to be stitched is divided into multiple 512*512 sub-images. Images that do not meet the 515*512 size requirement are also included as sub-images to be stitched. The corresponding region of each sub-image to be stitched in the final image is also divided in the same way. Finally, the sub-images to be stitched in the same position are merged into the corresponding sub-region to complete the image stitching.
[0123] There are several ways to improve image stitching efficiency. In another example:
[0124] Based on the boundary point coordinates of each image to be stitched, multiple images are combined into a fused image.
[0125] For example: Figure 6 The image shown is the image to be merged. The image to be merged includes a first image to be stitched, a second image to be stitched, a third image to be stitched, and a fourth image to be stitched.
[0126] Directly fusing the image to be fused with the resulting image results in high CPU usage and low image processing efficiency. Therefore, the image to be fused and the resulting image are segmented into images of preset sizes, and the segmented image to be fused is then fused with the segmented resulting image at the same location, thereby improving image processing efficiency.
[0127] It should be noted that the segmented sub-images to be fused have the same latitude and longitude as the corresponding sub-regions.
[0128] Please refer to Figure 7 This application embodiment also provides an application for Figure 1 The image stitching device 110 of the electronic device 100 includes:
[0129] The first acquisition module 111 is used to acquire multiple images to be stitched together and a result image representing the shooting range of the drone, captured by the drone.
[0130] The second acquisition module 112 is used to determine the coordinate position of each pixel in the result image;
[0131] The first determining module 113 is used to determine the parameter information of each of the images to be stitched together;
[0132] The second determining module 114 is used to determine the boundary point coordinate information of each image to be stitched based on the parameter information of the image to be stitched, wherein the boundary point coordinate information represents the coordinate information of the upper left vertex, lower left vertex, upper right vertex and lower right vertex of the image to be stitched.
[0133] The matching module 115 is used to match the boundary point coordinate information of each of the images to be stitched with the coordinate positions of each pixel in the result image to obtain the region range corresponding to each image to be stitched in the result image.
[0134] The fusion module 116 is used to fuse each of the images to be stitched with the corresponding regions in the result image to obtain the stitched target image.
[0135] It should be noted that the image stitching device provided in this embodiment has the same basic principle and technical effect as the image stitching method embodiment described above. For the sake of brevity, any parts not mentioned in this embodiment can be referred to the corresponding content in the above method embodiment.
[0136] In summary, this application acquires multiple images to be stitched by a drone and a result image representing the drone's shooting range. It determines the coordinates of each pixel in the result image and the parameter information of each image to be stitched. For each image to be stitched, based on its parameter information, it determines the coordinates of its boundary points. Image stitching is then performed based on these boundary point coordinates. The boundary point coordinates of each image to be stitched are matched with the coordinates of each pixel in the result image to obtain the corresponding region of each image to be stitched in the result image. Each image to be stitched is then fused with its corresponding region in the result image to obtain the stitched target image. This eliminates the need for multiple specialized software programs to obtain the feature points of each image to be stitched. Images with the same feature points are stitched together; image stitching can be achieved simply by determining the boundary point coordinates of the images to be stitched, thereby reducing the cost and improving the efficiency of image stitching.
[0137] This application also provides an electronic device 100, which includes a processor 130 and a memory 120. The memory 120 stores computer-executable instructions, which, when executed by the processor 130, implement the image stitching method.
[0138] This application embodiment also provides a computer-readable storage medium storing a computer program, which, when executed by the processor 130, implements the image stitching method.
[0139] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.
[0140] Furthermore, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part. If the function is implemented as a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes: USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, optical disks, and other media capable of storing program code.
[0141] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0142] The above descriptions are merely various embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
Claims
1. An image stitching method, characterized in that, The method includes: Acquire multiple images to be stitched together, captured by a drone, as well as a result image representing the drone's shooting range; Obtain the coordinates of each pixel in the resulting image; Determine the parameter information of each of the images to be stitched together; For each image to be stitched, the boundary point coordinate information of the image to be stitched is determined based on the parameter information of the image to be stitched. The boundary point coordinate information represents the coordinate information of the top left vertex, bottom left vertex, top right vertex and bottom right vertex of the image to be stitched. The boundary point coordinates of each image to be stitched are matched with the coordinates of each pixel in the result image to obtain the region range of each image to be stitched in the result image. Each of the images to be stitched is fused with the corresponding region in the result image to obtain the stitched target image. The parameter information includes: sensor information of the UAV that captured the images to be stitched, focal length, yaw angle, pitch angle, roll angle, shooting altitude, and center point coordinates of the images to be stitched. The step of determining the boundary point coordinates of each image to be stitched, based on the parameter information of that image, includes: Determine the width and height information of the sensor data; For each image to be stitched, the actual width and actual height of the image are determined based on the width and height information of the sensor and the focal length. Based on the actual width information, pitch angle, roll angle, shooting height, and actual height information, the vertical distance from the center point of each image to be stitched to the image edge is determined, wherein the image edge includes the top edge, bottom edge, left edge, and right edge; Based on the center point coordinates, the vertical distance from the center point to the image edge, and the yaw angle, the boundary point coordinate information of each image to be stitched is determined.
2. The method according to claim 1, characterized in that, The step of obtaining the coordinate positions of each pixel in the resulting image includes: Determine the operational range of the drone and the drone camera parameters; Acquire images captured by the drone; Determine the raster information of the captured image; Based on the grid information and the UAV camera parameters, an affine transformation is performed on the spatial operation range to obtain the coordinate positions of each pixel in the resulting image.
3. The method according to claim 1, characterized in that, The actual width and actual height information of each of the images to be stitched together satisfy the following formula: Actual width information = ; Actual height information = ; Where x represents the width of the sensor, y represents the height of the sensor, and f represents the focal length.
4. The method according to claim 1, characterized in that, The vertical distance from the center point of each of the images to be stitched to the edge of the image satisfies the following formula: bottom = ; top = ; left = ; right = ; Wherein, bottom is the vertical distance from the center point to the bottom edge, top is the vertical distance from the center point to the top edge, left is the vertical distance from the center point to the left edge, right is the vertical distance from the center point to the right edge, H is the shooting height, Pitch is the pitch angle, yView is the actual height information, xView is the actual width information, and Roll is the roll angle.
5. The method according to claim 1, characterized in that, The method further includes: feathering the edges of each of the images to be stitched together; The step of fusing each of the images to be stitched with the corresponding regions in the resulting image to obtain the stitched target image includes: Each feathered image to be stitched is fused with the corresponding region in the resulting image to obtain the stitched target image.
6. The method according to claim 1, characterized in that, The step of fusing each of the images to be stitched with the corresponding regions in the resulting image to obtain the stitched target image includes: For each image to be stitched, the image to be stitched and its corresponding region are divided according to a preset size to obtain multiple sub-images to be stitched and multiple sub-regions. Each sub-image to be stitched is merged into its corresponding sub-region to obtain the stitched target image.
7. An image stitching device, characterized in that, The device includes: The first acquisition module is used to acquire multiple images to be stitched together and a result image representing the shooting range of the drone, captured by the drone. The second acquisition module is used to determine the coordinate position of each pixel in the resulting image; The first determining module is used to determine the parameter information of each of the images to be stitched together; The second determining module is used to determine the boundary point coordinate information of each image to be stitched based on the parameter information of the image to be stitched, wherein the boundary point coordinate information represents the coordinate information of the upper left vertex, lower left vertex, upper right vertex and lower right vertex of the image to be stitched. The matching module is used to match the boundary point coordinate information of each image to be stitched with the coordinate position of each pixel in the result image to obtain the region range corresponding to each image to be stitched in the result image. The fusion module is used to fuse each of the images to be stitched with the corresponding regions in the result image to obtain the stitched target image. The parameter information includes: sensor information of the drone that captured the image to be stitched, focal length, yaw angle, pitch angle, roll angle, shooting altitude, and the coordinates of the center point of the image to be stitched. The second determining module is specifically used for: Determine the width and height information of the sensor data; For each image to be stitched, the actual width and actual height of the image are determined based on the width and height information of the sensor and the focal length. Based on the actual width information, pitch angle, roll angle, shooting height, and actual height information, the vertical distance from the center point of each image to be stitched to the image edge is determined, wherein the image edge includes the top edge, bottom edge, left edge, and right edge; Based on the center point coordinates, the vertical distance from the center point to the image edge, and the yaw angle, the boundary point coordinate information of each image to be stitched is determined.
8. An electronic device, characterized in that, It includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the method according to any one of claims 1-6.
9. A storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the steps of the method described in any one of claims 1-6.