Irregular large area uniform block adjustment positioning method and device of unmanned aerial vehicle image

Abstract

Embodiments of the present disclosure provide an unmanned aerial vehicle image block adjustment positioning method and device for irregular large area uniform block. The method comprises: acquiring a set of unmanned aerial vehicle images and POS data; acquiring geographical range information of a shooting area of the set of unmanned aerial vehicle images according to the POS data; performing block on the set of unmanned aerial vehicle images based on the geographical range information of the shooting area to obtain a set of unmanned aerial vehicle image blocks; performing pose estimation on each unmanned aerial vehicle image block in the set of unmanned aerial vehicle image blocks according to a structure from motion (SfM) algorithm to obtain a set of unmanned aerial vehicle image block information; performing global coordinate conversion on the set of unmanned aerial vehicle image block information to obtain a set of rough unmanned aerial vehicle image block information in a global coordinate system; and performing global block adjustment calculation on the set of rough unmanned aerial vehicle image block information to obtain final global three-dimensional coordinates of a physical point and global unmanned aerial vehicle image pose parameter information. The present application reduces the problem of block adjustment failure in pose solution caused by error accumulation through an adaptive block method.

Description

Technical Field

[0001] This invention relates to the field of image processing, and more particularly to a method, server, medium, and related equipment for adjusting and locating UAV images in a uniformly divided, irregular large area. Background Technology

[0002] In the process of rapid urbanization in my country, refined 3D city models, as key foundational data for urban planning, construction, management, and informatization, are becoming increasingly important and have become a core component of the urban spatial data framework. This surge in demand has powerfully driven the vigorous development of oblique photogrammetry technology. Oblique photogrammetry, with its large-scale, high-precision, and high-resolution characteristics, can comprehensively perceive complex scenes. The data generated by efficient data acquisition equipment and professional processing workflows can intuitively reflect the appearance, location, height, and other attributes of ground features, providing a solid guarantee for presenting realistic effects and achieving surveying-grade accuracy. However, compared to traditional photogrammetry, oblique photogrammetry is characterized by multi-angle shooting, massive data volume, and complex data conditions, placing higher demands on data processing technology. In particular, problems such as camera pose recovery for difficult data such as irregular large areas, long strips, and weak textures urgently need to be effectively solved. Summary of the Invention

[0003] To address the aforementioned problems, embodiments of the present invention provide a method for adjusting and locating UAV images in irregular large areas using uniformly segmented blocks, along with a server, media, and related equipment. A method for uniformly segmenting and adjusting irregular large areas and long strip regions is proposed.

[0004] One aspect of the present invention provides a method for adjusting and locating UAV images in a uniformly segmented, irregular large region, comprising:

[0005] Step 101: Acquire drone imagery and POS data;

[0006] Step 102: Based on the POS data above, obtain the geographical range information of the shooting area of ​​the above UAV image set;

[0007] Step 103: Divide the above UAV image set into blocks based on the geographical range information of the above shooting area to obtain a UAV image block set;

[0008] Step 104: Perform pose estimation on each UAV image block in the above UAV image block set according to the Structure of Motion (SfM) algorithm to obtain the UAV image block information set, wherein the above UAV image block information set includes the UAV image pose information and object point three-dimensional coordinates of the current block of each UAV image block.

[0009] Step 105: Perform global coordinate transformation on the above UAV image block information set to obtain a rough UAV image block information set in the global coordinate system;

[0010] Step 106: Perform global adjustment calculation on the above rough UAV image block information set to obtain the final global object point 3D coordinates and global UAV image pose parameter information.

[0011] Optionally, obtaining the geographic range information of the shooting area of ​​the UAV image set based on the POS data includes: establishing a global index of the UAV image set and the POS data based on a preset order to obtain associated coordinate information; and calculating the average positional interval between the UAV image sets based on the associated coordinate information to obtain the geographic range information of the shooting area of ​​the UAV image set.

[0012] Optionally, the above-mentioned segmentation of the UAV imagery into a UAV imagery block set based on the geographical extent information of the shooting area includes: determining the long side of the geographical extent information of the shooting area in the longitude or latitude direction to obtain long side information; performing initial segmentation of the UAV imagery based on the long side information to obtain a long side segmentation block set; further segmenting each long side segment in the long side segmentation block set into short side segments to obtain a resegmentation block set; generating the average number of images in all resegments within the resegmentation block set; and merging resegments with a number of images less than a preset multiple of the average number of images into adjacent blocks to obtain the UAV imagery block set.

[0013] Optionally, the above-mentioned pose estimation of each UAV image block in the above-mentioned UAV image block set according to the Structure of Motion (SfM) algorithm to obtain the UAV image block information set includes: performing the following steps on each UAV image block in the above-mentioned UAV image block set: generating the initial rotation matrix of each image in the UAV image block through corresponding point matching and essential matrix decomposition, and obtaining the rotation matrix of each image in the block through matrix optimization within the block; generating the initial translation vector of each image based on the rotation matrix and corresponding point matching results within the block, and obtaining the translation vector of each image in the block and the filtered three-view matching point set after calculating the reprojection error and eliminating outliers; using the rotation matrix and translation vector within the block as initial values, and the filtered three-view matching point set as observation values, performing bundle adjustment optimization to obtain the final UAV image pose information of the current block and the final three-dimensional coordinates of the object point of the current block; in response to performing the above steps on all UAV image blocks, the above-mentioned UAV image block information set is obtained.

[0014] Optionally, the above-mentioned coordinate transformation of the above-mentioned UAV image block information set to obtain the transformed UAV image block information set includes: determining blocks with common UAV images based on the above-mentioned UAV image block information set, and obtaining information on the block with the most intersection; determining a reference block and a global reference coordinate system based on the information on the block with the most intersection; for each block to be transformed that has common UAV images with the above-mentioned reference block, performing the following steps: based on a unified feature point index, obtaining object points that are jointly observed in the above-mentioned reference block and the current block to be transformed, and obtaining a common three-dimensional point pair set; using the above-mentioned common three-dimensional point pair set as observation values, generating spatial similarity transformation parameters from the coordinate system of the current block to be transformed to the above-mentioned global reference coordinate system; based on the above-mentioned spatial similarity transformation parameters, transforming the three-dimensional coordinates of all object points and the pose of UAV images in the current block to be transformed to the above-mentioned global reference coordinate system; in response to the completion of processing all blocks to be transformed that have common UAV images with the above-mentioned reference block, obtaining the above-mentioned transformed UAV image block information set.

[0015] Optionally, the above-mentioned global adjustment calculation of the rough UAV image patch information set to obtain the final global object point 3D coordinates and global UAV image pose parameter information includes: generating a global reprojection error equation based on the poses of all UAV images and the 3D coordinates of all object points in the transformed UAV image patch information set as initial values; solving the global reprojection error equation using a nonlinear least squares algorithm to obtain the optimized UAV image pose parameters and optimized object point 3D coordinates; and generating the global object point 3D coordinates and global UAV image pose parameter information based on the optimized UAV image pose parameters and optimized object point 3D coordinates.

[0016] Optionally, the initial block division is set to 65% overlap, and the preset multiple is 0.4.

[0017] Another aspect of the present invention provides a server, characterized in that the server runs the above-mentioned UAV image adjustment and positioning method for uniformly dividing irregular large areas.

[0018] Another aspect of the present invention provides a computer-readable storage medium having stored thereon computer program instructions, wherein the computer program instructions, when executed by a processor, cause the processor to perform any of the methods described above.

[0019] A third aspect of the present invention provides an electronic device comprising: a processor; and a memory storing instructions executable by the processor; wherein the processor is configured to execute the instructions to implement any of the methods described above.

[0020] The embodiments of the present invention provide a method, server, medium, and related equipment for uniformly segmented UAV image adjustment and localization of irregular large areas. Based on the Structure for Motion Restoration (SfM) algorithm and block adjustment, the present invention designs a method for uniformly segmented adjustment of irregular large areas and long strip areas. This method can effectively solve the problem of adjustment failure in pose calculation due to error accumulation in irregular large areas and long strip areas.

[0021] Further aspects and scope of adaptation become apparent from the description provided herein. It should be understood that various aspects of this application may be implemented individually or in combination with one or more other aspects. It should also be understood that the description and specific embodiments herein are for illustrative purposes and are not intended to limit the scope of this application. Attached Figure Description

[0022] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0023] Figure 1 This is an overall flowchart of a UAV image adjustment and positioning method for uniformly dividing an irregular large area according to an embodiment of the present invention. Detailed Implementation

[0024] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0025] like Figure 1 As shown, the UAV image adjustment and positioning method for irregular large-area uniformly divided regions in this embodiment of the invention includes:

[0026] Step 101: Acquire drone imagery and POS data.

[0027] In some embodiments, the executing entity may acquire at least one drone image and corresponding POS data through a drone aerial photography system. The executing entity may be a smart device (such as a computer or mobile phone) connected to the weighing equipment via wired or wireless means. The wireless connection methods may include, but are not limited to, 3G / 4G connections, WiFi connections, Bluetooth connections, WiMAX connections, Zigbee connections, UWB (ultra-wideband) connections, and other currently known or future wireless connection methods.

[0028] Step 102: Based on the POS data above, obtain the geographical range information of the shooting area of ​​the above UAV image set.

[0029] In some embodiments, the executing entity can obtain the geographical extent information of the shooting area of ​​the drone image set based on the POS data. The geographical extent information of the shooting area of ​​the drone image set can be the geographical extent of the area to be processed, for example, a rectangular bounding box including latitude and longitude information. The drone image set can be at least one drone image distributed within the geographical extent of the shooting area.

[0030] In some optional implementations of certain embodiments, the aforementioned executing entity may obtain the geographical range information of the shooting area of ​​the aforementioned UAV image set based on the aforementioned POS data through the following steps:

[0031] The first step involves establishing a global index for the aforementioned UAV image set and POS data based on a preset order, thereby obtaining associated coordinate information. This associated coordinate information can be a set of relationships between various UAV images within the UAV image set. In practice, the executing entity can create a project using the input images and corresponding POS data, establishing a global index for the images according to the flight order (e.g., CF041712, CF041713, etc.), and associating the UAV images with POS coordinates to obtain the aforementioned associated coordinate information.

[0032] The second step involves calculating the average positional interval between the aforementioned drone image sets based on the associated coordinate information, thereby obtaining the geographical extent information of the shooting area of ​​the drone image sets. In practice, the executing entity can calculate the average positional interval between images based on the POS information corresponding to each drone image to obtain the geographical extent information of the shooting area of ​​the drone image sets. The calculation formula is as follows:

[0033]

[0034]

[0035] in, The maximum index number, Here are the longitude coordinates corresponding to the image with index number i. This represents the average interval along the longitude direction. The same formula can be used to calculate the average interval along the latitude direction. This step also calculates the extent of the region and its maximum longitude. minimum longitude Maximum latitude minimum latitude .

[0036] Step 103: Divide the UAV image set into blocks based on the geographical range information of the shooting area to obtain the UAV image block set.

[0037] In some embodiments, the aforementioned executing entity may divide the aforementioned drone image set into blocks based on the geographical range information of the aforementioned shooting area to obtain a drone image block set.

[0038] In some optional implementations of certain embodiments, the aforementioned executing entity may divide the aforementioned UAV image set into blocks based on the geographical extent information of the aforementioned shooting area through the following steps to obtain a UAV image block set:

[0039] The first step is to determine the longer side of the geographical area being photographed, along the longitude or latitude direction, thus obtaining the longer side information. This longer side information can be the relative longer side information of the geographical area being photographed. In practice, the executing entity can determine the relative longer side using the following methods:

[0040]

[0041] If it is Equation 1, then the latitude direction is the direction of the longer side; otherwise, the longitude direction is the direction of the longer side.

[0042] The second step involves initially segmenting the UAV imagery based on the aforementioned long-side information, resulting in a long-side segment set. This long-side segment set can be a collection of at least two segments cropped along the long-side direction. In practice, the executing entity can perform initial segmentation along the long-side direction. For example, if the longitude direction is the long-side direction, segments can be divided into blocks of 20 intervals, and the distance between blocks along the longitude direction can be represented by the following formula:

[0043]

[0044] The number of blocks that can be divided along the longitude direction is:

[0045]

[0046] To ensure that each block can be accurately converted to a unified benchmark and merged after adjustment, the aforementioned execution block can be divided into blocks with a 65% overlap. Maximum longitude and minimum longitude for:

[0047]

[0048]

[0049] The above long-side block set is finally obtained.

[0050] The third step involves further subdividing each long-side block in the aforementioned long-side block set into short-side blocks, resulting in a re-block set. This re-block set may include at least one block whose geographical extent is approximately square, representing the geographical area of ​​the re-blocked long-side block set. In practice, the executing entity may, for each long-side block obtained in the previous step, no longer use the fixed short-side range of the entire region, but instead, based on the spatial distribution of all UAV images actually falling within that long-side block, recalculate and statistically analyze the actual range of that long-side block in the short-side (e.g., latitude) direction, i.e., [ Then, within this local short side range [ Within this area, the same segmentation logic and calculation formula as for the long-side segmentation are used to divide the area along the short side. Similarly, to ensure continuity between blocks, the short-side segmentation can also inherit and set a 65% overlap rate. Finally, each initial long strip segment is further subdivided into at least two final segments that are geographically close to a square, i.e., the aforementioned subdivided set.

[0051] The fourth step is to generate the average number of images across all sub-blocks within the aforementioned sub-block set. In practice, the executing entity can count the number of UAV images within each sub-block and calculate the average to obtain the aforementioned average number of images.

[0052] The fifth step involves merging any subdivided blocks that are less than a preset multiple of the average number of images into adjacent blocks to obtain the aforementioned UAV image block set. In practice, when the number of UAV images in a subdivided block is less than 0.4 times the average number of images, the executing entity can merge that block into its adjacent image blocks to ultimately obtain the aforementioned UAV image block set.

[0053] In some alternative implementations of certain embodiments, the initial blocks can be set to 65% overlap, and the preset multiple can be 0.4.

[0054] Step 104: Perform pose estimation on each UAV image block in the UAV image block set using the Structure of Motion (SfM) algorithm to obtain the UAV image block information set.

[0055] In some embodiments, the execution entity may perform pose estimation on each UAV image block in the above UAV image block set according to the Structure of Motion (SfM) algorithm to obtain a UAV image block information set, wherein the UAV image block information set includes the UAV image pose information and object point three-dimensional coordinates of the current block of each UAV image block.

[0056] In some optional implementations of certain embodiments, the aforementioned execution entity may perform pose estimation on each UAV image block in the aforementioned UAV image block set according to the Structure of Motion (SfM) algorithm through the following steps to obtain a UAV image block information set, including:

[0057] The first step is to perform the following steps on each drone image block in the above drone image block set:

[0058] The first sub-step involves generating the initial rotation matrix for each image within a UAV image block through corresponding point matching and essential matrix decomposition. Then, the rotation matrix of each image within the block is obtained through matrix optimization within the block. In practice, the aforementioned execution entity can first utilize the camera focal length and CCD size information to perform fast corresponding point matching in a normalized coordinate system using the SurfGPU algorithm, obtaining the corresponding point matching results. Here, the camera can be a model concept, referring to the set of imaging centers, poses, and internal parameters corresponding to each shooting action. Then, based on the matched point pairs, the initial rotation matrix between the image pairs is calculated. The formula is as follows:

[0059]

[0060] in, Let i be the relative rotation matrix between cameras i and j. Let i be the relative translation matrix between cameras i and j, and then, the multiple sets of relative rotation matrices obtained in the previous step. As observations, an error equation for global rotation is constructed. Finally, through least-squares optimization, the global rotation matrix of each UAV image in this block is obtained. .

[0061] The second sub-step, based on the rotation matrix and corresponding point matching results within the aforementioned block, generates the initial translation vector for each image. After calculating the reprojection error and eliminating gross errors, it obtains the translation vector of each image within the block and the filtered three-view matching point set. In practice, the aforementioned execution entity can use the aforementioned global rotation matrix... By matching the above corresponding points, we can obtain any UAV image (view) in any three-view group (set T), along with its own rotation matrix. Then, the aforementioned execution entity can solve for the initial translation vector of the camera center in the local coordinate system of the three views. Then, the reprojection error of the three-view matching points is calculated, and three-view matching points with errors exceeding a predetermined value are removed according to a corresponding threshold. This predetermined value can be an integer. Let any point in the three views... The projection error is , The error calculation formula is as follows:

[0062]

[0063] Finally, the aforementioned executing entity can use the translation average method to adjust the initial translation vector. Global optimization is performed to obtain the global translation vector of each UAV image within the current block. .

[0064] The third sub-step uses the rotation matrix and translation vector within the aforementioned block as initial values, and the selected three-view matching point set as observation values, to perform bundle adjustment optimization, obtaining the final UAV image pose information and the final 3D coordinates of the object points in the current block. In practice, the aforementioned execution entity can use the aforementioned translation vector... And the global rotation matrix mentioned above Based on the above camera projection error equation, the global projection error is minimized to obtain the three-dimensional coordinates of the object point corresponding to the same point and the image pose parameters, i.e., bundle adjustment, to obtain the final UAV image pose information of the current block and the final three-dimensional coordinates of the object point of the current block.

[0065] The second step involves performing the above steps on all drone image blocks to obtain the aforementioned drone image block information set.

[0066] Step 105: Perform a global coordinate transformation on the UAV image block information set to obtain a rough UAV image block information set in the global coordinate system.

[0067] In some embodiments, the aforementioned execution entity may perform global coordinate transformation on the aforementioned UAV image block information set to obtain a rough UAV image block information set in a global coordinate system.

[0068] In some optional implementations of certain embodiments, the aforementioned execution entity may perform a global coordinate transformation on the aforementioned UAV image block information set through the following steps to obtain the transformed UAV image block information set:

[0069] The first step is to identify blocks with shared UAV images based on the aforementioned UAV image block information set, thus obtaining information on blocks with the most intersections. This information can include the number of intersections between each UAV image block and other UAV image blocks. In practice, the executing entity can count the blocks with shared UAV images and record the block index and the index of the shared image.

[0070] The second step is to determine the reference block and the global reference coordinate system based on the information of the most intersecting blocks mentioned above. In practice, the executing entity can determine the UAV image block with the most intersecting blocks as the reference block and determine its coordinate system as the global reference coordinate system mentioned above.

[0071] Third, for each block to be converted that shares UAV imagery with the aforementioned baseline block, perform the following steps:

[0072] The first sub-step involves obtaining object-space points that are commonly observed in both the reference block and the current block to be converted, based on a unified feature point index, thus obtaining a common 3D point pair set. Here, the above can be a ( , A set of point pairs. Where... These are the coordinates of the point within the block to be converted. These are the coordinates of the same point in the reference block (global) coordinate system. In practice, the aforementioned execution entity can identify the commonly observed object points in the local coordinate system of the block to be transformed and the global coordinate system of the reference block through a unified feature point index, thus obtaining the aforementioned common three-dimensional point pair set.

[0073] The second sub-step uses the aforementioned set of common 3D point pairs as observations to generate spatial similarity transformation parameters from the current coordinate system to be transformed block to the aforementioned global reference coordinate system. In practice, the executing entity can use the common point pairs from the first step as observations and perform optimization calculations using the least squares principle to solve for a set of optimal scaling coefficients. 3x3 rotation matrix The sum is a 3*1 translation matrix That is, the spatial similarity transformation parameters mentioned above ( , , The conversion formula is as follows:

[0074]

[0075] The third sub-step involves transforming the 3D coordinates of all object points and the UAV image pose within the current block to be transformed to the aforementioned global reference coordinate system, based on the aforementioned spatial similarity transformation parameters. In practice, the aforementioned execution entity can use the aforementioned spatial similarity transformation parameters ( , , The 3D coordinates of all object points and the poses (positions and attitudes) of all UAV images within the conversion block are converted.

[0076] The fourth step involves processing all the unmanned aerial vehicle (UAV) image blocks that share the same UAV imagery as the aforementioned baseline blocks, to obtain the converted UAV image block information set.

[0077] Step 106: Perform global adjustment calculations on the rough UAV image block information set to obtain the final global object point 3D coordinates and global UAV image pose parameter information.

[0078] In some embodiments, the aforementioned execution entity may perform global adjustment calculations on the aforementioned coarse UAV image block information set to obtain the final global object point three-dimensional coordinates and global UAV image pose parameter information.

[0079] In some optional implementations of certain embodiments, the aforementioned execution entity can perform global adjustment calculations on the aforementioned coarse UAV image block information set through the following steps to obtain the final global object point 3D coordinates and global UAV image pose parameter information:

[0080] The first step is to generate a global reprojection error equation based on the poses of all UAV images and the 3D coordinates of all object points in the converted UAV image block information set.

[0081] The second step involves solving the global reprojection error equation using a nonlinear least squares algorithm to obtain the pose parameters of the optimized UAV image and the optimized 3D coordinates of the object points.

[0082] The third step involves generating the global object point 3D coordinates and global UAV image pose parameter information based on the optimized UAV image pose parameters and the optimized object point 3D coordinates.

[0083] It is understood that, in addition to the above, this invention also includes some conventional structures and methods, which are well known and will not be elaborated upon further. However, this does not mean that these structures and methods are absent from this invention.

[0084] Those skilled in the art will recognize that, although numerous exemplary embodiments of the invention have been shown and described in detail herein, many other variations or modifications conforming to the principles of the invention can be directly determined or derived from the disclosure of the invention without departing from its spirit and scope. Therefore, the scope of the invention should be understood and construed as covering all such other variations or modifications.

[0085] In this invention, a method, server, medium, and related equipment for UAV image adjustment and localization based on uniformly divided irregular large areas are designed. Based on the Structure for Motion Restoration (SfM) algorithm and block adjustment, this invention designs a method for uniformly divided block adjustment of irregular large areas and long strip areas. This method can effectively solve the problem of adjustment failure in pose calculation due to error accumulation in irregular large areas and long strip areas.

[0086] Those skilled in the art will understand that various aspects of this application can be implemented as a system, method, or program product. Therefore, various aspects of this application can be specifically implemented in the following forms: a completely hardware implementation, a completely software implementation (including firmware, microcode, etc.), or a combination of hardware and software implementations, collectively referred to herein as a "circuit," "module," or "system."

[0087] In embodiments of this disclosure, a computer-readable storage medium is also provided, on which computer program instructions are stored. When executed, for example, by a processor, these computer program instructions can implement the steps of the UAV image adjustment and localization method for uniformly segmented irregular large areas described in any of the above embodiments. In some possible implementations, various aspects of this application can also be implemented as a program product comprising program code. When the program product is run on a terminal device, the program code causes the terminal device to perform the steps described in the various exemplary embodiments of this application in the UAV image adjustment and localization method for uniformly segmented irregular large areas described in this specification.

[0088] The program product for implementing the above-described method according to embodiments of this application may employ a portable compact disc read-only memory (CD-ROM) and include program code, and may run on a terminal device, such as a personal computer. However, the program product of this application is not limited thereto. In this document, a readable storage medium may be any tangible medium that contains or stores a program that may be used by or in conjunction with an instruction execution system, apparatus, or device.

[0089] The above-described program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0090] The aforementioned computer-readable storage medium may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The readable storage medium may also be any readable medium other than a readable storage medium, capable of transmitting, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0091] Program code for performing the operations of this application can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0092] Those skilled in the art will understand that various aspects of this application can be implemented as a system, method, or program product. Therefore, various aspects of this application can be specifically implemented in the following forms: a completely hardware implementation, a completely software implementation (including firmware, microcode, etc.), or a combination of hardware and software implementations, collectively referred to herein as a "circuit," "module," or "system."

[0093] The above description is merely a preferred embodiment of this disclosure and is not intended to limit the scope of this disclosure. Various modifications and variations can be made to this disclosure by those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for adjusting and locating UAV images in a uniformly divided, irregular large area, characterized in that: The method includes: Acquire drone image sets and POS data; Based on the POS data, obtain the geographical range information of the shooting area of ​​the UAV image set; Based on the geographical range information of the shooting area, the UAV image set is divided into blocks to obtain a UAV image block set. The pose estimation of each UAV image block in the UAV image block set is performed according to the Structure of Motion (SfM) algorithm to obtain a UAV image block information set, wherein the UAV image block information set includes the UAV image pose information and object point three-dimensional coordinates of the current block of each UAV image block. A global coordinate transformation is performed on the UAV image block information set to obtain a coarse UAV image block information set in the global coordinate system; Global adjustment calculations are performed on the rough UAV image block information set to obtain the final global object point 3D coordinates and global UAV image pose parameters.

2. The method according to claim 1, wherein, The step of obtaining the geographical range information of the shooting area of ​​the UAV image set based on the POS data includes: Based on a preset order, a global index is established for the UAV image set and the POS data to obtain associated coordinate information; Based on the associated coordinate information, the average positional interval between the UAV image sets is calculated to obtain the geographical range information of the shooting area of ​​the UAV image sets.

3. The method according to claim 2, wherein, The drone image set is divided into blocks based on the geographical range information of the shooting area to obtain a drone image block set, including: Determine the longer side of the geographical range information of the shooting area in the longitude or latitude direction to obtain the longer side information; Based on the long side information, the UAV image is initially divided into blocks to obtain a long side block set; Each long-side block in the long-side block set is further divided into short-side blocks to obtain a re-block set; Generate the average number of images across all sub-blocks within the sub-block set; The number of subdivided blocks that are less than a preset multiple of the average number of images are merged into adjacent blocks to obtain the UAV image block set.

4. The method according to claim 1, wherein, The pose estimation of each UAV image block in the UAV image block set is performed using the Structure of Motion (SfM) algorithm to obtain a UAV image block information set, including: The following steps are performed on each drone image block in the drone image block set: By matching corresponding points and decomposing the essential matrix, the initial rotation matrix of each image within the UAV image block is generated, and the rotation matrix of each image within the block is obtained by matrix optimization within the block. Based on the rotation matrix and corresponding point matching results within the block, the initial translation vector of each image is generated. After calculating the reprojection error and eliminating gross errors, the translation vector of each image within the block and the filtered three-view matching point set are obtained. Using the rotation matrix and translation vector within the block as initial values, and the filtered three-view matching point set as observation values, bundle adjustment optimization is performed to obtain the final UAV image pose information and the final three-dimensional coordinates of the object points in the current block. In response to performing the steps described above on all UAV image blocks, the UAV image block information set is obtained.

5. The method according to claim 3, wherein, The step of performing a global coordinate transformation on the UAV image block information set to obtain the transformed UAV image block information set includes: Based on the UAV image block information set, determine the blocks that share common UAV images and obtain the information of the blocks with the most intersection; Based on the information of the block with the most intersection, determine the reference block and the global reference coordinate system; For each block to be converted that shares UAV imagery with the reference block, perform the following steps: Based on a unified feature point index, the object points that are commonly observed in the reference block and the current block to be converted are obtained, resulting in a common set of three-dimensional point pairs. Using the set of common 3D point pairs as observations, spatial similarity transformation parameters are generated from the current block coordinate system to be transformed to the global reference coordinate system; Based on the spatial similarity transformation parameters, the three-dimensional coordinates of all object points and the pose of UAV images within the current block to be transformed are transformed to the global reference coordinate system; In response to the completion of processing all the unmanned aerial vehicle (UAV) image blocks that share the same UAV image with the reference block, the converted UAV image block information set is obtained.

6. The method according to claim 3, wherein, The global adjustment calculation of the rough UAV image patch information set to obtain the final global object point 3D coordinates and global UAV image pose parameters includes: The global reprojection error equation is generated based on the pose of all UAV images and the three-dimensional coordinates of all object points in the converted UAV image block information set as initial values. The global reprojection error equation is solved using a nonlinear least squares algorithm to obtain the pose parameters of the optimized UAV image and the optimized three-dimensional coordinates of the object points. Based on the pose parameters of the optimized UAV imagery and the optimized 3D coordinates of the object points, the global object point 3D coordinates and the global UAV imagery pose parameter information are generated.

7. The method according to claim 3, wherein, The initial block is set with an overlap of 65%, and the preset multiple is 0.

4.

8. A server, characterized in that, The server runs the UAV image adjustment and positioning method for uniformly dividing irregular large areas as described in claims 1-7.

9. A computer-readable storage medium having computer program instructions stored thereon, wherein, When executed by a processor, the computer program instructions cause the processor to perform the method according to any one of claims 1 to 7.

10. An electronic device, comprising: processor; as well as A memory that stores instructions executable by the processor; The processor is configured to execute the instructions to implement the method according to any one of claims 1 to 7.

Images