A method and device for processing unmanned aerial vehicle photogrammetry data

Abstract

The present application discloses a method and device for processing data of drone photogrammetry, the method comprising: obtaining drone collected data, the collected data comprising a POS data set and a photo data set, the POS data set comprising the position information of each POS point, the photo data set comprising the photos collected by the aerial camera at each aerial point; based on the SFM algorithm, according to the photos in the photo data set, calculating the posture information of the aerial camera at each aerial point; according to the posture information, matching the photos in the photo data set with the position information of the POS point to obtain a matching relationship, the matching relationship is used to indicate the loss of the position information of the photos and POS points in the photo data set. Based on the photo texture information, the present application autonomously pairs the photo data and the POS data with the help of the SFM algorithm, avoiding the error caused by data pairing through time, and improving the data processing accuracy.

Description

Technical Field

[0001] The present application relates to the technical field of drone data collection processing, and in particular to a drone photogrammetry data processing method and device. Background Art

[0002] UAV photogrammetry has outstanding advantages such as flexibility, high efficiency, speed and low operating cost, and is increasingly favored by the market. UAVs are usually equipped with a global navigation satellite system (GNSS) and an inertial measurement unit (IMU) to provide the drone with position information and attitude information. That is, during the drone aerial photography process, the GNSS module and the IMU module store the recorded POS data and photos, and the POS data and photo data will be recorded sequentially at fixed time intervals according to the flight control software. In practice, during the field aerial survey process, due to occasional uncertain reasons such as external electromagnetic signal interference and internal access of the instrument, individual POS points or photo data records may be lost, resulting in the recorded POS data and photo data not corresponding to each other, and the amount of data recorded by the two is inconsistent or even the amount of data is consistent, but the content of the records is inconsistent, resulting in the failure of subsequent data processing.

[0003] At present, technicians need to check the POS file and the photo file at the same time to ensure that the quantitative relationship between the two is consistent. When inconsistency occurs, it is necessary to check the location information of the POS data and the photo data one by one, check the lost POS or photo data and delete the corresponding data to ensure that the two correspond one to one. Alternatively, in some related technologies, data matching is performed based on the time series of the collected data.

[0004] The above-mentioned manual processing method is mainly based on the shooting distance and shooting direction of the film, which requires a lot of time to visually detect the film data one by one, and relies on rich experience and fast logical reasoning ability, which makes the detection cost high, and the reliability and efficiency are low, and it cannot fundamentally solve the problem. When matching data based on time information, due to the large storage space of film data, the film data will be copied, rendered and cut many times. In this process, the data acquisition time of the film storage is likely to be lost, resulting in the inability to read the time series; in addition, the film acquisition time can only be accurate to integer seconds, and due to the existence of errors, the time interval of the aerial photography points must be greater than 2 seconds, so that when the time interval is less than 2 seconds, this method has errors. Summary of the invention

[0005] The purpose of this application is to provide a method and device for processing data collected by a drone, so as to achieve efficient and accurate processing of data collected by a drone.

[0006] To achieve the above objectives, this application provides the following solutions:

[0007] In a first aspect, the present application provides a method for processing drone photogrammetry data, the processing method comprising:

[0008] Acquire data collected by the drone, wherein the collected data includes a POS data set and a photo data set, wherein the POS data set includes location information of each POS point, and the photo data set includes photos collected by the aerial camera at each aerial point;

[0009] Based on the SFM algorithm, the position and posture information of the aerial camera at each aerial point is calculated according to the images in the image data set;

[0010] According to the posture information, the photos in the photo data set are matched with the position information of the POS point to obtain a matching relationship, and the matching relationship is used to indicate the loss of the photos in the photo data set and the position information of the POS point.

[0011] Optionally, in some embodiments of the present application, the method for processing drone photogrammetry data, the step of calculating the position and posture information of the aerial camera at each aerial point based on the photos in the photo data set based on the SFM algorithm includes:

[0012] Extracting characteristic data of each image;

[0013] According to the characteristic data, determine the pixels with the same name between the images;

[0014] Calculate the basic matrix and the essential matrix of each image feature point according to the pixels with the same name;

[0015] The position and posture information of the aerial camera is determined according to the basic matrix and the essential matrix, and the position and posture information represents the position information of the aerial camera when collecting images at each aerial point.

[0016] Optionally, in the drone photogrammetry data processing method in some embodiments of the present application, determining the pixels with the same name between the images according to the feature data comprises:

[0017] Determining the association relationship between the images based on the extracted feature data, wherein the association relationship represents the degree of overlap between the images;

[0018] Determine an image pair in the image data set according to the association relationship, wherein the image pair represents two images with the highest degree of overlap in the image data set;

[0019] Extract the feature points of two images in the image pair;

[0020] Based on the SIFT algorithm, the feature points of the two images in the image pair are matched to obtain the pixels with the same name between the images.

[0021] Optionally, in some embodiments of the present application, the method for processing drone photogrammetry data, matching the photos in the photo data set with the position information of the POS point according to the pose information to obtain a matching relationship includes:

[0022] Determine a target transformation matrix from the plane coordinate system of the aerial camera to the POS plane coordinate system according to the posture information and the POS data set;

[0023] Based on the target transformation matrix, the position information of the aerial camera at each aerial point is transformed into the POS plane coordinate system, and the position information of the position information of each aerial point in the POS plane coordinate system is obtained as the aerial point transformation set;

[0024] The position information of each aerial point in the aerial point conversion set is distance matched with each POS point in the POS data set, and the matching relationship between each photo in the photo data set and the POS point in the POS data set is determined.

[0025] Optionally, in the drone photogrammetry data processing method in some embodiments of the present application, determining the target transformation matrix from the plane coordinate system of the aerial camera to the POS plane coordinate system according to the pose information and the POS data set includes:

[0026] Determine a camera point vector set of each aerial camera point according to the position and posture information;

[0027] Determine a POS point vector set of each POS point according to the POS data;

[0028] Based on the Euclidean distance, the vectors in the camera point vector set and the POS point vector set are matched to obtain a target matching vector set;

[0029] The target transformation matrix is ​​determined according to the target matching vector set.

[0030] Optionally, in the drone photogrammetry data processing method in some embodiments of the present application, the matching of the camera point vector set and the vectors in the POS point vector set based on the Euclidean distance to obtain the target matching vector set includes:

[0031] Traversing the camera point vector set, determining the Euclidean distance between each camera point vector and each vector in the POS point vector set;

[0032] Based on the Euclidean minimum distance, matching the vectors in the camera point vector set and the POS point vector set to form an initial matching vector pair set;

[0033] Based on the principle of random sampling consistency, error screening is performed on the initial matching vector pair set to obtain a candidate matching vector pair set;

[0034] Determine an initial transformation matrix from the plane coordinate system of the aerial camera to the POS plane coordinate system according to the candidate matching vector pair set;

[0035] According to the initial transformation matrix, the position information of the aerial camera at each aerial point is transformed into the POS plane coordinate system, and the position information of the position information of each aerial point in the POS plane coordinate system is obtained as the initial aerial point transformation set;

[0036] Matching the position information in the initial aerial point conversion set with the POS point to obtain an initial matching relationship in the POS plane coordinate system;

[0037] Based on the consistency principle of random sampling, the initial matching relationship is screened for errors to obtain the target matching vector set.

[0038] Optionally, the drone photogrammetry data processing method in some embodiments of the present application, after acquiring the drone acquisition data, and before calculating the position and pose information of the aerial camera at each aerial point according to the photos in the photo data set based on the SFM algorithm, the method further includes:

[0039] Performing pixel value correction on each image in the image data set;

[0040] The corrected image is resampled to obtain a preprocessed image.

[0041] Optionally, in the drone photogrammetry data processing method in some embodiments of the present application, the pixel value correction of each image in the image data set includes:

[0042] Traverse the pixel points of the image to be corrected, and calculate the distortion amount of each pixel point based on the configured distortion model;

[0043] Determine each pixel point whose distortion amount is greater than a set distortion threshold as a pixel point to be corrected;

[0044] The sum of the pixel value of each pixel point to be corrected and the corresponding distortion amount is determined, and the sum is used as the corrected pixel value of each pixel point to be corrected.

[0045] Optionally, in the method for processing drone photogrammetry data in some embodiments of the present application, the resampling of the corrected image to obtain the preprocessed image comprises:

[0046] According to the size parameters of the pictures, each original picture is scaled according to the resampling ratio coefficient to obtain a reduced picture;

[0047] Traverse each pixel point in the reduced image to determine the original pixel value corresponding to each pixel point;

[0048] The original pixel value is assigned to the pixel value of the corresponding pixel point in the reduced image to obtain a resampled image.

[0049] In a second aspect, the present application provides a computer device, the computer device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the processor executing the computer program to implement the method for processing drone photogrammetry data as described in the first aspect.

[0050] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0051] The UAV photogrammetry data processing method and device provided in the present application first perform feature processing on each image data in the image data set by using the motion recovery structure technology (SFM) for the collected unmanned data including the POS data set and the picture data set, calculate the posture information at each aerial point when the UAV takes the picture, and then calculate the posture information, match the posture information with the POS data, retrieve the matching relationship between the aerial camera position point and the POS point, and finally realize the determination of the details of the loss of the picture and POS data. That is, the present application autonomously pairs the picture data and the POS data based on the picture texture information with the help of the SFM algorithm, avoids the errors and inefficiencies caused by data pairing through time information or manual work, and improves the data processing accuracy and processing efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0053] Figure 1 A schematic diagram of a sequence of POS data and photo data in some embodiments of the present application;

[0054] Figure 2 A schematic diagram of a process flow of a method for processing drone photogrammetry data according to some embodiments of the present application;

[0055] Figure 3A schematic diagram of a flow chart of a method for processing drone photogrammetry data according to some other embodiments of the present application;

[0056] Figure 4 A schematic diagram of a flow chart of a method for processing drone photogrammetry data according to some further embodiments of the present application;

[0057] Figure 5 A schematic diagram of the position information of the aerial camera at each aerial point in some embodiments of the present application;

[0058] Figure 6 A flowchart of a method for processing drone photogrammetry data according to some embodiments of the present application is provided;

[0059] Figure 7 A schematic diagram of a flow chart of a method for processing drone photogrammetry data according to other embodiments of the present application;

[0060] Figure 8 A schematic diagram of the matching results of POS data and photo data in some embodiments of the present application;

[0061] Fig. 9 A schematic diagram of a list of matching results of POS data and photo data in some embodiments of the present application;

[0062] Fig.10 A schematic diagram of a correction process of image data in some embodiments of the present application;

[0063] Fig.11 A schematic diagram of a resampling process of image data in some embodiments of the present application;

[0064] Fig.12 A schematic diagram of the structure of a drone photogrammetry data processing device according to some embodiments of the present application;

[0065] Fig.13 A schematic diagram of the structure of a computer device provided for some embodiments of the present application. DETAILED DESCRIPTION

[0066] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0067] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0068] It can be understood that drones have built-in GPS / POS systems. During the data collection and processing process of the drone, GPS / POS can be used as auxiliary positioning to reduce the number of ground control points, provide necessary topological information during feature point matching, and improve the speed of connection point extraction.

[0069] GPS / POS records the position and attitude information of the drone at the time of aerial photography and saves it as an independent file for later data processing. During the GPS / POS-assisted aerial triangulation data processing, the aerial point information recorded by GPS / POS and the photo information need to be matched one by one in sequence, so that the position and attitude information recorded by GPS / POS of a certain photo can be retrieved in sequence.

[0070] Usually, the aerial camera is a detachable device, independent of the positioning module and the inertial navigation module. At the aerial photography point, the camera receives the photo signal and takes pictures. After completion, the pictures are stored in the aerial photography folder. The name mainly includes a fixed letter part and a digital number that is automatically accumulated according to the order of shooting. The format is the universal JPG format.

[0071] In practice, under normal circumstances, the POS point data is consistent with the image data and corresponds one to one. When in use, the two are matched and used through sequence numbers.

[0072] For example, Figure 1 The basic logic of the correspondence between POS data and photo data shown in the figure is that if the POS data sequence is POS001, POS002, POS003, POS004, ..., and the photo data sequence is photo 001, photo 002, photo 003, photo 004, ..., the two are matched and used in sequence according to the numbering sequence. However, during the field aerial survey, due to occasional uncertain reasons such as external electromagnetic signal interference and internal access of the instrument, individual POS points or photo data records may be lost, resulting in the recorded POS data and photo data not corresponding, and the data volume recorded by the two is inconsistent or even the data volume is consistent, but the recorded content is inconsistent, resulting in the failure of subsequent data processing.

[0073] That is, the POS data and the photo data are generated in sequence respectively. When the photo data is processed subsequently, the POS data file and the photo data file are matched only by the sequence number. There is no other way to proofread and verify the two. If the POS data or the photo data is lost, the two cannot correspond to each other and the photo processing cannot be performed.

[0074] In this application, in order to improve the efficiency and accuracy of data processing, an aerial photography data matching method based on image texture information is proposed, and displayed through a visual interface, which can perform targeted matching processing and error elimination, ultimately improving matching accuracy and reducing errors.

[0075] In order to better understand the UAV photogrammetry data processing method provided in this application, it is described in detail below with reference to the accompanying drawings.

[0076] Figure 2 FIG. 1 is a flow chart of a method for processing drone photogrammetry data provided by some embodiments of the present application. Figure 2 As shown, the method specifically comprises the following steps:

[0077] S110, acquiring data collected by the drone, the collected data including a POS data set and a photo data set, the POS data set including the position information of each POS point, and the photo data set including photos collected by the aerial camera at each aerial point.

[0078] S120, based on the SFM algorithm, calculate the position and posture information of the aerial camera at each aerial point according to the images in the image data set.

[0079] S130, matching the image in the image data set with the position information of the POS point according to the posture information to obtain a matching relationship, wherein the matching relationship is used to indicate the loss of the image in the image data set and the position information of the POS point.

[0080] Specifically, combined Figure 2 and Figure 3 In the embodiment of the present application, the collected data of the drone can be obtained first, that is, the POS data set and the photo data set can be obtained.

[0081] The POS data set may include the position information at each POS point, and the photo data set may include photos captured by the aerial camera at each aerial point.

[0082] Furthermore, in an embodiment of the present application, after obtaining the above data, a series of operations such as feature extraction processing can be performed on each image in the image data set based on the SFM algorithm to calculate the position and posture information of the drone's aerial camera at each aerial point.

[0083] The position information may include position information and posture information.

[0084] That is, after processing the image data through the SFM algorithm, the position information of the aerial camera at each aerial photography point, such as coordinate information, can be calculated.

[0085] Finally, according to the posture information of the aerial camera, the images in the image data set and the POS data in the POS data set can be matched to obtain a matching relationship between the two, and finally the loss of image data and POS data can be reflected through the matching relationship.

[0086] It can be understood that in the embodiments of the present application, for the unmanned data collected including the POS data set and the photo data set, the feature processing is first performed on each photo data in the photo data set by adopting the motion recovery structure technology (SFM), and the posture information at each aerial point when the drone takes the photo is calculated, and then the posture information is calculated, and the posture information and the POS data are matched and processed, and the matching relationship between the aerial camera position point and the POS point is retrieved, and finally the detailed situation of the loss of the photo and POS data is determined. That is, the present application is based on the photo texture information and uses the SFM algorithm to autonomously pair the photo data and POS data, thereby avoiding the errors and inefficiencies caused by data matching through time information or manual work, and improving the data processing accuracy and efficiency.

[0087] Optionally, in some embodiments of the present application, in S120, when calculating the posture information of the aerial camera based on the SFM algorithm using the photographs in the acquired photograph data set, the steps of feature point extraction and matching, structure recovery, and camera posture parameter estimation may be included.

[0088] like Figure 4 As shown, the following steps may be specifically included:

[0089] S121, extracting feature data of each image.

[0090] S122, determining the pixels with the same name between the images according to the feature data.

[0091] S123, calculating the basic matrix and the essential matrix of each image feature point according to the pixels with the same name.

[0092] S124, determining the position and posture information of the aerial camera according to the basic matrix and the essential matrix, wherein the position and posture information represents the position information of the aerial camera when collecting images at each aerial point.

[0093] Specifically, in some embodiments of the present application, feature data of each acquired image data may be extracted first, and then pixels of the same name of each image may be determined based on the extracted feature data.

[0094] In practice, in S122, the following steps may be specifically included:

[0095] S01, determining the association relationship between the images based on the extracted feature data, where the association relationship represents the degree of overlap between the images.

[0096] S02: determining an image pair in the image data set according to the association relationship, where the image pair represents two images with the highest degree of overlap in the image data set.

[0097] S03, extracting feature points of two images in the image pair.

[0098] S04, matching the feature points of the two images in the image pair based on the SIFT algorithm to obtain the pixel points with the same name between the images.

[0099] Specifically, after extracting the feature data of each image, the overlapping area between any two images in the acquired image data set is determined according to the feature data of the image, so as to determine the association relationship between each image and all other images. That is, by extracting the feature points of each image, the overlapping area of ​​the two images is drawn, and the area of ​​the overlapping area is calculated, and finally, an image undirected graph is constructed according to the size of the overlapping area and the corresponding relationship of the overlapping images to describe the association relationship between the images.

[0100] That is, the association relationship may include the corresponding relationship between the two images, and the area of ​​the overlapping region between the two images.

[0101] Furthermore, after calculating the association relationship between each image and any other image, the image pairs in the image data set can be determined according to the association relationship. That is, the most related photos can be found in the image undirected graph, and the images with the largest overlap with the image can be found according to the image association relationship to form an image pair. That is, according to the size of the area of ​​the overlapping area in the association relationship, the two images with the largest area can be regarded as a photo pair.

[0102] Finally, after determining the image pairs in the image data set, the feature points of the images in each image pair can be extracted for each image pair, and then the feature points of the image pairs are matched based on the SIFT algorithm to obtain the pixels with the same name between the images.

[0103] That is, the SIFT method can be used to extract key points from each image, and the brute force matching method can be used to complete the initial matching processing of the key points. Then, the random sampling consistency method can be used to eliminate the matching errors and retain the key point data with correct matching.

[0104] Further, in an embodiment of the present application, after determining the same-name pixels between each image in S122, the basic matrix and the intrinsic matrix of the feature points of each image can be calculated based on the same-name pixels, and then the camera's posture data can be determined based on the basic matrix and the intrinsic matrix.

[0105] In practice, the camera parameters and 3D coordinates of feature points can be calculated by triangulation, and then the camera parameters and 3D coordinates can be optimized by the BA beam method.

[0106] Finally, after completing the above steps, the camera pose data is obtained, including the position and attitude of the camera. Generally, this data is represented by quaternions Q and T, which are converted into three-dimensional spatial coordinates. The coordinates are relative coordinates and dimensionless data. Subsequent data processing only uses its plane coordinates.

[0107] like Figure 5 As shown, the calculated relative plane position of the aerial camera when taking the picture.

[0108] It can be understood that through the above steps, after using the acquired image data set, the three-dimensional scene is reconstructed by analyzing the motion and structural information in multiple images, that is, the SMF algorithm, and the images collected by the drone with overlapping areas are input to finally complete the sparse point cloud reconstruction and the determination of the internal and external parameters of the camera, that is, the determination of the posture information of the drone aerial camera is realized.

[0109] Optionally, in some embodiments of the present application, after calculating the camera's posture information through the above steps, the camera's posture information can be used to match the image data and POS data to determine the correspondence between the two, that is, to determine the specific missing data.

[0110] Understandably, combined Figure 6 and Figure 7 As shown in the figure, the camera plane coordinates of the aerial point calculated by SFM are dimensionless, while the POS data is a spatial coordinate system. There are transformations such as scaling and rotation between the two. The two conform to the principle of perspective transformation. The transformation matrix is ​​a homography matrix, that is, the aerial camera point matches the POS data. The core is to solve the corresponding homography matrix, that is, the conversion matrix.

[0111] like Figure 6 As shown, the method may specifically include the following steps:

[0112] S131, determining a target transformation matrix from the plane coordinate system of the aerial camera to the POS plane coordinate system according to the position and posture information and the POS data set.

[0113] S132, based on the target transformation matrix, transform the posture information of the aerial camera at each aerial point into the POS plane coordinate system, and obtain the position information of the posture information of each aerial point in the POS plane coordinate system as the aerial point transformation set.

[0114] S133, distance matching is performed between the position information of each aerial point in the aerial point conversion set and each POS point in the POS data set, and a matching relationship between each image in the image data set and the POS point in the POS data set is determined.

[0115] Specifically, combined Figure 7As shown, in some embodiments of the present application, determining the conversion matrix from the aerial camera plane coordinate system to the POS plane coordinate system may specifically include the following steps:

[0116] S04: Determine a camera point vector set of each aerial camera point according to the position and posture information.

[0117] S06: Determine a POS point vector set of each POS point based on the POS data.

[0118] S07, based on the Euclidean distance, matching the vectors in the camera point vector set and the POS point vector set to obtain a target matching vector set.

[0119] S08: Determine the target transformation matrix according to the target matching vector set.

[0120] Specifically, Figure 7 As shown, for the data set of the solved aerial camera pose information, take its plane coordinates and establish a spatial index SpatialIndex_M, that is, the camera point set of each aerial camera point. Then traverse each camera point in the aerial camera point set M, obtain the coordinates of the nearest q camera points around the aerial camera point, such as point m, such as 50 point coordinates, and construct a Delaunay triangulation. Use the lengths of all triangle edges of the triangulation to form a vector m_V of m points, that is, the camera point vector set of each aerial camera point.

[0121] Similarly, for POS data, the first choice is to build the spatial index SpatialIndex_P of all POS points, establish the spatial index SpatialIndex_P, traverse the aerial point set P, obtain the coordinates of the nearest q points around the POS point, such as 50 points, and build a Delaunay triangulation. Use the lengths of all triangle edges of the triangulation to form a vector p_V of point p, that is, the POS point vector set.

[0122] Furthermore, after the aerial point vector set and the POS point vector set are determined by solving the pose information and POS data, in S03, the target matching vector set of the camera point vector set and the POS point vector set is determined by the Euclidean distance, which may specifically include:

[0123] S001, traverse the camera point vector set, and determine the Euclidean distance between each camera point vector and each vector in the POS point vector set.

[0124] S002, based on the Euclidean minimum distance, matching the vectors in the camera point vector set and the POS point vector set to form an initial matching vector pair set.

[0125] S003, based on the random sampling consistency principle, error screening is performed on the initial matching vector pair set to obtain a candidate matching vector pair set.

[0126] S003: Determine an initial transformation matrix from the plane coordinate system of the aerial camera to the POS plane coordinate system according to the candidate matching vector pair set.

[0127] S004, according to the initial transformation matrix, transform the posture information of the aerial camera at each aerial point into the POS plane coordinate system, and obtain the position information of the posture information of each aerial point in the POS plane coordinate system as the initial aerial point transformation set.

[0128] S005, matching the position information in the initial aerial point conversion set with the POS point to obtain an initial matching relationship in the POS plane coordinate system.

[0129] S006: Based on the random sampling consistency principle, the initial matching relationship is screened for errors to obtain the target matching vector set.

[0130] Specifically, Figure 7 As shown, the vector m_V of the aerial camera point set M is traversed, and its Euclidean distance with the vector p_V of each POS point is compared. The minimum distance is used to match the aerial point and the POS point to form a matching pair M_P_PAIRS, that is, the initial matching vector pair set.

[0131] Furthermore, due to the existence of errors, there are mismatches in the matching pairs M_P_PAIRS. The random sampling consistency principle is used to eliminate the errors of the matching pairs to form matching pairs M_P_PAIRS_RANSAC, that is, a set of candidate matching vector pairs.

[0132] Furthermore, the error-eliminating matching pair M_P_PAIRS_RANSAC is used to calculate the homography matrix H of the left side system of the POS plane that transforms the aerial camera point plane coordinate system, that is, the initial transformation matrix, and according to this matrix H, the aerial point plane coordinates are transformed into the plane coordinate system of POS, that is, the initial aerial point transformation set. Assume that the converted aerial point set is set to M_C.

[0133] Finally, the converted aerial point set M_C is traversed, and the POS point with the smallest Euclidean distance is retrieved from the POS point set, and a matching set MC_P_PAIRS is formed, i.e., the initial matching relationship. Then, the matching pair error is eliminated using the random sampling consistency principle, and a matching pair MC_P_PAIRS_RANSAC is formed, i.e., the target matching relationship.

[0134] Further, after the target matching relationship is calculated, S04 is continued to be executed, that is, the calculated target matching pairs MC_P_PAIRS_RANSAC can be used to calculate the homography matrix H, that is, the target transformation matrix.

[0135] Furthermore, after obtaining the target transformation matrix through the above steps, continue to execute S132, based on the transformation matrix, transform the plane coordinates of the aerial point into the POS plane coordinate system, and obtain the aerial point set of the aerial camera point in the POS plane coordinate system; and then in S133, perform distance matching between the transformed aerial point and the POS point of the POS data to determine the correspondence between the film data and the POS data.

[0136] Specifically, after the target conversion matrix is ​​calculated, the target conversion matrix can be used to convert the coordinates of the aerial point into POS coordinates to obtain a target point set of the aerial point in the POS coordinate system.

[0137] Finally, traverse the converted aerial point set MC_C, and search for the POS point with the smallest Euclidean distance from it in the POS point set, and form a matching set MCC_P_PAIRS. Search for unpaired data in the aerial camera point set MC_C and the POS point set P, output the matching results and unmatched data, and complete the matching retrieval results of the image and POS point data.

[0138] like Figure 8 The figure shows the matching result between the image data and the POS data.

[0139] like Fig. 9 The figure shows a list of matching results between the output aerial photograph data and the POS data.

[0140] It can be understood that in the embodiment of the present application, the posture information of the aerial point camera and the POS data are matched to obtain the transformation matrix between the two, and then the aerial point plane coordinates are converted to the POS plane coordinates through the transformation matrix to achieve the unification of the coordinate system. Finally, the aerial point and the POS point are matched in the same coordinate system to determine the corresponding relationship between the two, that is, to achieve the matching of the film data and the POS data, and finally the loss situation of the drone collected data is obtained through the matching relationship.

[0141] Optionally, in some embodiments of the present application, in order to improve data processing accuracy, after acquiring the collected data of the drone, the image data therein may be preprocessed first, and then subsequent processing such as feature extraction may be performed.

[0142] That is Figure 3 As shown, after S110, the method may further include:

[0143] S115, performing pixel value correction on each image in the image data.

[0144] S116, resampling the corrected image data to obtain pre-processed image data.

[0145] Specifically, the acquired original image data is corrected for the distortion and mechanical errors of the optical lens.

[0146] In practice, the errors of digital cameras are mainly caused by the distortion of the optical lens and mechanical errors. Optical distortion includes radial distortion and tangential distortion. Mechanical error refers to the error generated when the image captured by the optical lens is converted to a digital array image.

[0147] Correspondingly, the distortion model formula of the image is:

[0148]

[0149] Among them, the d generated by the pixel point in the x and y directions x d y Distortion amount, k1, k2 and k3 represent radial distortion parameters, p1 and p2 represent tangential distortion parameters, and r represents the distance from the current pixel to the origin of the coordinate system.

[0150] If Fig.10 As shown, for a certain image in the acquired image data (i.e., the image to be corrected), when the correction process is performed based on the above distortion model formula, the following steps may be specifically included:

[0151] S11, traversing the pixel points of the image to be corrected, and calculating the distortion amount of each pixel point based on the configured distortion model.

[0152] S12, determining each pixel point whose distortion amount is greater than a set distortion threshold as a pixel point to be corrected.

[0153] S13, determining the sum of the pixel value of each pixel point to be corrected and the corresponding distortion amount, and the sum is used as the corrected pixel value of each pixel point to be corrected.

[0154] Specifically, in the embodiment of the present application, for the image to be corrected, the pixel points of the image to be corrected, such as a certain pixel point (i, j), can be traversed first, and then the distortion amounts dx and dy of each pixel point can be determined based on the above distortion model.

[0155] The corrected pixel point can be expressed as: i=i+dx, j=j+dy.

[0156] Furthermore, the determined distortion amounts dx and dy of each pixel point may be compared with a set distortion threshold to determine a pixel point whose distortion amount is greater than the distortion threshold value as a pixel point to be corrected.

[0157] Furthermore, for the determined pixel points to be corrected, the sum of the pixel value of each pixel point to be corrected and the corresponding distortion amount is determined, and the sum is used as the corrected target pixel value of each pixel point to be corrected.

[0158] It can be understood that for a pixel point whose distortion amount is greater than the distortion threshold, it means that the distortion amount of the pixel point exceeds the set threshold and needs to be corrected, while for other pixel points that are not greater than the distortion threshold, their distortion amounts can be ignored, that is, their original pixel values ​​can be output as target pixel values.

[0159] Finally, according to the above processing, an image composed of the target pixel values ​​of each pixel point is output as a corrected image.

[0160] It can be understood that in the actual processing process, the above correction process will be repeatedly executed in sequence for all acquired image data to complete the correction of each image.

[0161] Optionally, in some embodiments of the present application, in order to increase the data speed, after completing the above-mentioned image correction processing, further resampling preprocessing may be performed.

[0162] That is, drone aerial photography generally adopts low-altitude flight, with a ground resolution of up to 0.3cm and rich texture. Each picture data storage is up to 10MB-20MB, and the picture storage of the shooting area is up to 50GB or more. Since this application only needs to solve the approximate coordinates of the camera when processing by SFM technology, therefore, in order to improve data processing efficiency, the pictures can be resampled.

[0163] It can be understood that the smaller the resampling ratio, the greater the compression and the greater the loss of detailed texture. Conversely, the smaller the compression, the smaller the loss of detailed texture. For example, the present application can set the resampling ratio to 0.1 to scale the length and width of the original image to 1 / 10, so that a single image is less than 1MB.

[0164] If Fig.11 As shown, the method may also include:

[0165] S14, scaling each original image according to the size parameter of the original image data according to the resampling ratio coefficient to obtain reduced image data.

[0166] S15, traversing each pixel point in the reduced image to determine the original pixel value corresponding to each pixel point.

[0167] S16, assigning the original pixel value to the pixel value of the corresponding pixel point in the reduced image to obtain the resampled image data.

[0168] Specifically, for the original image data or the corrected image data obtained, as the initial image before resampling, the initial image can first be scaled according to the size parameters of the initial image (such as the width (W) and height (H) parameters of the image) and the set resampling ratio coefficient (such as 0.1).

[0169] That is, the product of the width and height of the initial image and 0.1 can be determined as the target size after scaling, and then the initial image is scaled according to the target size to obtain a reduced image.

[0170] Furthermore, each pixel point (i, j) of the reduced image after the scaling process is traversed, and then the pixel point (I, J) of the initial image corresponding to each pixel point of the reduced image is determined.

[0171] Finally, the pixel values ​​of the pixels of the initial image corresponding to the pixels of the determined reduced image are assigned to the pixels of the reduced image as the resampled image, so that the pixel values ​​of the pixels of the reduced image still maintain the pixel values ​​of the initial image to ensure the characteristic data of each image.

[0172] It can be understood that in the actual processing process, for all the original images or corrected images obtained, the above-mentioned resampling process is repeated in sequence to complete the resampling of each image, that is, to complete the resampling data processing process.

[0173] It can also be understood that in some embodiments, the original image data obtained may be first rectified, and then the rectified image may be resampled. Alternatively, only the rectified or resampled process may be performed, which is not limited in the present application and may be adjusted according to the actual situation.

[0174] It can be understood that based on the image texture information, by covering the overlapping images of the same area, the motion recovery structure (SFM) technology is used to calculate the posture information of the drone at the time of taking the image. Its posture information includes position and attitude. Its position contains XYZ three-dimensional coordinates, which are the image coordinate system. Then, through the matching processing of the coordinates and POS data, the correspondence between the aerial camera position point and the POS point is retrieved, and then the detailed information of the loss of the image and POS data is obtained.

[0175] On the other hand, the present application also provides a drone photogrammetry data processing device, such as Fig.12 As shown, the device comprises:

[0176] The acquisition module 210 is used to acquire the data collected by the drone, wherein the collected data includes a POS data set and a photo data set, wherein the POS data set includes the position information of each POS point, and the photo data set includes photos collected by the aerial camera at each aerial point;

[0177] A calculation module 220, configured to calculate the position and posture information of the aerial camera at each aerial point according to the images in the image data set based on an SFM algorithm;

[0178] The matching module 230 is used to match the images in the image data set with the position information of the POS point according to the posture information to obtain a matching relationship, wherein the matching relationship is used to indicate the loss of the images in the image data set and the position information of the POS point.

[0179] Optionally, in the drone photogrammetry data processing device provided in the embodiment of the present application, the computing module is specifically used for:

[0180] Extracting characteristic data of each image;

[0181] According to the characteristic data, determine the pixels with the same name between the images;

[0182] Calculate the basic matrix and the essential matrix of each image feature point according to the pixels with the same name;

[0183] The position and posture information of the aerial camera is determined according to the basic matrix and the essential matrix, and the position and posture information represents the position information of the aerial camera when collecting images at each aerial point.

[0184] Optionally, in the drone photogrammetry data processing device provided in the embodiment of the present application, the computing module is specifically used for:

[0185] Determining the association relationship between the images based on the extracted feature data, wherein the association relationship represents the degree of overlap between the images;

[0186] Determine an image pair in the image data set according to the association relationship, wherein the image pair represents two images with the highest degree of overlap in the image data set;

[0187] Extract the feature points of two images in the image pair;

[0188] Based on the SIFT algorithm, the feature points of the two images in the image pair are matched to obtain the pixels with the same name between the images.

[0189] Optionally, in the drone photogrammetry data processing device provided in the embodiment of the present application, the matching module is specifically used for:

[0190] Determine a target transformation matrix from the plane coordinate system of the aerial camera to the POS plane coordinate system according to the posture information and the POS data set;

[0191] Based on the target transformation matrix, the position information of the aerial camera at each aerial point is transformed into the POS plane coordinate system, and the position information of the position information of each aerial point in the POS plane coordinate system is obtained as the aerial point transformation set;

[0192] The position information of each aerial point in the aerial point conversion set is distance matched with each POS point in the POS data set, and the matching relationship between each photo in the photo data set and each POS point in the POS data set is determined.

[0193] Optionally, in the drone photogrammetry data processing device provided in the embodiment of the present application, the matching module is specifically used for:

[0194] Determine a camera point vector set of each aerial camera point according to the position and posture information;

[0195] Determine a POS point vector set of each POS point according to the POS data;

[0196] Based on the Euclidean distance, the vectors in the camera point vector set and the POS point vector set are matched to obtain a target matching vector set;

[0197] The target transformation matrix is ​​determined according to the target matching vector set.

[0198] Optionally, in the drone photogrammetry data processing device provided in the embodiment of the present application, the matching module is specifically configured to include:

[0199] Traversing the camera point vector set, determining the Euclidean distance between each camera point vector and each vector in the POS point vector set;

[0200] Based on the Euclidean minimum distance, matching the vectors in the camera point vector set and the POS point vector set to form an initial matching vector pair set;

[0201] Based on the principle of random sampling consistency, error screening is performed on the initial matching vector pair set to obtain a candidate matching vector pair set;

[0202] Determine an initial transformation matrix from the plane coordinate system of the aerial camera to the POS plane coordinate system according to the candidate matching vector pair set;

[0203] According to the initial transformation matrix, the position information of the aerial camera at each aerial point is transformed into the POS plane coordinate system, and the position information of the position information of each aerial point in the POS plane coordinate system is obtained as the initial aerial point transformation set;

[0204] Matching the position information in the initial aerial point conversion set with the POS point to obtain an initial matching relationship in the POS plane coordinate system;

[0205] Based on the consistency principle of random sampling, the initial matching relationship is screened for errors to obtain the target matching vector set.

[0206] Optionally, the drone photogrammetry data processing device provided in the embodiment of the present application further includes:

[0207] A correction module 240, used for performing pixel value correction on each image in the image data set;

[0208] The resampling module 250 is used to perform resampling processing on the corrected image to obtain a pre-processed image.

[0209] Optionally, in the drone photogrammetry data processing device provided in the embodiment of the present application, the correction module is specifically configured to include:

[0210] Traverse the pixel points of the image to be corrected, and calculate the distortion amount of each pixel point based on the configured distortion model;

[0211] Determine each pixel point whose distortion amount is greater than a set distortion threshold as a pixel point to be corrected;

[0212] The sum of the pixel value of each pixel point to be corrected and the corresponding distortion amount is determined, and the sum is used as the corrected pixel value of each pixel point to be corrected.

[0213] Optionally, in the drone photogrammetry data processing device provided in the embodiment of the present application, the resampling module is specifically used for:

[0214] According to the size parameters of the pictures, each original picture is scaled according to the resampling ratio coefficient to obtain a reduced picture;

[0215] Traverse each pixel point in the reduced image to determine the original pixel value corresponding to each pixel point;

[0216] The original pixel value is assigned to the pixel value of the corresponding pixel point in the reduced image to obtain a resampled image.

[0217] In an exemplary embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Fig.13As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store video tag processing data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for processing drone photogrammetry data is implemented.

[0218] Those skilled in the art will understand that Fig.13 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0219] In an exemplary embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0220] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0221] In an exemplary embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0222] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0223] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0224] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a computer device, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but is not limited thereto.

[0225] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0226] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A method for processing unmanned aerial vehicle photogrammetry data, characterized in that: The UAV photogrammetry data processing method comprises: Acquire data collected by the drone, wherein the collected data includes a POS data set and a photo data set, wherein the POS data set includes location information of each POS point, and the photo data set includes photos collected by the aerial camera at each aerial point; Based on the SFM algorithm, the position and posture information of the aerial camera at each aerial point is calculated according to the images in the image data set; According to the posture information, matching the photos in the photo data set with the position information of the POS point to obtain a matching relationship, wherein the matching relationship is used to indicate the loss of the photos in the photo data set and the position information of the POS point; Wherein, matching the photos in the photo data set with the position information of the POS point according to the posture information to obtain a matching relationship includes: Determine a target transformation matrix from the plane coordinate system of the aerial camera to the POS plane coordinate system according to the posture information and the POS data set; Based on the target transformation matrix, the position information of the aerial camera at each aerial point is transformed into the POS plane coordinate system, and the position information of the position information of each aerial point in the POS plane coordinate system is obtained as the aerial point transformation set; The position information of each aerial point in the aerial point conversion set is distance matched with each POS point in the POS data set, and the matching relationship between each photo in the photo data set and the POS point in the POS data set is determined.

2. The method for processing UAV photogrammetry data according to claim 1, characterized in that: The calculating of the position and posture information of the aerial camera at each aerial point based on the photos in the photo data set based on the SFM algorithm comprises: Extracting characteristic data of each image; Determine the pixels with the same name between the images according to the characteristic data; Calculate the basic matrix and the essential matrix of each image feature point according to the pixels with the same name; The position and posture information of the aerial camera is determined according to the basic matrix and the essential matrix, and the position and posture information represents the position information of the aerial camera when collecting images at each aerial point.

3. The method for processing UAV photogrammetry data according to claim 2, characterized in that: Determining the pixels with the same name between the images according to the feature data comprises: Determining the association relationship between the images based on the extracted feature data, wherein the association relationship represents the degree of overlap between the images; Determine an image pair in the image data set according to the association relationship, wherein the image pair represents two images with the highest degree of overlap in the image data set; Extract the feature points of two images in the image pair; Based on the SIFT algorithm, the feature points of the two images in the image pair are matched to obtain the pixels with the same name between the images.

4. The method for processing UAV photogrammetry data according to claim 1, characterized in that: Determining the target transformation matrix from the plane coordinate system of the aerial camera to the POS plane coordinate system according to the posture information and the POS data set includes: Determine a camera point vector set of each aerial camera point according to the position and posture information; Determine a POS point vector set of each POS point according to the POS data; Based on the Euclidean distance, the vectors in the camera point vector set and the POS point vector set are matched to obtain a target matching vector set; The target transformation matrix is ​​determined according to the target matching vector set.

5. The method for processing UAV photogrammetry data according to claim 4, characterized in that: The method of matching the vectors in the camera point vector set and the POS point vector set based on the Euclidean distance to obtain the target matching vector set includes: Traversing the camera point vector set, determining the Euclidean distance between each camera point vector and each vector in the POS point vector set; Based on the Euclidean minimum distance, matching the vectors in the camera point vector set and the POS point vector set to form an initial matching vector pair set; Based on the principle of random sampling consistency, error screening is performed on the initial matching vector pair set to obtain a candidate matching vector pair set; Determine an initial transformation matrix from the plane coordinate system of the aerial camera to the POS plane coordinate system according to the candidate matching vector pair set; According to the initial transformation matrix, the position information of the aerial camera at each aerial point is transformed into the POS plane coordinate system, and the position information of the position information of each aerial point in the POS plane coordinate system is obtained as the initial aerial point transformation set; Matching the position information in the initial aerial point conversion set with the POS point to obtain an initial matching relationship in the POS plane coordinate system; Based on the consistency principle of random sampling, the initial matching relationship is screened for errors to obtain the target matching vector set.

6. The method for processing UAV photogrammetry data according to any one of claims 1 to 5, characterized in that: After acquiring the drone data, and before calculating the position and posture information of the aerial camera at each aerial point based on the photos in the photo data set based on the SFM algorithm, the method further includes: Performing pixel value correction on each image in the image data set; The corrected image is resampled to obtain a preprocessed image.

7. The method for processing UAV photogrammetry data according to claim 6, characterized in that: The pixel value correction of each image in the image data set comprises: Traverse the pixel points of the image to be corrected, and calculate the distortion amount of each pixel point based on the configured distortion model; Determine each pixel point whose distortion amount is greater than a set distortion threshold as a pixel point to be corrected; The sum of the pixel value of each pixel point to be corrected and the corresponding distortion amount is determined, and the sum is used as the corrected pixel value of each pixel point to be corrected.

8. The method for processing UAV photogrammetry data according to claim 7, characterized in that: The resampling of the corrected image to obtain the preprocessed image comprises: According to the size parameters of the pictures, each original picture is scaled according to the resampling ratio coefficient to obtain a reduced picture; Traverse each pixel point in the reduced image to determine the original pixel value corresponding to each pixel point; The original pixel value is assigned to the pixel value of the corresponding pixel point in the reduced image to obtain a resampled image.

9. A computer device, characterized in that: The computer device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method for processing unmanned aerial vehicle photogrammetry data as described in any one of claims 1 to 8.

Images