Surveying and mapping image processing method based on remote sensing of unmanned aerial vehicle

Abstract

The invention discloses a surveying and mapping image processing method based on unmanned aerial vehicle remote sensing, and relates to the technical field of map surveying and mapping. Before formal surveying and mapping, a plurality of benchmark reference areas are arranged in a target surveying and mapping area, and test flight surveying and mapping information of the benchmark reference areas obtained under different flight attitudes of an unmanned aerial vehicle is obtained by utilizing test flight of the unmanned aerial vehicle; and comparing the obtained test flight surveying and mapping information with the actual position information of the benchmark reference area so as to obtain surveying and mapping errors of the unmanned aerial vehicle in different flight attitudes, and correcting the surveying and mapping process of the unmanned aerial vehicle in different flight attitudes through the obtained surveying and mapping errors. Therefore, surveying and mapping errors possibly caused by the unmanned aerial vehicle in various flight attitudes are reduced, and the accuracy of remote sensing surveying and mapping of the unmanned aerial vehicle is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of map surveying and mapping, and specifically to a method for processing surveying and mapping images based on UAV remote sensing. Background Art

[0002] As a new surveying and mapping method, UAV remote sensing technology is gradually becoming an important tool for obtaining geospatial information. Traditional surveying and mapping methods, such as ground surveying, aerial photogrammetry, etc., have certain limitations in practical applications. UAV remote sensing technology effectively makes up for the deficiencies of traditional surveying and mapping methods with its advantages of flexibility, convenience, low cost, fast response speed, etc. UAVs can fly at low altitudes, are not overly restricted by complex terrains and weather conditions, and can quickly and accurately obtain high-resolution remote sensing images.

[0003] When using UAV remote sensing technology for surveying and mapping, the data obtained under different flight postures of the UAV often have certain deviations, resulting in a decrease in surveying and mapping accuracy. To reduce the surveying and mapping deviations caused by the UAV flight posture, a method for processing surveying and mapping images based on UAV remote sensing is provided herein. Summary of the Invention

[0004] The purpose of the present invention is to provide a method for processing surveying and mapping images based on UAV remote sensing.

[0005] The purpose of the present invention can be achieved through the following technical solutions: A method for processing surveying and mapping images based on UAV remote sensing includes: Construct a plane coordinate system for the target surveying and mapping area, and set a reference reference area within the target surveying and mapping area; Set a test flight route, obtain test flight surveying and mapping information of each reference reference area through the UAV on the test flight route, and calibrate the surveying and mapping accuracy of the UAV through the obtained test flight surveying and mapping information; Set a surveying and mapping flight route for the UAV after completing the surveying and mapping accuracy calibration, obtain real-time surveying and mapping information through the UAV on the surveying and mapping flight route, process the obtained real-time surveying and mapping information to obtain a registered image and surveying and mapping parameters of the target surveying and mapping area, and the real-time surveying and mapping information includes real-time image information and real-time remote sensing information; Based on the obtained registered image, fill in simulation elements in the plane coordinate system, configure the layers of the filled simulation elements, and combine the surveying and mapping parameters to obtain a regional surveying and mapping model of the target surveying and mapping area.

[0006] Further, the process of constructing a plane coordinate system for the target surveying and mapping area and setting a reference reference area within the target surveying and mapping area is as follows: Select the target surveying and mapping area, and construct a plane coordinate system based on the selected target surveying and mapping area; Set several reference areas within the target survey area, obtain the relative positions between the reference areas, generate corresponding virtual reference areas in the plane coordinate system according to the relative positions between the reference areas, and obtain the coordinate ranges of the virtual reference areas in the plane coordinate system, denoted as reference coordinate ranges.

[0007] Further, the process of obtaining the flight test survey information of each reference area is as follows: Set a flight test route, set survey areas on the flight test route, and associate each survey area with a reference area; Preset several UAV flight postures, and record the flight posture of the UAV when it reaches the survey area as the initial posture; Based on the initial posture, change the posture of the UAV in the survey area according to the preset UAV flight postures, and obtain the flight test survey information of the reference area in different flight postures. The flight test survey information includes flight test image information and flight test remote sensing information.

[0008] Further, the process of calibrating the survey accuracy of the UAV through the obtained flight test survey information is as follows: Intercept the flight test image information obtained in the same flight posture, convert it into corresponding image frames, rasterize each image frame, and then obtain the corresponding grayscale image; Extract the feature values of each grayscale image regarding the area where the reference area is located, and overlap the grayscale images to obtain the feature fusion area of the area where the reference area is located; Obtain the flight test remote sensing information obtained in the same flight posture, and obtain the relative position between the reference area and the UAV according to the obtained flight test remote sensing information and the current location of the UAV, so as to obtain the flight test survey coordinates of the reference area in the current flight posture; Map the obtained feature fusion area to the corresponding position in the plane coordinate system according to the obtained flight test survey coordinates; Compare the feature fusion area with the reference coordinate range to obtain the coordinate offset; Associate the obtained coordinate offset with this flight posture, and summarize the coordinate offsets of the same flight posture of each reference area to obtain the corresponding mean value as the survey calibration value of this flight posture.

[0009] Further, convert the obtained real-time image information into image frames, and input the converted image frames into the trained neural network model for feature extraction to obtain each image feature within the image frames; Obtain whether any of the image features extracted from each image frame is a reference area, so as to obtain the captured image frames and off-screen image frames; Summarize the captured image frames containing image features corresponding to the same reference area to obtain a set of image frames associated with the reference area; Traverse the image features extracted from each captured image frame in the set of image frames, and mark the same image features; For the relative position relationship between the marked same image feature and the feature corresponding to the reference area in each image frame, if the relative position relationship remains unchanged, it indicates that the corresponding image feature is a solid feature, otherwise it is a dynamic feature, and all dynamic features are removed; Fit each image frame, and perform feature fusion on the same image feature in the fitted image frames to obtain a registered image associated with the reference area.

[0010] Furthermore, obtain the relative position distances between each solid feature in the registered image and the reference area. Taking the reference area as the center, select the solid features with the farthest relative position distances at both ends along the mapping flight route, and record them as out-of-frame reference features; Check whether there are image features in the out-of-frame image frames that are the same as the out-of-frame reference features. If so, summarize all the out-of-frame image frames with out-of-frame reference features, and obtain the relative position distances between each image feature in the summarized out-of-frame image frames and the out-of-frame reference features. Determine the solid features and dynamic features in the out-of-frame image frames according to the relative position distances, remove the dynamic features, and then fit the summarized out-of-frame image frames with the registered image to obtain a new registered image; Then select new out-of-frame reference features from the new registered image and process the remaining out-of-frame image frames, and so on; Map the registered image associated with the reference area obtained to the corresponding position in the plane coordinate system.

[0011] Furthermore, based on the flight attitude of the unmanned aerial vehicle during flight, obtain the corresponding mapping calibration value, and calibrate the obtained real-time remote sensing information based on the mapping calibration value; According to the calibrated real-time remote sensing information, obtain the position information corresponding to each image feature; Obtain the relative position with the reference area with the closest distance according to the position information of each image feature; Compare the position information of the reference area obtained according to the calibrated real-time remote sensing information with the reference coordinate range of the corresponding reference area, and obtain the secondary mapping calibration value according to the comparison result; Calibrate the position information of each image feature according to the secondary mapping calibration value and the relative position of each image feature with the reference area; Map the position information of each calibrated image feature to each image feature in the planar coordinate system, and update the positions and coordinate information of each image feature in the planar coordinate system.

[0012] Further, the process of obtaining the regional mapping model of the target mapping area includes: Construct a feature - element association database, which includes different image features, and each image feature is associated with a simulation element; Traverse each image feature in the planar coordinate system, index the corresponding simulation element in the feature - element association database according to the traversed image feature, and replace the simulation element with the corresponding image feature in the planar coordinate system; Read the coordinate information of each simulation element, and check whether there is coordinate overlap between each simulation element and other simulation elements; If there is no coordinate overlap with other simulation elements, mark this simulation element as a single - layer element. If there is coordinate overlap with other simulation elements, mark this simulation element as a composite - layer element; Obtain the coordinate information corresponding to each simulation element in the composite - layer element, compare the coordinate information of each simulation element, sort the coordinate overlapping parts of the simulation elements according to the comparison results, sort the simulation elements according to the sorting results, generate an element layer corresponding to each simulation element according to the sorting results, and fit the corresponding positions of the element layer in the planar coordinate system to obtain a composite layer.

[0013] Compared with the prior art, the beneficial effects of the present invention are: Before formal mapping, set several reference areas in the target mapping area, and use the test flight of the unmanned aerial vehicle (UAV) to obtain the test - flight mapping information of the reference areas obtained under different flight postures of the UAV. Then compare the obtained test - flight mapping information with the actual location information of the reference areas, so as to obtain the mapping errors existing in the UAV under different flight postures. Correct the mapping process of different flight postures of the UAV through the obtained mapping errors, thereby reducing the mapping errors that may be caused by the UAV under various flight postures and improving the accuracy of UAV remote - sensing mapping. Brief Description of the Drawings

[0014] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments recorded in the present invention. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings.

[0015] Figure 1 It is the flowchart of the method of the present invention; Figure 2 This is the flight test mapping flowchart of the present invention; Figure 3 This is the actual mapping flowchart of the present invention. Detailed implementation manners

[0016] As Figure 1 shown, a mapping image processing method based on UAV remote sensing includes: Construct a plane coordinate system for the target mapping area, and set a reference reference area within the target mapping area. The specific process is as follows: Select the target mapping area, and construct a plane coordinate system based on the selected target mapping area; Set several reference reference areas within the target mapping area, and obtain the relative positions between the reference reference areas. According to the relative positions between the reference reference areas, generate corresponding virtual reference areas in the plane coordinate system, and obtain the coordinate ranges of the virtual reference areas in the plane coordinate system, denoted as reference coordinate ranges.

[0017] As Figure 2 shown, in another embodiment of the present invention, flight test mapping of the UAV will be performed, that is: by setting a flight test route, obtaining flight test mapping information of each reference reference area through the UAV on the flight test route, and calibrating the mapping accuracy of the UAV through the obtained flight test mapping information. The specific process is as follows: Set a flight test route, and the flight test route can sequentially pass through each reference reference area without repetition; Set mapping areas on the flight test route, and each mapping area is associated with a reference reference area; Preset several UAV flight postures, and record the flight posture of the UAV when it reaches the mapping area as the initial posture; Based on the initial posture, change the posture of the UAV according to the preset UAV flight postures within the mapping area, and obtain flight test mapping information of the reference reference area under different flight postures. The flight test mapping information includes flight test image information and flight test remote sensing information; Intercept the flight test image information obtained under the same flight posture, and convert it into corresponding image frames. After rasterizing each image frame, obtain the corresponding grayscale image; Extract the feature values of each grayscale image regarding the area where the reference reference area is located, and overlap the grayscale images to obtain the feature fusion area of the area where the reference reference area is located; Obtain the flight test remote sensing information obtained under the same flight attitude. According to the obtained flight test remote sensing information and the current location of the UAV, obtain the relative position between the reference area and the UAV, so as to obtain the flight test mapping coordinates of the reference area under the current flight attitude; Map the obtained feature fusion area to the corresponding position in the plane coordinate system according to the obtained flight test mapping coordinates; Compare the feature fusion area with the reference coordinate range to obtain the coordinate offset; Associate the obtained coordinate offset with the flight attitude, and so on, to obtain the coordinate offsets within the mapping area of each flight attitude in the reference area; Summarize the coordinate offsets of the same flight attitude of each reference area to obtain the corresponding average value as the mapping calibration value of the flight attitude; It should be noted that in the specific implementation process, the flight process of the UAV usually maintains a fixed flight altitude.

[0018] As Figure 3 shown, in another embodiment of the present invention, during actual mapping, a mapping flight route is set for the UAV that has completed mapping accuracy calibration, and real-time mapping information is obtained through the UAV on the mapping flight route, and the obtained real-time mapping information is processed to obtain the registration image and mapping parameters of the target mapping area. The specific process is as follows: The real-time mapping information includes real-time image information and real-time remote sensing information; Convert the obtained real-time image information into image frames, and input the converted image frames into the trained neural network model for feature extraction to obtain various image features within the image frames; Obtain whether any of the image features extracted from each image frame is any reference area; If the image frame contains image features corresponding to any reference area, mark the image frame as a captured image frame, and record the remaining image frames that do not contain image features corresponding to any reference area as off-screen image frames; Summarize the image frames containing image features corresponding to the same reference area to obtain an image frame set associated with the reference area; Traverse the image features extracted from each image frame within the image frame set, and mark the same image features; If the relative position relationship between the marked same image feature and the feature corresponding to the reference area within each image frame has not changed, it means that the corresponding image feature is a solid feature, otherwise it is a dynamic feature, and all dynamic features are removed; Fit each image frame and perform feature fusion on the same image features within the fitted image frames to obtain a registered image associated with the reference reference area; Obtain the relative position distances between each solid feature in the registered image and the reference reference area. With the reference reference area as the center, select the solid features with the farthest relative position distances at both ends along the mapping flight route, and record them as out-of-frame reference features; Obtain whether there are image features in the out-of-frame image frames that are the same as the out-of-frame reference features. If so, summarize all the out-of-frame image frames with out-of-frame reference features, and obtain the relative position distances between each image feature in the summarized out-of-frame image frames and the out-of-frame reference features. Determine the solid features and dynamic features in the out-of-frame image frames according to the relative position distances, eliminate the dynamic features, and then fit the summarized out-of-frame image frames with the registered image to obtain a new registered image; Then select new out-of-frame reference features from the new registered image and process the remaining out-of-frame image frames, and so on; Map the registered image associated with the obtained reference reference area to the corresponding position in the plane coordinate system.

[0019] In another embodiment of the present invention, based on the flight attitude of the unmanned aerial vehicle during flight, obtain the corresponding mapping calibration value, and calibrate the obtained real-time remote sensing information based on the mapping calibration value; According to the calibrated real-time remote sensing information, obtain the position information corresponding to each image feature; Obtain the relative position with the reference reference area with the closest distance according to the position information of each image feature; Compare the position information of the reference reference area obtained according to the calibrated real-time remote sensing information with the reference coordinate range of the corresponding reference reference area, and obtain the secondary mapping calibration value according to the comparison result; Calibrate the position information of each image feature according to the secondary mapping calibration value and the relative position of each image feature reference reference area; Map the position information of each calibrated image feature to each image feature in the plane coordinate system, and update the positions and coordinate information of each image feature in the plane coordinate system.

[0020] Based on the obtained registered image, perform simulation element filling in the plane coordinate system, configure the layers of the filled simulation elements, and combine the mapping parameters to obtain the regional mapping model of the target mapping area. The specific process includes: Construct a feature-element association database, where the feature-element association database includes different image features, and each image feature is associated with a simulation element; Traverse each image feature in the plane coordinate system. According to the traversed image features, index the corresponding simulation elements in the feature - element association database, map the simulation elements to the plane coordinate system, and replace the corresponding image features in the plane coordinate system; After completing the replacement of all image features, read the coordinate information of each simulation element, and determine whether there is coordinate overlap between each simulation element and other simulation elements according to the coordinate information of each simulation element; If there is no coordinate overlap with other simulation elements, mark this simulation element as a single - layer element. If there is coordinate overlap with other simulation elements, mark this simulation element as a composite - layer element; Obtain the coordinate information corresponding to each simulation element in the composite - layer element, compare the coordinate information of each simulation element, sort the coordinate overlapping parts of the simulation elements according to the comparison results, sort the simulation elements according to the sorting results, generate an element layer corresponding to each simulation element according to the sorting results, and fit the corresponding positions of the element layer in the plane coordinate system to obtain a composite layer. And so on until the layer configuration of all image features is completed, thereby completing the construction of the regional mapping model.

[0021] In another embodiment of the present invention, after completing the layer configuration of all image features, associate the positions and coordinate information of each image feature in the latest plane coordinate system with the corresponding layers to generate information index items. The user can complete the retrieval of the layer and coordinate information corresponding to the information index item by clicking on each information index item.

[0022] The above - mentioned are only the preferred embodiments of the present invention, and do not impose any form of limitation on the present invention. Although the present invention has been disclosed above with the preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some changes or modifications to equivalent embodiments with equivalent changes within the scope of the technical solution of the present invention. However, as long as it does not depart from the content of the technical solution of the present invention, any modification or equivalent replacement made to the above - mentioned embodiments based on the technical essence of the present invention still falls within the scope of the technical solution of the present invention.

Claims

1. A mapping image processing method based on UAV remote sensing, characterized in that: include: Construct a plane coordinate system for the target surveying and mapping area, and set a benchmark reference area within the target surveying and mapping area; Set up a test flight route, use the UAV to obtain the test flight surveying and mapping information of each benchmark reference area on the test flight route, and calibrate the surveying and mapping accuracy of the UAV through the obtained test flight surveying and mapping information; Setting a surveying and mapping flight route for the UAV that has completed the surveying and mapping accuracy calibration, acquiring real-time surveying and mapping information on the surveying and mapping flight route through the UAV, processing the acquired real-time surveying and mapping information, and acquiring a registration image and surveying and mapping parameters of the target surveying and mapping area, wherein the real-time surveying and mapping information includes real-time image information and real-time remote sensing information; Based on the obtained registration image, simulation elements are filled in the plane coordinate system, and the filled simulation elements are configured in layers, and the regional mapping model of the target mapping area is obtained in combination with the mapping parameters.

2. The method for mapping images based on UAV remote sensing according to claim 1, characterized in that: The process of constructing the plane coordinate system of the target surveying and mapping area and setting the benchmark reference area within the target surveying and mapping area is as follows: Select a target surveying and mapping area, and construct a plane coordinate system based on the selected target surveying and mapping area; Several benchmark reference areas are set in the target surveying area, and the relative positions between the benchmark reference areas are obtained. According to the relative positions between the benchmark reference areas, corresponding virtual reference areas are generated in the plane coordinate system, and the coordinate range of each virtual reference area in the plane coordinate system is obtained, which is recorded as the reference coordinate range.

3. The method for mapping images based on UAV remote sensing according to claim 2, characterized in that: The process of obtaining the flight test mapping information for each benchmark reference area is as follows: Set a test flight route, set a surveying and mapping area on the test flight route, and associate each surveying and mapping area with a benchmark reference area; Preset several UAV flight attitudes, and record the flight attitude of the UAV when it arrives at the surveying area as the initial attitude; Based on the initial attitude, the attitude of the UAV is changed in the surveying and mapping area according to the preset UAV flight attitude, and the test flight surveying and mapping information of the benchmark reference area under different flight attitudes is obtained, and the test flight surveying and mapping information includes test flight image information and test flight remote sensing information.

4. The method for mapping image processing based on UAV remote sensing according to claim 3 is characterized in that: The process of calibrating the UAV's mapping accuracy through the test flight mapping information obtained is as follows: The test flight image information obtained under the same flight attitude is intercepted and converted into corresponding image frames, and each image frame is rasterized to obtain the corresponding grayscale image; Extracting feature values ​​of each grayscale image with respect to the area where the benchmark reference area is located, and overlapping each grayscale image to obtain a feature fusion area where the benchmark reference area is located; Acquire the test flight remote sensing information obtained under the same flight attitude, and obtain the relative position between the benchmark reference area and the UAV according to the obtained test flight remote sensing information and the current position of the UAV, so as to obtain the test flight mapping coordinates of the benchmark reference area under the current flight attitude; Mapping the obtained feature fusion area to a corresponding position in the plane coordinate system according to the obtained test flight mapping coordinates; Compare the feature fusion area with the reference coordinate range to obtain the coordinate offset; The obtained coordinate offset is associated with the flight attitude, and the coordinate offsets of the same flight attitude in each benchmark reference area are summarized to obtain the corresponding average value as the surveying and mapping calibration value of the flight attitude.

5. The method for processing surveying and mapping images based on UAV remote sensing according to claim 4, characterized in that: The obtained real-time image information is converted into image frames, and the converted image frames are input into the trained neural network model for feature extraction to obtain various image features in the image frames; Determine whether the image features extracted in each image frame are any benchmark reference area, thereby obtaining a captured image frame and an off-screen image frame; Aggregating captured image frames containing image features corresponding to the same benchmark reference area to obtain an image frame set associated with the benchmark reference area; Traversing the image features extracted from each captured image frame in the image frame set, and marking the same image features; The relative position relationship between the marked same image feature and the feature corresponding to the benchmark reference area in each image frame, if the relative position relationship does not change, it means that the corresponding image feature is a solid feature, otherwise it is a dynamic feature, and all dynamic features are eliminated; Each image frame is fitted, and the same image features in the fitted image frames are fused to obtain a registered image associated with the benchmark reference area.

6. The method for mapping image processing based on UAV remote sensing according to claim 5, characterized in that: The relative position distance between each solid feature in the registration image and the benchmark reference area is obtained. Taking the benchmark reference area as the center, the solid features with the farthest relative position distance at both ends of the surveying flight route are selected and recorded as off-screen reference features. Obtain whether there is an image feature identical to the off-picture comparison feature in the off-picture image frame; if so, aggregate all off-picture image frames with the off-picture comparison feature, and obtain the relative position distance between each image feature in the aggregated off-picture image frame and the off-picture comparison feature; determine the solid features and dynamic features in the off-picture image frame according to the relative position distance, remove the dynamic features, and then fit the aggregated off-picture image frame with the registration image to obtain a new registration image; Then, new off-screen comparison features are selected from the new registered image, and the remaining off-screen image frames are processed, and so on; The obtained registered image associated with the fiducial reference region is mapped to a corresponding position in the plane coordinate system.

7. The method for mapping image processing based on UAV remote sensing according to claim 6, characterized in that: Based on the flight attitude of the UAV during the flight, the corresponding surveying and mapping calibration value is obtained, and the obtained real-time remote sensing information is calibrated based on the surveying and mapping calibration value; According to the calibrated real-time remote sensing information, the position information corresponding to each image feature is obtained; Obtaining a relative position to the nearest benchmark reference area according to the position information of each image feature; Compare the position information of the benchmark reference area obtained according to the real-time remote sensing information after calibration with the reference coordinate range of the corresponding benchmark reference area, and obtain the secondary surveying and mapping calibration value according to the comparison result; Calibrate the position information of each image feature according to the secondary surveying and mapping calibration value and the relative position of the benchmark reference area of ​​each image feature; The position information of each calibrated image feature is mapped to each image feature in the plane coordinate system, and the position and coordinate information of each image feature in the plane coordinate system is updated.

8. The method for mapping image processing based on UAV remote sensing according to claim 7, characterized in that: The process of obtaining the regional mapping model of the target mapping area includes: Constructing a feature-element association database, wherein the feature-element association database includes different image features, each image feature is associated with a simulation element; Traversing each image feature in the plane coordinate system, indexing the corresponding simulation element in the feature-element association database according to the traversed image feature, and replacing the simulation element with the corresponding image feature in the plane coordinate system; Read the coordinate information of each simulation element to determine whether the coordinates of each simulation element overlap with those of other simulation elements; If there is no overlap with other simulation element coordinates, the simulation element is marked as a single layer element; if there is overlap with other simulation element coordinates, the simulation element is marked as a composite layer element; Obtain the coordinate information corresponding to each simulation element in the composite layer element, and compare the coordinate information of each simulation element, sort the overlapping parts of the coordinates of the simulation elements according to the comparison result, sort the simulation elements according to the sorting result, generate element layers corresponding to each simulation element according to the sorting result, and fit the corresponding positions of the element layers in the plane coordinate system to obtain the composite layer.

Images