A method for constructing an integrated three-dimensional scene on water and underwater based on multi-source data

Abstract

The application discloses a kind of water surface and underwater integrated three-dimensional scene construction method based on multi-source data, belong to geographic information surveying and mapping technical field, including the following steps: S1: determine scheme;S2: determine motion trajectory;S3: obtain water surface, revetment and river and lake surrounding environment data;S4: obtain underwater riverbed data;S5: water surface, revetment and river and lake surrounding environment data preprocessing and generate point cloud data and three-dimensional model;S6: underwater riverbed data preprocessing and generate underwater riverbed three-dimensional model;S7: fusion output water surface and underwater integrated three-dimensional scene model;The application is obtained by fusing unmanned aerial vehicle carrying center orthographic angle camera and four direction optical axis and horizontal plane 45 ° angle camera water surface, revetment and river and lake surrounding environment data and underwater riverbed data such as multi-source data obtained by unmanned ship carrying multibeam echo sounder and GNSS-RTK receiver, and constructs water surface and underwater integrated three-dimensional scene model, can provide basis for comprehensive, fine management of natural resource three-dimensional space.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of geographic information surveying and mapping, and particularly relates to a water and underwater integrated three-dimensional scene construction method based on multi-source data. BACKGROUND

[0002] With the development of cities and the increasingly clear nature of resource ownership work, natural resources will enter the era of fine and information management. However, the main results of the natural resource ownership work at the present stage are still two-dimensional map results made by relying on existing geographic conditions survey results, orthophotos and other results, and traditional surveying and mapping operation modes relying on RTK and on-site surveying and mapping means. This mode cannot form a natural resource three-dimensional map based on spatial reference, and therefore it is difficult to meet the requirements of fine management of natural resources.

[0003] In view of this, a water and underwater integrated three-dimensional scene construction method based on multi-source data is designed to solve the above problems. SUMMARY

[0004] To solve the problems in the background art, the application provides a water and underwater integrated three-dimensional scene construction method based on multi-source data, which has the characteristics of being able to provide a basis for comprehensive and fine management of natural resource three-dimensional space.

[0005] To achieve the above purpose, the application provides the following technical scheme: a water and underwater integrated three-dimensional scene construction method based on multi-source data, comprising the following steps:

[0006] S1: determining a UAV photography scheme and a UAV ship navigation scheme according to a three-dimensional scene construction area;

[0007] S2: determining a UAV movement trajectory and a UAV ship navigation trajectory through the UAV photography scheme and the UAV ship navigation scheme;

[0008] S3: moving a UAV carrying a central orthographic angle camera and four direction optical axes at an angle of 45° with the horizontal plane to obtain water surface, revetment and river and lake surrounding environment data through the determined movement trajectory;

[0009] S4: moving a UAV ship carrying a multi-beam depth sounder and a GNSS-RTK receiver to obtain underwater riverbed data through the determined navigation trajectory;

[0010] S5: pre-processing the images obtained by UAV oblique photography and close-range photography, and then performing aerial triangulation to obtain an analysis report, judging the accuracy, and then judging whether the re-projection error meets the accuracy requirement, if the requirement is met, generating point cloud data and a three-dimensional model of the water surface, revetment and river and lake surrounding environment through the oblique photography and close-range photography images, if the requirement is not met, re-measuring the UAV;

[0011] S6: The underwater riverbed data obtained by the unmanned ship carrying the multi-beam echo sounder and the GNSS-RTK receiver are preprocessed and then subjected to multi-profile analysis to obtain an analysis report and determine the accuracy thereof. If the accuracy meets the requirements, point cloud registration is performed based on a manual coarse registration and an ICP point cloud matching algorithm automatic fine registration combined method, and a three-dimensional underwater riverbed model is generated based on the obtained underwater riverbed data. If the accuracy does not meet the requirements, the unmanned ship is re-measured;

[0012] S7: The unmanned aerial photography data and close-range photography data and the multi-beam echo sounder three-dimensional point cloud data after registration are simultaneously imported into the ContextCapture three-dimensional modeling software, and a water and underwater integrated three-dimensional scene model is generated by using the software;

[0013] S8: The accuracy of the generated water surface, revetment and surrounding environment three-dimensional model and the underwater riverbed three-dimensional model and the water and underwater integrated three-dimensional scene model is compared. If the accuracy is the same, the water and underwater integrated three-dimensional scene model is output to a display terminal for display to realize visualization. If the accuracy is different, steps S5-S7 are adjusted and repeated until the accuracy is the same and the water and underwater integrated three-dimensional scene model is output and stopped.

[0014] Preferably, in step S3, the unmanned aerial photography includes the following steps: the unmanned aerial vehicle performs oblique photography according to the determined motion trajectory. If an object blocks the oblique photography area during the shooting process, the position and area of the object blocking area are determined and transmitted to the control terminal. The control terminal determines the motion trajectory of the close-range photography of the unmanned aerial vehicle and transmits it back to the unmanned aerial vehicle. The unmanned aerial vehicle performs close-range photography according to the determined motion trajectory until the close-range photography of the blocking object is completed and returns to the oblique photography motion trajectory.

[0015] Preferably, in step S3, the object blocking detection is based on height, and specifically includes the following steps: it is judged whether a certain object point is visible on the image. On its search path, the elevations of the object points on the search path and the elevations of the projected light beams at the corresponding positions of the object points are compared in turn from the object point to the ground point direction. If the elevation of any point on the search path is higher than the projected light beam height, the object point is blocked during imaging and has no imaging. Otherwise, there is no blocking. The projected light beam elevation value is calculated as follows:

[0016]

[0017] In the formula, i is the i-th point on the search path, d is the search step length on the search path, (x s , y s , z s ) and (x o , y o , z o ) are the projection center and the matching point. s , y s , z s ) and (x o , y o , z o ) are the projection center and the matching point.

[0018] Preferably, in the step S5, before generating the point cloud data of the water surface, revetment and the surrounding environment of the river and lake, the planning of the land and resources management bureau for the three-dimensional scene construction area is acquired, if there is no land planning change plan in the three-dimensional scene construction area, the oblique photography and close-range photography image data are allocated as a unified identifier, if there is a short-term land planning change plan in the three-dimensional scene construction area, the oblique photography and close-range photography image data with the change plan are allocated a first identifier, and the image data without the change plan are allocated a second identifier, and then the point cloud data is generated through the oblique photography and close-range photography image data allocated with the first identifier and the second identifier.

[0019] Preferably, in the step S6, the ICP point cloud matching algorithm specifically includes the following steps: first, the feature points of two point sets are acquired, data matching is performed according to the feature points, and the matching points are set as the corresponding points, then the motion parameters are solved according to the corresponding relationship, and finally the data conversion is performed by using the parameters, which is to minimize the objective function S 2 by the corresponding points, and the calculation formula is as follows:

[0020]

[0021] In the formula, N is the iteration number, Q i is the point in the reference point cloud, P i is the corresponding point in the target point cloud, and R and t are the rotation and translation matrices to be calculated.

[0022] Preferably, in the step S8, in the water surface and underwater integrated three-dimensional scene model display process, when the local area needs to be changed due to the land planning change, the image data allocated with a third identifier is generated by acquiring the image data through the unmanned aerial vehicle carrying the central orthographic angle camera and the four direction optical axes with the horizontal plane at an angle of 45°, the point cloud data allocated with the first identifier is updated to the corresponding associated point cloud data allocated with the third identifier, and then the model is reconstructed and displayed.

[0023] Compared with the prior art, the beneficial effects of the present application are:

[0024] 1、The present application constructs the water surface and underwater integrated three-dimensional scene model by fusing the water surface, revetment and the surrounding environment data of the river and lake obtained by the unmanned aerial vehicle carrying the central orthographic angle camera and the four direction optical axes with the horizontal plane at an angle of 45° and the underwater riverbed data obtained by the unmanned ship carrying the multi-beam depth sounder and the GNSS-RTK receiver, which can provide a basis for the comprehensive and fine management of the natural resource three-dimensional space.

[0025] 2. The present invention acquires water surface, revetment, and river and lake surrounding environment data by combining oblique photography and close-range photography. This enables comprehensive acquisition of water surface, revetment, and river and lake surrounding environment data, avoiding blind spots from affecting the construction of the 3D scene model. At the same time, the close-range photography switches based on height-based occlusion detection, enabling accurate and comprehensive acquisition of water surface, revetment, and river and lake surrounding environment data, thus improving the accuracy of 3D scene model construction.

[0026] 3. Before constructing the 3D scene model, this invention obtains the land planning information of the region from the Land and Resources Administration Bureau, and when there is a land change plan in the short term, it achieves local rapid update and replacement during the later changes by assigning different identifiers, thereby reducing the amount of data processing during subsequent 3D scene model updates. Attached Figure Description

[0027] Figure 1 This is a flowchart of the method for constructing an integrated three-dimensional scene above and below water based on multi-source data according to the present invention. Detailed Implementation

[0028] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0029] Please see Figure 1 The present invention provides the following technical solution: a method for constructing an integrated three-dimensional scene above and below water based on multi-source data, comprising the following steps:

[0030] S1: Determine the drone photography scheme and the unmanned vessel navigation scheme based on the 3D scene area construction;

[0031] S2: Determine the drone's motion trajectory and the unmanned vessel's navigation trajectory through drone photography schemes and unmanned vessel navigation schemes;

[0032] S3: The drone is equipped with a central orthogonal angle camera and four cameras with optical axes at 45° angles to the horizontal plane. It performs oblique photography according to a determined motion trajectory. During the shooting process, if an object occludes the oblique shooting area, the location and area of ​​the object occlusion area are determined and transmitted to the control terminal. The drone waits for the control terminal to determine the close-up photography motion trajectory of the drone and transmit it back to the drone. The drone performs close-up photography according to the determined motion trajectory until the close-up photography of the occluded object is completed. It then returns to the oblique photography motion trajectory to continue shooting to obtain environmental data of the water surface, revetment and surrounding rivers and lakes.

[0033] The object occlusion detection is height-based and includes the following steps: Determining whether a given object point is visible on the image; along its search path, comparing the elevation of each object point along the path with the elevation of the corresponding projection beam. If the elevation of any point along the search path is higher than the projection beam height, the object point is occluded during imaging and not captured; otherwise, it is not occluded. The formula for calculating the projection beam elevation is:

[0034]

[0035] In the formula: i is the i-th point on the search path, d is the search step size on the search path, (x s y s , z s ) and (x o y o , z o () represents the projection center and the point to be matched;

[0036] S4: The unmanned surface vessel is equipped with a multibeam echo sounder and a GNSS-RTK receiver to acquire underwater riverbed data by moving along a defined navigation path;

[0037] S5: The images acquired by UAV oblique photography and close-range photography are preprocessed and then subjected to aerial triangulation to obtain an analysis report. The accuracy is then assessed, and the reprojection error is checked to see if it meets the accuracy requirements. If it does, the planning of the 3D scene construction area by the Land and Resources Bureau is obtained. If there is no land planning change plan in the 3D scene construction area, the oblique photography and close-range photography image data are assigned a unified identifier. If there is a land planning change plan in the short term in the 3D scene construction area, the oblique photography and close-range photography image data with the change plan are assigned a first identifier, and the image data without the change plan are assigned a second identifier. Then, the oblique photography and close-range photography image data with the first and second identifiers are used to simultaneously generate point cloud data and 3D models of the water surface, revetment, and surrounding environment of rivers and lakes. If the requirements are not met, the UAV is re-measured.

[0038] S6: The underwater riverbed data acquired by the unmanned surface vessel equipped with a multibeam echo sounder and GNSS-RTK receiver is first preprocessed and then subjected to multi-profile analysis to obtain an analysis report. The accuracy is then judged. If the requirements are met, point cloud registration is performed using a combination of manual coarse registration and automatic fine registration using the ICP point cloud matching algorithm. At the same time, an underwater riverbed 3D model is generated using the acquired underwater riverbed data. If the requirements are not met, the unmanned surface vessel is re-measured.

[0039] The ICP point cloud matching algorithm specifically includes the following steps: First, acquire feature points from two point sets; perform data matching based on the feature points; and set these matched points as hypothetical corresponding points. Then, solve for motion parameters based on this correspondence. Finally, use these parameters to perform data transformation, which aims to make the objective function S... 2 Minimize, the calculation formula is as follows:

[0040]

[0041] In the formula: N is the number of iterations, Q i P is a reference point in the point cloud. i R and t are the corresponding points in the target point cloud, and R and t are the rotation and translation matrices that need to be calculated.

[0042] S7: Simultaneously import the UAV oblique photography and close-range photography data and the registered multibeam echo sounder 3D point cloud data into ContextCapture 3D modeling software, and use the software to generate an integrated 3D scene model of the surface and underwater.

[0043] S8: Compare the accuracy of the generated 3D models of the water surface, revetment, and surrounding environment of rivers and lakes, and the underwater riverbed with the generated integrated 3D scene model of the water surface and underwater. If the accuracy is the same, output the integrated 3D scene model of the water surface and underwater to the display terminal for visualization. If the accuracy is different, adjust and repeat steps S5-S7 until the accuracy is the same and output the integrated 3D scene model of the water surface and underwater. During the display of the integrated 3D scene model of the water surface and underwater, if local areas need to be changed due to changes in land planning, use a drone equipped with a central orthophoto camera and four cameras with optical axes at 45° angles to the horizontal plane to acquire image data and generate point cloud data with a third identifier. Update the point cloud data with the first identifier to the corresponding associated point cloud data with the third identifier, reconstruct the model, and then display it.

[0044] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for constructing an integrated 3D scene above and below water based on multi-source data, characterized in that, Includes the following steps: S1: Determine the drone photography scheme and the unmanned vessel navigation scheme based on the 3D scene area construction; S2: Determine the drone's motion trajectory and the unmanned vessel's navigation trajectory through drone photography schemes and unmanned vessel navigation schemes; S3: The drone is equipped with a central orthogonal angle camera and four cameras whose optical axes are at a 45° angle to the horizontal plane. It moves along a defined trajectory to acquire data on the water surface, revetment, and surrounding environment of rivers and lakes. S4: The unmanned surface vessel is equipped with a multibeam echo sounder and a GNSS-RTK receiver to acquire underwater riverbed data by moving along a defined navigation path; S5: The images acquired by the UAV oblique photography and close-range photography are preprocessed and then aerial triangulation is performed to obtain an analysis report, determine its accuracy, and then determine whether the reprojection error meets the accuracy requirements. If the requirements are met, point cloud data and 3D models of the water surface, revetment and surrounding environment of rivers and lakes are generated simultaneously from the oblique photography and close-range photography images. If the requirements are not met, the UAV is re-measured. S6: The underwater riverbed data acquired by the unmanned surface vessel equipped with a multibeam echo sounder and GNSS-RTK receiver is first preprocessed and then subjected to multi-profile analysis to obtain an analysis report. The accuracy is then judged. If the requirements are met, point cloud registration is performed using a combination of manual coarse registration and automatic fine registration using the ICP point cloud matching algorithm. At the same time, an underwater riverbed 3D model is generated using the acquired underwater riverbed data. If the requirements are not met, the unmanned surface vessel is re-measured. S7: Simultaneously import the UAV oblique photography and close-range photography data and the registered multibeam echo sounder 3D point cloud data into ContextCapture 3D modeling software, and use the software to generate an integrated 3D scene model of the surface and underwater. S8: Compare the accuracy of the generated 3D models of the water surface, revetment, and surrounding environment of the river and lake, and the underwater riverbed with the generated integrated 3D scene model of the water surface and underwater. If the accuracy is the same, output the integrated 3D scene model of the water surface and underwater to the display terminal for visualization. If the accuracy is different, adjust and repeat steps S5-S7 until the accuracy is the same and output the integrated 3D scene model of the water surface and underwater.

2. The method for constructing an integrated three-dimensional scene above and below water based on multi-source data according to claim 1, characterized in that: In step S3, the drone photography includes the following steps: the drone performs oblique photography according to a determined motion trajectory. During the shooting process, if an object occludes the oblique shooting area, the location and area of ​​the object occlusion area are determined and transmitted to the control terminal. The drone waits for the control terminal to determine the close-up photography motion trajectory of the drone and transmit it back to the drone. The drone performs close-up photography according to the determined motion trajectory until the close-up photography of the occluded object is completed and it returns to the oblique photography motion trajectory.

3. The method for constructing an integrated three-dimensional scene above and below water based on multi-source data according to claim 2, characterized in that: In step S3, object occlusion detection is based on height and specifically includes the following steps: determining whether a certain object point is visible on the image; on its search path, sequentially comparing the elevation of the object point along the search path with the elevation of the projection beam at the corresponding position of the object point; if the elevation of any point on the search path is higher than the height of the projection beam, then the object point is occluded during the imaging process and is not imaged; otherwise, there is no occlusion. The formula for calculating the projection beam elevation is: In the formula: i is the i-th point on the search path, d is the search step size on the search path, (x s y s , z s ) and (x o y o , z o () represents the projection center and the point to be matched.

4. The method for constructing an integrated three-dimensional scene above and below water based on multi-source data according to claim 1, characterized in that: In step S5, before generating point cloud data of the water surface, revetment, and surrounding environment of rivers and lakes, the planning of the 3D scene construction area by the Land and Resources Administration Bureau is obtained. If there is no land planning change plan in the 3D scene construction area, the oblique photography and close-range photography image data are assigned a unified identifier. If there is a land planning change plan in the short term in the 3D scene construction area, the oblique photography and close-range photography image data with the change plan are assigned a first identifier, and the image data without the change plan are assigned a second identifier. Then, point cloud data is generated by using the oblique photography and close-range photography image data assigned with the first identifier and the second identifier.

5. The method for constructing an integrated three-dimensional scene above and below water based on multi-source data according to claim 1, characterized in that: In step S6, the ICP point cloud matching algorithm specifically includes the following steps: first, acquire feature points from two point sets; perform data matching based on the feature points; and set these matched points as hypothetical corresponding points. Then, solve for motion parameters based on this correspondence. Finally, use these parameters to perform data transformation, which is to make the objective function S through the corresponding points. 2 Minimize, the calculation formula is as follows: In the formula: N is the number of iterations, Q i P is a reference point in the point cloud. i R and t are the corresponding points in the target point cloud, and R and t are the rotation and translation matrices that need to be calculated.

6. The method for constructing an integrated three-dimensional scene above and below water based on multi-source data according to claim 1, characterized in that: In step S8, during the display of the integrated three-dimensional scene model above and below water, when a local area needs to be changed due to changes in land planning, image data is acquired by a drone equipped with a central orthophoto camera and four cameras whose optical axes are at a 45° angle to the horizontal plane to generate point cloud data with a third identifier. The point cloud data with the first identifier is then updated to the corresponding associated point cloud data with the third identifier for model reconstruction and display.

Images