Spatial three-dimensional constraint augmented reality three-dimensional modeling method and system

By arranging multiple image control points at different heights of the target area and performing data processing, the problem of low accuracy and accuracy of real scene three-dimensional modeling in the prior art is solved, and higher model graphing accuracy and accuracy are achieved.

CN119579810AInactive Publication Date: 2025-03-07SUZHOU CHENGFANG INFORMATION TECH CO LTD
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202510141460.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-02-08
Publication Date
2025-03-07
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

In the existing three-dimensional modeling methods of real scenes, the model graphing accuracy and accuracy are low, resulting in problems such as warping, deformation, and positional offset of the model.

Method used

By arranging at least four image control points in a plane-like distribution at different heights of the target area, the point coordinate data and image data of these points are obtained, and internal data processing is carried out to obtain a real-life three-dimensional model.

Benefits of technology

The accuracy and accuracy of model drawing are improved, and the possibility of warping, deformation, position shifting and other problems of three-dimensional models caused by accuracy problems are reduced.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119579810A_ABST
    Figure CN119579810A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of unmanned aerial vehicle photogrammetry, and particularly provides a spatial three-dimensional constraint augmented live-action three-dimensional modeling method and system, and the method comprises the steps: arranging at least four image control points in a target region; wherein the image control points are in planar distribution in the target area, and the image control points are arranged at different heights relative to the reference points, so that the geometric accuracy of the live-action three-dimensional model of the target area is restrained through the image control points; acquiring point coordinate data of the photo control point; acquiring image data of the target area according to the image control point; and performing interior work data processing according to the point coordinate data and the image data to obtain the live-action three-dimensional model. According to the method, the precision and accuracy of model mapping can be effectively improved, and the problems of warping, deformation, position offset and the like of the model are avoided.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicle photogrammetry, and in particular to a method and system for spatial stereo constraint enhanced real-scene three-dimensional modeling. Background Art

[0002] Obtaining oblique images through drones is one of the important methods for building real-life 3D models. Although the cameras carried by drones have high resolution, the installation position and angle of the oblique cameras will affect the quality and geometric accuracy of the captured images. If the camera installation position is unstable or the angle setting is inaccurate, the captured images may be distorted or offset, thus affecting the geometric accuracy of the real-life 3D model. In the process of oblique photogrammetry, since multiple cameras shoot at the same time and generate a large amount of data, errors may accumulate during the processing and fusion of these data. The accumulated errors will cause the geometric accuracy of the real-life 3D model to gradually decrease, which in turn will cause problems such as warping, deformation, and position offset in the model. Summary of the invention

[0003] The spatial stereo constraint enhanced real-scene 3D modeling method and system provided in the embodiments of the present invention at least solve the problem of low model drawing precision and accuracy in the existing real-scene 3D modeling methods, effectively improve the precision and accuracy of model drawing, and avoid problems such as warping, deformation, and position offset of the model.

[0004] In a first aspect, the present invention provides a method for spatial stereo constraint enhanced real-scene three-dimensional modeling, comprising arranging at least four image control points in a target area; wherein the image control points are distributed in a planar manner in the target area, and relative to a reference point, the image control points are arranged at different heights, so as to constrain the geometric accuracy of the real-scene three-dimensional model of the target area through the image control points; obtaining point coordinate data of the image control points; obtaining image data of the target area based on the image control points; performing internal data processing based on the point coordinate data and the image data to obtain the real-scene three-dimensional model.

[0005] In one embodiment of the present invention, at least four image control points are arranged in the target area, including: according to the accuracy requirements of the target model, the image control point arrangement positions are designed to obtain first position data; wherein the image control points include conventional image control points and constrained image control points, and the conventional image control points and the constrained image control points are arranged at different heights; the first position data includes four corner arrangement points and at least one internal arrangement point, and the corner arrangement points and the internal arrangement points are evenly distributed in the target area in a planar manner; the four corner arrangement points are respectively arranged at the four corners of the square target area, and the .... The points are used to arrange the conventional image control points; the internal arrangement points are traversed, and each of the internal arrangement points is updated to a constraint arrangement point, or updated to a conventional arrangement point, to obtain second position data; wherein the constraint arrangement points are used to arrange the constraint image control points, and the conventional arrangement points are used to arrange the conventional image control points, and along two horizontal directions perpendicular to each other in the target area, the points adjacent to the constraint arrangement points are the conventional arrangement points, or the corner arrangement points; according to the second position data, the conventional image control points and the constraint image control points are arranged in the target area.

[0006] In one embodiment of the present invention, traversing the internal layout points, updating each of the internal layout points to a constraint layout point, or updating it to a regular layout point, and obtaining the second position data, includes: traversing the internal layout points, and determining whether the current internal layout point satisfies a constraint condition; wherein the constraint condition is that, along two horizontal directions perpendicular to each other in the target area, the points adjacent to the current internal layout point are the corner layout points, or regular layout points, or unupdated internal layout points; if the current internal layout point satisfies the constraint condition, updating the current internal layout point to the constraint layout point, and determining the next internal layout point; if the current internal layout point does not satisfy the constraint condition, updating the current internal layout point to the regular layout point, and determining the next internal layout point; completing the traversal, and obtaining the second position data.

[0007] In one embodiment of the present invention, at least part of the image control points are arranged at preset heights, and the preset heights include building roof surfaces, building corners and building sides in the target area.

[0008] In one embodiment of the present invention, internal data processing is performed according to the point coordinate data and the image data to obtain the real-scene three-dimensional model, including: performing aerial triangulation processing on the image data according to the point coordinate data to obtain aerial triangulation image data; wherein the aerial triangulation processing includes free network aerial triangulation processing, image control point processing and regional network aerial triangulation processing; performing model processing on the aerial triangulation image data to obtain the real-scene three-dimensional model; wherein the model processing includes point cloud generation, mesh construction and texture mapping.

[0009] In one embodiment of the present invention, the image data is subjected to aerial triangulation processing according to the point coordinate data to obtain aerial triangulation image data, including: preprocessing the image data to obtain a preprocessed image; subjecting the preprocessed image to the free network aerial triangulation processing, extracting and matching feature points, and calculating the exterior orientation element result; subjecting the image control point puncture processing according to the point coordinate data and the exterior orientation element result to add the image control points to the preprocessed image to determine the absolute position and attitude position of the image, and obtain an image control point image; subjecting the image control point image to the regional network aerial triangulation processing to obtain the aerial triangulation image data; wherein the aerial triangulation image data includes encrypted image control point coordinate data; expressed as: , , , , in, is the image point coordinate matrix, is the control point coordinate matrix, is the self-calibration parameter matrix, is the correction matrix of the camera station coordinates, is the encrypted pixel coordinate increment, is the increment of the image exterior orientation element, For self-calibration parameters, is the antenna phase center deviation, is the drift error correction number, In the formula The coefficient matrix of In the formula The coefficient matrix of are all unit matrices, are all constant matrices, They are all power arrays.

[0010] In one embodiment of the present invention, image data of the target area is obtained according to the image control points, including: designing a UAV route and generating a flight mission according to target mapping requirements; wherein the target mapping requirements include image control point distribution and target model accuracy requirements; and according to the flight mission, controlling the UAV to collect flight data of the target area to obtain the image data.

[0011] In one embodiment of the present invention, it also includes: performing quality inspection on the current image data; wherein the quality inspection includes image shadow and reflection inspection, image ghosting and blur inspection, image loss inspection, image cloud penetration inspection, image loss inspection, and image damage inspection; determining whether the current image data passes the quality inspection; if the image data fails the quality inspection, generating a new flight mission and acquiring new image data.

[0012] In one embodiment of the present invention, obtaining the point coordinate data of the image control point includes: simultaneously receiving the satellite signal of the first target satellite and the satellite signal of the second target satellite through the base station and the mobile station to obtain the observation value; wherein the mobile station is set at the image control point, and the observation value includes a first base observation value, a second base observation value, a first mobile observation value and a second mobile observation value, the first base observation value is the observation value of the base station on the first target satellite, the second base observation value is the observation value of the base station on the second target satellite, the first mobile observation value is the observation value of the mobile station on the first target satellite, and the second mobile observation value is the observation value of the mobile station on the second target satellite; according to the observation values, calculating the first station difference and the second station difference; wherein the first station difference is the station difference between the first target satellite and the base station and the mobile station, and the second station difference is the station difference between the second target satellite and the base station and the mobile station; expressed as: , , The first station difference The first target satellite The inter-station difference between the base station and the rover, The first target satellite The observation equation with the first flow observation, The first target satellite The observation equation with the first benchmark observation value; the second station difference is Second target satellite The inter-station difference between the base station and the rover, The second target satellite The observation equation with the second flow observation, The second target satellite The observation equation with the second reference observation value, is the satellite epoch; According to the first station difference and the second station difference, a double difference is calculated. ; expressed as: , Calculating and obtaining the point coordinate data of the current image control point according to the double difference; or, The satellite signal of the target satellite is received simultaneously by a base station and a mobile station to obtain an observation value; wherein the mobile station is set at the image control point; the observation values ​​are linearly combined to obtain a virtual carrier phase observation value; based on the coordinate data of the base station and the virtual carrier phase observation value, the point coordinate data of the current image control point is calculated.

[0013] In the second aspect, the present invention also provides a spatial stereo constraint enhanced real-scene three-dimensional modeling system, which is applied to the spatial stereo constraint enhanced real-scene three-dimensional modeling method as described in any one of the above; the spatial stereo constraint enhanced real-scene three-dimensional modeling system includes a layout module, which is used to layout at least four image control points in the target area; wherein the image control points are distributed in a planar manner in the target area, and relative to the reference point, the image control points are arranged at different heights to constrain the geometric accuracy of the real-scene three-dimensional model of the target area through the image control points; a coordinate acquisition module, which is used to acquire the point coordinate data of the image control points; an image acquisition module, which is used to acquire the image data of the target area according to the image control points; a model construction module, which is used to perform internal data processing according to the point coordinate data and the image data to obtain the real-scene three-dimensional model.

[0014] The above technical solution of the present invention has the following beneficial effects compared with the prior art: The spatial stereo constraint enhanced real-scene 3D modeling method and system described in the present invention can constrain the geometric accuracy of the real-scene 3D model corresponding to the target area, especially the geometric position accuracy of the high-level model, by arranging at least four image control points distributed in a planar manner at different heights in the target area. The geometric state of the model itself is restored, the accuracy and precision of the model drawing are improved, and the possibility of warping, deformation, position offset and other problems of the 3D model caused by accuracy problems is reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other embodiments can be obtained based on these drawings without creative work. In the drawings:

[0016] Figure 1 It is a flow chart of a method for enhancing real-scene three-dimensional modeling by spatial stereo constraints in a preferred embodiment of the present invention.

[0017] Figure 2 It is a structural schematic diagram of a spatial stereo constraint enhanced real-scene three-dimensional modeling system in a preferred embodiment of the present invention.

[0018] The above drawings include the following reference numerals:

[0019] 10. Layout module; 20. Coordinate acquisition module; 30. Image acquisition module; 40. Model building module. DETAILED DESCRIPTION

[0020] Embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present invention. It should be understood that the drawings and embodiments of the present invention are only for exemplary purposes and are not intended to limit the scope of protection of the present invention.

[0021] In order to solve the problem of low model drawing precision and accuracy of existing real-scene 3D modeling methods, Figure 1 As shown, the present invention provides a spatial stereo constraint enhanced real scene 3D modeling method. The spatial stereo constraint enhanced real scene 3D modeling method comprises:

[0022] At least four image control points are arranged in the target area, wherein the image control points are distributed in a planar manner in the target area and are arranged at different heights relative to the reference points, so as to constrain the geometric accuracy of the real-scene three-dimensional model of the target area through the image control points.

[0023] Get the point coordinate data of the image control point.

[0024] According to the image control points, the image data of the target area is obtained.

[0025] Based on the point coordinate data and image data, internal data processing is performed to obtain a real-scene three-dimensional model.

[0026] The target area is the area where a real - scene 3D model needs to be established. Preferably, the target area is an urban - level area, and correspondingly, the real - scene 3D model is an urban - level oblique 3D model.

[0027] The layout of image control points is the core of the method. In the existing - technology modeling methods, image control points are usually laid on the ground, and ground control points are collected for in - house data processing. Due to the influence of many factors such as measuring instruments, data - processing algorithms, and model - reconstruction methods, there are certain errors in the positions of the constructed model points, and the propagation of these errors among the points in the model matching is cumulative. That is, as the distance from the position of the image control point increases, the constraint effect of the control point will gradually weaken, and the error will gradually amplify. Therefore, in the real - scene 3D model obtained by using the existing - technology modeling methods, the accuracy of the high - level position is often much lower than that of the ground position, which in turn leads to low mapping accuracy and accuracy of the model.

[0028] It can be understood that three points form a plane. In the present invention, at least four image control points are distributed in a planar shape in the target area, that is, at least four image control points are not collinear.

[0029] In the actual application process, the specific number of image control points is related to the area of the target area and the accuracy requirements of the target model. The accuracy requirement of the target model is the ground resolution requirement of the model, that is, the size of the ground object represented by each pixel in the model. For example, the model accuracy is 3 cm, 5 cm, etc.

[0030] When the area of the target area remains unchanged, the higher the accuracy requirement of the target model, the more image control points are collected. Correspondingly, the distance between each image control point is closer. Similarly, when the accuracy requirement of the target model remains unchanged, the larger the area of the target area, the more points need to be collected while keeping the distance between the image control points within a certain range unchanged. Exemplarily, for a 3 - cm accuracy requirement and an area of dozens of square kilometers, the number of image control points required can reach hundreds.

[0031] Relative to the reference point, these image control points are respectively laid at different heights. The reference point is the point of the model coordinate system of the corresponding real - scene 3D model. Preferably, the model coordinate system is set to the CGCS2000 coordinate system (China Geodetic Coordinate System 2000) or a local coordinate system.

[0032] Exemplarily, among the laid image control points, some of the image control points are laid on the ground of the target area, and some of the image control points are laid at a preset height. Taking four image control points as an example, the target area is divided into a "field" - shaped grid, and the four image control points are respectively set in one of the four grids of the "field" - shaped grid. Two diagonal image control points are both laid on the ground or both laid at a preset height.

[0033] The preset heights include building roof surfaces, building corners, and building sides in the target area.

[0034] Preferably, the roof of the building is selected as the preset height. For buildings where it is impossible to collect image control points on the roof, replaceable points are selected near the building for collection, and building corner points can also be collected by a total station.

[0035] By selecting the building roof surface as the preset height, elevation errors are less likely to occur during the subsequent image control point processing, effectively reducing the possibility of artificially increasing the error accuracy of the image control points, thereby ensuring high model mapping accuracy.

[0036] By arranging these image control points, the geometric accuracy of the real-life 3D model corresponding to the target area can be constrained, especially the geometric position accuracy of the model's high-level layers, thereby changing the model's physical state and achieving the purpose of enhancing the effect. The geometric state of the model itself is restored, the accuracy and precision of the model drawing are improved, and the possibility of warping, deformation, positional offset and other problems of the 3D model caused by accuracy problems is reduced. Among them, the physical state of the model includes position, shape and structure.

[0037] Before arranging the image control points, it is necessary to design the arrangement of the image control points. Preferably, when designing, the corresponding image control points can be automatically generated or manually adjusted.

[0038] After completing the layout of the image control points, it is necessary to obtain the point coordinate data of the image control points to prepare for subsequent internal data processing. Preferably, the point coordinate data of the image control points is collected in the field. After obtaining the point coordinate data, the point coordinate data can be imported through certain structural rules, or added by manually adding longitude, latitude, and elevation.

[0039] In addition, image data of the target area is obtained according to the image control points. Preferably, the image data is collected by using an unmanned aerial vehicle in the field.

[0040] Finally, according to the point coordinate data and image data, the internal data processing is performed to obtain the real scene 3D model. Those skilled in the art can perform corresponding internal data processing according to actual needs, so as to obtain real scene 3D models of different formats, including but not limited to DOM, DSM, point cloud, 3Dtiles, obj and other format types. Exemplarily, orthophoto image data within the modeling range can be produced by DOM, and the model is in TIFF format; digital surface model within the modeling range can be produced by DSM, and the model is in TIFF format; point cloud data within the modeling range can be produced by point cloud, and the model is in LAS format.

[0041] The spatial stereo constraint enhanced real-scene 3D modeling method of the present invention can constrain the geometric accuracy of the real-scene 3D model corresponding to the target area, especially the geometric position accuracy of the high-level model, by arranging at least four image control points distributed in a planar manner at different heights in the target area. The geometric state of the model itself is restored, the accuracy and precision of the model drawing are improved, and the possibility of warping, deformation, position offset and other problems of the 3D model caused by accuracy problems is reduced.

[0042] In some embodiments of the spatial stereo constraint enhanced real scene 3D modeling method of the present invention, the image control points include conventional image control points and constrained image control points, and the conventional image control points and the constrained image control points are arranged at different heights. Among them, the conventional image control points are arranged on the ground of the target area, and the constrained image control points are arranged at a preset height. The preset height includes the roof surface, building corners and building sides in the target area.

[0043] At least four image control points are deployed in the target area, including:

[0044] Firstly, according to the accuracy requirement of the target model, the layout positions of the image control points are designed to obtain the first position data.

[0045] The first position data includes four corner layout points and at least one internal layout point, and the corner layout points and the internal layout points are evenly distributed in the target area.

[0046] Specifically, four corner layout points are respectively set at the four corners of the square target area, and the corner layout points are used to layout conventional image control points. Different internal layout points are used to set different image control points.

[0047] Preferably, when the initial target area is not a square, corresponding processing is performed to obtain a square target area. It should be noted that when only one internal layout point is set, the internal layout point is set at the geometric center of the target area.

[0048] Secondly, the internal layout points are traversed, and each internal layout point is updated to a constraint layout point or a regular layout point to obtain second position data.

[0049] Among them, the constraint layout point is used to layout the constraint image control point, and the conventional layout point is used to layout the conventional image control point. Along two mutually perpendicular horizontal directions in the target area, the points adjacent to the constraint layout point are conventional layout points or corner layout points.

[0050] The two horizontal directions perpendicular to each other in the target area can be set according to actual needs, for example, set to be along the length direction and the width direction of the square.

[0051] It can be imagined that along two mutually perpendicular horizontal directions in the target area, there are usually four points adjacent to the internally arranged points; in some cases, there are only three adjacent points.

[0052] Preferably, along two mutually perpendicular horizontal directions in the target area, image control points adjacent to each constrained image control point are regular image control points, and image control points adjacent to each regular image control point are constrained image control points, so as to achieve the best constraint enhancement effect.

[0053] Finally, according to the second position data, conventional image control points and constrained image control points are arranged in the target area.

[0054] By arranging the corresponding conventional image control points and constraint image control points, the geometric accuracy of the real-life 3D model corresponding to the target area, especially the geometric position accuracy of the model's high-level layers, can be constrained. This achieves the restoration of the model's own geometric state, improves the accuracy and precision of the model's mapping, and reduces the possibility of warping, deformation, positional offset, and other problems in the 3D model caused by accuracy issues.

[0055] Further, in some embodiments of the spatial stereo constraint enhanced real scene 3D modeling method of the present invention, traversing the internal layout points, updating each internal layout point to a constraint layout point, or updating to a regular layout point, to obtain second position data, includes:

[0056] Traverse the internal layout points to determine whether the current internal layout points meet the constraints.

[0057] The constraint condition is that, along two mutually perpendicular horizontal directions in the target area, the points adjacent to the current internal layout points are corner layout points, regular layout points, or unupdated internal layout points. It is sufficient to ensure that the points adjacent to the current internal layout points are not constraint layout points.

[0058] If the current internal layout point meets the constraint condition, the current internal layout point is updated to the constraint layout point, and the next internal layout point is determined.

[0059] If the current internal layout point does not meet the constraint conditions, the current internal layout point is updated to a regular layout point, and the next internal layout point is determined.

[0060] After the traversal is completed, the second position data is obtained.

[0061] Preferably, when traversing the internal layout points, screening is performed in sequence in an alternate manner, which can quickly and effectively achieve the update of the internal layout points and improve efficiency.

[0062] It should be noted that in some other embodiments, adjacent constraint image control points can also be arranged. However, from the perspective of practical application, adjacent, high-to-high combination of constraint image control points will make the constraints redundant.

[0063] The adjacent, high-low combination of conventional image control points and constrained image control points can well achieve the effect of high-level constraints on the model. This layout method can not only improve the accuracy of the model, but also effectively improve efficiency and reduce costs.

[0064] Of course, in practical applications, the accuracy of the ground model still needs to be controlled first, so adjacent image control points can all be conventional image control points.

[0065] The spatial stereo constraint enhanced real-scene 3D modeling method described in the present invention, in some embodiments, can measure image control points through differential RTK technology to obtain point coordinate data of the image control points.

[0066] Among them, differential RTK technology does not refer to a single technology. It includes real-time differential RTK (Real-time kinematic, real-time dynamic carrier phase differential technology) and post-differential PPK (post processed kinematic, positioning technology using carrier phase for post-differentiation).

[0067] Exemplarily, obtaining point coordinate data of image control points by real-time differential RTK technology includes:

[0068] First, the base station and the rover receive simultaneously (for example, both The base station and the mobile station belong to the prior art, and the specific working principles of the two are not described in detail. The mobile station is set at the image control point.

[0069] The observation values ​​include a first reference observation value, a second reference observation value, a first mobile observation value, and a second mobile observation value. The first reference observation value is the observation value of the reference station on the first target satellite, the second reference observation value is the observation value of the reference station on the second target satellite, the first mobile observation value is the observation value of the mobile station on the first target satellite, and the second mobile observation value is the observation value of the mobile station on the second target satellite.

[0070] Secondly, based on the observed values, we calculate Time difference between the first stop The difference between the second station , expressed as:

[0071] ,

[0072] ,

[0073] The first station difference The first target satellite The inter-station difference between the base station and the rover, The first target satellite The observation equation with the first flow observation, The first target satellite The observation equation with the first benchmark observation value. The second station difference is Second target satellite The inter-station difference between the base station and the rover, The second target satellite The observation equation with the second flow observation, The second target satellite The observation equation with the second reference observation value, is the satellite epoch.

[0074] Then, the double difference is calculated based on the difference between the first station and the difference between the second station. , expressed as:

[0075] .

[0076] Finally, according to the double difference , calculate the point coordinate data of the current image control point. Calculating the point coordinate data of the current image control point belongs to common knowledge of those skilled in the art and will not be described in detail.

[0077] Through real-time differential RTK technology, the signals received by two or more receivers can be compared, the differences between them can be calculated, and these differences can be applied to the positioning calculation, thereby eliminating the influence of errors such as ephemeris error, satellite clock error, ionosphere error, tropospheric error and multipath error, thereby improving positioning accuracy.

[0078] Alternatively, the point coordinate data of the image control point obtained by the post-differentiation PPK technology includes:

[0079] First, the base station and the mobile station simultaneously receive the satellite signals of the target satellite to obtain the observation value. Among them, the mobile station is set at the image control point.

[0080] Secondly, the observations are linearly combined to obtain virtual carrier phase observations. Based on the virtual carrier phase observations, the relative position between the mobile station and the base station can be calculated.

[0081] Finally, based on the virtual carrier phase observations and the known coordinate data of the base station, the precise three-dimensional coordinates of the mobile station can be calculated, and the point coordinate data of the current image control point can be obtained.

[0082] The post-differential PPK technology can also achieve the effect of improving positioning accuracy.

[0083] In some embodiments of the spatial stereo constraint enhanced real scene 3D modeling method of the present invention, obtaining image data of a target area according to image control points includes:

[0084] Design UAV routes and generate flight missions based on target mapping requirements.

[0085] Among them, the target mapping requirements include the distribution of image control points and the accuracy requirements of the target model, and the distribution of image control points and the accuracy requirements of the target model are known quantities. In addition, the target mapping requirements also include model results requirements, including but not limited to DOM, DSM, point cloud, 3Dtiles, obj and other format types. For example, orthophoto image data within the modeling range can be produced through DOM, and the model is in TIFF format; digital surface models within the modeling range can be produced through DSM, and the model is in TIFF format; point cloud data within the modeling range can be produced through point cloud, and the model is in LAS format.

[0086] The design of the drone route includes setting the drone flight altitude, image ground resolution, camera parameters, heading overlap, lateral overlap, photo spacing and route spacing. Of course, it is also necessary to select a suitable low-altitude photogrammetry drone equipment model. The ground resolution of the collected image is determined by the flight altitude combined with the camera parameter performance of the drone. Exemplarily, the acquisition equipment carried by the drone can be a single-lens or five-lens aerial survey camera.

[0087] The flight mission includes the setting and checking of the flight altitude, speed, margin, heading overlap rate and lateral overlap rate.

[0088] Before collecting flight data, it is also necessary to check the drone status, flight environment and weather conditions.

[0089] The drone status includes the camera photo ratio, white balance and other parameters of the drone, as well as the setting and checking of the battery power, storage card status and gimbal status.

[0090] The flight environment and weather conditions include checks on ground flatness, obstructions, wind speed and direction, electromagnetic interference, ground control and no-fly zones.

[0091] Then, according to the flight mission, the UAV is controlled to collect flight data of the target area to obtain image data.

[0092] It is worth noting that after completing the flight data collection and obtaining the image data, the quality of the current image data needs to be tested.

[0093] Among them, quality inspection includes image shadow and reflection inspection, image ghosting and blur inspection, image loss inspection, image cloud penetration inspection, image loss inspection, and image damage inspection. Taking image ghosting as an example, when shadows or ghosting appear in many images, then in the corresponding area of ​​the real-life 3D model generated by modeling, there will also be large shadows or ghosting effects, affecting the modeling effect.

[0094] Therefore, it is necessary to determine whether the current image data has passed the quality inspection. If the image data has not passed the quality inspection, a new flight mission is generated and new image data is acquired. When generating a new flight mission, the drone route should be consistent with the original route.

[0095] In some embodiments, the spatial stereo constraint enhanced real scene 3D modeling method of the present invention performs office data processing based on point coordinate data and image data to obtain a real scene 3D model, including:

[0096] According to the point coordinate data, the image data is processed by aerial triangulation to obtain aerial triangulation image data.

[0097] Among them, aerial triangulation processing includes free network aerial triangulation processing, image control point puncture point processing and regional network aerial triangulation processing.

[0098] Specifically, the image data is processed by aerial triangulation according to the point coordinate data to obtain aerial triangulation image data, including:

[0099] First, the image data is preprocessed to obtain a preprocessed image. Preprocessing the image data is the basic step in building a tilted real-scene 3D model. By performing a series of processing on the original image data, such as overlap inspection, distortion correction, image stitching, image intensity processing, etc., it can reduce the noise and distortion in the image, improve the clarity and contrast of the image, and make the subsequent 3D modeling process more accurate and efficient.

[0100] Subsequently, the preprocessed image is subjected to free-net aerial triangulation processing. Free-net aerial triangulation processing can perform aerial triangulation solution on the preprocessed image, extract feature points, and associate and match the same feature points in different photos. According to the matching results of the feature points, the position and attitude of the camera are adjusted so that the error of the intersection of feature points in three-dimensional space is minimized. Finally, the exterior orientation element results can be output, which include the parameters of the camera position and attitude at the moment of photography, such as line elements, heading angle, pitch angle, and roll angle.

[0101] Next, image control point puncturing is performed according to the point coordinate data and the results of the exterior orientation elements, and image control points are added to the preprocessed image to determine the absolute position and attitude position of the image, and an image control point image is obtained.

[0102] By puncturing the image control points, the accurate correspondence between the image and the actual terrain can be ensured. After loading the image control points, the necessary control point information can be provided for aerial triangulation.

[0103] Finally, the image of the image control point is processed by regional network aerial triangulation and absolute orientation is performed to obtain aerial triangulation image data; wherein, the aerial triangulation image data includes the encrypted image control point coordinate data, including elevation and plane position.

[0104] For example, after completing the image control point puncture processing and adding each image control point, the beam method is used to analyze the regional network aerial triangulation so that the light rays of the common point can achieve the best intersection, thereby obtaining the encrypted image control point coordinate data, which is expressed as:

[0105] ,

[0106] ,

[0107] ,

[0108] ,

[0109] in, is the image point coordinate matrix, is the control point coordinate matrix, is the self-calibration parameter matrix, is the correction matrix of the camera station coordinates, is the encrypted pixel coordinate increment, is the increment of the image exterior orientation element, For self-calibration parameters, is the antenna phase center deviation, is the drift error correction number, In the formula The coefficient matrix of In the formula The coefficient matrix of are all unit matrices, are all constant matrices, They are all power arrays.

[0110] By taking the similar projection beam of each image as the adjustment unit and the collinear equation of the central projection as the basic equation of adjustment, the light rays of the common point can be optimally intersected and the entire area can be optimally incorporated into the known control point coordinate system. This can more conveniently take into account the influence of image system errors and is most convenient for introducing non-photogrammetric additional observation values.

[0111] Preferably, the plane and elevation error accuracy of the reprojection error control points need to be checked to ensure that they meet the requirements. If they do not meet the requirements, it is necessary to check the image control point punctures and detect whether the image control point accuracy meets the requirements.

[0112] If the error of the image control points exceeds the limit due to human error during the puncturing of the image control points, it is necessary to re-puncture the points and perform aerial triangulation. If the accuracy of the image control points does not meet the requirements, it is necessary to re-collect the point coordinate data of the image control points, and then re-puncture the points and perform aerial triangulation.

[0113] After obtaining the aerial triangulation image data, the aerial triangulation image data is subjected to model processing to obtain a real-scene three-dimensional model; wherein the model processing includes point cloud generation, mesh construction and texture mapping.

[0114] Of course, model processing is related to production parameters, including model coordinate system, modeling range, tile division, geometric accuracy selection, result type, and origin setting. For example, the model coordinate system is usually set to the CGCS2000 coordinate system or the local coordinate system; the design of the tile size needs to take into account the memory capacity of the server to ensure that the calculation will not fail or reduce efficiency due to insufficient memory during the modeling process.

[0115] On the other hand, referring to Figure 2 As shown, an embodiment of the present invention further provides a spatial stereo constraint enhanced real-scene 3D modeling system, which is applied to the spatial stereo constraint enhanced real-scene 3D modeling method as described in any one of the above embodiments.

[0116] The spatial stereo constraint enhanced real-scene three-dimensional modeling system comprises a layout module 10 , a coordinate acquisition module 20 , an image acquisition module 30 and a model construction module 40 .

[0117] The layout module 10 is used to layout at least four image control points in the target area, wherein the image control points are distributed in a planar manner in the target area and are arranged at different heights relative to the reference points, so as to constrain the geometric accuracy of the real-scene three-dimensional model of the target area through the image control points.

[0118] The coordinate acquisition module 20 is used to acquire point coordinate data of image control points.

[0119] The image acquisition module 30 is used to acquire image data of the target area according to the image control points.

[0120] The model building module 40 is used to perform internal data processing based on point coordinate data and image data to obtain a real-scene three-dimensional model.

[0121] The target area is the area where the real-scene 3D model needs to be established. Preferably, the target area is a city-level area, and accordingly, the real-scene 3D model is a city-level inclined 3D model.

[0122] The layout of image control points is the core of the method. In the modeling method of the prior art, image control points are usually laid out on the ground, and internal data processing is performed by collecting ground control points. Due to the influence of many factors such as measuring instruments, data processing algorithms, and model reconstruction methods, there are certain errors in constructing model points, and the propagation of these errors in the model matching points is cumulative. That is, as the distance from the image control point increases, the constraint effect of the control point will gradually weaken, and the error will gradually increase. Therefore, in the real-life three-dimensional model obtained by the modeling method of the prior art, the accuracy of the high-level position is often much lower than that of the ground position, which leads to low model mapping precision and accuracy.

[0123] It can be understood that three points constitute a surface. In the present invention, at least four image control points are distributed in a surface in the target area, that is, at least four image control points are not collinear.

[0124] In actual application, the specific number of image control points is related to the area of ​​the target area and the target model accuracy requirements. The target model accuracy requirements refer to the model ground resolution requirements, the size of the ground object represented by each pixel in the model, such as the model accuracy of 3cm, 5cm, etc.

[0125] When the area of ​​the target area remains unchanged, the higher the accuracy requirement of the target model, the more image control points are collected, and accordingly, the closer the distance between each image control point is. Similarly, when the accuracy requirement of the target model remains unchanged, the larger the area of ​​the target area, the more points need to be collected to control the distance between image control points to remain within a certain range. For example, the number of image control points required for a 3cm accuracy requirement and an area of ​​dozens of square kilometers can reach hundreds.

[0126] These image control points are arranged at different heights relative to the reference point. The reference point is a point in the model coordinate system of the corresponding real-scene 3D model. Preferably, the model coordinate system is set to the CGCS2000 coordinate system or the local coordinate system.

[0127] Exemplarily, among the surveyed image control points, some of them are arranged on the ground of the target area, and the other part of the image control points are arranged at a preset height. Taking four image control points as an example, the target area is divided into a "field" grid, and the four image control points are respectively set in one of the four grids of the "field" grid. Two diagonal image control points are both arranged on the ground or at a preset height.

[0128] Among them, the preset height includes the building roof surface, building corner points and building sides in the target area.

[0129] Preferably, the building roof surface is selected as the preset height. For buildings where it is impossible to collect image control points on the roof surface, replaceable points can be selected near the building for collection, and the building corner points can also be collected by total station.

[0130] By selecting the building roof surface as the preset height, during the subsequent image control point piercing process, it is not easy to generate elevation errors, effectively reducing the possibility of artificially increasing the error accuracy of image control points, and thus ensuring high model mapping accuracy and accuracy.

[0131] By arranging these image control points, the geometric accuracy of the real scene three-dimensional model corresponding to the target area can be constrained, especially the geometric position accuracy on the model height, thereby changing the physical state of the model and achieving the purpose of enhancing the effect. The restoration of the geometric state of the model itself is realized, the accuracy and accuracy of model mapping are improved, and the possibility of problems such as warping, deformation, and position deviation of the three-dimensional model caused by accuracy problems is reduced. Among them, the physical state of the model includes position, shape, and structure, etc.

[0132] Before arranging the image control points, it is necessary to carry out the layout design of the image control points. Preferably, during the design, the corresponding image control points can be automatically generated or manually adjusted.

[0133] After the arrangement of the image control points is completed, the point coordinate data of the image control points is obtained through the coordinate acquisition module 20 to prepare for subsequent in-house data processing. Preferably, the point coordinate data of the image control points is collected in the field. After obtaining the point coordinate data, the point coordinate data can be imported through a certain structural rule, or added by manually adding longitude, latitude, and elevation.

[0134] In addition, through the image acquisition module 30, the image data of the target area is obtained according to the image control points. Preferably, the image data is collected in the field by an unmanned aerial vehicle. Traverse the internal layout points

[0135] Finally, through the model construction module 40, in-house data processing is carried out according to the point coordinate data and the image data to obtain a real scene three-dimensional model.

[0136] Those skilled in the art can perform corresponding in-house data processing according to actual needs, so as to obtain real-scene 3D models of different formats, including but not limited to DOM, DSM, point cloud, 3Dtiles, obj, etc. For example, orthophoto data within the modeling range can be produced through DOM, and the model is in TIFF format; digital surface models within the modeling range can be produced through DSM, and the model is in TIFF format; point cloud data within the modeling range can be produced through point cloud, and the model is in LAS format.

[0137] The spatial stereo constraint enhanced real-scene 3D modeling system of the present invention can constrain the geometric accuracy of the real-scene 3D model corresponding to the target area, especially the geometric position accuracy of the high-level model, by arranging at least four image control points distributed in a planar manner at different heights in the target area. The geometric state of the model itself is restored, the accuracy and precision of the model drawing are improved, and the possibility of warping, deformation, position offset and other problems of the 3D model caused by accuracy problems is reduced.

[0138] It should be noted that the term "including" and its variations used in the embodiments of the present invention are open inclusions, that is, "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one other embodiment"; the term "some embodiments" means "at least some embodiments". The modifications of "one" and "multiple" mentioned in the embodiments of the present invention are illustrative and not restrictive. Those skilled in the art should understand that unless the context clearly indicates otherwise, it should be understood as "one or more".

[0139] The various steps described in the method implementation methods provided by the embodiments of the present invention may be performed in different orders and / or in parallel. In addition, the method implementation methods may include additional steps and / or omit the steps shown. The scope of protection of the present invention is not limited in this respect.

[0140] The term "embodiment" in this specification refers to specific features, structures or characteristics described in conjunction with the embodiment that can be included in at least one embodiment of the present invention. The appearance of this phrase in various places in the specification does not necessarily mean the same embodiment, nor does it mean that it is mutually exclusive with other embodiments and is independent or optional. The various embodiments in this specification are described in a related manner, and the same or similar parts between the various embodiments refer to each other. In particular, for the device, equipment, and system embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts refer to the partial description of the method embodiment.

[0141] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of protection. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be subject to the attached claims.

Claims

1. A spatial constraint enhanced real scene three-dimensional modeling method, characterized in that: include: At least four image control points are arranged in the target area; wherein the image control points are distributed in a planar manner in the target area, and are arranged at different heights relative to the reference point, so as to constrain the geometric accuracy of the real-scene three-dimensional model of the target area through the image control points; Acquire point coordinate data of the image control point; Acquiring image data of the target area according to the image control points; Based on the point coordinate data and the image data, internal data processing is performed to obtain the real-scene three-dimensional model.

2. The spatial constraint enhanced real scene 3D modeling method according to claim 1, characterized in that: At least four image control points are deployed in the target area, including: According to the accuracy requirements of the target model, the image control point layout positions are designed to obtain the first position data; wherein the image control points include conventional image control points and constrained image control points, and the conventional image control points and the constrained image control points are arranged at different heights; the first position data includes four corner layout points and at least one internal layout point, and the corner layout points and the internal layout points are evenly distributed in the target area in a planar manner; the four corner layout points are respectively set at the four corners of the square target area, and the corner layout points are used to layout the conventional image control points; Traversing the internal layout points, updating each of the internal layout points to a constraint layout point, or updating each of the internal layout points to a conventional layout point, and obtaining second position data; wherein the constraint layout point is used to layout the constraint image control point, and the conventional layout point is used to layout the conventional image control point, and along two mutually perpendicular horizontal directions in the target area, the points adjacent to the constraint layout point are the conventional layout point, or the corner layout point; The conventional image control points and the constrained image control points are arranged in the target area according to the second position data.

3. The spatial constraint enhanced real scene 3D modeling method according to claim 2, characterized in that: Traversing the internal layout points, updating each of the internal layout points to a constraint layout point, or updating each of the internal layout points to a regular layout point, to obtain second position data, including: Traversing the internal layout points, determining whether the current internal layout points satisfy a constraint condition; wherein the constraint condition is that, along two mutually perpendicular horizontal directions in the target area, the points adjacent to the current internal layout points are the corner layout points, or the regular layout points, or the internal layout points that have not been updated; If the current internal layout point satisfies the constraint condition, the current internal layout point is updated to the constraint layout point, and the next internal layout point is determined; If the current internal layout point does not meet the constraint condition, the current internal layout point is updated to the conventional layout point, and the next internal layout point is determined; The traversal is completed and the second position data is obtained.

4. The method for spatial stereoscopic constraint enhanced real scene 3D modeling according to any one of claims 1 to 3, characterized in that: At least part of the image control points are arranged at preset heights, and the preset heights include building roofs, building corners and building sides in the target area.

5. The spatial constraint enhanced real scene 3D modeling method according to claim 1, characterized in that: According to the point coordinate data and the image data, internal data processing is performed to obtain the real scene three-dimensional model, including: According to the point coordinate data, the image data is subjected to aerial triangulation processing to obtain aerial triangulation image data; wherein the aerial triangulation processing includes free network aerial triangulation processing, image control point puncture point processing and regional network aerial triangulation processing; The aerial triangulation image data is subjected to model processing to obtain the real-scene three-dimensional model; wherein the model processing includes point cloud generation, mesh construction and texture mapping.

6. The spatial constraint enhanced real scene 3D modeling method according to claim 5, characterized in that: According to the point coordinate data, the image data is subjected to aerial triangulation processing to obtain aerial triangulation image data, including: Preprocessing the image data to obtain a preprocessed image; The pre-processed image is subjected to the free-net aerial triangulation process, feature points are extracted and matched, and exterior orientation element results are calculated; Performing the image control point puncturing process according to the point coordinate data and the external orientation element result, adding the image control points to the pre-processed image to determine the absolute position and posture position of the image, and obtaining an image control point image; The image of the image of the control point is processed by the regional network aerial triangulation to obtain the aerial triangulation image data; wherein the aerial triangulation image data includes the encrypted image control point coordinate data; expressed as: , , , , in, is the image point coordinate matrix, is the control point coordinate matrix, is the self-calibration parameter matrix, is the correction matrix of the camera station coordinates, is the encrypted pixel coordinate increment, is the increment of the image exterior orientation element, For self-calibration parameters, is the antenna phase center deviation, is the drift error correction number, In the formula The coefficient matrix of In the formula The coefficient matrix of are all unit matrices, are all constant matrices, They are all power arrays.

7. The spatial constraint enhanced real scene 3D modeling method according to claim 1, characterized in that: Acquiring image data of the target area according to the image control points includes: Design the UAV route and generate the flight mission according to the target mapping requirements, wherein the target mapping requirements include the distribution of image control points and the accuracy requirements of the target model; According to the flight mission, the UAV is controlled to collect flight data of the target area to obtain the image data.

8. The spatial constraint enhanced real scene 3D modeling method according to claim 7, characterized in that: Also includes: Performing quality inspection on the current image data; wherein the quality inspection includes image shadow and reflection inspection, image ghosting and blur inspection, image loss inspection, image cloud penetration inspection, image loss inspection, and image damage inspection; Determining whether the current image data passes the quality detection; If the image data fails the quality inspection, a new flight mission is generated and new image data is acquired.

9. The spatial constraint enhanced real scene 3D modeling method according to claim 1, characterized in that: Acquiring point coordinate data of the image control point includes: The observation values ​​are obtained by simultaneously receiving the satellite signal of the first target satellite and the satellite signal of the second target satellite through the base station and the mobile station; wherein the mobile station is arranged at the image control point, and the observation values ​​include a first base observation value, a second base observation value, a first mobile observation value, and a second mobile observation value, the first base observation value is the observation value of the base station on the first target satellite, the second base observation value is the observation value of the base station on the second target satellite, the first mobile observation value is the observation value of the mobile station on the first target satellite, and the second mobile observation value is the observation value of the mobile station on the second target satellite; According to the observation values, a first inter-station difference and a second inter-station difference are calculated; wherein the first inter-station difference is the inter-station difference between the first target satellite and the reference station and the mobile station, and the second inter-station difference is the inter-station difference between the second target satellite and the reference station and the mobile station; expressed as: , , The first station difference The first target satellite The inter-station difference between the base station and the rover, The first target satellite The observation equation with the first flow observation, The first target satellite The observation equation with the first benchmark observation value; the second station difference is Second target satellite The inter-station difference between the base station and the rover, The second target satellite The observation equation with the second flow observation, The second target satellite The observation equation with the second reference observation value, is the satellite epoch; According to the first station difference and the second station difference, a double difference is calculated. ; expressed as: , Calculating and obtaining the point coordinate data of the current image control point according to the double difference; or, The satellite signal of the target satellite is received by the base station and the mobile station at the same time to obtain the observation value; wherein the mobile station is arranged at the image control point; Linearly combining the observed values ​​to obtain a virtual carrier phase observation; The point coordinate data of the current image control point is calculated based on the coordinate data of the reference station and the virtual carrier phase observation value.

10. A spatial stereo constraint enhanced real scene 3D modeling system, applied to the spatial stereo constraint enhanced real scene 3D modeling method according to any one of claims 1 to 9, characterized in that: include: A layout module, the layout module is used to layout at least four image control points in the target area; wherein the image control points are distributed in a planar manner in the target area, and relative to the reference point, the image control points are arranged at different heights to constrain the geometric accuracy of the real-scene three-dimensional model of the target area through the image control points; A coordinate acquisition module, the coordinate acquisition module is used to acquire point coordinate data of the image control point; An image acquisition module, the image acquisition module is used to acquire image data of the target area according to the image control points; A model building module is used to perform internal data processing based on the point coordinate data and the image data to obtain the real-scene three-dimensional model.

Citation Information

Patent Citations

  • Refined live-action three-dimensional modeling method based on air-ground fusion

    CN111540048A

  • Intelligent explosion pit data acquisition method

    CN119310597A

  • Data processing method and apparatus

    US20210183100A1