A modeling method and electronic device based on aerial photography and lidar.

Abstract

The present application relates to the technical field of three-dimensional modeling, and discloses a modeling method and electronic equipment based on aerial photography and laser radar, the method comprising: establishing a first three-dimensional model through aerial photography pictures, creating a first list through the first three-dimensional model, obtaining a second point cloud collected by the laser radar, and establishing a second three-dimensional model based on the second point cloud; correcting the coordinates of the second vertex in the second three-dimensional model according to the first vertex in the first three-dimensional model to obtain a fourth three-dimensional model; obtaining a third three-dimensional model by deleting vertices and faces that do not meet preset rules in the first three-dimensional model; merging the third three-dimensional model and the fourth three-dimensional model to obtain a reconstructed three-dimensional model, thereby being able to reconstruct the bottom three-dimensional shape of the scene that is ignored by aerial photography, minimizing the data redundancy of the laser radar point cloud, and being suitable for buildings and terrains with relatively complex structures.

Description

Technical Field

[0001] The present application relates to the technical field of three-dimensional modeling, and in particular to a modeling method and electronic equipment based on aerial photography and laser radar. Background Art

[0002] In the existing technology, aerial photography has become a common way of 3D modeling technology. Due to the characteristics of aerial photography, the 3D modeling method based on aerial photography is very effective for the higher parts of the scene, but it is difficult to restore the 3D shape details of the bottom. Therefore, relying solely on aerial photography for 3D modeling cannot achieve good modeling effects. Summary of the invention

[0003] The present application provides a modeling method based on aerial photography and lidar to solve one or more technical problems existing in the prior art, and at least provide a beneficial option or create conditions.

[0004] The modeling method based on aerial photography and laser radar according to the first aspect of the technical solution of the present application comprises the following steps:

[0005] Acquire a plurality of aerial images, and establish a first three-dimensional model based on the plurality of aerial images;

[0006] According to the height coordinate value of the first vertex of the first three-dimensional model, the first vertex is divided into a plurality of height groups according to a preset step length;

[0007] Creating a first list according to the height group, the abscissa value of the first vertex, and the ordinate value of the first vertex;

[0008] Acquire a second point cloud, and establish a second three-dimensional model based on the second point cloud;

[0009] Correcting the coordinates of the second vertex in the second three-dimensional model according to the first vertex to obtain a fourth three-dimensional model;

[0010] Deleting vertices and faces that do not meet preset rules in the first three-dimensional model to obtain a third three-dimensional model;

[0011] Combining the third three-dimensional model and the fourth three-dimensional model to obtain a reconstructed three-dimensional model;

[0012] The first vertex is a vertex of the first three-dimensional model, the second vertex is a vertex of the second three-dimensional model, and the second point cloud is acquired by laser radar scanning.

[0013] According to some technical solutions of the present application, before dividing the first vertices into a plurality of height groups according to a preset step length, the method further includes:

[0014] Rotate the first three-dimensional model horizontally so that the due north direction of the first three-dimensional model is the positive direction of the horizontal coordinate axis, and the due east direction of the first three-dimensional model is the positive direction of the vertical coordinate axis;

[0015] The first three-dimensional model is horizontally translated so that the abscissa value of the first vertex and the ordinate value of the first vertex are positive values.

[0016] According to some technical solutions of the present application, the first vertex of the first three-dimensional model is divided into a plurality of height groups according to a preset step length according to the height coordinate value of the first vertex, including:

[0017] Find the maximum and minimum height coordinates of the first vertex, denoted by Z max and Z min ;

[0018] According to the preset step length, the first vertices are divided into a plurality of height groups according to their height coordinate values, and the height groups are represented as follows: nth group: [z max -(n-1)H,z max -nH), n is a positive integer, and H is the preset step size;

[0019] Among them, the last group is expressed as: [z max -(n-1)H,z min ].

[0020] According to some technical solutions of the present application, creating a first list according to the height group, the abscissa value of the first vertex, and the ordinate value of the first vertex includes:

[0021] Set the height group that does not include the first vertex as an empty group, and enter the range of the height coordinate values ​​of the empty group into the first list;

[0022] An area multiplier reduction parameter is set by the maximum value and the minimum value of the height coordinate of the first vertex, which is denoted as R;

[0023] Query the height group of each non-empty group, and if the first height group is adjacent to the second height group, record the first list according to the abscissa value of the first vertex in the two adjacent height groups, the ordinate value of the first vertex and the area multiplier decreasing parameter;

[0024] Among them, the first height group is the upper limit of the height coordinate value is z max -jH, the second height group is the upper limit of the height coordinate value is z max -(j-1)H, where j is an arbitrary integer.

[0025] According to some technical solutions of the present application, the first list is recorded according to the abscissa value of the first vertex in two adjacent height groups, the ordinate value of the first vertex and the area multiplication rate decreasing parameter, including:

[0026] If the maximum value of the horizontal coordinate of the first vertex in the first height group is less than (1-R) ​​times the maximum value of the horizontal coordinate of the first vertex in the second height group, the first height group is entered into the first list, otherwise, a second judgment is performed;

[0027] The second judgment is: if the maximum value of the ordinate of the first vertex in the first height group is less than (1-R) ​​times the maximum value of the ordinate of the first vertex in the second height group, then the first height group is entered into the first list; otherwise, a third judgment is performed;

[0028] The third judgment is: if the minimum value of the horizontal coordinate of the first vertex in the first height group is greater than (1+R) times the maximum value of the horizontal coordinate of the first vertex in the second height group, then the first height group is entered into the first list; otherwise, the fourth judgment is performed;

[0029] The fourth judgment is: if the minimum vertical coordinate value of the first vertex in the first height group is greater than (1+R) times the maximum vertical coordinate value of the first vertex in the second height group, then the first height group is entered into the first list.

[0030] According to some technical solutions of the present application, obtaining the second point cloud includes:

[0031] Sending the first list to a ground device, where the ground device carries a laser radar;

[0032] Acquire the second point cloud;

[0033] Among them, the second point cloud is point cloud data after correction based on the height coordinate value of the position coordinates of the ground device, the height coordinate value of the ground device is not higher than a first set height, and the first set height is set based on the upper boundary coordinates of the group with the lowest upper boundary of the height coordinate value in the first list.

[0034] According to some technical solutions of the present application, establishing a second three-dimensional model based on the second point cloud includes:

[0035] Eliminate point cloud data in the second point cloud whose height coordinate value is higher than a preset height;

[0036] Establishing a second three-dimensional model according to the remaining point cloud data;

[0037] The preset height is set according to the sum of the maximum value of the height coordinate in the first list and the preset step value.

[0038] According to some technical solutions of the present application, the step of correcting the coordinates of the second vertex in the second three-dimensional model according to the first vertex includes:

[0039] Setting a second vertex that meets a preset feature in the second three-dimensional model as a fourth vertex;

[0040] The vertex whose height coordinate value among the first vertices falls within the interval between the first height value and the second height value is set as the third vertex, the first height value is the minimum value of the height coordinate among the fourth vertices, and the second height value is the maximum value of the height coordinate among the fourth vertices;

[0041] Correcting the coordinates of the second vertex according to the first formula;

[0042] The first formula is expressed as:

[0043]

[0044] Among them, (x k ,y k ,z k ) are the initial coordinates of the second vertex, is the corrected coordinate of the second vertex, x maxA is the maximum value of the abscissa of the third vertex, x maxB is the maximum value of the horizontal coordinate of the fourth vertex, y maxA is the maximum ordinate value of the third vertex, y maxB is the maximum value of the ordinate among the fourth vertices;

[0045] The preset feature refers to a vertex whose height coordinate value in the second vertex is greater than the maximum height coordinate value in the first list.

[0046] According to some technical solutions of the present application, deleting vertices and faces in the first three-dimensional model that do not meet preset rules includes:

[0047] Deleting a first triangular face in the first three-dimensional model, wherein the height coordinate values ​​of the three first vertices of the first triangular face are all smaller than the maximum height coordinate value of the fourth vertex;

[0048] The first vertices that do not belong to any triangular surface in the first three-dimensional model are deleted.

[0049] According to the second technical solution of the present application, an electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the method described in the first aspect when executing the computer program.

[0050] This application has at least the following beneficial effects:

[0051] The present application establishes a first three-dimensional model through aerial photos, creates a first list through the first three-dimensional model; obtains a second point cloud collected by a laser radar, and establishes a second three-dimensional model based on the second point cloud; corrects the coordinates of the second vertex in the second three-dimensional model according to the first vertex in the first three-dimensional model to obtain a fourth three-dimensional model; obtains a third three-dimensional model by deleting vertices and faces in the first three-dimensional model that do not meet preset rules; merges the third three-dimensional model and the fourth three-dimensional model to obtain a reconstructed three-dimensional model, thereby being able to reconstruct the bottom three-dimensional shape of the scene that is ignored by aerial photography.

[0052] It can be understood that the beneficial effects of the second aspect mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here.

[0053] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0055] Figure 1 A schematic diagram of a method according to an embodiment of the present application;

[0056] Figure 2 A schematic block diagram of a module of an apparatus according to an embodiment of the present application;

[0057] Figure 3 A schematic diagram of the structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is further described in detail below in conjunction with the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the embodiments of the present application. They are only examples of devices and methods consistent with some aspects of the embodiments of the present application as detailed in the attached claims.

[0059] It is understood that the terms "first", "second", etc. used in this application can be used to describe various concepts in this article, but unless otherwise specified, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another concept. For example, without departing from the scope of the embodiment of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the words "if" and "if" as used herein can be interpreted as "at the time of" or "when" or "in response to determination".

[0060] The terms "at least one", "multiple", "each", "any", etc. used in this application, at least one includes one, two or more, multiple includes two or more, each refers to each of the corresponding multiple, and any refers to any one of the multiple.

[0061] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0062] The modeling method based on aerial photography and laser radar provided in the embodiment of the present application relates to the field of three-dimensional modeling technology. The modeling method based on aerial photography and laser radar provided in the embodiment of the present application can be applied to a server, and may also be applied to a terminal, and may be software running in a terminal or a server. In some embodiments, the terminal may be a smart phone, a tablet computer, a laptop computer, a desktop computer, a vehicle-mounted terminal, etc., but is not limited thereto; the server side may be configured as an independent physical server, or may be configured as a server cluster or a distributed system consisting of multiple physical servers, or may be configured as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The server may also be a node server in a blockchain network; the software may be an application that implements a modeling method based on aerial photography and laser radar, etc., but is not limited to the above forms.

[0063] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0064] In the prior art, aerial photography is a common 3D modeling technology approach, including technical means such as drone oblique photography. In recent years, LiDAR technology has also been widely used in the field of 3D reconstruction. However, the applicant has found that aerial photography alone or a single modeling technology method in LiDAR is often unable to cope with buildings and terrains with more complex structures. For example, the 3D reconstruction method based on aerial photography is very effective for the higher parts of the scene, but it is difficult to restore the 3D shape details of the bottom. LiDAR technology is often deployed on ground equipment platforms, which is very effective for the parts of the scene close to the ground, but cannot scan the space near the top of the scene.

[0065] In view of this, an embodiment of the present application provides a modeling method based on aerial photography and lidar, which uses aerial photography to reconstruct the scene, and uses the lidar of ground equipment to reconstruct the bottom part of the scene that is ignored by aerial photography, minimizes the data redundancy of the lidar point cloud, and then matches and splices the two parts. The modeling method based on aerial photography and lidar provided in an embodiment of the present application is suitable for buildings and terrains with more complex structures.

[0066] Reference Figure 1 , the method of the embodiment of the present application comprises the following steps:

[0067] Acquire or receive a plurality of aerial images (photos), and create a first three-dimensional model according to the plurality of aerial images.

[0068] Specifically, the present application involves two types of data acquisition equipment: aerial photography equipment and ground equipment with laser radar. Among them, the aerial photography equipment can be a device such as a drone with aerial photography function that can fly at a certain altitude. The ground equipment can be an unmanned vehicle or robot, etc., which carries a laser radar module and can move to a set position as required to scan the modeling scene. Based on the data obtained by the scan, the specific modeling calculation process needs to rely on the server or terminal described above.

[0069] In the present application, the site to be reconstructed may be an outdoor scene. The result of the reconstruction is a three-dimensional model in a three-dimensional space, such as a building or a ship. The coordinate system of the modeling space includes an x-axis and a y-axis of the horizontal plane and a z-axis representing the height, and the unit may be meters. Among them, the higher the height, the larger the value of the z-axis coordinate. It should be understood that the modeled three-dimensional model is usually represented by a certain number of vertices and triangular faces.

[0070] During the aerial photography modeling stage, the aerial photography equipment is controlled to fly over the site to be reconstructed to take pictures. The aerial photography equipment needs to fly level over the site to be reconstructed and take pictures at different positions at the same height. The specific flight path of the aerial photography equipment is not limited in this article. The flight altitude of the aerial photography equipment can be set according to the actual situation, weather conditions and environmental conditions of the scene to be modeled. This article does not limit this. Normally, the aerial photography equipment can fly level at one altitude and collect multiple photos at different positions at this altitude. However, this article does not exclude the situation of flying level at different altitudes and collecting multiple photos. There is no limit on the number of photos collected in this article. For example, the total number of photos collected should be no less than 100.

[0071] If the aerial photography equipment used to collect aerial photos has a positioning function, the aerial photography equipment can mark the location coordinates of the time of shooting for each photo, including the x-axis coordinate, y-axis coordinate and altitude coordinate (z-axis coordinate). If the aerial photography equipment does not have a positioning function, the aerial photography equipment can only mark the altitude coordinate for each photo.

[0072] After collecting all the aerial photos, the aerial photography equipment sends all the photos to the electronic device responsible for modeling, which can be a terminal or a server. The terminal or server uses a 3D modeling method suitable for the photo shooting method to reconstruct and calculate the modeling scene to obtain a first 3D model. The first 3D model is a 3D model established based on the aerial photos. For example, the 3D modeling can be performed by using a motion structure recovery (SFM, Structure from motion) algorithm. This application does not limit the specific method of 3D modeling through pictures. All methods based on real-scene 3D reconstruction can be adaptively applied to this application.

[0073] In some embodiments of the present application, it also includes: horizontally rotating the first three-dimensional model so that the north direction of the first three-dimensional model is aligned with the positive direction of the horizontal coordinate axis, and the east direction is aligned with the positive direction of the vertical coordinate axis; horizontally translating the first three-dimensional model so that the horizontal and vertical coordinate values ​​of the vertices in the first three-dimensional model are positive values.

[0074] Specifically, for the needs of subsequent data processing and modeling, the terminal or server uses a three-dimensional modeling method suitable for the photo shooting method to reconstruct and calculate the modeling scene to obtain a first three-dimensional model. For the three-dimensional model (first three-dimensional model) established based on the aerial image, it is necessary to first rotate the three-dimensional model established based on the aerial image in the horizontal direction to ensure that the due north direction of the three-dimensional model established based on the aerial image is the positive direction of the x-axis (horizontal axis), and the due east direction of the three-dimensional model established based on the aerial image is the positive direction of the y-axis (vertical axis). The three-dimensional model established based on the aerial image is translated in the horizontal direction to ensure that the x-axis coordinates and y-axis coordinates of all vertices of the three-dimensional model established based on the aerial image are positive.

[0075] In some embodiments of the present application, the height coordinate values ​​(z-axis coordinates) of the vertices in the three-dimensional model established based on the aerial image are also included, and the first vertex is divided into multiple height groups according to a preset step size.

[0076] Specifically, after rotating and translating the three-dimensional model built based on the aerial image, the coordinate values ​​of all vertices of the three-dimensional model built based on the aerial image on the z-axis (height coordinate axis) are counted to find the maximum value z max With the minimum value z min , according to the maximum value z max With the minimum value z min , set a suitable height step parameter H (preset step length), the preset step length can be flexibly set according to the actual modeling needs, and this application does not limit this. For example, the preset step length can be set to 1 meter. Using the height step parameter H, all vertices of the three-dimensional model established according to the aerial image are grouped according to the size of their respective z-axis coordinate values. For example, n groups can be obtained. Among them, the z-axis coordinate value range of the vertices contained in each group can be expressed as an interval. For example, each group can be expressed as: the first group is [z max ,z max -H), the second group is [z max -H,z max -2H), the third group is [z max -2H,z max -3H), according to the above rules, the last group can be expressed as [z max -(n-1)H,z min ].

[0077] In some embodiments of the present application, a first list is created according to the divided height groups, the horizontal coordinate values ​​of the vertices in the three-dimensional model established according to the aerial pictures, and the vertical coordinate values ​​of the vertices in the three-dimensional model established according to the aerial pictures.

[0078] Specifically, after all vertices of the 3D model built based on the aerial image are grouped according to the size of their respective z-axis coordinate values ​​according to the height step parameter H, the x-axis and y-axis coordinate values ​​of the vertices included in each group are counted, and the maximum and minimum values ​​of the vertex coordinates on the x-axis and y-axis are found to obtain the coordinate value range of the vertex on the x-axis and y-axis. If a group does not contain any vertices, then this group is marked as an empty group.

[0079] Afterwards, a range list (the first list) ignored by aerial photography is created. The table is an empty table at the beginning of creation and does not include any grouping data.

[0080] Each value range marked as an empty group is recorded in the range list ignored by aerial photography (the first list).

[0081] According to the maximum value z max With the minimum value z min , set a suitable area reduction ratio parameter, denoted as R. The area reduction ratio parameter can be flexibly set according to the specific model and the maximum coordinate value and the minimum coordinate value. For example, it can be set to 0.1.

[0082] Check each group that is not marked as an empty group, if a certain height upper bound is z max -jH grouping and height upper bound is z max -(j-1)H groups are adjacent, where j is an arbitrary integer, then the following judgment needs to be performed:

[0083] Judgment 1: If the upper bound of the height is z max -jH's maximum x-axis coordinate is less than the upper limit of height z max -(j-1)H is (1-R) ​​times the maximum x-axis coordinate of the group, then the height upper limit is z max -jH group is recorded in the range list ignored by aerial photography (first list), and all judgments of these two groups are terminated. Otherwise, the next judgment (judgment 2) is performed on these two groups.

[0084] Judgment 2: If the upper bound of the height is z max -jH's maximum y-axis coordinate is less than the upper limit of height z max -(j-1)H is (1-R) ​​times the maximum value of the grouped y-axis coordinate, then the height upper limit is z max -jH group is recorded in the range list ignored by aerial photography, and all judgments of these two groups are terminated. Otherwise, the next judgment (judgment three) is performed on these two groups.

[0085] Judgment 3: If the upper bound of the height is z max -jH's minimum x-axis coordinate is greater than the upper limit of height z max-(j-1)H is (1+R) times the maximum x-axis coordinate of the group, then the height upper limit is z max -jH group is recorded in the range list ignored by aerial photography, and all judgments of these two groups are terminated. Otherwise, the next judgment (judgment four) is performed on these two groups.

[0086] Judgment 4: If the upper bound of the height is z max -jH's minimum y-axis coordinate is greater than the upper limit of the height z max -(j-1)H is (1+R) times the maximum value of the grouped y-axis coordinate, then the upper bound of the height is z max -jH's group is recorded in the range list ignored by aerial photography, and all judgments of these two groups are ended.

[0087] After performing the above operations, the server sends the range list (first list) ignored by the aerial photography to the ground device with the laser radar.

[0088] In the ground scanning phase, the ground device with laser radar performs the following steps based on the received range list (first list) ignored by the aerial photography:

[0089] First, find the group with the lowest upper bound in the range list ignored by aerial photography, and then control the ground equipment to move to the same height or a lower height to scan the scene that needs to be modeled. The above-mentioned same height refers to the upper bound coordinate value of the group with the lowest upper bound in the range list ignored by aerial photography. Exemplarily, if the groups in the range list ignored by aerial photography are sorted according to the size of the height coordinate value of the upper bound, the first group is the group with the highest upper bound, the coordinate height of the second group's upper bound is second, and the last group is the group with the lowest upper bound. Then the ground equipment is controlled to move to a height position consistent with the height coordinate of the upper bound of the last group, or to a position lower than the height coordinate of the upper bound of the last group, that is, the height coordinate value of the final moving position of the ground equipment should be less than the height coordinate value of the upper bound of the last group in the range list ignored by aerial photography. The ground equipment scans the scene that needs to be modeled at the final moving position. Among them, the value of the moving height can be measured by the aerial photography equipment flying to the position where the ground equipment is located.

[0090] The laser radar scan results obtained by the ground device are usually a certain number of points. For these points, the z-axis coordinate values ​​of these points need to be superimposed on the height value of the ground device, and then these modified points are sent to the server.

[0091] In some embodiments of the present application, it also includes obtaining a second point cloud and establishing a second three-dimensional model based on the second point cloud, wherein the second point cloud includes points scanned by a laser radar obtained by a ground device, and these points have been corrected according to the height coordinate values ​​in the position coordinates of the ground device. The height coordinate values ​​in the position coordinates of the ground device can be set according to the methods in the above embodiments of the present application, which will not be repeated here.

[0092] Specifically, the ground device superimposes the z-axis coordinate values ​​of the points scanned by the laser radar with the height value of the ground device, and then sends these modified points to the server. Based on these points, the server first finds the group G with the highest upper limit in the range list ignored by aerial photography (the first list), (according to the example of the above embodiment, the group G with the highest upper limit can be the first group of groups in the range list ignored by aerial photography), and then the server removes all points received that are higher than the upper limit of the height of group G by a preset step value H. It can be understood that the height coordinate of the upper limit of the height of group G is the maximum value of the height coordinates in the first list, that is, the height coordinate values ​​of the coordinates of the points removed by the server should all satisfy the requirement of being greater than the sum of the maximum value of the height coordinate in the first list and the preset step value H. Afterwards, the server performs three-dimensional modeling on the remaining points after the elimination to obtain a second three-dimensional model. The specific modeling method is not limited in this application, and all methods applied to point cloud modeling can be applied to this application.

[0093] The second three-dimensional model obtained (the three-dimensional model established by the lidar point cloud) is rotated in the horizontal direction to ensure that the north direction of the three-dimensional model established by the lidar point cloud is the positive direction of the x-axis, and the east direction of the three-dimensional model established by the lidar point cloud is the positive direction of the y-axis.

[0094] In some embodiments of the present application, the method further includes modifying the coordinates of the second vertex in the second three-dimensional model according to the first vertex to obtain a fourth three-dimensional model;

[0095] Specifically, among all vertices (second vertices) of the three-dimensional model (second three-dimensional model) established by the laser radar point cloud, find each vertex with a higher height upper bound than the group G (the group with the highest height upper bound in the first list), that is, the vertices (fourth vertices) whose height coordinate values ​​in the second vertices are greater than the maximum height coordinate value in the first list, and find the x-axis coordinate maximum value x of these vertices (fourth vertices). maxB , the maximum value of the y-axis coordinate y maxB 、The maximum value of the z-axis coordinate z maxB The minimum value of the z-axis coordinate minB .

[0096] Then, among the vertices (first vertices) of the three-dimensional model (first three-dimensional model) established based on the aerial image, find the coordinate value of the z-axis in [z minB ,z maxB ], count and get the maximum x-axis coordinates of all vertices (third vertices) in the interval maxA and the maximum y-axis coordinate y maxA .

[0097] For all vertices (second vertices) of the 3D model built from the LiDAR point cloud, modify the coordinates of each second vertex. If the coordinates of a second vertex before modification are (x k ,y k ,z k ), the modified coordinates are The specific calculation process is:

[0098]

[0099] Among them, x maxA is the maximum horizontal coordinate of the third vertex, x maxB is the maximum horizontal coordinate of the fourth vertex, y maxA is the maximum ordinate value of the third vertex, y maxB is the maximum value of the ordinate in the fourth vertex. By correcting the coordinates of the second vertex, the fourth three-dimensional model can be obtained.

[0100] In some embodiments of the present invention, the method further includes: deleting vertices and faces in the first three-dimensional model that do not meet preset rules to obtain a third three-dimensional model.

[0101] Specifically, in the 3D model built based on the aerial image, all three vertices whose coordinates on the z-axis are lower than z maxB In the three-dimensional model established based on the aerial image, all vertices that do not belong to any triangular surface are removed, and the three-dimensional model obtained is recorded as the third three-dimensional model.

[0102] The third three-dimensional model and the fourth three-dimensional model are combined to obtain a reconstructed three-dimensional model, and the reconstructed three-dimensional model is output as a final modeling result.

[0103] A modeling method based on aerial photography and laser radar proposed in an embodiment of the present application, establishes a first 3D model through aerial photography pictures, creates a first list through the first 3D model, obtains a second point cloud collected by the laser radar, and establishes a second 3D model based on the second point cloud; corrects the coordinates of the second vertex in the second 3D model according to the first vertex in the first 3D model to obtain a fourth 3D model; obtains a third 3D model by deleting vertices and faces that do not meet preset rules in the first 3D model; and merges the third 3D model and the fourth 3D model to obtain a reconstructed 3D model, thereby reconstructing the bottom 3D shape of the scene that is ignored by aerial photography, minimizing the data redundancy of the laser radar point cloud, and is suitable for buildings and terrains with more complex structures.

[0104] For the modeling method based on aerial photography and laser radar provided above, the embodiment of the present application also provides a modeling device based on aerial photography and laser radar, which is used to implement the above method, such as Figure 2 As shown, a device based on aerial photography and laser radar includes:

[0105] A first three-dimensional model building module, used to obtain a plurality of aerial pictures and build a first three-dimensional model based on the plurality of aerial pictures;

[0106] A grouping module, configured to group the first vertex into a plurality of height groups according to a preset step length according to a height coordinate value of the first vertex of the first three-dimensional model;

[0107] A first list creation module, used to create a first list according to the height group, the abscissa value of the first vertex and the ordinate value of the first vertex;

[0108] A second three-dimensional model building module, used for acquiring a second point cloud and building a second three-dimensional model based on the second point cloud;

[0109] A fourth three-dimensional model building module, used for correcting the coordinates of the second vertex in the second three-dimensional model according to the first vertex to obtain a fourth three-dimensional model;

[0110] A third three-dimensional model building module, used to delete vertices and faces that do not meet preset rules in the first three-dimensional model to obtain a third three-dimensional model;

[0111] A three-dimensional model reconstruction module, used for combining the third three-dimensional model and the fourth three-dimensional model to obtain a reconstructed three-dimensional model;

[0112] The first vertex is a vertex of the first three-dimensional model, the second vertex is a vertex of the second three-dimensional model, and the second point cloud is acquired through laser radar scanning.

[0113] It can be understood that the contents of the above method embodiments are all applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0114] The present application also provides an electronic device, such as Figure 3 As shown, the electronic device includes a memory, one or more processors ( Figure 3 Only one is shown) and a computer program stored in the memory and executable on the processor. Wherein: the memory is used to store software programs and units, and the processor executes various functional applications and data processing by running the software programs and units stored in the memory to obtain the resources corresponding to the above-mentioned preset events. Specifically, the processor implements the above-mentioned modeling method based on aerial photography and laser radar by running the above-mentioned computer program stored in the memory. The electronic device can be a server and a terminal.

[0115] It can be understood that the contents of the above method embodiments are all applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0116] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the above-mentioned modeling method based on aerial photography and lidar.

[0117] It can be understood that the contents of the above method embodiments are all applicable to the present storage medium embodiments, the functions specifically implemented by the present storage medium embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0118] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0119] An embodiment of the present application also provides a computer program product, which includes a computer program. When the computer program is executed by one or more processors, it can implement the steps of the above-mentioned modeling method based on aerial photography and lidar.

[0120] The modeling method, device, electronic device and medium based on aerial photography and laser radar provided in the embodiments of the present application establish a first 3D model through aerial photography pictures, create a first list through the first 3D model, obtain a second point cloud collected by the laser radar, and establish a second 3D model based on the second point cloud; correct the coordinates of the second vertex in the second 3D model according to the first vertex in the first 3D model to obtain a fourth 3D model; obtain a third 3D model by deleting vertices and faces that do not meet preset rules in the first 3D model; merge the third 3D model and the fourth 3D model to obtain a reconstructed 3D model, so as to reconstruct the bottom 3D shape of the scene that is ignored by aerial photography, minimize the data redundancy of the laser radar point cloud, and is suitable for buildings and terrains with more complex structures.

[0121] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0122] Although specific embodiments are described herein, those of ordinary skill in the art will recognize that many other modifications or alternative embodiments are also within the scope of the present disclosure. For example, any of the functions and / or processing capabilities described in conjunction with a particular device or component may be performed by any other device or component. In addition, although various exemplary implementations and architectures have been described according to embodiments of the present disclosure, those of ordinary skill in the art will recognize that many other modifications to the exemplary implementations and architectures herein are also within the scope of the present disclosure.

[0123] Some aspects of the present disclosure are described above with reference to the block diagram and flow chart of the system, method, system and / or computer program product according to exemplary embodiments.It should be understood that the combination of one or more blocks in the block diagram and flow chart and the blocks in the block diagram and flow chart can be realized by executing computer executable program instructions respectively.Equally, according to some embodiments, some blocks in the block diagram and flow chart may not need to be executed in the order shown, or may not need to be executed in full.In addition, additional components and / or operations beyond those components and / or operations shown in the blocks in the block diagram and flow chart may be present in certain embodiments.

[0124] Therefore, the blocks in the block diagrams and flow charts support the combination of devices for performing the specified functions, the combination of elements or steps for performing the specified functions, and the program instruction devices for performing the specified functions. It should also be understood that each block in the block diagrams and flow charts and the combination of blocks in the block diagrams and flow charts can be implemented by a special-purpose hardware computer system or a combination of special-purpose hardware and computer instructions that performs a specific function, element or step.

[0125] The program modules, applications, etc. described herein may include one or more software components, including, for example, software objects, methods, data structures, etc. Each such software component may include computer-executable instructions that, in response to execution, cause at least a portion of the functionality described herein (e.g., one or more operations of the illustrative methods described herein) to be performed.

[0126] Software components can be encoded with any of various programming languages. An exemplary programming language can be a low-level programming language, such as an assembly language associated with a specific hardware architecture and / or an operating system platform. The software component comprising assembly language instructions may need to be converted to executable machine code by an assembler before being executed by hardware architecture and / or platform. Another exemplary programming language can be a higher-level programming language, which can be transplanted across multiple architectures. The software component comprising a higher-level programming language may need to be converted to an intermediate representation by an interpreter or a compiler before execution. Other examples of programming languages ​​include but are not limited to macro languages, shells or command languages, job control languages, scripting languages, database queries or search languages ​​or report writing languages. In one or more exemplary embodiments, the software component comprising the instruction of one of the above-mentioned programming language examples can be directly executed by an operating system or other software components, without first being converted into another form.

[0127] Software components may be stored as files or other data storage structures. Software components of similar type or related functions may be stored together, such as in a specific directory, folder, or library. Software components may be static (e.g., preset or fixed) or dynamic (e.g., created or modified at execution time).

[0128] The embodiments of the present application are described in detail above in conjunction with the accompanying drawings, but the present application is not limited to the above embodiments, and various changes can be made within the knowledge scope of ordinary technicians in the relevant technical field without departing from the purpose of the present application.

Claims

1. A modeling method based on aerial photography and lidar, It is characterized in that The following steps are involved: Acquire a plurality of aerial images, and establish a first three-dimensional model based on the plurality of aerial images; According to the height coordinate value of the first vertex of the first three-dimensional model, the first vertex is divided into a plurality of height groups according to a preset step length; Creating a first list according to the height group, the abscissa value of the first vertex, and the ordinate value of the first vertex; Acquire a second point cloud, and establish a second three-dimensional model based on the second point cloud; Correcting the coordinates of the second vertex in the second three-dimensional model according to the first vertex to obtain a fourth three-dimensional model; Deleting vertices and faces that do not meet preset rules in the first three-dimensional model to obtain a third three-dimensional model; Combining the third three-dimensional model and the fourth three-dimensional model to obtain a reconstructed three-dimensional model; The first vertex is a vertex of the first three-dimensional model, the second vertex is a vertex of the second three-dimensional model, and the second point cloud is acquired by laser radar scanning; The step of correcting the coordinates of the second vertex in the second three-dimensional model according to the first vertex includes: Setting a second vertex that meets a preset feature in the second three-dimensional model as a fourth vertex; The vertex whose height coordinate value among the first vertices falls within the interval between the first height value and the second height value is set as the third vertex, the first height value is the minimum value of the height coordinate among the fourth vertices, and the second height value is the maximum value of the height coordinate among the fourth vertices; Correcting the coordinates of the second vertex according to the first formula; The first formula is expressed as: Among them, (x k ,y k ,z k ) are the initial coordinates of the second vertex, is the corrected coordinate of the second vertex, x maxA is the maximum value of the abscissa of the third vertex, x maxB is the maximum value of the horizontal coordinate of the fourth vertex, y maxA is the maximum ordinate value of the third vertex, y maxB is the maximum value of the ordinate among the fourth vertices; The preset feature refers to a vertex whose height coordinate value in the second vertex is greater than the maximum height coordinate value in the first list.

2. The modeling method based on aerial photography and laser radar according to claim 1, It is characterized in that Before dividing the first vertices into a plurality of height groups according to a preset step length, the method further includes: Rotate the first three-dimensional model horizontally so that the due north direction of the first three-dimensional model is the positive direction of the horizontal coordinate axis, and the due east direction of the first three-dimensional model is the positive direction of the vertical coordinate axis; The first three-dimensional model is horizontally translated so that the abscissa value of the first vertex and the ordinate value of the first vertex are positive values.

3. The modeling method based on aerial photography and laser radar according to claim 1, It is characterized in that The step of dividing the first vertex into a plurality of height groups according to a preset step length according to the height coordinate value of the first vertex of the first three-dimensional model includes: Find the maximum and minimum height coordinates of the first vertex, denoted by Z max and Z min ; According to the preset step length, the first vertices are divided into a plurality of height groups according to their height coordinate values, and the height groups are represented as follows: nth group: [z max -(n-1)H,z max -nH), n is a positive integer, and H is the preset step size; Among them, the last group is expressed as: [z max -(n-1)H,z min ].

4. The modeling method based on aerial photography and laser radar according to claim 3, It is characterized in that The step of creating a first list according to the height group, the abscissa value of the first vertex, and the ordinate value of the first vertex comprises: Set the height group that does not include the first vertex as an empty group, and enter the range of the height coordinate values ​​of the empty group into the first list; An area multiplier reduction parameter is set by the maximum value and the minimum value of the height coordinate of the first vertex, which is denoted as R; Query the height group of each non-empty group, and if the first height group is adjacent to the second height group, record the first list according to the abscissa value of the first vertex in the two adjacent height groups, the ordinate value of the first vertex and the area multiplier decreasing parameter; Among them, the first height group is the upper limit of the height coordinate value is z max -jH, the second height group is the upper limit of the height coordinate value is z max -(j-1)H, where j is an arbitrary integer.

5. The modeling method based on aerial photography and laser radar according to claim 4, It is characterized in that The step of recording the first list according to the abscissa value of the first vertex in two adjacent height groups, the ordinate value of the first vertex and the area multiplication rate decreasing parameter comprises: If the maximum value of the horizontal coordinate of the first vertex in the first height group is less than (1-R) ​​times the maximum value of the horizontal coordinate of the first vertex in the second height group, the first height group is entered into the first list, otherwise, a second judgment is performed; The second judgment is: if the maximum value of the ordinate of the first vertex in the first height group is less than (1-R) ​​times the maximum value of the ordinate of the first vertex in the second height group, then the first height group is entered into the first list; otherwise, a third judgment is performed; The third judgment is: if the minimum value of the horizontal coordinate of the first vertex in the first height group is greater than (1+R) times the maximum value of the horizontal coordinate of the first vertex in the second height group, then the first height group is entered into the first list; otherwise, the fourth judgment is performed; The fourth judgment is: if the minimum vertical coordinate value of the first vertex in the first height group is greater than (1+R) times the maximum vertical coordinate value of the first vertex in the second height group, then the first height group is entered into the first list.

6. The modeling method based on aerial photography and laser radar according to claim 1, It is characterized in that The obtaining of the second point cloud comprises: Sending the first list to a ground device, where the ground device carries a laser radar; Acquire the second point cloud; Among them, the second point cloud is point cloud data after correction based on the height coordinate value of the position coordinates of the ground device, the height coordinate value of the ground device is not higher than a first set height, and the first set height is set based on the upper boundary coordinates of the group with the lowest upper boundary of the height coordinate value in the first list.

7. The modeling method based on aerial photography and laser radar according to claim 1, It is characterized in that The establishing of a second three-dimensional model based on the second point cloud comprises: Eliminate point cloud data in the second point cloud whose height coordinate value is higher than a preset height; Establishing a second three-dimensional model according to the remaining point cloud data; The preset height is set according to the sum of the maximum value of the height coordinate in the first list and the preset step value.

8. The modeling method based on aerial photography and laser radar according to claim 1, It is characterized in that The deleting of vertices and faces in the first three-dimensional model that do not meet preset rules includes: Deleting a first triangular face in the first three-dimensional model, wherein the height coordinate values ​​of the three first vertices of the first triangular face are all smaller than the maximum height coordinate value of the fourth vertex; The first vertices that do not belong to any triangular surface in the first three-dimensional model are deleted.

9. An electronic device, It is characterized in that The electronic device comprises a memory and a processor, the memory stores a computer program, and the processor implements the method according to any one of claims 1 to 8 when executing the computer program.

Images