A Construction Progress Management Method, Device, Medium and Equipment Based on Aerial Photography

Abstract

The present application provides a construction progress management method, device, medium and equipment based on aerial photography. An unmanned aerial vehicle is used for oblique photography to obtain an image sequence of a construction site at different times, and the image sequence includes GNSS data and IMU data; based on the image sequence, a three-dimensional point cloud of a target building is established; based on two sets of point cloud data at different times, differential point cloud data of the target building is generated; based on the differential point cloud data, GNSS data and IMU data, the construction progress is determined; that is, by using an unmanned aerial vehicle to photograph the construction site at different times, and establishing a three-dimensional point cloud of the target building based on the images, differential point cloud data of the target building at different times is generated through two sets of point cloud data at different times, and according to the differential point cloud data, GNSS data and IMU data, the construction progress of the target building is determined, so as to realize unmanned supervision of the construction progress of the target building, and at the same time, three-dimensional image information with good visualization effect can be generated by using the three-dimensional point cloud data.

Description

Technical Field

[0001] This application relates to the technical field of construction progress management, and particularly relates to a construction progress management method, device, medium and equipment based on aerial photography. Background Art

[0002] Construction progress monitoring is an important part of construction project management. By monitoring the construction progress, it can help project managers timely discover problems and take appropriate measures to ensure that the construction process proceeds according to the plan. Therefore, it has important significance.

[0003] However, at present, most construction units control the overall progress of the project through the registration of raw material arrivals and conduct statistics by comprehensively analyzing the consumption of steel bars and concrete combined with construction experience. This method can basically solve the problem for general civil construction projects with low construction technical content and low construction difficulty. However, for large-scale engineering projects, it is difficult to quickly and accurately obtain the construction progress information of the building, and the accuracy cannot be guaranteed. Moreover, the single display of data cannot comprehensively reflect the actual progress during the construction process, resulting in incomplete construction data and making it difficult to conduct accurate construction progress management. Therefore, there is an urgent need for a method that can efficiently, safely and comprehensively obtain the construction progress. Summary of the Invention

[0004] In order to solve the above technical problems, this application is proposed. Embodiments of this application provide a construction progress management method, device, medium and equipment based on aerial photography.

[0005] According to one aspect of this application, a construction progress management method based on aerial photography is provided, including: using an unmanned aerial vehicle for oblique photography to obtain an image sequence of the construction site at different times; wherein, the image sequence includes GNSS (Global Navigation Satellite System) data and IMU (Inertial Measurement Unit) data; based on the image sequence, establishing a three-dimensional point cloud of the target building; based on two sets of point cloud data of the target building at different times, generating differential point cloud data of the target building at different times; based on the differential point cloud data, the GNSS data and the IMU data, determining the construction progress of the target building.

[0006] In one embodiment, the method of using an unmanned aerial vehicle (UAV) for oblique photography to obtain image sequences of a construction site at different times includes: correcting the camera exposure delay time of the UAV according to the cubic spline interpolation method; differentiating the GNSS data according to the camera exposure delay time to obtain the relationship between the antenna phase center coordinates and the camera station coordinates; and based on the relationship between the antenna phase center coordinates and the camera station coordinates, using the UAV for oblique photography to obtain the image sequences of the construction site at different times.

[0007] In one embodiment, the step of correcting the camera exposure delay time of the UAV according to the cubic spline interpolation method includes: obtaining the flight sampling points of the UAV according to GNSS sampling; based on the flight sampling points, performing cubic polynomial fitting using a cubic spline function to obtain the flight trajectory of the UAV; obtaining the exposure moment of the antenna phase center coordinates of the UAV according to the interpolation calculation method; and based on the exposure moment of the antenna phase center coordinates of the UAV, correcting the camera exposure delay time of the UAV.

[0008] In one embodiment, the step of generating the differential point cloud data of the target building at different times based on two sets of point cloud data of the target building at different times includes: extracting the feature data of the two sets of point cloud data; based on the feature data, performing feature matching on the two sets of point cloud data to generate a set of matching relationships; and based on the set of matching relationships, calculating the differences between the two sets of point cloud data to generate the differential point cloud data.

[0009] In one embodiment, the step of performing feature matching on the two sets of point cloud data based on the feature data to generate a set of matching relationships includes: for any first point cloud in one set of the two sets of point cloud data, querying for the second point cloud with the closest distance to the first point cloud in the other set of the two sets of point cloud data, and adding the first point cloud and the corresponding second point cloud as a point cloud pair to an alternative set; for any point cloud pair in the alternative set, calculating the first distance value between the second point cloud of the point cloud pair and all the first point clouds in the alternative set; if the first distance value is equal to the distance between the first point cloud and the second point cloud of the point cloud pair, determining the point cloud pair as an alternative point cloud pair; arbitrarily selecting three pairs of alternative point cloud pairs and calculating the side lengths of a first triangle and a second triangle; wherein, the first triangle is composed of the first point clouds of the three pairs of alternative point cloud pairs, and the second triangle is composed of the second point clouds of the three pairs of alternative point cloud pairs; if the side lengths of the first triangle and the second triangle meet a preset condition, adding the three pairs of alternative point cloud pairs to the set of matching relationships; wherein, the preset condition is: ; wherein, is a given threshold, , i= 1, 2, 3, which are the side lengths of the three sides of the first triangle, which are the side lengths of the three sides of the second triangle respectively.

[0010] In one embodiment, calculating the difference between the two sets of point cloud data based on the set of matching relationships to generate the differential point cloud data includes: for each point cloud pair in the set of matching relationships, calculating the Euclidean distance between the first point cloud and the second point cloud; wherein, the calculation formula of the Euclidean distance is: , where is the first point cloud, is the second point cloud, is the scaling factor, is the rotation matrix, is the translation vector; calculating the error contribution of the point cloud pair through an error metric function; wherein, the error metric function is , and the calculation formula of the error contribution is: , where is the scaling coefficient of the error metric function; accumulating the error contributions of all point cloud pairs in the set of matching relationships to obtain the total error; wherein, the calculation formula of the total error is: ; calculating the optimal scaling factor, the optimal rotation matrix and the optimal translation vector based on minimizing the total error; based on the optimal scaling factor, the optimal rotation matrix and the optimal translation vector, transforming the second point cloud of all point cloud pairs in the set of matching relationships into the transformed point cloud in the coordinate system of the first point cloud; wherein, the calculation formula of the transformed point cloud is: , where is the transformed point cloud, is the set of second point clouds; calculating the second distance value between the transformed point cloud and the corresponding first point cloud; if the second distance value is greater than a preset distance threshold, then taking the corresponding first point cloud and the transformed point cloud as a differential point cloud pair.

[0011] In one embodiment, determining the construction progress of the target building based on the differential point cloud data, the GNSS data and the IMU data includes: aligning the differential point cloud data, the GNSS data and the IMU data; determining the construction progress of the target building based on the aligned differential point cloud data, the GNSS data and the IMU data.

[0012] According to another aspect of the present application, there is provided a construction progress management device based on aerial photography, including: an image acquisition module for performing oblique photography using a drone to acquire an image sequence of a construction site at different times; wherein the image sequence includes GNSS data and IMU data; a point cloud establishment module for establishing a three-dimensional point cloud of a target building based on the image sequence; a difference generation module for generating difference point cloud data of the target building at different times based on two sets of point cloud data of the target building at different times; and a progress determination module for determining the construction progress of the target building based on the difference point cloud data, the GNSS data, and the IMU data.

[0013] According to another aspect of the present application, there is provided a computer-readable storage medium storing a computer program for executing any of the above methods.

[0014] According to another aspect of the present application, there is provided an electronic device, including: a processor; a memory for storing executable instructions of the processor; and the processor for executing any of the above methods.

[0015] A construction progress management method, device, medium, and device provided by the present application use a drone to perform oblique photography to acquire an image sequence of a construction site at different times; wherein the image sequence includes GNSS data and IMU data; based on the image sequence, a three-dimensional point cloud of a target building is established; based on two sets of point cloud data of the target building at different times, difference point cloud data of the target building at different times is generated; based on the difference point cloud data, the GNSS data, and the IMU data, the construction progress of the target building is determined; that is, by taking images of the construction site at different times by a drone and establishing a three-dimensional point cloud of the target building based on the images, difference point cloud data of the target building at different times is generated through two sets of point cloud data at different times, and the construction progress of the target building is determined according to the difference point cloud data, the GNSS data, and the IMU data, thereby realizing unmanned supervision of the construction progress of the target building, and at the same time, three-dimensional image information with good visualization effect can be generated using the three-dimensional point cloud data. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other objects, features, and advantages of the present application will become more obvious. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation to the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0017] Figure 1It is a schematic flowchart of a construction progress management method based on aerial photography provided by an exemplary embodiment of the present application.

[0018] Figure 2 It is a schematic diagram of an oblique photography method provided by an exemplary embodiment of the present application.

[0019] Figure 3 It is a schematic diagram of a point cloud structure provided by an exemplary embodiment of the present application.

[0020] Figure 4 It is a schematic diagram of the structure of a construction progress management device based on aerial photography provided by an exemplary embodiment of the present application.

[0021] Figure 5 It is a structural diagram of an electronic device provided by an exemplary embodiment of the present application. Detailed implementation manners

[0022] Next, exemplary embodiments of the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. It should be understood that the present application is not limited by the exemplary embodiments described herein.

[0023] Figure 1 It is a schematic flowchart of a construction progress management method based on aerial photography provided by an exemplary embodiment of the present application. As Figure 1 shown, the construction progress management method based on aerial photography includes the following steps:

[0024] Step 110: Use a drone to perform oblique photography to obtain an image sequence of the construction site at different times.

[0025] Among them, the image sequence includes GNSS data and IMU data. The present application uses a drone to perform oblique photography to obtain an image sequence of the construction site at different times. Since oblique photography can provide richer surface information and improve the accuracy of 3D point clouds, the present application regularly or as needed uses a drone to cruise over the construction site to capture the image sequence of the construction site (as Figure 2 shown) to obtain the image information of the construction buildings within the construction site, so as to realize the unmanned monitoring of the construction progress.

[0026] Step 120: Based on the image sequence, establish a three-dimensional point cloud of the target building.

[0027] After the present application obtains the image sequence of the construction site, based on the obtained image sequence, differential GNSS and IMU data are used to generate a high-precision 3D model through image registration, attitude estimation and optimization, and assisted aerial triangulation to establish a three-dimensional point cloud of the target building.

[0028] Step 130: Generate differential point cloud data of the target building at different times based on two sets of point cloud data of the target building at different times.

[0029] After establishing the three-dimensional point cloud of the target building in this application, by comparing the point cloud data of the target building at different times, differential point cloud data of the target building at different times is obtained, so as to know the construction progress of the target building during the time period between different times.

[0030] Step 140: Determine the construction progress of the target building based on the differential point cloud data, GNSS data, and IMU data.

[0031] In this application, based on the differential point cloud data, GNSS data, and IMU data, the construction progress of the target building is comprehensively determined to achieve real-time and accurate monitoring of the target building.

[0032] A construction progress management method based on aerial photography provided by this application uses an unmanned aerial vehicle (UAV) for oblique photography to obtain image sequences of the construction site at different times; among them, the image sequences include GNSS data and IMU data; based on the image sequences, a three-dimensional point cloud of the target building is established; based on two sets of point cloud data of the target building at different times, differential point cloud data of the target building at different times is generated; based on the differential point cloud data, GNSS data, and IMU data, the construction progress of the target building is determined; that is, by taking images of the construction site at different times with a UAV and establishing a three-dimensional point cloud of the target building based on the images, differential point cloud data of the target building at different times is generated through two sets of point cloud data at different times, and according to the differential point cloud data, GNSS data, and IMU data, the construction progress of the target building is determined, so as to achieve unmanned supervision of the construction progress of the target building, and at the same time, three-dimensional image information with good visualization effects can be generated using the three-dimensional point cloud data.

[0033] In one embodiment, the specific implementation manner of the above step 110 may be: according to the cubic spline interpolation method, correct the camera exposure delay time of the UAV; differential GNSS data according to the camera exposure delay time to obtain the relationship between the antenna phase center coordinates and the camera station coordinates; based on the relationship between the antenna phase center coordinates and the camera station coordinates, use the UAV for oblique photography to obtain image sequences of the construction site at different times.

[0034] This application uses the cubic spline interpolation method to correct the camera exposure delay time of the UAV to determine the actual camera exposure time of the UAV, and combines GNSS data to obtain the relationship between the antenna phase center coordinates and the camera station coordinates. After using the UAV for oblique photography, according to the relationship between the antenna phase center coordinates and the camera station coordinates, the captured images are converted to obtain image sequences of the construction site at different times.

[0035] In one embodiment, the specific implementation of the above step 110 may be: obtaining flight sampling points of the UAV according to GNSS sampling; performing cubic polynomial fitting using a cubic spline function based on the flight sampling points to obtain the flight trajectory of the UAV; obtaining the exposure moment for acquiring the antenna phase center coordinates of the UAV according to an interpolation calculation method; and correcting the camera exposure delay time of the UAV based on the exposure moment for acquiring the antenna phase center coordinates of the UAV.

[0036] This application obtains multiple flight sampling points of the UAV according to GNSS sampling, and performs cubic polynomial fitting using a cubic spline function based on the multiple flight sampling points to obtain the flight trajectory of the UAV. Through an interpolation calculation method, the exposure moment for acquiring the antenna phase center coordinates of the UAV is obtained, and the camera exposure delay time of the UAV is corrected according to the exposure moment for acquiring the antenna phase center coordinates of the UAV, so as to obtain an accurate correspondence between time and spatial position in the image sequence.

[0037] In one embodiment, the specific implementation of the above step 130 may be: extracting feature data of two sets of point cloud data; performing feature matching on the two sets of point cloud data based on the feature data to generate a set of matching relationships; and calculating the difference between the two sets of point cloud data based on the set of matching relationships to generate differential point cloud data.

[0038] This application extracts the feature data of two sets of point cloud data at different times and performs feature matching to calculate the difference between the two sets of point cloud data. Specifically, denote the two sets of point cloud data as point cloud group P and point cloud group Q . Extract the SHOT features of point cloud group P and point cloud group Q . Denote the feature data of point cloud P in point cloud group p as , and the feature data of point cloud Q in point cloud group q as .

[0039] In one embodiment, the specific implementation of the above step 130 may be as follows: For any first point cloud in one set of point cloud data among the two sets of point cloud data, query for the second point cloud with the closest distance to the first point cloud in the other set of point cloud data among the two sets of point cloud data, and add the first point cloud and the corresponding second point cloud as a point cloud pair to the alternative set; for any pair of point cloud pairs in the alternative set, calculate the first distance value between the second point cloud of the point cloud pair and all the first point clouds in the alternative set; if the first distance value is equal to the distance between the first point cloud and the second point cloud of the point cloud pair, determine the point cloud pair as an alternative point cloud pair; randomly select three pairs of alternative point cloud pairs and calculate the side lengths of the first triangle and the second triangle; wherein, the first triangle is composed of the first point clouds of the three pairs of alternative point cloud pairs, and the second triangle is composed of the second point clouds of the three pairs of alternative point cloud pairs; if the side lengths of the first triangle and the second triangle meet the preset conditions, add the three pairs of alternative point cloud pairs to the matching relationship set; wherein, the preset conditions are: ; wherein, is a given threshold, , i = 1, 2, 3, are respectively the side lengths of the three sides of the first triangle, are respectively the side lengths of the three sides of the second triangle.

[0040] Specifically, for each point cloud P in the point cloud group p , query in the feature set of the point cloud group Q for the p with the closest distance to the point cloud , and add all such points to the set . For each , calculate the distance between the feature vectors of and all the points in the point cloud group , find the point with the smallest distance. If , then add to the filtered alternative point cloud pair set . Randomly sample 3 pairs of points in the set , denoted as . Construct two triangles composed of the point pairs , denoted as and respectively, and denote the corresponding sides as . Calculate the ratio of the corresponding side lengths , i = 1, 2, 3. If it satisfies , that is, the ratio of the square of any side to the product of the other two sides meets the above range, then add these 3 pairs of points to the final matching relationship set 。

[0041] In one embodiment, the specific implementation of the above step 130 may be: for each point cloud pair in the matching relationship set, calculate the Euclidean distance between the first point cloud and the second point cloud; wherein, the calculation formula of the Euclidean distance is: , where is the first point cloud, is the second point cloud, is the scaling factor, is the rotation matrix, is the translation vector; calculate the error contribution of the point cloud pair through the error metric function; wherein, the error metric function is , and the calculation formula of the error contribution is: , where is the scaling coefficient of the error metric function; accumulate the error contributions of all point cloud pairs in the matching relationship set to obtain the total error; wherein, the calculation formula of the total error is: ; based on the minimization of the total error, calculate the optimal scaling factor, the optimal rotation matrix, and the optimal translation vector; based on the optimal scaling factor, the optimal rotation matrix, and the optimal translation vector, transform all the second point clouds in the matching relationship set into the transformed point clouds in the coordinate system of the first point cloud; wherein, the calculation formula of the transformed point cloud is: , where is the transformed point cloud, is the set of second point clouds; calculate the second distance value between the transformed point cloud and the corresponding first point cloud; if the second distance value is greater than the preset distance threshold, then use the corresponding first point cloud and the transformed point cloud as the differential point cloud pair.

[0042] Specifically, for each point cloud pair in the set , calculate the Euclidean distance between the transformed point cloud and the point cloud ; calculate the error contribution of each point pair through the error metric function , where the scaling coefficient is used to control the scaling of the error metric function; accumulate the error contributions of all point cloud pairs to obtain the total error:

[0043] ;

[0044] Based on the minimization of the total error, optimize and calculate the optimal scaling factor, the optimal rotation matrix, and the optimal translation vector; transform the point cloud group into the coordinate system of the point cloud group : , and the transformed point cloud group is denoted as ; For each point cloud in the point cloud group find the point cloud in the transformed point cloud group that is closest to it calculate the distance between each pair of point clouds set a distance threshold if the distance between the point clouds is greater than this distance threshold, then it is considered that there is a significant difference between the points and generate a set of differential point cloud pairs, where the set of differential point cloud pairs includes all point pairs with significant differences . .

[0045] In one embodiment, the specific implementation manner of the above step 140 may be: align the differential point cloud data, GNSS data, and IMU data; based on the aligned differential point cloud data, GNSS data, and IMU data, determine the construction progress of the target building.

[0046] Specifically, the present application obtains the geographical information data of the construction area through the GIS platform, including topographic maps, planning maps, etc., registers the GIS data with the point cloud data, and the aligned GIS data can be used as a reference to provide geographical reference for subsequent analysis; and integrates the set of differential point cloud pairs with the GIS data, and aligns them through the geographical coordinate system to ensure that all data is within the same reference framework. Specifically, the present application converts the point cloud data from the local coordinate system to the global coordinate system using the following conversion formula:

[0047] ；

[0048] where is the global coordinate, is the local coordinate, is the rotation matrix, is the translation vector; further, a fine alignment is performed between the point cloud data and the GIS data, and the specific alignment formula is as follows:

[0049] ；

[0050] where is the point cloud data, is the corresponding GIS data, N is the total number of the point cloud data.

[0051] After processing its data, the present application displays the superimposed three-dimensional model on the GIS platform (such as Figure 3As shown in the figure, the changed part of the construction progress is highlighted, and the point cloud pairs in the differential point cloud set are color-coded according to the size of the difference. For example, the part with the largest change is marked in red, the part with a smaller change is marked in yellow, and the unchanged part is marked in green. The GIS tool is used to perform spatial analysis on the superimposed 3D model to analyze the point cloud differences at different positions within the construction area, enabling construction management personnel to intuitively see the changes in the construction progress.

[0052] Figure 4 FIG. is a schematic structural diagram of a construction progress management device based on aerial photography provided by an exemplary embodiment of the present application. As Figure 4 shown, the construction progress management device 40 based on aerial photography includes: an image acquisition module 41, configured to use an unmanned aerial vehicle (UAV) for oblique photography to acquire an image sequence of a construction site at different times; wherein, the image sequence includes GNSS data and IMU data; a point cloud generation module 42, configured to establish a three-dimensional point cloud of a target building based on the image sequence; a difference generation module 43, configured to generate differential point cloud data of the target building at different times based on two sets of point cloud data of the target building at different times; and a progress determination module 44, configured to determine the construction progress of the target building based on the differential point cloud data, GNSS data, and IMU data.

[0053] For a construction progress management device based on aerial photography provided by the present application, the image acquisition module 41 uses an unmanned aerial vehicle for oblique photography to acquire an image sequence of a construction site at different times; wherein, the image sequence includes GNSS data and IMU data; the point cloud generation module 42 establishes a three-dimensional point cloud of a target building based on the image sequence; the difference generation module 43 generates differential point cloud data of the target building at different times based on two sets of point cloud data of the target building at different times; the progress determination module 44 determines the construction progress of the target building based on the differential point cloud data, GNSS data, and IMU data; that is, by taking images of the construction site at different times with an unmanned aerial vehicle and establishing a three-dimensional point cloud of the target building based on the images, generating differential point cloud data of the target building at different times through two sets of point cloud data at different times, and determining the construction progress of the target building according to the differential point cloud data, GNSS data, and IMU data, so as to realize unmanned supervision of the construction progress of the target building, and at the same time, three-dimensional image information with good visualization effect can be generated using the three-dimensional point cloud data.

[0054] In one embodiment, the above image acquisition module 41 may be further configured to: correct the camera exposure delay time of the unmanned aerial vehicle according to the cubic spline interpolation method; obtain the relationship between the antenna phase center coordinate and the camera station coordinate by differentiating the GNSS data according to the camera exposure delay time; and use the unmanned aerial vehicle for oblique photography based on the relationship between the antenna phase center coordinate and the camera station coordinate to acquire an image sequence of the construction site at different times.

[0055] In one embodiment, the above image acquisition module 41 may be further configured to: obtain the flight sampling points of the UAV according to GNSS sampling; based on the flight sampling points, perform cubic polynomial fitting using the cubic spline function to obtain the flight trajectory of the UAV; according to the interpolation calculation method, obtain the exposure moment of the antenna phase center coordinates of the UAV; based on the exposure moment of the antenna phase center coordinates of the UAV, correct the camera exposure delay time of the UAV.

[0056] In one embodiment, the above difference generation module 43 may be further configured to: extract the feature data of two sets of point cloud data; based on the feature data, perform feature matching on the two sets of point cloud data to generate a set of matching relationships; based on the set of matching relationships, calculate the difference between the two sets of point cloud data to generate difference point cloud data.

[0057] In one embodiment, the above difference generation module 43 may be further configured to: for any first point cloud in one of the two sets of point cloud data, query the second point cloud with the closest distance to the first point cloud in the other set of the two sets of point cloud data, and add the first point cloud and the corresponding second point cloud as a point cloud pair to the alternative set; for any pair of point cloud pairs in the alternative set, calculate the first distance value between the second point cloud of the point cloud pair and all the first point clouds in the alternative set; if the first distance value is equal to the distance between the first point cloud and the second point cloud of the point cloud pair, determine the point cloud pair as an alternative point cloud pair; randomly select three pairs of alternative point cloud pairs and calculate the side lengths of the first triangle and the second triangle; wherein, the first triangle is composed of the first point clouds of the three pairs of alternative point cloud pairs, and the second triangle is composed of the second point clouds of the three pairs of alternative point cloud pairs; if the side lengths of the first triangle and the second triangle meet the preset conditions, add the three pairs of alternative point cloud pairs to the set of matching relationships; wherein, the preset conditions are: ; wherein, is a given threshold, , i = 1, 2, 3, are respectively the side lengths of the three sides of the first triangle, are respectively the side lengths of the three sides of the second triangle.

[0058] In one embodiment, the above difference generation module 43 may be further configured to: for each point cloud pair in the set of matching relationships, calculate the Euclidean distance between the first point cloud and the second point cloud; wherein, the calculation formula of the Euclidean distance is: , wherein, is the first point cloud, is the second point cloud, is the scaling factor, is the rotation matrix, is the translation vector; calculating the error contribution of the point cloud pair through an error metric function; wherein, the error metric function is , and the calculation formula for the error contribution is: , where is the scaling coefficient of the error metric function; accumulating the error contributions of all point cloud pairs in the matching relationship set to obtain the total error; wherein, the calculation formula for the total error is: ; based on minimizing the total error, calculating the optimal scaling factor, the optimal rotation matrix, and the optimal translation vector; based on the optimal scaling factor, the optimal rotation matrix, and the optimal translation vector, transforming the second point cloud of all point cloud pairs in the matching relationship set into the transformed point cloud in the coordinate system of the first point cloud; wherein, the calculation formula for the transformed point cloud is: , where is the transformed point cloud, is the second point cloud set; calculating the second distance value between the transformed point cloud and the corresponding first point cloud; if the second distance value is greater than a preset distance threshold, then taking the corresponding first point cloud and the transformed point cloud as a pair of differential point clouds.

[0059] In one embodiment, the above progress determination module 44 can be further configured to: align the differential point cloud data, the GNSS data, and the IMU data; and determine the construction progress of the target building based on the aligned differential point cloud data, GNSS data, and IMU data.

[0060] Next, referring to Figure 5 to describe the electronic device according to an embodiment of the present application. The electronic device can be any one or both of the first device and the second device, or a stand-alone device independent of them, and the stand-alone device can communicate with the first device and the second device to receive the input signals collected from them.

[0061] Figure 5 illustrates a block diagram of an electronic device according to an embodiment of the present application.

[0062] As Figure 5 shown, the electronic device 10 includes one or more processors 11 and a memory 12.

[0063] The processor 11 can be a central processing unit (CPU) or other form of processing unit with data processing capabilities and / or instruction execution capabilities, and can control other components in the electronic device 10 to perform desired functions.

[0064] The memory 12 may include one or more computer program products, and the computer program products may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory, etc. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage media, and the processor 11 may run the program instructions to implement the methods of the various embodiments of the present application described above and / or other desired functions. Various contents such as input signals, signal components, noise components, etc. may also be stored in the computer-readable storage media.

[0065] In one example, the electronic device 10 may further include: an input device 13 and an output device 14, and these components are interconnected through a bus system and / or other forms of connection mechanisms (not shown).

[0066] When the electronic device is a stand-alone device, the input device 13 may be a communication network connector for receiving the collected input signals from the first device and the second device.

[0067] In addition, the input device 13 may further include, for example, a keyboard, a mouse, and so on.

[0068] The output device 14 may output various information to the outside, including the determined distance information, direction information, etc. The output device 14 may include, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.

[0069] Of course, for simplicity, Figure 5 only some of the components related to the present application in the electronic device 10 are shown, and components such as buses, input / output interfaces, etc. are omitted. In addition, according to specific application scenarios, the electronic device 10 may further include any other appropriate components.

[0070] In addition to the above methods and devices, the embodiments of the present application may also be computer program products, which include computer program instructions that, when run by a processor, cause the processor to execute the steps in the methods according to the various embodiments of the present application described in the "Exemplary Method" section of this specification.

[0071] The computer program product may be written in any combination of one or more programming languages for programming code to perform the operations of the embodiments of the present application. The programming languages include object-oriented programming languages such as Java, C++, etc., and also include conventional procedural programming languages such as the "C" language or similar programming languages. The programming code may be executed entirely on the user computing device, partially on the user device, executed as a stand-alone software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0072] In addition, an embodiment of the present application may also be a computer-readable storage medium having computer program instructions stored thereon, and when the computer program instructions are run by a processor, the processor is caused to execute the steps in the methods according to various embodiments of the present application described in the "Exemplary Method" section above of this specification.

[0073] The computer-readable storage medium may employ any combination of one or more readable media. The readable media may be a readable signal medium or a readable storage medium. The readable storage medium may, for example, include but is not limited to an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0074] The basic principles of the present application have been described above in conjunction with specific embodiments. However, it should be noted that the advantages, benefits, effects, etc. mentioned in the present application are only examples and not limitations, and it cannot be considered that these advantages, benefits, effects, etc. are essential for each embodiment of the present application. In addition, the above-disclosed specific details are only for the purposes of illustration and easy understanding, and not for limitation. The above details do not limit the present application to necessarily adopt the above specific details for implementation.

[0075] The block diagrams of the devices, apparatuses, equipment, and systems involved in this application are only illustrative examples, and do not require or imply that they must be connected, arranged, and configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including", "comprising", "having", etc. are open-ended terms, meaning "including but not limited to", and can be used interchangeably with each other. The words "or" and "and" used herein refer to the phrase "and / or", and can be used interchangeably with it, unless the context clearly indicates otherwise. The word "such as" used herein refers to the phrase "such as but not limited to", and can be used interchangeably with it.

[0076] It should also be noted that in the devices, equipment, and methods of this application, each component or each step can be decomposed and / or recombined. These decompositions and / or recombinations should be regarded as equivalent solutions of this application.

[0077] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of this application. Therefore, this application is not limited to the aspects shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0078] The above description has been given for purposes of illustration and description. In addition, this description does not limit the embodiments of this application to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. A construction progress management method based on aerial photography, characterized in that: include: Using a drone to perform oblique photography to obtain image sequences of the construction site at different times; wherein the image sequence includes GNSS data and IMU data; Based on the image sequence, establishing a three-dimensional point cloud of the target building; Based on the two sets of point cloud data of the target building at different periods, generating difference point cloud data of the target building at different periods; Determining the construction progress of the target building based on the difference point cloud data, the GNSS data and the IMU data; Wherein, generating difference point cloud data of the target building at different periods based on the two sets of point cloud data of the target building at different periods includes: Extracting feature data of the two sets of point cloud data; Based on the feature data, feature matching is performed on the two sets of point cloud data to generate a matching relationship set; Based on the matching relationship set, calculating the difference between the two sets of point cloud data to generate the difference point cloud data; The performing feature matching on the two sets of point cloud data based on the feature data to generate a matching relationship set includes: For any first point cloud in one of the two sets of point cloud data, query the other set of point cloud data in the two sets of point cloud data for a second point cloud that is closest to the first point cloud, and add the first point cloud and the corresponding second point cloud as a point cloud pair to a candidate set; For any pair of point cloud pairs in the candidate set, calculating a first distance value between a second point cloud of the point cloud pair and all first point clouds in the candidate set; If the first distance value is equal to the distance between the first point cloud and the second point cloud of the point cloud pair, determining the point cloud pair as a candidate point cloud pair; Select any three pairs of candidate point cloud pairs, and calculate the side lengths of the first triangle and the second triangle; wherein the first triangle is composed of the first point clouds of the three pairs of candidate point cloud pairs, and the second triangle is composed of the second point clouds of the three pairs of candidate point cloud pairs; If the side lengths of the first triangle and the second triangle meet a preset condition, the three pairs of candidate point cloud pairs are added to the matching relationship set; wherein the preset condition is: Among them, τ is a given threshold, a1, a2, a3 are the lengths of the three sides of the first triangle, b1, b2, b3 are the lengths of the three sides of the second triangle; The calculating the difference between the two sets of point cloud data based on the matching relationship set to generate the difference point cloud data comprises: For each point cloud pair in the matching relationship set, the Euclidean distance between the first point cloud and the second point cloud is calculated; wherein the calculation formula of the Euclidean distance is: |sRq+Tp|, wherein p is the first point cloud, q is the second point cloud, s is the scaling factor, R is the rotation matrix, and T is the translation vector; The error contribution of the point cloud pair is calculated by the error metric function; wherein the error metric function is x is the independent variable, and the error contribution is calculated as: ρ(|sRq+Tp|), where μ is the scaling factor of the error metric function; The error contributions of all point cloud pairs in the matching relationship set are accumulated to obtain the total error; wherein the calculation formula of the total error is: E(s,R,T)=∑ρ(|s·Rq+Tp|); Based on the minimization of the total error, an optimal scaling factor, an optimal rotation matrix and an optimal translation vector are calculated; Based on the optimal scaling factor, the optimal rotation matrix and the optimal translation vector, the second point cloud of all point cloud pairs in the matching relationship set is transformed into a transformed point cloud in the first point cloud coordinate system; wherein the calculation formula of the transformed point cloud is: Q ′ ={sRq+T|q∈Q}, where Q ′ is the transformed point cloud, Q is the second point cloud set; Calculating a second distance value between the transformed point cloud and the corresponding first point cloud; If the second distance value is greater than a preset distance threshold, the corresponding first point cloud and the transformed point cloud are taken as a difference point cloud pair.

2. The construction progress management method based on aerial photography according to claim 1 is characterized in that: The use of a drone for oblique photography to obtain image sequences of the construction site at different times includes: Correcting the exposure delay time of the camera of the UAV according to the cubic spline interpolation method; Differentiate the GNSS data according to the camera exposure delay time to obtain the relationship between the antenna phase center coordinates and the camera station coordinates; Based on the relationship between the antenna phase center coordinates and the camera station coordinates, the drone is used to perform oblique photography to obtain image sequences of the construction site at different times.

3. The construction progress management method based on aerial photography according to claim 2 is characterized in that: The method of correcting the exposure delay time of the camera of the drone according to the cubic spline interpolation method includes: According to GNSS sampling, the flight sampling points of the UAV are obtained; Based on the flight sampling points, a cubic spline function is used to perform cubic polynomial fitting to obtain the flight trajectory of the UAV; According to the interpolation calculation method, the exposure time for obtaining the antenna phase center coordinates of the UAV is obtained; Based on the exposure time of the antenna phase center coordinates of the drone, the camera exposure delay time of the drone is corrected.

4. The construction progress management method based on aerial photography according to claim 1 is characterized in that: The determining the construction progress of the target building based on the difference point cloud data, the GNSS data and the IMU data includes: Aligning the difference point cloud data, the GNSS data, and the IMU data; Based on the aligned difference point cloud data, the GNSS data and the IMU data, the construction progress of the target building is determined.

5. A construction progress management device based on aerial photography, characterized in that: include: An image acquisition module is used to use a drone to perform oblique photography to acquire image sequences of the construction site at different times; wherein the image sequence includes GNSS data and IMU data; A point cloud building module, used to build a three-dimensional point cloud of the target building based on the image sequence; A difference generation module, used for generating difference point cloud data of the target building at different periods based on the two sets of point cloud data of the target building at different periods; A progress determination module, configured to determine the construction progress of the target building based on the difference point cloud data, the GNSS data and the IMU data; Wherein, the difference generation module is further configured as follows: Extracting feature data of the two sets of point cloud data; Based on the feature data, feature matching is performed on the two sets of point cloud data to generate a matching relationship set; Based on the matching relationship set, calculating the difference between the two sets of point cloud data to generate the difference point cloud data; The difference generation module is further configured as follows: For any first point cloud in one of the two sets of point cloud data, query the other set of point cloud data in the two sets of point cloud data for a second point cloud that is closest to the first point cloud, and add the first point cloud and the corresponding second point cloud as a point cloud pair to a candidate set; For any pair of point cloud pairs in the candidate set, calculating a first distance value between a second point cloud of the point cloud pair and all first point clouds in the candidate set; If the first distance value is equal to the distance between the first point cloud and the second point cloud of the point cloud pair, determining the point cloud pair as a candidate point cloud pair; Select any three pairs of candidate point cloud pairs, and calculate the side lengths of the first triangle and the second triangle; wherein the first triangle is composed of the first point clouds of the three pairs of candidate point cloud pairs, and the second triangle is composed of the second point clouds of the three pairs of candidate point cloud pairs; If the side lengths of the first triangle and the second triangle meet a preset condition, the three pairs of candidate point cloud pairs are added to the matching relationship set; wherein the preset condition is: Among them, τ is a given threshold, a1, a2, a3 are the lengths of the three sides of the first triangle, b1, b2, b3 are the lengths of the three sides of the second triangle; The difference generation module is further configured as follows: For each point cloud pair in the matching relationship set, the Euclidean distance between the first point cloud and the second point cloud is calculated; wherein the calculation formula of the Euclidean distance is: |sRq+Tp|, wherein p is the first point cloud, q is the second point cloud, s is the scaling factor, R is the rotation matrix, and T is the translation vector; The error contribution of the point cloud pair is calculated by the error metric function; wherein the error metric function is x is the independent variable, and the error contribution is calculated as: ρ(|sRq+Tp|), where μ is the scaling factor of the error metric function; The error contributions of all point cloud pairs in the matching relationship set are accumulated to obtain the total error; wherein the calculation formula of the total error is: E(s,R,T)=∑ρ(|s·Rq+Tp|); Based on the minimization of the total error, an optimal scaling factor, an optimal rotation matrix and an optimal translation vector are calculated; Based on the optimal scaling factor, the optimal rotation matrix and the optimal translation vector, the second point cloud of all point cloud pairs in the matching relationship set is transformed into a transformed point cloud in the first point cloud coordinate system; wherein the calculation formula of the transformed point cloud is: Q ′ ={sRq+T|q∈Q}, where Q ′ is the transformed point cloud, Q is the second point cloud set; Calculating a second distance value between the transformed point cloud and the corresponding first point cloud; If the second distance value is greater than a preset distance threshold, the corresponding first point cloud and the transformed point cloud are taken as a difference point cloud pair.

6. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and the computer program is used to execute the method according to any one of claims 1 to 4.

7. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is used to execute the method described in any one of claims 1 to 4.

Images