Progress recognition method based on multi-modal remote sensing data

Abstract

The invention relates to the field of engineering construction progress identification, and provides a progress identification method based on multi-modal remote sensing data in order to improve the efficiency and the reliability at the same time, which comprises the following steps: establishing a component standard database and a target detection model; during project construction, regularly performing remote sensing image shooting on a construction area by using an unmanned aerial vehicle; obtaining overall three-dimensional point cloud data of the construction area based on the remote sensing image of the single flight; splicing the remote sensing images of the single flight to obtain an integral image of the construction area; target detection and point cloud identification are carried out; and performing cross validation on the two identification results to extract coordinate information, and determining the installation progress according to the extracted coordinate information. By adopting the mode, the reliability is improved while the efficiency is improved.

Description

Technical Field

[0001] The invention relates to the field of engineering construction progress identification, and in particular to a progress identification method based on multi-modal remote sensing data. Background Art

[0002] At present, the construction progress information of engineering construction mainly relies on the daily reports of subcontracting teams, but there are a lot of false reports, omissions and lies among subcontracting teams. In addition, the construction area is huge, and the limited project management personnel cannot check the construction progress manually. In summary, traditional methods are not only inefficient but also unreliable. Summary of the invention

[0003] In order to improve efficiency and reliability at the same time, the present application provides a progress identification method based on multimodal remote sensing data.

[0004] The technical solution adopted by the present invention to solve the above problems is:

[0005] The progress identification method based on multimodal remote sensing data includes:

[0006] Step 1: Establish a component standard database to store the size parameters and feature parameters of each component; establish and train a target detection model;

[0007] Step 2: During the project construction period, drones are used regularly to take remote sensing images of the construction area;

[0008] Step 3: Acquire the three-dimensional point cloud data of the entire construction area based on the remote sensing images of a single flight; stitch the remote sensing images of a single flight to obtain an image of the entire construction area;

[0009] Step 4: Target detection: Crop the overall image of the construction area into small photo datasets for target detection, and retain the actual location information corresponding to the photos; then input the photo dataset into the target detection model for detection and recognition;

[0010] Perform point cloud recognition: calculate the overall three-dimensional point cloud data of the construction area and perform detection and recognition in combination with the component standard database;

[0011] Step 5: Compare the results:

[0012] When both target detection and point cloud recognition detect the target object: if the target detection result is consistent with the point cloud recognition result, the target detection recognition frame coordinate information is extracted; if the target detection result is inconsistent with the point cloud recognition result, the point cloud data of the target detection recognition frame area is extracted and the point cloud recognition is performed again. If it is consistent with the target detection result, the target detection recognition frame coordinate information is extracted; if it is inconsistent, the point cloud data recognition result shall prevail and the target detection recognition frame coordinate information is extracted;

[0013] If target detection alone fails to detect the target object, then the point cloud coordinate information is extracted;

[0014] If the target object is not detected by point cloud recognition alone, the recognition result of the target detection is discarded;

[0015] Step 6: Determine the construction progress based on the extracted coordinate information.

[0016] Furthermore, the project is a photovoltaic project.

[0017] Furthermore, the target detection model is trained based on a model training database, which includes remote sensing images of different backgrounds, different weather conditions and different objects, wherein the different objects refer to photovoltaic panels, brackets and piles.

[0018] Furthermore, the characteristic parameters are:

[0019] The pile is a dense point cloud in the vertical direction with a fixed height above the ground;

[0020] The photovoltaic panel is a plane inclined to the ground;

[0021] The support includes columns, diagonal beams and purlins, among which the point cloud features of columns are similar to those of piles, but with a higher height;

[0022] The diagonal beam is a diagonal brace between columns, often accompanying columns, and inclined at a certain angle to the ground;

[0023] The purlin is the only horizontal component and is parallel to the ground.

[0024] Further, step 6 is specifically as follows: step 61, analyzing the string information in the CAD plan layout of the photovoltaic array area, extracting the coordinate information of each array and each string; and projecting and transforming the CAD coordinates into longitude and latitude coordinates;

[0025] Step 62, according to the analysis results of step 5 and the longitude and latitude coordinates, if a photovoltaic panel with a complete string length is detected, the string is installed; if a photovoltaic panel less than the complete string length is identified, the photovoltaic panel is being installed; if four complete purlins are identified, the bracket is installed; if a column or inclined beam or less than four purlins are identified, the bracket is being installed; if a pile is identified, the fixed-point piling is completed.

[0026] Furthermore, the UAV flies by imitating the ground and setting the altitude.

[0027] Furthermore, the UAV flight routes have heading overlap and lateral overlap.

[0028] Compared with the prior art, the present invention has the following beneficial effects:

[0029] 1. Improve efficiency: By analyzing multi-modal remote sensing image data and automatically identifying it, the project managers' ability to control the construction progress of the project site is greatly improved.

[0030] 2. Improve accuracy: Cross-validation is carried out by using point cloud data calculation and image set target detection. While ensuring efficiency, the accuracy of the recognition results is greatly improved, effectively avoiding problems such as construction personnel not working according to plan, misreporting, false reporting, and omission of daily construction progress, so that managers can grasp the construction progress more accurately.

[0031] 3. Cost savings: Through drone technology and multimodal remote sensing data recognition technology, efficient and accurate progress management can be achieved without a large increase in personnel investment.

[0032] 4. Widely applicable: This technical solution is applicable to various sites, whether it is different scenes such as grass, mud, snow, water, or different weather such as sunny, cloudy, and rainy days, this technical solution can be used for effective progress management. It has a wide range of applications and strong adaptability.

[0033] 5. Real-time monitoring: Relying on drone remote sensing technology, project managers can use drone remote sensing images of the site at any time to achieve real-time monitoring of construction progress. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 Flowchart of the progress identification method based on multimodal remote sensing data. DETAILED DESCRIPTION

[0035] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0036] Take the photovoltaic project as an example. Figure 1 As shown, the progress recognition method based on multimodal remote sensing data includes:

[0037] Step 1, establish a component standard database, which needs to store the size parameters and feature parameters of each component required in the installation process of the photovoltaic project array area string, which is used for point cloud computing processing in subsequent steps, where the feature parameters refer to the definition of each component. The pile is a dense point cloud in the vertical direction, and the height from the ground is relatively fixed; the bracket is divided into columns, inclined beams and purlins, among which the point cloud map features of the column are similar to those of the pile, but the height is higher. The inclined beam is the diagonal support between the columns, which often appears with the columns and is inclined at a certain angle to the ground. The purlin is the only horizontal component, parallel to the ground, and has a long length; the photovoltaic panel is a plane inclined to the ground, with a high point cloud density, which can be directly determined by extracting size information.

[0038] Establish and train a target detection model; in this embodiment, the target detection model is trained based on a model training database, which includes remote sensing images of different backgrounds (grass, mud, snow, water, etc.), different weather (sunny, cloudy, rainy) and different objects (photovoltaic panels, brackets, piles).

[0039] Step 2: During the project construction, drones are used to regularly take remote sensing images of the construction area. The flight altitude is set by imitating the terrain, and the flight altitude can be adjusted according to the actual accuracy and efficiency. When planning the route, a certain degree of heading overlap and lateral overlap must be ensured to ensure sufficient image information for subsequent 3D modeling.

[0040] Step 3: Obtain the three-dimensional point cloud data of the entire construction area based on the remote sensing images of a single flight; stitch the remote sensing images of a single flight to obtain an image of the entire construction area.

[0041] By processing and calculating the overlap, distortion and other information of the photos taken by the drone at different locations and times, the three-dimensional data of the site is obtained, and modeling is performed based on the three-dimensional data to complete the extraction of the site point cloud data.

[0042] The image data is synthesized, and the complete image pattern is cropped into a small photo data set for target detection, and the actual location information corresponding to the photo is retained (image cropping ensures that the detection of the component is a large target detection and improves the detection success rate).

[0043] Step 4: Target detection: Regularly crop the overall image of the construction area into small photo datasets for target detection, and retain the actual location information corresponding to the photos; then input the photo dataset into the target detection model for detection and identification; the detection objects include photovoltaic panels, purlins, columns, inclined beams, and piles.

[0044] Perform point cloud recognition: calculate the overall three-dimensional point cloud data of the construction area, and perform detection and recognition in combination with the component standard database.

[0045] Step 5: Compare the results:

[0046] When both target detection and point cloud recognition detect the target object: if the target detection result is consistent with the point cloud recognition result, the target detection recognition frame coordinate information is extracted; if the target detection result is inconsistent with the point cloud recognition result, the point cloud data of the target detection recognition frame area is extracted and the point cloud recognition is performed again. If it is consistent with the target detection result, the target detection recognition frame coordinate information is extracted; if it is inconsistent, the point cloud data recognition result shall prevail and the target detection recognition frame coordinate information is extracted;

[0047] If target detection alone fails to detect the target object, then the point cloud coordinate information is extracted;

[0048] If the target object is not detected by point cloud recognition alone, the recognition result of the target detection is discarded.

[0049] When re-performing point cloud recognition on the point cloud data in the target detection recognition box area, the point cloud data can be directly compared with the component standard database to obtain the recognition result, and then the recognition result can be verified with the recognition result of the target detection; the recognition result of the target detection can also be directly verified. If the verification results are inconsistent, they are compared with the component standard database to obtain the final recognition result.

[0050] Step 6: Determine the installation progress based on the extracted coordinate information, specifically:

[0051] Step 61, analyzing the string information in the CAD plan layout of the photovoltaic array area, extracting the coordinate information of each array and each string; and projecting and transforming the CAD coordinates into longitude and latitude coordinates;

[0052] Step 62, according to the analysis results of step 5 and the longitude and latitude coordinates, if a photovoltaic panel with a complete string length is detected, the string is installed; if a photovoltaic panel less than the complete string length is identified, the photovoltaic panel is being installed; if four complete purlins are identified, the bracket is installed; if a column or inclined beam or less than four purlins are identified, the bracket is being installed; if a pile is identified, the fixed-point piling is completed.

[0053] Single target detection is easily affected by external factors such as weather, resulting in missed detection, and the detection success rate of small targets such as piles is low. Secondly, the random placement of photovoltaic panels, brackets, etc. will cause false detection of target detection. When only point cloud data is used for component detection, in order to meet the recognition accuracy requirements, the drone needs to be equipped with a laser radar. The laser point cloud data has a large amount of calculation and cannot guarantee efficiency, which does not meet the actual progress management needs. Therefore, the present invention is based on the image set and three-dimensional point cloud data obtained by remote sensing image processing, and uses target detection and point cloud computing results cross-validation to perform final component detection. Compared with single target detection, the accuracy is higher; compared with single point cloud data detection, the efficiency is higher.

Claims

1. A progress identification method based on multimodal remote sensing data, characterized in that: include: Step 1: Establish a component standard database to store the size parameters and feature parameters of each component; Build and train the target detection model; Step 2: During the project construction period, drones are used regularly to take remote sensing images of the construction area; Step 3: Acquire the three-dimensional point cloud data of the entire construction area based on the remote sensing images of a single flight; stitch the remote sensing images of a single flight to obtain an image of the entire construction area; Step 4: Target detection: Crop the overall image of the construction area into small photo datasets for target detection, and retain the actual location information corresponding to the photos; then input the photo dataset into the target detection model for detection and recognition; Perform point cloud recognition: calculate the overall three-dimensional point cloud data of the construction area and perform detection and recognition in combination with the component standard database; Step 5: Compare the results: When both target detection and point cloud recognition detect the target object: if the target detection result is consistent with the point cloud recognition result, the coordinate information of the target detection recognition box is extracted; If the target detection result is inconsistent with the point cloud recognition result, the point cloud data of the target detection recognition frame area is extracted and the point cloud recognition is performed again. If it is consistent with the target detection result, the coordinate information of the target detection recognition frame is extracted; If there is any inconsistency, the point cloud data recognition result shall prevail, and the coordinate information of the target detection recognition frame shall be extracted; If target detection alone fails to detect the target object, then the point cloud coordinate information is extracted; If the target object is not detected by point cloud recognition alone, the recognition result of the target detection is discarded; Step 6: Determine the construction progress based on the extracted coordinate information.

2. The progress identification method based on multimodal remote sensing data according to claim 1, characterized in that: The project is a photovoltaic project.

3. The progress identification method based on multimodal remote sensing data according to claim 2, characterized in that: The target detection model is trained based on the model training database, which includes remote sensing images of different backgrounds, different weather conditions and different objects, where the different objects refer to photovoltaic panels, brackets and piles.

4. The progress identification method based on multimodal remote sensing data according to claim 3 is characterized in that: The characteristic parameters are: The pile is a dense point cloud in the vertical direction with a fixed height above the ground; The photovoltaic panel is a plane inclined to the ground; The support includes columns, diagonal beams and purlins, among which the point cloud features of columns are similar to those of piles, but with a higher height; The diagonal beam is a diagonal brace between columns, often accompanying columns, and inclined at a certain angle to the ground; The purlin is the only horizontal component and is parallel to the ground.

5. The progress identification method based on multimodal remote sensing data according to claim 4 is characterized in that: Step 6 is as follows: Step 61, analyzing the string information in the CAD plan layout of the photovoltaic array area, extracting the coordinate information of each array and each string; and projecting and transforming the CAD coordinates into longitude and latitude coordinates; Step 62, according to the analysis results of step 5 and the longitude and latitude coordinates, if a photovoltaic panel with a complete string length is detected, the string is installed; if a photovoltaic panel less than the complete string length is identified, the photovoltaic panel is being installed; if four complete purlins are identified, the bracket is installed; if a column or inclined beam or less than four purlins are identified, the bracket is being installed; if a pile is identified, the fixed-point piling is completed.

6. The progress identification method based on multimodal remote sensing data according to claim 1, characterized in that: The UAV flies by imitating the ground and setting the altitude.

7. The progress identification method based on multimodal remote sensing data according to any one of claims 1 to 6, characterized in that: The UAV flight routes have heading overlap and lateral overlap.

Images