Construction progress intelligent monitoring method for multi-temporal images of unmanned aerial vehicle

Abstract

The present application belongs to the field of artificial intelligence and computer vision technology, and relates to a kind of unmanned aerial vehicle multi-temporal image construction progress intelligent monitoring method.The method comprises the following steps:S1, data preparation and pretreatment, including unmanned aerial vehicle orthophoto cutting and data set construction and stage division;S2, model training and segmentation prediction, including model architecture judgment selection and model training optimization and inference output;S3, image stitching and grid conversion, including image block stitching to restore the original image size and segmentation result processing and grid conversion;S4, period calibration and construction progress demonstration.The present application realizes high-precision identification for different construction component forms, improves the robustness and adaptability of the model;secondly, combined with the spatial overlap relationship of surface elements, the first appearance time of each construction component surface stage is automatically determined, effectively reflecting the dynamic evolution of the construction process;finally, the results can be output in standard geographic vector data format, with good expandability.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence and computer vision technology, and relates to a method for intelligent monitoring of construction progress using multi-temporal images from unmanned aerial vehicles. Background Technology

[0002] Construction progress is a core element of project management, directly reflecting the efficiency of construction plan execution, resource allocation capabilities, and safety risk control level. An efficient and accurate construction progress monitoring system not only helps in the effective control of schedule and cost but also provides solid support for subsequent quality acceptance, completion filing, and operation and maintenance management. Traditional methods of obtaining progress mainly rely on manual inspections, on-site reports, and drawing annotations, which suffer from high labor intensity, strong subjectivity, and delayed updates, making it difficult to meet the high-frequency, fast-paced demands of current construction sites.

[0003] With the continuous development of the smart construction site concept, automated monitoring of construction progress based on emerging technologies such as artificial intelligence and remote sensing has become a research hotspot. In recent years, breakthroughs in deep learning in the field of image recognition have made it possible to monitor construction progress using orthophotos from unmanned aerial vehicles (UAVs). However, in practical applications, several challenges remain: on the one hand, the construction process involves frequent phase changes, and single-phase images are difficult to present continuous progress evolution, limiting the expression of the overall project dynamics; on the other hand, activities such as bridge pier construction, beam and slab laying, and road construction have clear structural logic and evolutionary sequence, and current systems have not effectively utilized this engineering semantic information, resulting in a lack of coherence, accuracy, and interpretability in the analysis of multi-temporal construction states.

[0004] Therefore, there is an urgent need to build an intelligent monitoring method that integrates semantic recognition and construction progress evolution analysis capabilities, so as to realize the transformation of the construction process from "static recording" to "dynamic perception" and from "manual judgment" to "intelligent analysis", and promote the development of project management towards a more efficient, accurate and systematic direction. Summary of the Invention

[0005] This invention proposes a construction progress monitoring method for smart construction site scenarios, which integrates the intelligent recognition of UAV orthophotos with the ability to analyze the evolution of the construction process. It aims to achieve accurate monitoring of construction components (such as bridge piers, beams, slabs, and road surfaces) at different time stages through automatic recognition and geographic location labeling, and output standard geographic vector data format. This solves the problems of high reliance on manual labor, lack of structural semantic support for recognition results, and difficulty in effectively presenting the dynamic evolution of the construction process in existing technologies.

[0006] The technical solution adopted in this invention is,

[0007] A method for intelligent monitoring of construction progress using multi-temporal imagery from unmanned aerial vehicles (UAVs) includes the following steps:

[0008] S1. Data Preparation and Preprocessing

[0009] Step S1.1: Cropping of UAV orthophotos

[0010] Using orthophoto data acquired by a UAV as input, preprocessing operations are performed on the original image. First, the original orthophoto is loaded via a path, and its basic information such as resolution and image size is extracted. The initial patch size is set to a fixed size, and the entire original image is cropped into several image patches of the same size. If the original image size is not divisible by the set patch size, zero-padding is applied to the original image to expand its size to a size divisible by the patch size.

[0011] The method for calculating image padding size is as follows:

[0012]

[0013] Where pad_height represents the pixels used to pad the height, pad_width represents the pixels used to pad the width, H represents the original image height, W represents the original image width, P is the patch size, all in pixels, and mod is the modulo operation.

[0014] Calculate the pixels needed for padding, then calculate the new height and width, using the following formula:

[0015]

[0016] Where H' is the height after padding, and W' is the width after padding.

[0017] The size of the padded image is an integer multiple of the patch size. Then, the number of rows and columns N of the patch is calculated. row With N col The expression is as follows:

[0018]

[0019] After completing the image, crop it into patches and calculate the coordinates of each patch. For the patch in row 'row' and column 'col', the following steps can be taken:

[0020]

[0021] Where start_row and end_row are the row ranges of the original image, start_col and end_col are the column ranges of the original image, and P is the patch size.

[0022] Each cropped patch undergoes standardization, normalizing pixel values ​​to the range [0, 255] and converting the data type to uint8. Subsequently, based on the number of image channels, different storage modes are used to save the results. For single-channel images, they are saved in grayscale mode; for three-dimensional images with only one channel, dimensionality compression is performed before saving as grayscale; for three-channel images, they are saved in RGB mode; in four-channel images, only the first three channels are retained for RGB saving, discarding transparency information; if the image has more than four channels, the first three channels are also truncated for saving; all images are saved in PNG format.

[0023] Step S1.2: Constructing the dataset and dividing the stages.

[0024] The manually labeled JSON files are converted into corresponding mask images, generating label images that correspond one-to-one with the cropped patches. The data is divided into training, validation, and test sets according to a set ratio, and saved uniformly in PNG format.

[0025] S2, Model Training and Segmentation Prediction

[0026] Step S2.1: Determining and Selecting the Model Architecture

[0027] During the task judgment phase, the required segmentation and recognition type T is determined based on the component attributes of the target object, and then the most suitable network model structure is matched. The judgment logic is as follows:

[0028]

[0029] Where M represents the selected model structure and T represents the current task type label.

[0030] Step S2.2: Model Training Optimization and Inference Output

[0031] During the model adaptation phase, a corresponding training process is constructed based on the selected model architecture. The cross-entropy loss function is used throughout the training process to measure model performance, taking into account both classification and regression error optimization.

[0032] L cls Cross-entropy loss, used to evaluate the accuracy of the model in multi-class discrimination tasks, is defined as follows:

[0033]

[0034] Among them, y i For real category labels, To predict category labels, C represents the number of categories;

[0035] For both types of models mentioned above, the Adam optimizer is used, and a cosine annealing learning rate decay strategy is introduced. The learning rate update strategy is defined as follows:

[0036]

[0037] Where, η t Let η be the current learning rate. max For the maximum learning rate, η min To minimize the learning rate, T cur For the current number of times, T i This represents the total number of training sessions.

[0038] After the model achieves stable convergence on the validation set, the optimal weight file is automatically saved, and the model enters the inference phase. During inference, the optimal weights obtained in the training phase are loaded, and prediction operations are performed on the test images in batches, outputting semantic segmentation results. Each result image is represented in the form of a mask image, marking the location and category information of the construction area.

[0039] All inference results are saved in a pixel-level format as .PNG, along with the index coordinates (i,j) of each pixel block in the whole image.

[0040] S3, Image stitching and raster conversion

[0041] Step S3.1: Image blocks are stitched together to restore the original image size.

[0042] After the model completes pixel-level inference for each image patch, the prediction results of all image patches need to be spatially stitched and cropped to finally output a segmentation result map with geographic reference information.

[0043] The specific process is as follows:

[0044] Each image patch generated during the inference stage is saved according to the naming convention image_name_row_col.png, where row and col represent the row and column indices of the image patch in the original image grid, respectively. Before stitching the images, the size of the patch grid structure must first be calculated based on the index information to facilitate subsequent embedding of each patch into the grid according to its index. The calculation formula is as follows:

[0045]

[0046] Where, N r N c These represent the total number of rows and columns of the patch grid, respectively.

[0047] Subsequently, the size information of each image patch is extracted from the input patch file, obtaining the pixel size P and the number of channels C for each image patch. At the same time, the geographic image information of the original image is read, including the original image size, number of bands, data type, coordinate reference system, and geographic boundaries.

[0048] Based on the patch grid structure and original image parameters, an array structure for image reconstruction is created. For grayscale or single-channel images, the image reconstruction array is defined as follows:

[0049] I rec =zeros((N r ×P,N c ×P,C)) (9)

[0050] Among them, I rec To reconstruct the image array, N r ×P represents the height of the reconstructed image, N c ×P represents the width of the reconstructed image, and C represents the number of channels;

[0051] Based on the row and column indices of each image patch in the grid, the corresponding insertion region in the reconstructed image is calculated as follows:

[0052]

[0053] Where ROW and COL are the row and column coordinates of the current image patch, min(rows) and min(cols) are the minimum row and column values ​​in the patch grid, P is the pixel size of the corresponding image patch, start_row and end_row are the row range of the reconstructed image, and start_col and end_col are the column range of the reconstructed image.

[0054] To ensure that the image segmentation prediction results remain consistent with the original image format during the stitching and reconstruction process, an intelligent adaptation mechanism for the number of image channels was designed. For a single-channel original image, if the image patch has multiple channels, only the first channel is used; for a three-channel (RGB) original image, if the image patch has a single channel, it is copied three times to form an RGB format; if the number of channels exceeds three, the first three channels are retained; for a four-channel (RGBA) image, if the image patch has a single channel or three channels, an alpha channel is automatically added and set to opaque; if it exceeds four channels, the first four channels are truncated; for other special channel cases, adaptation is performed through dimensional expansion or channel truncation.

[0055] Subsequently, each image patch is sequentially filled into the reconstructed image array according to its position in the reconstructed image. The images are processed differently depending on the number of channels: when the original image is a single-channel image (grayscale), a two-dimensional array index is used for cropping to directly remove the padding area and maintain the single-channel structure; when the original image is a multi-channel image, a three-dimensional array index is used for cropping to remove the padding area. The specific formulas are as follows:

[0056] Single-channel image:

[0057] I cropped =I rec [0:H original ,0:W original (11)

[0058] Multi-channel images:

[0059] I cropped =I rec [0:H original ,0:W original ,:] (12)

[0060] Among them, H original W represents the height of the original input image. original I is the width of the original input image. cropped The final image after cropping, i.e., removing the padding area, I rec Reconstruct the image containing the padding region.

[0061] Finally, the original georeferenced information is reassigned to the stitched and cropped images, and the images are saved in GeoTIFF format to generate a semantic segmentation result map containing coordinate information.

[0062] Step S3.2: Segmentation Result Processing and Raster Conversion

[0063] The results of inference on the network model are analyzed and processed to extract the raster features and geographic information of the target area, and the GeoTIFF file saved by the inference results is converted into a shapefile file.

[0064] The specific processing procedure is as follows:

[0065] First, load the GeoTIFF format semantic segmentation file, count the total number of target pixels in the file, and calculate its proportion in the entire image. The specific formula is as follows:

[0066]

[0067] Where, N target N represents the target number of pixels. totalI represents the total number of pixels. raster (i,j) represents the pixel value at position (i,j);

[0068] Subsequently, a binary mask is created, marking pixels with a value of 1 in the target region as 1 and all other pixels as 0. The specific formula is as follows:

[0069]

[0070] Among them, M binany Let i be a binary mask matrix, and j be the pixel coordinates.

[0071] Based on this binary mask, an 8-connectivity algorithm is used to identify independent regions. By excluding background labels with a pixel value of 0, independent target regions with a pixel value of 1 are obtained and represented as a label set:

[0072] L = {l1, l2, ..., l n} (15)

[0073] Where L is the set of all independent regions, l i The label for the i-th independent region;

[0074] For each individual region, extract its boundary contour and generate the corresponding polygon object:

[0075] M=Polygon(C)={(x1,y1),(x2,y2),...,(x n ,y n (16)

[0076] Where C is the contour coordinate sequence, (x i ,y i ) represents the coordinates of the i-th vertex on the contour, and M is the polygonal geometry object that generates the pair.

[0077] Based on the outline, the area and perimeter of each polygon are calculated to quantify the completed construction area. The calculation formula is as follows:

[0078]

[0079] Where A is the area of ​​the polygon, Z is the perimeter, (x i ,y i ) represents the vertex coordinates, and n represents the number of vertices;

[0080] Next, the pixel coordinates of each extracted independent region's contour coordinates are transformed into geographic coordinates. The transformation process is based on the geographic radiometric transformation relationship of the image, and the mathematical expression is as follows:

[0081] (x geo ,y geo)=T·(x pixel ,y pixel (18)

[0082] Among them, (x geo ,y geo (x) represents the transformed geographic coordinates. pixel ,y pixel ) represents the image pixel coordinates, and T represents the geographic transformation matrix;

[0083] To ensure that the output vector data has consistent and accurate measurement information in a spatial reference system, it is necessary to convert the area and perimeter calculation results from the geographic coordinate system to the metric system. The mathematical expression for the unit conversion is as follows:

[0084] Area unit conversion formula:

[0085]

[0086] Among them, area m2 area represents square meters. deg2 Represented as area in square degrees, lat tom This indicates the number of meters corresponding to 1 degree of latitude. tom It represents the number of meters corresponding to 1 degree of longitude.

[0087] Length unit conversion formula:

[0088]

[0089] Where (lat1, lon1) are the latitude and longitude coordinates of the starting point of the line segment, (lat2, lon2) are the latitude and longitude coordinates of the ending point of the line segment, and L is the length of the line segment.

[0090] Finally, the number of targets, total area, and average area in the raster are extracted and saved as GeoDataFrame files. Then, the GeoDataFrame files are output as vector-format shapefiles.

[0091] S4. Time Marking and Construction Progress Demonstration

[0092] To achieve precise tracking of the evolution of construction areas over time, this method uses multi-period Shapefiles to perform spatial overlay analysis on key construction targets. By determining the spatial overlap between targets in different periods and following the rule of "overlapping targets are assigned to earlier periods," the method achieves accurate identification of the period to which components belong.

[0093] In practice, the process first iterates through all Shapefiles in the specified directory, automatically extracts the phase identifiers contained in the filenames, and loads the vector data for each construction phase in chronological order. For the data of each phase, the geometric information of the specified target object is extracted, including but not limited to the polygon boundary coordinates and corresponding area, and combined with its phase label, a standardized data structure is constructed for subsequent comparison and attribution determination.

[0094] In the period attribution determination process, all loaded target objects are first organized into a target list in chronological order, and a unique label identifier and its corresponding geographic coordinates are generated for each target object in the list. Then, spatial overlap analysis is performed on each object in the current period and all objects recorded in all historical periods: if the current target object has a spatial overlap area with a historical target in the record list, it is determined that the area covered by the target has been completed in an earlier construction phase, and the target is automatically assigned to the earliest period in the historical record; if there is no spatial overlap, the target is determined to be a newly added construction area and assigned to the current period Ti. This mechanism strictly follows the principle of "overlapping areas are assigned to earlier periods," ensuring that any spatial area is assigned to only one construction period within the construction cycle, thereby avoiding duplicate statistics and improving the accuracy of construction progress identification and the consistency of temporal logic.

[0095] In the spatial overlap and clipping stage, vector data from each stage is processed sequentially according to a set time order, extracting all valid geometric objects and their attributes from each stage's layer. For each object in the current stage, a spatial difference operation is performed with all previously assigned geometric sets to remove areas overlapping with historical stages, retaining only the newly added parts. If the clipped area is a polyhedral structure, the area ratio of each sub-polygon is calculated, and its area attribute field is updated. Simultaneously, an area threshold control mechanism is introduced to remove fragmented areas smaller than the minimum area threshold, improving the geometric quality and expressive effectiveness of the results. After clipping and filtering, a geometric merging operation is performed on all valid objects in the current stage to generate the final spatial calibration result, retaining the attribute information of each component, including the actual clipped area and the normalized ratio value, providing standardized and unique spatial data support for accurate analysis and visualization of construction progress.

[0096] In the construction completion rate calculation stage, object identification and quantity statistics are performed based on the YOLOv11 semantic segmentation model. To ensure the consistency and temporal comparability of the statistical results, a strategy of "prioritizing overlapping and assigning to earlier times" is adopted, assigning duplicate detected targets to the time node of their first appearance. Based on this rule, the number of newly added objects at each time node and their proportion to the total number of targets can be automatically calculated.

[0097] Simultaneously, in the spatial overlap clipping process, spatial difference operations are performed on the repeated regions in different periods. Then, the area of ​​the clipped vectors is calculated, and their proportion in the entire image is determined.

[0098]

[0099] Among them, P road P represents the area of ​​the road surface after cutting. total Let A be the area of ​​the entire graph. road This represents the road surface coverage rate for the current period.

[0100] By combining the segmentation label map, the beam and slab laying areas can be identified, and their areas can be compared with the total laying area in the construction plan to calculate the current beam and slab laying completion rate. The calculation formula is as follows:

[0101]

[0102] Among them, A beam A represents the actual area of ​​the area where beams and slabs have been laid after spatial trimming. beam_target R is the total planned area for the planned laying. beam The completion rate of beam and slab laying.

[0103] By comprehensively analyzing the number of bridge piers, road surface coverage, and beam and slab paving completion rate, the construction progress and spatial evolution of key components at the construction site are fully reflected, providing scientific and reliable data support and decision-making basis for construction management and dynamic adjustments.

[0104] The beneficial effects of this invention are reflected in the following aspects: First, a deep semantic segmentation and recognition framework integrating YOLOv11 and DeepLabv3+ is constructed, achieving high-precision recognition for different construction component shapes and improving the robustness and adaptability of the model; Second, by combining the spatial overlap relationship of surface elements, the first occurrence phase of each construction component surface stage is automatically determined, effectively reflecting the dynamic evolution of the construction process; Finally, it supports outputting results in a standard geographic vector data format, has good scalability, provides standardized data support for construction reporting and project management, and promotes the technological innovation and application of smart construction, digital supervision, and multi-department collaborative construction. Attached Figure Description

[0105] Figure 1 This is a flowchart of the present invention;

[0106] Figure 2 The image shows the prediction results for an example. Detailed Implementation

[0107] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0108] The complete steps of the method of the present invention are as follows:

[0109] A method for intelligent monitoring of construction progress using multi-temporal imagery from unmanned aerial vehicles (UAVs) includes the following steps:

[0110] S1. Data Preparation and Preprocessing

[0111] Step S1.1: Cropping of UAV orthophotos

[0112] Using orthophoto data acquired by UAVs as input, the original images are first preprocessed to meet the size specifications and data consistency requirements of the subsequent semantic segmentation model during the training and inference stages.

[0113] Specifically, the original orthophoto is first loaded via a path, and its basic information, such as resolution and image size, is extracted to provide data support for subsequent image cropping. The initial cropping patch size is set to a fixed size (the size parameter can be set according to the proportion of actual ground features in the image), and the entire original image is cropped into several image patches of the same size. If the original image size is not divisible by the set patch size, to avoid loss of image edge information or increased processing complexity, the original image is zero-padded (i.e., background areas with pixel values ​​of 0 are filled on the right and bottom sides), expanding its size to a size divisible by the patch size.

[0114] For example, if the input image size is 1024×1024 pixels and the patch is set to 512×512 pixels, it can be directly cropped into 4 image blocks. If the input image size is 1280×1280 pixels, cropping will result in 4 complete 512×512 image blocks and a residual area of ​​256×256. To preserve all image information and maintain the uniformity of patch division, background areas with pixel values ​​of 0 will be automatically filled on the right and bottom sides of the image, expanding the entire image size to 1536×1536 pixels, making it divisible into 9 complete 512×512 image blocks. The image padding size calculation method is as follows:

[0115]

[0116] Where pad_height represents the pixels used to pad the height, pad_width represents the pixels used to pad the width, H represents the original image height, W represents the original image width, P is the patch size, all in pixels, and mod is the modulo operation.

[0117] Calculate the pixels needed for padding, then calculate the new height and width, using the following formula:

[0118]

[0119] Where H' is the height after padding, and W' is the width after padding.

[0120] The size of the padded image is an integer multiple of the patch size. Then, the number of rows and columns N of the patch is calculated. row With N col The expression is as follows:

[0121]

[0122] After completing the image, crop it into patches and calculate the coordinates of each patch. For the patch in row 'row' and column 'col', the following steps can be taken:

[0123]

[0124] Where start_row and end_row are the row ranges of the original image, start_col and end_col are the column ranges of the original image, and P is the patch size.

[0125] Each cropped patch needs to be standardized, normalizing pixel values ​​to the range [0, 255] and converting the data type to uint8. Then, based on the number of image channels, different storage modes are used to save the results. For single-channel images, they are saved in grayscale mode; for three-dimensional images with only one channel, dimensionality compression is performed before saving as grayscale; for three-channel images, they are saved in RGB mode; for four-channel images, only the first three channels are retained for RGB saving, discarding transparency information; if the number of image channels is greater than 4, the first three channels are also truncated and saved; all images are saved in PNG format.

[0126] Step S1.2: Constructing the dataset and dividing the stages.

[0127] The manually labeled JSON files are converted into corresponding mask images, generating label images that correspond one-to-one with the cropped patches. The data is divided into training, validation, and test sets according to a set ratio (e.g., 7:2:1) and saved uniformly in PNG format for easy and effective use by subsequent network models.

[0128] S2, Model Training and Segmentation Prediction

[0129] Step S2.1: Determining and Selecting the Model Architecture

[0130] During the task judgment phase, the required segmentation and recognition type T is determined based on the component attributes of the target object, and then the most suitable network model structure is matched. The judgment logic is as follows:

[0131]

[0132] Where M represents the selected model structure and T represents the current task type label.

[0133] Step S2.2: Model Training Optimization and Inference Output

[0134] During the model adaptation phase, a corresponding training process is constructed based on the selected model architecture. The cross-entropy loss function is used throughout the training process to measure model performance, taking into account both classification and regression error optimization.

[0135] L cls Cross-entropy loss, used to evaluate the accuracy of the model in multi-class discrimination tasks, is defined as follows:

[0136]

[0137] Among them, y i For real category labels, To predict category labels, C represents the number of categories;

[0138] For both types of models mentioned above, the Adam optimizer is used, and a cosine annealing learning rate decay strategy is introduced to improve training efficiency and model generalization ability. The learning rate update strategy is defined as follows:

[0139]

[0140] Where, η t Let η be the current learning rate. max For the maximum learning rate, η min To minimize the learning rate, T cur For the current number of times, T i This represents the total number of training sessions.

[0141] After the model achieves stable convergence on the validation set, the optimal weight file is automatically saved, and the model enters the inference phase. During inference, the optimal weights obtained in the training phase are loaded, and prediction operations are performed on the test images in batches, outputting semantic segmentation results. Each result image is represented in the form of a mask, marking the location and category information of the construction area (such as bridge piers, road surfaces, beams, etc.).

[0142] All inference results are saved in a pixel-level format as .PNG, along with the index coordinates (i,j) of each pixel block in the whole image, to facilitate downstream modules in performing image stitching and reconstruction of construction areas (piers, road surfaces, beams, etc.).

[0143] S3, Image stitching and raster conversion

[0144] Step S3.1: Image blocks are stitched together to restore the original image size.

[0145] After the model completes pixel-level inference for each image patch, this step, in order to achieve the restoration and unified representation of the entire UAV orthophoto, requires spatial stitching and size cropping of the prediction results for all image patches, ultimately outputting a segmentation result map with georeferenced information. The specific process is as follows:

[0146] Each image patch generated during the inference stage is saved according to the naming convention image_name_row_col.png, where row and col represent the row and column indices of the image patch in the original image grid, respectively. Before stitching the images, the size of the patch grid structure must first be calculated based on the index information to facilitate subsequent embedding of each patch into the grid according to its index. The calculation formula is as follows:

[0147]

[0148] Where, N r N c These represent the total number of rows and columns of the patch grid, respectively.

[0149] Subsequently, the size information of each image patch is extracted from the input patch file, obtaining the pixel size P and the number of channels C for each image patch. At the same time, the geographic image information of the original image is also read, including the original image size, number of bands, data type, coordinate reference system, and geographic boundaries, providing necessary basic information support for subsequent image reconstruction.

[0150] Based on the patch grid structure and original image parameters, an array structure for image reconstruction is created. For grayscale or single-channel images, the image reconstruction array is defined as follows:

[0151] I rec =zeros((N r ×P,N c ×P,C)) (9)

[0152] Among them, I rec To reconstruct the image array, N r ×P represents the height of the reconstructed image, N c ×P represents the width of the reconstructed image, and C represents the number of channels;

[0153] Based on the row and column indices of each image patch in the grid, the corresponding insertion region in the reconstructed image is calculated as follows:

[0154]

[0155] Where ROW and COL are the row and column coordinates of the current image patch, min(rows) and min(cols) are the minimum row and column values ​​in the patch grid, P is the pixel size of the corresponding image patch, start_row and end_row are the row range of the reconstructed image, and start_col and end_col are the column range of the reconstructed image. Using these index ranges, each image patch can be accurately stitched to its corresponding position in the reconstructed image.

[0156] To ensure that the image segmentation prediction results remain consistent with the original image format during the stitching and reconstruction process, an intelligent adaptation mechanism for the number of image channels was designed. For a single-channel original image, if the image patch is multi-channel, only the first channel is used; for a three-channel (RGB) original image, if the image patch is single-channel, it is copied three times to form an RGB format; if the number of channels exceeds three, the first three channels are retained; for a four-channel (RGBA) image, if the image patch is single-channel or three-channel, an alpha channel is automatically added and set to opaque; if it exceeds four channels, the first four channels are truncated; for other special channel cases, adaptation is performed through dimensional expansion or channel truncation. This mechanism ensures that it can adapt to various formats of the input image patch and correctly restore it to a format consistent with the original image, significantly improving the compatibility and stability in deep learning prediction result processing, image format conversion, and image patch stitching.

[0157] Subsequently, each image patch is sequentially filled into the reconstructed image array according to its position in the reconstructed image. Since some parts of the original input image have padding regions added during the initial reconstruction, these regions need to be cropped during the reconstruction process. The image is processed differently depending on the number of channels: when the original image is a single-channel image (grayscale), two-dimensional array indexing is used for cropping to directly remove the padding regions and maintain the single-channel structure; when the original image is a multi-channel image, three-dimensional array indexing is used for cropping to remove the padding regions. The specific formula is as follows:

[0158] Single-channel image:

[0159] I cropped =I rec [0:H original ,0:W original (11)

[0160] Multi-channel images:

[0161] I cropped =I rec [0:H original ,0:W original ,:] (12)

[0162] Among them, Horiginal W represents the height of the original input image. original I is the width of the original input image. cropped The final image after cropping, i.e., removing the padding area, I rec Reconstruct the image containing the padding region.

[0163] Finally, the original georeferenced information is reassigned to the stitched and cropped images, and the images are saved in GeoTIFF format to generate a semantic segmentation result map containing coordinate information.

[0164] Step S3.2: Segmentation Result Processing and Raster Conversion

[0165] This step primarily involves analyzing and processing the results of the network model inference, extracting the raster features and geographic information of the target area, and converting the GeoTIFF file saved from the inference results into a shapefile file. This provides fundamental support for subsequent period determination and construction progress assessment. The specific processing flow is as follows:

[0166] First, load the GeoTIFF format semantic segmentation file, count the total number of target pixels (e.g., pixel value 255) in the file, and calculate its proportion in the entire image. The specific formula is as follows:

[0167]

[0168] Where, N target N represents the target number of pixels. total I represents the total number of pixels. raster (i,j) represents the pixel value at position (i,j);

[0169] Next, a binary mask is created, marking pixels with a value of 255 in the target area as 1, and all other pixels as 0. The specific formula is as follows:

[0170]

[0171] Among them, M binany Let i be a binary mask matrix, and j be the pixel coordinates.

[0172] Based on this binary mask, an 8-connectivity algorithm is used to identify independent regions. By excluding background labels with a pixel value of 0, independent target regions with a pixel value of 1 are obtained and represented as a label set:

[0173] L = {l1, l2, ..., l n} (15)

[0174] Where L is the set of all independent regions, l i The label for the i-th independent region;

[0175] For each individual region, extract its boundary contour and generate the corresponding polygon object:

[0176] M=Polygon(C)={(x1,y1),(x2,y2),...,(x n ,y n (16)

[0177] Where C is the contour coordinate sequence, (x i ,y i ) represents the coordinates of the i-th vertex on the contour, and M is the polygonal geometry object that generates the pair.

[0178] Based on the outline, the area and perimeter of each polygon are calculated to quantify the completed construction area. The calculation formula is as follows:

[0179]

[0180] Where A is the area of ​​the polygon, Z is the perimeter, (x i ,y i ) represents the vertex coordinates, and n represents the number of vertices;

[0181] Next, the pixel coordinates of each extracted independent region's contour coordinates are transformed into geographic coordinates. The transformation process is based on the geographic radiometric transformation relationship of the image, and the mathematical expression is as follows:

[0182] (x geo ,y geo )=T(x pixel ,y pixel (18)

[0183] Among them, (x geo ,y geo (x) represents the transformed geographic coordinates. pixel ,y pixel ) represents the image pixel coordinates, and T represents the geographic transformation matrix;

[0184] To ensure that the output vector data (such as shapefiles) has consistent and accurate measurement information in a spatial reference system, it is necessary to convert the area and perimeter calculation results from the geographic coordinate system (units in degrees) to the metric system. The mathematical expression for unit conversion is as follows:

[0185] Area unit conversion formula:

[0186]

[0187] Among them, area m2 area represents square meters. deg2It is expressed as area in square degrees. This indicates the number of meters corresponding to 1 degree of latitude. tom It represents the number of meters corresponding to 1 degree of longitude.

[0188] Length unit conversion formula:

[0189]

[0190] Where (lat1, lon1) are the latitude and longitude coordinates of the starting point of the line segment, (lat2, lon2) are the latitude and longitude coordinates of the ending point of the line segment, and L is the length of the line segment.

[0191] Finally, the number of targets, total area, and average area in the raster are extracted and saved as GeoDataFrame files. Then, the GeoDataFrame files are converted into vector-format shapefiles to provide accurate data support for subsequent engineering statistics and construction progress reports.

[0192] S4. Time Marking and Construction Progress Demonstration

[0193] This step is mainly used to achieve accurate tracking of the evolution of the construction area over time, based on multi-period Shapefile files (such as T1, T2, T3, ..., T...). n This method performs spatial overlay analysis on key construction targets (bridge piers, road surfaces, beams, etc.). By determining the spatial overlap between targets at different times and following the rule of "prioritizing earlier overlaps," the method achieves accurate identification of the component's time period.

[0194] In the specific process, firstly, all Shapefiles in the specified directory are traversed, and the period identifier (such as T) contained in the filename is automatically extracted. x The vector data of each construction phase is loaded sequentially according to the time sequence. For the data of each period, the geometric information of the specified target object (such as bridge pier, beam, pavement, etc.) is extracted, including but not limited to the polygon boundary coordinates and corresponding area. Combined with its period label, a standardized data structure is constructed for subsequent comparison and attribution determination.

[0195] In the period attribution determination process, all loaded target objects are first organized into a target list in chronological order. A unique tag identifier (ID) and its corresponding geographic coordinates are generated for each target object in the list (e.g., the bridge pier in period T1 is labeled ID 0, with a location of (x, y)). Subsequently, spatial overlap analysis is performed on each object in the current period and all objects recorded in all historical periods: if the current target object has a spatial overlap area with historical targets in the record list, it is determined that the area covered by the target has been completed in an early construction phase, and the target is automatically assigned to the earliest period in the historical record; if there is no spatial overlap, the target is determined to be a newly added construction area and assigned to the current period Ti. This mechanism strictly follows the principle of "overlapping areas are assigned to earlier periods first," ensuring that any spatial area is assigned to only one construction period within the construction cycle, thereby avoiding duplicate statistics and improving the accuracy of construction progress identification and the consistency of temporal logic.

[0196] In the spatial overlap and clipping stage, vector data from each stage is processed sequentially according to a set time order, extracting all valid geometric objects and their attributes from each stage's layer. For each object in the current stage, a spatial difference operation is performed with all previously assigned geometric sets to remove areas overlapping with historical stages, retaining only the newly added parts. If the clipped area is a polyhedral structure, the area ratio of each sub-polygon is calculated, and its area attribute field is updated. Simultaneously, an area threshold control mechanism is introduced to remove fragmented areas smaller than the minimum area threshold, improving the geometric quality and expressive effectiveness of the results. After clipping and filtering, a geometric merging operation is performed on all valid objects in the current stage to generate the final spatial calibration result, retaining the attribute information of each component, including the actual clipped area and the normalized ratio value, providing standardized and unique spatial data support for accurate analysis and visualization of construction progress.

[0197] In the construction completion rate calculation stage, taking bridge piers as an example, the YOLOv11 semantic segmentation model is used for pier identification and quantity statistics. Taking a certain construction area as an example, the total target number of bridge piers is set at 10. In time T1, 5 bridge piers are detected; in time T2, 7 are detected, of which 5 overlap with the time of T1, and 2 are newly added; in time T3, 9 are identified, of which 7 overlap with the previous identification results, and 2 are newly added. To ensure the consistency and temporal comparability of the statistical results, a strategy of "prioritizing overlapping and assigning to earlier times" is adopted, assigning duplicate detection targets to the time node of their first appearance. Based on this rule, the number of newly added bridge piers at each time node and their proportion to the total target number can be automatically calculated, thereby accurately depicting the construction progress and temporal evolution trend during the bridge pier construction process.

[0198] Simultaneously, in spatial overlap cropping, spatial difference operations are performed on repeated regions in different periods. Taking the road surface region as an example, pixel-level semantic segmentation of the road surface region is performed based on the DeepLabv3+ model, and the number of road surface pixels in the segmentation image is counted. During processing, the process is carried out in period order, from T1 to T... n Layers are loaded one by one. For each road surface object in the current layer, spatial difference clipping is performed with historical layers, retaining only newly added areas that have not yet been covered. Then, the area of ​​the clipped vectors is calculated, and their proportion in the overall image is determined.

[0199]

[0200] Among them, P road P represents the area of ​​the road surface after cutting. total Let A be the area of ​​the entire graph. road This represents the road surface coverage rate for the current period.

[0201] By combining the segmentation label map, the beam and slab laying areas can be identified, and their areas can be compared with the total laying area in the construction plan to calculate the current beam and slab laying completion rate. The calculation formula is as follows:

[0202]

[0203] Among them, A beam A represents the actual area of ​​the area where beams and slabs have been laid after spatial trimming. beam_target R is the total planned area for the planned laying. beam The completion rate of beam and slab laying.

[0204] By comprehensively analyzing the number of bridge piers, road surface coverage, and beam and slab paving completion rate, the construction progress and spatial evolution of key components at the construction site are fully reflected, providing scientific and reliable data support and decision-making basis for construction management and dynamic adjustments.

[0205] Using generalization evaluation results from real-world conditions, such as Figure 2 As shown.

Claims

1. A method for intelligent monitoring of construction progress using multi-temporal imagery from unmanned aerial vehicles (UAVs), characterized in that, Includes the following steps: S1. Data Preparation and Preprocessing Step S1.1: Cropping of UAV orthophotos The original images are preprocessed using orthophoto data acquired by the UAV as input. First, load the original orthophoto image via the path and extract its resolution and basic image size information; set the initial cropping patch size to a fixed size and crop the entire original image into several image patches of the same size; if the original image size cannot be divided by the set patch size, zero-padded the original image to expand its size to a size that can be divided by the patch size. The padded image is cropped into patches, and the coordinates of each patch are calculated; Each cropped patch is then normalized, with pixel values ​​normalized to [0, ...]. The range is defined, and the data type is converted to uint8. Subsequently, the results are saved using different storage modes based on the number of image channels; Step S1.2: Constructing the dataset and dividing the process into stages; The manually labeled JSON files are converted into corresponding mask images, generating label images that correspond one-to-one with the cropped patches; the data is divided into training, validation, and test sets according to a set ratio, and saved uniformly in PNG format; S2, Model Training and Segmentation Prediction Step S2.1: Determining and Selecting the Model Architecture During the task judgment phase, the type of segmentation and recognition required is determined based on the component attributes of the target object. The judgment logic is as follows: （5） in, Indicates the selected model structure. Indicates the current task type label; Step S2.2: Model Training Optimization and Inference Output During the model adaptation phase, a corresponding training process is constructed based on the selected model architecture; the cross-entropy loss function is used to measure model performance throughout the training process, taking into account the optimization of classification and regression errors. All models employ the Adam optimizer and incorporate a cosine annealing learning rate decay strategy. After the model achieves stable convergence on the validation set, the optimal weight file is automatically saved and the model enters the inference phase. During inference, the optimal weights obtained during the training phase are loaded, and prediction operations are performed on the test images in batches to output semantic segmentation results. Each result image is represented in the form of a mask image, which marks the location and category information of the construction area. All inference results are saved in pixel-level format as .PNG files, along with the index coordinates of each pixel block within the entire image. ; S3, Image stitching and raster conversion Step S3.1: Image blocks are stitched together to restore the original image size. After the model completes pixel-level inference for each image patch, the prediction results of all image patches are spatially stitched and cropped to finally output a segmentation result map with geographic reference information. Step S3.2: Segmentation Result Processing and Raster Conversion The results of inference on the network model are analyzed and processed to extract the raster features and geographic information of the target area, and the GeoTIFF file saved by the inference results is converted into a shapefile file. S4. Time Marking and Construction Progress Demonstration In the specific process, firstly, all Shapefile files in the specified directory are traversed, the phase identifier contained in the file name is automatically extracted, and the vector data of each construction phase is loaded in chronological order; for the data of each phase, the geometric information of the specified target object is extracted, including the polygon boundary coordinates and corresponding area, and a standardized data structure is constructed by combining it with its phase label. During the period attribution determination process, all loaded target objects are first organized into a target list in chronological order, and a unique label identifier and its corresponding geographic location coordinates are generated for each target object in the list. Then, spatial overlap analysis is performed on each object in the current period and all objects recorded in all historical periods: if there is a spatial overlap between the current target object and a historical target in the record list, it is determined that the area covered by the target has been completed in an early construction phase, and the target is automatically assigned to the earliest period in the historical record; if there is no spatial overlap, the target is determined to be a newly added construction area and assigned to the current period Ti. This mechanism strictly follows the principle of "overlapping priority assigned to earlier periods" to ensure that any spatial area is assigned to only one construction period within the construction cycle. In the spatial overlap and clipping stage, vector data from each stage is processed sequentially according to the set time order, extracting all valid geometric objects and their attributes from each layer. For each object in the current stage, a spatial difference operation is performed between it and all previously assigned geometric sets to remove areas overlapping with historical stages, retaining only the truly newly added parts. If the clipped area is a polyhedral structure, the area ratio of each sub-polygon is calculated, and its area attribute field is updated. At the same time, an area threshold control mechanism is introduced to remove fragmented areas smaller than the minimum area threshold. After clipping and filtering, a geometric merging operation is performed on all valid objects in the current stage to generate the final spatial calibration result, while retaining the attribute information of each component, including the actual clipped area and the normalized ratio value. In the construction completion rate calculation stage, object identification and quantity statistics are performed based on the YOLOv11 semantic segmentation model. To ensure the consistency and temporal comparability of the statistical results, the strategy of "overlapping priority is assigned to earlier times" is adopted, and duplicate detection targets are assigned to the time node of their first appearance. According to this rule, the number of new objects at each time node and their proportion to the total number of targets can be automatically calculated. Simultaneously, in the spatial overlap clipping process, spatial difference operations are performed on the repeated regions in different periods; subsequently, the area of ​​the clipped vectors is calculated, and their proportion in the entire image is determined. （21） in, This represents the area of ​​the road surface after trimming. The area is the whole map. This represents the current road surface coverage rate. By combining the segmentation label map, the beam and slab laying areas are identified, and their areas are compared with the total laying area in the construction plan to calculate the current beam and slab laying completion rate. The calculation formula is as follows: （22） in, This represents the actual area of ​​the area where beams and slabs have been laid after spatial trimming. The total planned area for the project is to be laid out. The completion rate of beam and slab laying; By comprehensively analyzing the number of bridge piers, road surface coverage, and beam and slab paving completion rate, the construction progress and spatial evolution of key components at the construction site are fully reflected.

2. The intelligent monitoring method for construction progress based on multi-temporal imagery from unmanned aerial vehicles (UAVs) according to claim 1, characterized in that, In step S1.1, the method for calculating the image padding size is as follows: （1） in, This represents the pixels used to pad the height. This represents the number of pixels used to pad the width. Indicates the original image height. Indicates the width of the original image. The patch size is in pixels. For modulo operation; Calculate the pixels needed for padding, then calculate the new height and width, using the following formula: （2） in, The height after completion To fill in the width; The size of the padded image is an integer multiple of the patch size, and then the number of rows and columns of the patch is calculated. and The expression is as follows: （3） The padded image is cropped into patches, and the coordinates of each patch are calculated. For the i-th... line, number The column patch can be performed as follows: （4） in For the original image row range, , For the column range of the original image, This refers to the patch size.

3. The intelligent monitoring method for construction progress based on multi-temporal imagery from unmanned aerial vehicles (UAVs) according to claim 2, characterized in that, In step S1.1, the results are saved using different storage modes according to the number of image channels: for single-channel images, they are saved in grayscale mode; if it is a three-dimensional image but contains only one channel, it is first compressed in dimension and then saved as a grayscale image; for three-channel images, they are saved in RGB mode; in four-channel images, only the first three channels are retained for RGB saving, and the transparency information is discarded; if the number of image channels is greater than 4, the first 3 channels are also truncated and saved; all the above images are saved in PNG format.

4. The intelligent monitoring method for construction progress based on multi-temporal imagery from unmanned aerial vehicles (UAVs) according to claim 1, characterized in that, In step S2.2, Cross-entropy loss, used to evaluate the accuracy of the model in multi-class discrimination tasks, is defined as follows: （6） in, For real category labels, To predict category labels, Number of categories; The learning rate update strategy is defined as follows: （7） in, The current learning rate, To achieve the maximum learning rate, To minimize the learning rate, For the current number of times, This represents the total number of training sessions.

5. The intelligent monitoring method for construction progress based on multi-temporal imagery from unmanned aerial vehicles according to claim 4, characterized in that, The specific process of step S3.1 is as follows: Each image patch generated during the inference stage is saved according to the naming convention image_name_row_col.png, where row and col represent the row and column indices of the image patch in the original image grid, respectively. Before stitching the images, the size of the patch grid structure must first be calculated based on the index information to facilitate subsequent embedding of each patch into the grid according to its index. The calculation formula is as follows: （8） in, These represent the total number of rows and columns of the patch grid, respectively. Subsequently, the size information of each image patch is extracted from the input patch file to obtain the pixel size P and the number of channels C of each image patch; at the same time, the geographic image information of the original image is read, including the original image size, number of bands, data type, coordinate reference system and geographic boundary content. Based on the understanding of the patch grid structure and original image parameters, an array structure for image reconstruction is created. For grayscale images or single-channel images, the image reconstruction array is defined as follows: （9） in, To reconstruct the image array, To reconstruct the image height, To reconstruct the image width, Number of channels; Based on the row and column indices of each image patch in the grid, the corresponding insertion region in the reconstructed image is calculated as follows: （10） in, , These are the row and column coordinates of the current image patch, respectively. , The minimum row and column values ​​in the patch grid. The pixel size of the corresponding image block. , To reconstruct the row range of the image, , To reconstruct the column range of the image; To ensure that the image segmentation prediction results remain consistent with the original image format during the stitching and reconstruction process, an intelligent adaptation mechanism for the number of image channels was designed. For a single-channel original image, if the image patch has multiple channels, only the first channel is taken. For a three-channel (RGB) original image, if the image patch has a single channel, it is copied three times to form an RGB format; if the number of channels exceeds three, the first three channels are retained. For a four-channel RGBA image, if the image patch has a single channel or three channels, an alpha channel is automatically added and set to opaque; if it has more than four channels, the first four channels are truncated. For other special channel cases, adaptation is performed through dimensional expansion or channel truncation. Subsequently, each image patch is sequentially filled into the reconstructed image array according to its position in the reconstructed image; the images are processed according to the number of channels; when the original image is a single-channel image, i.e., a grayscale image, a two-dimensional array index is used for cropping to directly remove the padding area and maintain the single-channel structure; when the original image is a multi-channel image, a three-dimensional array index is used for cropping to remove the padding area; the specific formula is as follows: Single-channel image: （11） Multi-channel images: （12） in, The height of the original input image. The width of the original input image. The final image after cropping, i.e., removing the padding area. Reconstruct the image containing the padding region; Finally, the original georeferenced information is reassigned to the stitched and cropped images, and the images are saved in GeoTIFF format to generate a semantic segmentation result map containing coordinate information.

6. The intelligent monitoring method for construction progress based on multi-temporal imagery from unmanned aerial vehicles according to claim 5, characterized in that, The specific processing flow for step S3.2 is as follows: First, load the GeoTIFF format semantic segmentation file, count the total number of target pixels in the file, and calculate its proportion in the entire image. The specific formula is as follows: （13） in, For the target number of pixels, This represents the total number of pixels. For position Pixel values; Subsequently, a binary mask is created, marking pixels with a value of 1 in the target region as 1 and all other pixels as 0. The specific formula is as follows: （14） in, It is a binary mask matrix. These are pixel coordinates; Based on this binary mask, an 8-connectivity algorithm is used to identify independent regions. By excluding background labels with a pixel value of 0, independent target regions with a pixel value of 1 are obtained and represented as a label set: （15） Where L is the set of all independent regions, For the first Labels for each independent area; For each individual region, extract its boundary contour and generate the corresponding polygon object: （16） in, It is a sequence of contour coordinates. For the first on the outline vertex coordinates To generate a pair of polygonal geometric objects; Based on the outline, the area and perimeter of each polygon are calculated to quantify the completed construction area. The calculation formula is as follows: （17） Where A is the area of ​​the polygon. Let the vertex coordinates be... The number of vertices; Next, the pixel coordinates to geographic coordinates are converted for the contour coordinates of each extracted independent region; the conversion process is based on the geographic radiometric transformation relationship of the image, and the mathematical expression is as follows: （18） in, The converted geographic coordinates Image pixel coordinates, This is the geographic transformation matrix; To ensure that the output vector data has consistent and accurate measurement information in the spatial reference system, it is also necessary to convert the area and perimeter calculation results from the geographic coordinate system to the metric system; the mathematical expression of the unit conversion is as follows: Area unit conversion formula: （19） in, Indicates area in square meters. It is expressed as area in square degrees. This indicates the number of meters corresponding to 1 degree of latitude. This indicates the number of meters corresponding to 1 degree of longitude. Length unit conversion formula: （20） in,( , ) represents the latitude and longitude coordinates of the starting point of the line segment. , () represents the latitude and longitude coordinates of the endpoint of the line segment. The length of the line segment; Finally, the number of targets, total area, and average area in the raster are extracted and saved as GeoDataFrame files. Then, the GeoDataFrame files are output as vector-format shapefiles.

Images