Method, device and equipment for monitoring progress of construction project and storage medium

Abstract

The application belongs to the field of finance and the field of image processing, and particularly relates to a building project progress monitoring method, device, equipment and storage medium. The application determines the height of a target building included in a satellite image based on a pre-trained shadow instance segmentation model and meta information of the satellite image, the shadow instance segmentation model being used to determine the shadow of the target building relative to the ground; determines the planar area of the target building based on a pre-trained contour instance segmentation model and the satellite image, the contour instance segmentation model being used to determine the planar contour of the target building relative to the ground; if the height and / or the planar area meet preset conditions, generates an alarm information, the alarm information being used to alarm the project progress of the target building. The application can achieve the purpose of improving the efficiency of monitoring the project progress of the building.

Description

Technical Field

[0001] This application relates to the fields of finance and image processing, and in particular to a method, apparatus, equipment and storage medium for monitoring the progress of construction projects. Background Technology

[0002] Generally speaking, engineering construction projects require a large financial foundation, so they usually borrow from banks. However, due to the large scale of loans and long time periods involved in engineering construction loans, banks face high uncertainty in credit management. Therefore, banks need to conduct post-loan monitoring of such credit projects to promptly identify various risks and avoid or reduce bank losses.

[0003] In related technologies, banks typically monitor project progress by periodically dispatching specialists to construction sites. However, this method is resource-intensive and requires significant manpower and materials. Furthermore, for construction projects with complex geographical environments, such as hydropower stations, bridges, and dams, relying solely on manual on-site inspections presents physical limitations. Therefore, these technologies suffer from low efficiency in monitoring construction project progress. Summary of the Invention

[0004] This application provides a method, apparatus, equipment, and storage medium for monitoring the progress of construction projects, so as to improve the efficiency of monitoring the progress of construction projects.

[0005] In a first aspect, this application provides a method for monitoring the progress of a construction project, comprising: determining the height of a target building included in a satellite image based on a pre-trained shadow instance segmentation model and metadata of a satellite image, wherein the shadow instance segmentation model is used to determine the shadow of the target building relative to the ground; determining the planar area of ​​the target building based on a pre-trained contour instance segmentation model and the satellite image, wherein the contour instance segmentation model is used to determine the planar contour of the target building relative to the ground; and generating an alarm message if the determined height and / or planar area meet preset conditions, wherein the preset conditions include the height being equal to a first threshold and / or the planar area being equal to a second threshold, or the difference between the height and the first threshold being less than a third threshold and / or the difference between the planar area and the second threshold being less than a fourth threshold, wherein the alarm message is used to issue an alarm for the project progress of the target building.

[0006] In one possible implementation, the shadow instance segmentation model is trained by: augmenting image samples with image sizes smaller than a preset value in the obtained MS-COCO dataset to obtain an augmented dataset; training a path aggregation network model using the augmented dataset to obtain a pre-trained model; and fine-tuning the network weights of the pre-trained model using a target dataset to obtain the shadow instance segmentation model, wherein the target dataset includes satellite images containing shadow instance segmentation annotations.

[0007] In one possible implementation, augmentation processing is performed on image samples in the obtained MS-COCO dataset whose image size is smaller than a preset value to obtain an augmented dataset. This includes: selecting image samples from the MS-COCO dataset to obtain a base dataset; for each image sample in the base dataset, constructing a multidimensional random transformation probability combination vector based on multiple floating-point numbers generated by a called random number generator, where each vector in the multidimensional random transformation probability combination vector corresponds to the selection probability value of a processing operator; and performing stepwise processing on the selection probability values ​​of multiple processing operators in the multidimensional random transformation probability combination vector to obtain a random... The transformation and combination vector is processed as follows: If a first target processing operator is found in the random transformation and combination vector, the first target processing corresponding to the first target processing operator is applied to the segmentation annotation pixel mask of the image sample to obtain the target segmentation mask; if a second target processing operator is found in the random transformation and combination vector, the second target processing corresponding to the second target processing operator is applied to the image sample to obtain the target image; based on the target segmentation mask, the target image is subjected to contour cropping processing to obtain the target contour of the target image; multiple contour copies are generated for the target contour according to a preset ratio; the base dataset is expanded using multiple contour copies to obtain the augmented dataset.

[0008] In one possible implementation, the metadata includes solar azimuth angle, solar altitude angle, satellite altitude angle, and same / opposite side identification information. The same / opposite side identification information is used to determine whether the satellite and the sun are located on the same side or opposite side of the target building, respectively. Based on the metadata of the pre-trained shadow instance segmentation model and the satellite image, the height of the target building included in the satellite image is determined, including: inputting the satellite image into the shadow instance segmentation model for instance segmentation processing to obtain a first segmentation result image including the shadow; determining the height projection direction of the shadow according to the solar azimuth angle; determining the shadow length of the shadow according to the first segmentation result image and the height projection direction; and determining the height based on the solar altitude angle, satellite altitude angle, same / opposite side identification information, and shadow length.

[0009] In one possible implementation, determining the shadow length of a shadow based on a first segmentation result image and a height projection direction includes: rotating the first segmentation result image according to the height projection direction to obtain an angle-corrected segmentation result image; binarizing the angle-corrected segmentation result image to obtain a first image; determining the minimum bounding rectangle of the angle-corrected shadow contour extracted in the first image; and determining the shadow length of the shadow based on the minimum bounding matrix.

[0010] In one possible implementation, determining the planar area of ​​a target building based on a pre-trained contour instance segmentation model and satellite imagery includes: inputting the satellite image into the contour instance segmentation model for instance segmentation processing to obtain a second segmentation result image including the planar contour; binarizing the second segmentation result image to obtain a second image; extracting the planar contour from the second image; and determining the planar area based on the planar contour.

[0011] In one possible implementation, if the height and / or planar area are determined to meet preset conditions, an alarm message is generated, including: storing the height and planar area in an indicator queue, the indicator queue including multiple heights and multiple planar areas arranged in chronological order of acquisition time; determining a target height as a first threshold among the multiple heights, and determining a target planar area as a second threshold among the multiple planar areas; storing a first comparison result of the height with the first threshold and / or a second comparison result of the planar area with the second threshold in a difference queue, the first comparison result indicating that the height is equal to the first threshold or the difference between the height and the first threshold is less than a third threshold, and the second comparison result indicating that the planar area is equal to the second threshold or the difference between the planar area and the second threshold is less than a fourth threshold; if the difference queue is determined to meet preset conditions, an alarm message is generated.

[0012] Secondly, this application provides a monitoring device for the progress of a construction project, comprising: a first determining module, used to determine the height of a target building included in a satellite image based on a pre-trained shadow instance segmentation model and metadata of a satellite image, wherein the shadow instance segmentation model is used to determine the shadow of the target building relative to the ground; a second determining module, used to determine the planar area of ​​the target building based on a pre-trained contour instance segmentation model and a satellite image, wherein the contour instance segmentation model is used to determine the planar contour of the target building relative to the ground; and an alarm module, used to generate alarm information if the determined height and / or planar area meet preset conditions, wherein the preset conditions include the height being equal to a first threshold and / or the planar area being equal to a second threshold, or the difference between the height and the first threshold being less than a third threshold and / or the difference between the planar area and the second threshold being less than a fourth threshold, wherein the alarm information is used to alarm the project progress of the target building.

[0013] Thirdly, this application provides an electronic device, including: a processor and a memory communicatively connected to the processor; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory to implement the construction project progress monitoring method of the first aspect.

[0014] Fourthly, this application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the construction project progress monitoring method as described in the first aspect.

[0015] Fifthly, embodiments of this application provide a computer program product, including a computer program, which, when executed by a processor, implements the method for monitoring the progress of a construction project as described in the first aspect.

[0016] The method, apparatus, equipment, and storage medium for monitoring the progress of construction projects provided in this application utilize satellite imagery. The satellite imagery is input into a pre-trained shadow instance segmentation model, and the height of the target building is obtained using the metadata of the satellite imagery. The satellite imagery is then input into a pre-trained contour instance segmentation model to obtain the planar contour of the target building relative to the ground, thereby determining the planar area of ​​the target building. By monitoring the height and planar area of ​​the target building, the project progress of the target building can be determined. If an abnormality in the project progress is detected, an alarm message can be generated to alert staff. Therefore, the project progress of the target building can be determined without manual on-site exploration, improving the efficiency of monitoring the progress of construction projects. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0018] Figure 1 A schematic diagram of a scenario for a method of monitoring the progress of a construction project provided in an embodiment of this application;

[0019] Figure 2 A flowchart illustrating a method for monitoring the progress of a construction project as provided in an embodiment of this application;

[0020] Figure 3 A schematic diagram of the solar azimuth angle provided in an embodiment of this application;

[0021] Figure 4 A schematic diagram showing the satellite and the sun on the same side of the target building, provided for an embodiment of this application;

[0022] Figure 5A schematic diagram showing the satellite and the sun located on opposite sides of the target building, as provided in an embodiment of this application;

[0023] Figure 6 A schematic diagram of the structure of the construction project progress monitoring device provided in the embodiments of this application;

[0024] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0025] The accompanying drawings have illustrated specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concepts of this application to those skilled in the art through reference to specific embodiments. Other drawings can be obtained from these drawings by those skilled in the art without any inventive effort. Detailed Implementation

[0026] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0027] First, let me explain the terms used in this application:

[0028] Meta-information: Information about information, used to describe the structure, semantics, purpose, and usage of the information. Meta-information of satellite imagery can include satellite imagery attribute information and acquisition data, such as solar azimuth angle, solar altitude angle, satellite altitude angle, and information indicating whether the satellite and the sun are on the same or opposite side relative to the target building.

[0029] MS-COCO dataset: A dataset built by Microsoft that includes object detection, segmentation, and captioning.

[0030] The related technologies provided in the background section have at least the following technical problems:

[0031] Due to the large scale and long duration of loans related to construction projects, banks need to conduct post-loan monitoring to promptly identify various risks and mitigate or reduce bank losses. Post-loan monitoring typically involves banks periodically dispatching specialists to construction sites to monitor project progress. However, this method is resource-intensive and requires significant manpower and materials. Furthermore, for construction projects with complex geographical environments, such as hydropower stations, bridges, and dams, relying solely on manual on-site inspections faces physical limitations, resulting in low efficiency in monitoring project progress.

[0032] With the significant decrease in launch and manufacturing costs, satellite services available to the public are gradually increasing. Satellite services offer advantages such as being contactless, having wide coverage, and providing accurate data. They are also largely unaffected by the geographical conditions of the target area. Remote sensors mounted on orbiting satellites can repeatedly observe targets at specific intervals (ranging from 3 to 58 days), with shorter intervals providing more detailed insights into the dynamic changes of ground features. Therefore, at the same imaging precision, remote sensing satellite imagery is more economical, saving manpower and resources. Furthermore, with the advancements in digital technology in recent years, the types of remote sensing satellite data have continuously increased, data acquisition is easier, and costs have continued to decrease. Remote sensing satellite image recognition technology based on image processing is increasingly valued by banks. Connecting satellite services with bank needs can provide a new way to understand and serve customers. Therefore, this application introduces remote sensing satellite imagery to avoid the need for manual monitoring of construction project progress, thereby improving the efficiency of monitoring construction project progress.

[0033] To address the problems in related technologies, this application proposes a method for monitoring the progress of construction projects. By extracting the target building and its shadow outline from remote sensing satellite images, and combining metadata such as solar azimuth angle, satellite elevation angle, and solar elevation angle from the remote sensing satellite, the method estimates the height, area, and other dimensional indicators of the target building in real time. By recording and tracking changes in building dimensions, an alarm message is promptly pushed to credit personnel once the alarm conditions are met, thereby achieving the purpose of regularly monitoring the progress of project construction and timely identifying risks.

[0034] In one embodiment, the method for monitoring the progress of a construction project can be applied in an application scenario. Figure 1 This is a schematic diagram of a scenario for a method to monitor the progress of a construction project provided in an embodiment of this application, such as... Figure 1As shown, the data acquisition device 100 can be used to acquire satellite images and store them in the database 101. The shadow instance segmentation model 102 can obtain satellite images from the database 101 for instance segmentation processing to obtain the height of the target building in the satellite image. The contour instance segmentation model 103 can obtain satellite images from the database 101 for instance segmentation processing to obtain the planar area of ​​the target building, that is, the area of ​​the projected contour of the target building on the ground. The monitoring module 104 monitors the height and planar area of ​​the target building to promptly alert to any risks that arise and push alarm messages to business personnel.

[0035] In light of the above scenarios, the technical solutions of this application and how they solve the aforementioned technical problems will be described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0036] This application provides a method for monitoring the progress of a construction project. Figure 2 A flowchart of a method for monitoring the progress of a construction project provided in this application embodiment is shown below. Figure 2 As shown, the method includes the following steps:

[0037] S201: Based on a pre-trained shadow instance segmentation model and metadata from satellite imagery, determine the height of target buildings included in the satellite imagery.

[0038] In this step, the shadow instance segmentation model is used to determine the shadow of the target building relative to the ground.

[0039] Specifically, satellite images can be input into the shadow instance segmentation model to obtain the shadow of the target building relative to the ground in the satellite image. Then, based on the solar azimuth angle, solar altitude angle, satellite altitude angle, etc. in the metadata, the height of the target building can be calculated.

[0040] S202: Determine the planar area of ​​the target building based on a pre-trained contour instance segmentation model and satellite imagery.

[0041] In this step, the contour instance segmentation model is used to determine the planar contour of the target building relative to the ground.

[0042] Specifically, satellite images can be input into the contour instance segmentation model to obtain the planar contour of the target building relative to the ground, that is, the contour of the target building's projection onto the ground vertically, and the area of ​​this planar contour is also the planar area of ​​the target building.

[0043] S203: If the height and / or plane area meet the preset conditions, an alarm message is generated.

[0044] In this step, the preset conditions include the height being equal to the first threshold and / or the floor area being equal to the second threshold, or the difference between the height and the first threshold being less than the third threshold and / or the difference between the floor area and the second threshold being less than the fourth threshold. The alarm information is used to issue an alarm on the project progress of the target building.

[0045] Specifically, the first threshold can be the height value obtained from the most recent monitoring time, and the second threshold can be the floor area value obtained from the most recent monitoring time. If the height equals the first threshold and / or the floor area equals the second threshold, it indicates that the project progress of the current target building is at a standstill. If the difference between the height and the first threshold is less than the third threshold and / or the difference between the floor area and the second threshold is less than the fourth threshold, it indicates that the project progress of the current target building is slow. Both of these situations indicate that there is a risk in the current project, and therefore an alert needs to be issued to the business personnel so that timely countermeasures can be taken against the risks.

[0046] The construction project progress monitoring method provided in this embodiment utilizes satellite imagery. The satellite imagery is input into a pre-trained shadow instance segmentation model, and the height of the target building is obtained using the metadata of the satellite imagery. Then, the satellite imagery is input into a pre-trained contour instance segmentation model to obtain the planar contour of the target building relative to the ground, thereby determining the planar area of ​​the target building. By monitoring the height and planar area of ​​the target building, the project progress can be determined. If an abnormality in the project progress is detected, an alarm message can be generated to alert staff. Therefore, the project progress of the target building can be determined without manual on-site exploration, improving the efficiency of construction project progress monitoring.

[0047] In one embodiment, the shadow instance segmentation model is trained by: augmenting image samples with image sizes smaller than a preset value in the obtained MS-COCO dataset to obtain an augmented dataset; training a path aggregation network model using the augmented dataset to obtain a pre-trained model; and fine-tuning the network weights of the pre-trained model using a target dataset to obtain the shadow instance segmentation model, wherein the target dataset includes satellite images containing shadow instance segmentation annotations.

[0048] In this approach, a community-developed open-source pre-trained PathAggregation Network (PANet) model based on the MS-COCO dataset can be used as the initial state for the shadow instance segmentation model. Then, the MS-COCO dataset is acquired. Considering the operational characteristic of small targets in remote sensing satellite images, targeted augmentation of image samples containing small targets can be performed based on the MS-COCO dataset to obtain a small target augmented dataset. This addresses the problem of insufficient remote sensing satellite image samples during real-world project implementation and avoids poor monitoring generalization due to an imbalance in the proportion of small targets. Then, using the community-developed open-source pre-trained PANet model as the initial state, transfer learning is performed on the entire PANet model based on the small target augmented dataset to obtain a PANet pre-trained model optimized for small targets. The weights of the backbone convolutional network in the PANet pre-trained model optimized for small targets are fixed. Based on a target dataset of real remote sensing satellite images containing 2D building shadow instance segmentation annotations, the network weights of the later layers of the PANet pre-trained model are fine-tuned and trained. Finally, a shadow instance segmentation model suitable for remote sensing satellite building shadow scenarios can be obtained.

[0049] Optionally, the training process of the contour instance segmentation model is the same as that of the shadow instance segmentation model, but the shadow instance segmentation model is used to determine the shadow of the target building relative to the ground, while the contour instance segmentation model is used to determine the planar contour of the target building relative to the ground.

[0050] Optionally, by using pre-trained shadow instance segmentation models and contour instance segmentation models, the project progress of the target building can be obtained quickly and accurately, thereby enabling rapid monitoring of the project progress of the target building. The project progress of the target building can be determined without manual on-site exploration, thus improving the efficiency of monitoring the construction project progress.

[0051] In one embodiment, augmentation processing is performed on image samples in the obtained MS-COCO dataset whose image size is smaller than a preset value to obtain an augmented dataset. This includes: selecting image samples from the MS-COCO dataset to obtain a base dataset; for each image sample in the base dataset, constructing a multidimensional random transformation probability combination vector based on multiple floating-point numbers generated by a called random number generator, where each vector in the multidimensional random transformation probability combination vector corresponds to the selection probability value of a processing operator; and performing stepwise processing on the selection probability values ​​of multiple processing operators in the multidimensional random transformation probability combination vector to obtain a random transformation... Combine vectors; if it is determined that a first target processing operator exists in the randomly transformed combined vectors, then the first target processing corresponding to the first target processing operator is performed on the segmentation annotation pixel mask of the image sample to obtain the target segmentation mask; if it is determined that a second target processing operator is selected in the randomly transformed combined vectors, then the second target processing corresponding to the second target processing operator is performed on the image sample to obtain the target image; based on the target segmentation mask, contour cropping processing is performed on the target image to obtain the target contour of the target image; multiple contour copies are generated for the target contour according to a preset ratio; the base dataset is expanded using multiple contour copies to obtain the augmented dataset.

[0052] In this scheme, when augmenting image samples with image sizes smaller than a preset value in the obtained MS-COCO dataset, image samples containing small targets can be selected from the MS COCO dataset first (considering that the image size in the MS COCO dataset is 640*480, the criterion for judging small targets is that the pixel mask area in the segmentation annotation of the target is less than 1024 = 32*32 pixels) as the base dataset for targeted augmentation.

[0053] In the above scheme, for each image sample in the basic dataset, three copies of the image can be made. Then, for each image copy, the random number generator can be called multiple times. Taking six calls as an example, calling the random number generator six times can generate six floating-point numbers between 0 and 1. These six floating-point numbers are used to construct a six-dimensional random transformation probability combination vector P = [p1p2p3p4p5p6]. The six dimensions p1, p2, p3, p4, p5, and p6 can represent the probability values ​​of the six processing operators: translation, rotation, saturation transformation, noise addition, brightness transformation, and contrast transformation, respectively. Then, the probability values ​​of the six dimensions are processed by stepping, that is, if pi is between 0 and 0.5, the result is 0; if pi is between 0.5 and 1, the result is 1. After processing, a random transformation combination vector T = [t1t2t3t4t5t6] is generated, where ti has a value range of {0, 1} and i = 1, 2, ..., 6.

[0054] In the above scheme, for the generated random transformation combination vector T, it is possible to check whether there is a translation or rotation operator in vector T. That is, to check whether the first dimension t1 or the second dimension t2 of vector T is 1. If the detection result is 1 (that is, there is a translation or rotation operator in vector T), that is, there is a first target processing operator in vector T, then the image segmentation annotation pixel mask of the image copy is also subjected to corresponding translation or rotation processing to obtain the target segmentation mask. Then, according to the selection status of the processing operators corresponding to the third dimension t3 to the sixth dimension t6 of vector T (selected is 1, unselected is 0), that is, if the second target processing operator is selected in vector T, the image copy is processed accordingly, including saturation transformation processing, noise addition processing, brightness transformation processing, contrast transformation processing, etc., to obtain the target image.

[0055] In the above scheme, after obtaining the target segmentation mask and the target image, the contour of the original small-sized target in the target image can be cropped based on the target segmentation mask. Then, multiple small target contour copies are generated at a 1:15 ratio, thereby expanding the number of small-sized targets in the image copies and reducing the imbalance problem of small-sized samples. Specifically, a random position can be selected on the current image copy as the pasting position for the contour copy of the small-sized target. Based on the pasting position and the labeled pixel mask of the original small-sized target, the labeled pixel mask information of the contour copy of the pasted small-sized target is obtained. Finally, Gaussian blur smoothing is applied to the pasting position area on the pasted contour copy to improve the realism of the augmented image.

[0056] Optionally, by augmenting the image samples in the acquired MS-COCO dataset whose image size is smaller than a preset value, the problem of insufficient remote sensing satellite image samples during the implementation of real projects can be solved. At the same time, the problem of poor monitoring generalization effect caused by the imbalance of small-sized sample ratio can be avoided, thereby improving the accuracy of monitoring the progress of construction projects.

[0057] In one embodiment, the metadata includes solar azimuth angle, solar altitude angle, satellite altitude angle, and same / opposite side identification information. The same / opposite side identification information is used to determine whether the satellite and the sun are located on the same side or opposite side of the target building, respectively. Based on the metadata of the pre-trained shadow instance segmentation model and the satellite image, the height of the target building included in the satellite image is determined, including: inputting the satellite image into the shadow instance segmentation model for instance segmentation processing to obtain a first segmentation result image including the shadow; determining the height projection direction of the shadow according to the solar azimuth angle; determining the shadow length of the shadow according to the first segmentation result image and the height projection direction; and determining the height based on the solar altitude angle, satellite altitude angle, same / opposite side identification information, and shadow length.

[0058] In this scheme, when determining the height of the target building included in the satellite image based on the pre-trained shadow instance segmentation model and the metadata of the satellite image, the satellite image can be input into the shadow instance segmentation model for instance segmentation processing to obtain the instance segmentation result map containing the 2D shadow of the target building, which is also the first segmentation result map.

[0059] In the above scheme, when determining the height projection direction of the shadow based on the solar azimuth angle, the solar azimuth angle can be obtained from the metadata first, and then the height projection direction of the shadow can be determined using the formula: A_shadow = A_solar – 180. Here, A_shadow represents the height projection direction of the shadow, and A_solar represents the solar azimuth angle. The solar azimuth angle can be calculated as follows: Figure 3 As shown, Figure 3 This is a schematic diagram of the solar azimuth angle provided in an embodiment of this application. Figure 3 In mathematics, the solar azimuth angle reflects the position of the sun. It refers to the angle between the projection of sunlight onto the horizon and the local meridian. It can be approximated as the angle between the shadow of a vertical line on the ground and the direction of true north along the meridian. Figure 3 Angle A. The solar elevation angle reflects the altitude of the sun and refers to the angle between the direction of sunlight incidence and the horizontal plane. It is complementary to the solar zenith angle. The solar elevation angle is shown as... Figure 3 The median angle h, the solar zenith angle, is as follows: Figure 3 The median angle is z. Similarly, the satellite elevation angle can refer to the angle between the incident direction of the satellite image and the ground plane.

[0060] In the above scheme, after obtaining the first segmentation result image and the height projection direction of the shadow, the shadow length can be determined. Based on the shadow length and the solar altitude angle, satellite altitude angle, and same-side and opposite-side identification information obtained from the metadata, the height of the target building can be determined.

[0061] Specifically, if the satellite and the sun are currently on the same side of the target building, then as follows: Figure 4 As shown, Figure 4 This is a schematic diagram showing the satellite and the sun on the same side of the target building, provided in an embodiment of this application. Figure 4 In this context, α represents the current satellite elevation angle, β represents the solar elevation angle, H represents the height of the target building, L2 represents the shadow length, L1 represents the distance between the satellite's incident point and the target building, and S represents the distance between the sun's incident point and the target building. The relevant geometric relationships can then be expressed by the following formula:

[0062]

[0063] Therefore, the height H of the target building can be obtained using the following formula:

[0064]

[0065] Specifically, if the satellite and the sun are currently on the same side of the target building, then as follows: Figure 5 As shown, Figure 5 This is a schematic diagram showing the satellite and the sun located on opposite sides of the target building, as provided in an embodiment of this application. Figure 5 In this context, the relevant geometric relationships can be represented by the following formula:

[0066]

[0067] Therefore, the height H of the target building can be obtained using the following formula:

[0068] H = L2tanβ

[0069] Alternatively, by calculating the height of the target building at different positions of the satellite and the sun, the height value can be made more accurate, thus improving the accuracy of monitoring the progress of the construction project.

[0070] In one embodiment, determining the shadow length of a shadow based on a first segmentation result image and a height projection direction includes: rotating the first segmentation result image according to the height projection direction to obtain an angle-corrected segmentation result image; binarizing the angle-corrected segmentation result image to obtain a first image; determining the minimum bounding rectangle of the angle-corrected shadow contour extracted in the first image; and determining the shadow length of the shadow based on the minimum bounding matrix.

[0071] In this scheme, the first segmentation result image can be rotated in a plane according to the height projection direction of the 2D shadow of the target building. The rotation reference direction can be counterclockwise, that is, if the height projection direction is positive, it is rotated counterclockwise, and if the height projection direction is negative, it is rotated clockwise. Then, an angle-corrected shadow instance segmentation image with the height projection direction pointing due north is obtained, which is the angle-corrected segmentation result image. The angle-corrected segmentation result image is binarized to obtain a binary image, which is the first image. Then, the angle-corrected shadow contour in the first image can be detected and extracted through the relevant interface of the image processing library (OpenCV). The corresponding minimum bounding rectangle of the angle-corrected shadow contour is obtained, and the height value of the minimum bounding rectangle is calculated. This height value is the shadow length of the 2D projection of the height of the target building observed by the satellite image.

[0072] Optionally, binarizing the angle-corrected segmentation result image can reduce the memory footprint of the angle-corrected segmentation result image, thereby accelerating shadow detection processing and meeting the performance requirements of engineering practice. Therefore, it can improve the efficiency of monitoring the progress of construction projects.

[0073] In one embodiment, determining the planar area of ​​a target building based on a pre-trained contour instance segmentation model and satellite imagery includes: inputting the satellite image into the contour instance segmentation model for instance segmentation processing to obtain a second segmentation result image including the planar contour; performing binarization processing on the second segmentation result image to obtain a second image; performing extraction processing on the second image to obtain the planar contour; and determining the planar area based on the planar contour.

[0074] In this scheme, satellite images can be input into a contour instance segmentation model for instance segmentation processing to obtain a segmentation result image including the 2D contour of the target building, i.e., the second segmentation result image; then, the second segmentation result image is binarized to obtain a binary image, i.e., the second image; then, through the relevant interface of the image processing library (OpenCV), the 2D contour of the target building in the second image is detected and extracted, i.e., the planar contour of the target building; finally, the area of ​​the planar contour can be determined as the planar area of ​​the target building.

[0075] Optionally, binarizing the second segmentation result image can reduce its memory footprint, thereby accelerating contour detection processing and meeting the performance requirements of engineering practice. Therefore, it can improve the efficiency of monitoring the progress of construction projects.

[0076] In one embodiment, if the height and / or planar area are determined to meet preset conditions, an alarm message is generated, including: storing the height and planar area in an indicator queue, the indicator queue including multiple heights and multiple planar areas arranged in chronological order of acquisition time; determining a target height as a first threshold among the multiple heights, and determining a target planar area as a second threshold among the multiple planar areas; storing a first comparison result of the height with the first threshold and / or a second comparison result of the planar area with the second threshold in a difference queue, the first comparison result indicating that the height is equal to the first threshold or the difference between the height and the first threshold is less than a third threshold, and the second comparison result indicating that the planar area is equal to the second threshold or the difference between the planar area and the second threshold is less than a fourth threshold; if the difference queue is determined to meet preset conditions, an alarm message is generated.

[0077] In this scheme, satellite imagery may include multiple buildings. For each building, its height and planar area can be obtained using the aforementioned method, and the following monitoring methods can be executed:

[0078] A queue of indicators for the target building is acquired, and the height and floor area of ​​the currently monitored target building are stored in the indicator queue. That is, the height and floor area values ​​detected for each target building can be stored in the indicator queue for observing the project progress of the target building. Then, the height with the smallest time interval from the current monitoring in the indicator queue is used as a first threshold, and the floor area with the smallest time interval from the current monitoring in the indicator queue is used as a second threshold. The currently monitored height is compared with the first threshold to obtain a first comparison result, which can be used to determine the project progress status of the target building, such as whether the project has stopped or is progressing slowly. The currently monitored floor area is compared with the second threshold to obtain a second comparison result, which can also be used to determine the project progress status of the target building, such as whether the project has stopped or is progressing slowly.

[0079] In the above scheme, after obtaining the first comparison result and the second comparison result, the first comparison result and the second comparison result can be stored in a difference queue. If either the first comparison result or the second comparison result meets a preset condition, an alarm message is generated. Alternatively, by monitoring multiple comparison results detected in the difference queue, the current project progress of the target building can be comprehensively considered to determine whether it has reached the trigger condition for generating an alarm message. That is, exceeding a preset number of comparison results indicates that the project progress of the target building has stopped or is slow. If the trigger condition is met, an alarm message is generated and pushed to business personnel.

[0080] In the above scheme, when determining the first and second thresholds, multiple heights and planar areas can be selected from the indicator queue in chronological order as the first and second thresholds, respectively. When determining the first and second thresholds for the first time, the height and planar area with the smallest relative time to the current monitoring time can be selected; when determining the first and second thresholds for the second time, the height and planar area with the second smallest relative time to the current monitoring time can be selected, and so on. This improves the accuracy of the first and second comparison results, thereby increasing the accuracy of monitoring the progress of construction projects.

[0081] The construction project progress monitoring method provided in this embodiment first trains a dedicated deep learning model based on remote sensing satellite samples to obtain models for instance segmentation of building outlines and building shadows in remote sensing satellite images. The dedicated deep learning model can be a MASK R-CNN model based on semantic segmentation methods, such as the PANet model. Considering the acquisition cost of remote sensing satellite images and the limited sample size of real satellite images, data augmentation methods are needed to expand the training samples to improve model performance.

[0082] Secondly, considering that the ground objects to be detected in real-world remote sensing satellite images are mostly small-sized objects, the augmentation process can selectively expand the samples of small targets to obtain better model generalization results.

[0083] Furthermore, changes in the geometric dimensions of a building under construction are often the most direct and effective way to measure the progress of a construction project. Therefore, in engineering practice, building dimensions can be specifically quantified into two indicators: height and floor area. For the 2D footprint of a building, i.e., its floor area, the building's planar outline can be extracted from satellite images, and then the area of ​​the building's planar outline can be directly obtained using image processing tools. For the building's height, based on the extracted shadow outline of the building in the satellite image, the height can be estimated by combining the solar azimuth angle, solar altitude angle, satellite altitude angle, and the building's shadow length. After estimating the building's floor area and height, the changes in building dimensions can be monitored by comparing them with preset values ​​and recording the differences in area and height, thereby determining the project's progress.

[0084] Overall, the technical solution provided in this application is one that can improve both the efficiency and accuracy of monitoring the progress of construction projects.

[0085] It is important to note that the construction project progress monitoring method proposed in this application can automatically detect the outlines of construction features in remote sensing satellite images and further estimate information such as the height and area of ​​the target buildings. This allows for regular monitoring of real-time changes in the construction progress and timely detection of risks. Therefore, this application can be applied, but is not limited to, monitoring changes in the construction progress of loan-funded projects in terms of real-time construction scope, construction area, number of buildings, number of construction machines, and number of completed projects, thereby providing a basis for post-loan risk early warning analysis for banks.

[0086] This application also provides a device for monitoring the progress of a construction project. Figure 6 A schematic diagram of the structure of the construction project progress monitoring device provided in the embodiments of this application is shown below. Figure 6 As shown, the monitoring device 600 for the construction project progress includes:

[0087] The first determining module 601 is used to determine the height of a target building included in a satellite image based on a pre-trained shadow instance segmentation model and the metadata of the satellite image. The shadow instance segmentation model is used to determine the shadow of the target building relative to the ground.

[0088] The second determining module 602 is used to determine the planar area of ​​the target building based on a pre-trained contour instance segmentation model and satellite images. The contour instance segmentation model is used to determine the planar contour of the target building relative to the ground.

[0089] The alarm module 603 is used to generate alarm information if the height and / or floor area meet preset conditions. The preset conditions include the height being equal to a first threshold and / or the floor area being equal to a second threshold, or the difference between the height and the first threshold being less than a third threshold and / or the difference between the floor area and the second threshold being less than a fourth threshold. The alarm information is used to alarm the project progress of the target building.

[0090] Optionally, the project progress monitoring device 600 can train the shadow instance segmentation model using the following method: augmenting the image samples with image sizes smaller than a preset value in the acquired MS-COCO dataset to obtain an augmented dataset; training the path aggregation network model using the augmented dataset to obtain a pre-trained model; and fine-tuning the network weights of the pre-trained model using the target dataset to obtain the shadow instance segmentation model, wherein the target dataset includes satellite images containing shadow instance segmentation annotations.

[0091] Optionally, when the monitoring device 600 for the construction project progress performs augmentation processing on image samples with image sizes smaller than a preset value in the acquired MS-COCO dataset to obtain an augmented dataset, it specifically performs the following: selecting image samples from the MS-COCO dataset to obtain a basic dataset; for each image sample in the basic dataset, constructing a multidimensional random transformation probability combination vector based on multiple floating-point numbers generated by the invoked random number generator, where each vector in the multidimensional random transformation probability combination vector corresponds to the selection probability value of a processing operator; and performing stepwise processing on the selection probability values ​​of multiple processing operators in the multidimensional random transformation probability combination vector. The process involves: obtaining a random transformation combination vector; if a first target processing operator is found in the random transformation combination vector, performing the first target processing corresponding to the first target processing operator on the segmentation annotation pixel mask of the image sample to obtain a target segmentation mask; if a second target processing operator is found in the random transformation combination vector, performing the second target processing corresponding to the second target processing operator on the image sample to obtain a target image; based on the target segmentation mask, performing contour cropping processing on the target image to obtain the target contour of the target image; generating multiple contour copies of the target contour according to a preset ratio; and using multiple contour copies to expand the basic dataset to obtain an augmented dataset.

[0092] Optionally, the metadata includes solar azimuth angle, solar altitude angle, satellite altitude angle, and same / opposite side identification information. The same / opposite side identification information is used to determine whether the satellite and the sun are located on the same side or opposite side of the target building, respectively. When the first determining module 601 determines the height of the target building included in the satellite image based on the pre-trained shadow instance segmentation model and the metadata of the satellite image, it is specifically used to: input the satellite image into the shadow instance segmentation model for instance segmentation processing to obtain a first segmentation result image including the shadow; determine the height projection direction of the shadow according to the solar azimuth angle; determine the shadow length of the shadow according to the first segmentation result image and the height projection direction; and determine the height based on the solar altitude angle, satellite altitude angle, same / opposite side identification information, and shadow length.

[0093] Optionally, when determining the shadow length of the shadow based on the first segmentation result image and the height projection direction, the first determining module 601 is specifically used to: rotate the first segmentation result image according to the height projection direction to obtain an angle-corrected segmentation result image; binarize the angle-corrected segmentation result image to obtain a first image; determine the minimum bounding rectangle of the angle-corrected shadow contour extracted in the first image; and determine the shadow length of the shadow based on the minimum bounding matrix.

[0094] Optionally, when determining the planar area of ​​a target building based on a pre-trained contour instance segmentation model and satellite images, the second determining module 602 is specifically used to: input the satellite image into the contour instance segmentation model for instance segmentation processing to obtain a second segmentation result image including the planar contour; perform binarization processing on the second segmentation result image to obtain a second image; perform extraction processing on the second image to obtain the planar contour; and determine the planar area based on the planar contour.

[0095] Optionally, when generating alarm information if the height and / or plane area meet preset conditions, the alarm module 603 specifically performs the following steps: storing the height and plane area in an indicator queue, the indicator queue including multiple heights and multiple plane areas arranged in chronological order of acquisition time; determining a target height as a first threshold among the multiple heights, and determining a target plane area as a second threshold among the multiple plane areas; storing a first comparison result of the height with the first threshold and / or a second comparison result of the plane area with the second threshold in a difference queue, the first comparison result indicating that the height is equal to the first threshold or the difference between the height and the first threshold is less than a third threshold, and the second comparison result indicating that the plane area is equal to the second threshold or the difference between the plane area and the second threshold is less than a fourth threshold; and generating alarm information if the difference queue meets preset conditions.

[0096] The construction project progress monitoring device provided in this embodiment is used to execute the technical solution of the construction project progress monitoring method in the aforementioned method embodiment. Its implementation principle and technical effect are similar, and will not be described again here.

[0097] This application also provides an electronic device. Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application, such as... Figure 7 As shown, the electronic device 700 includes:

[0098] The processor 711, the memory 712 which is communicatively connected to the processor 711, and the interaction interface 713;

[0099] The memory 712 is used to store computer-executable instructions that can be executed by the processor 711;

[0100] The processor 711 is configured to execute computer instructions stored in the execution memory 712 to implement the technical solution of the above-mentioned method for monitoring the progress of construction projects.

[0101] In the aforementioned electronic device 700, the memory 712, processor 711, and interaction interface 713 are electrically connected directly or indirectly to achieve data transmission or interaction. For example, these components can be electrically connected to each other via one or more communication buses or signal lines, such as a bus connection. The memory 712 stores computer-executable instructions for implementing a method to monitor the progress of a construction project, including at least one software functional module that can be stored in the memory in the form of software or firmware. The processor 711 executes various functional applications and data processing by running the software program and module stored in the memory 712.

[0102] The memory can be, but is not limited to, Random Access Memory (RAM), Read Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), and Electrically Erasable Programmable Read-Only Memory (EEPROM). The memory stores programs, which are then executed by the processor upon receiving execution instructions. Furthermore, the software programs and modules within the memory may include an operating system, which can include various software components and / or drivers for managing system tasks (e.g., memory management, storage device control, power management), and can communicate with various hardware or software components to provide an operating environment for other software components.

[0103] A processor can be an integrated circuit chip with signal processing capabilities. The aforementioned processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor.

[0104] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the technical solution of the construction project progress monitoring method provided in the foregoing method embodiments.

[0105] This application also provides a computer program product, including a computer program, which, when executed by a processor, is used to implement the technical solution of the construction project progress monitoring method provided in the foregoing method embodiments.

[0106] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the claims.

[0107] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A method of monitoring progress of a construction project, characterized by, The method comprises the following steps: determining the height of a target building included in a satellite image based on a pre-trained shadow instance segmentation model and meta information of the satellite image, the shadow instance segmentation model being used to determine the shadow of the target building relative to the ground; determining the planar area of the target building based on a pre-trained contour instance segmentation model and the satellite image, the contour instance segmentation model being used to determine the planar contour of the target building relative to the ground; generating an alarm information if the height and / or the planar area meet a preset condition, the preset condition comprising that the height is equal to a first threshold value and / or the planar area is equal to a second threshold value, or the difference between the height and the first threshold value is less than a third threshold value and / or the difference between the planar area and the second threshold value is less than a fourth threshold value, the alarm information being used to alarm the project progress of the target building; the meta information comprises a solar azimuth angle, a solar elevation angle, a satellite elevation angle and same- or different-side identification information, the same- or different-side identification information being used to determine whether the satellite and the sun are located on the same side or different sides of the target building; the method for determining the height of the target building included in the satellite image based on the pre-trained shadow instance segmentation model and the meta information of the satellite image comprises the following steps: inputting the satellite image into the shadow instance segmentation model for instance segmentation processing to obtain a first segmentation result image including the shadow; determining the height projection direction of the shadow according to the solar azimuth angle; performing rotation processing on the first segmentation result image according to the height projection direction to obtain an angle-corrected segmentation result image; performing binaryzation processing on the angle-corrected segmentation result image to obtain a first image; determining the minimum circumscribed rectangle of the angle-corrected shadow contour extracted in the first image; determining the shadow length of the shadow according to the minimum circumscribed matrix; and determining the height based on the solar elevation angle, the satellite elevation angle, the same- or different-side identification information and the shadow length.

2. The method of claim 1, wherein The shadow instance segmentation model is obtained by the following method: performing augmentation processing on the image samples with the image size smaller than a preset value in the acquired MS-COCO dataset to obtain an augmented dataset; training a path aggregation network model by using the augmented dataset to obtain a pre-training model; performing fine-tuning training on the network weights of the pre-training model by using a target dataset to obtain the shadow instance segmentation model, the target dataset comprising satellite images with shadow instance segmentation labels.

3. The method of claim 2, wherein The method for performing augmentation processing on the image samples with the image size smaller than a preset value in the acquired MS-COCO dataset to obtain an augmented dataset comprises the following steps: screening the image samples in the MS-COCO dataset to obtain a basic dataset; for each image sample in the basic dataset, constructing a multi-dimensional random transformation probability combination vector according to a plurality of floating-point numbers generated by a called random number generator, each vector in the multi-dimensional random transformation probability combination vector corresponding to a selected probability value of a processing operator; The selected probability values of a plurality of processing operators in the multi-dimensional random transformation probability combination vector are processed by step grading to obtain a random transformation combination vector; If it is determined that the first target processing operator exists in the random transformation combination vector, a first target processing corresponding to the first target processing operator is performed on a segmentation annotation pixel mask of the image sample to obtain a target segmentation mask; If it is determined that a second target processing operator is selected in the random transformation combination vector, a second target processing corresponding to the second target processing operator is performed on the image sample to obtain a target image; Based on the target segmentation mask, contour cutting processing is performed on the target image to obtain a target contour of the target image; According to a preset ratio, a plurality of contour copies are generated for the target contour; The plurality of contour copies are used to expand the basic data set to obtain the augmented data set.

4. The method of monitoring progress of a construction project according to any one of claims 1 to 3, wherein, The determination of the planar area of the target building based on the pre-trained contour instance segmentation model and the satellite image includes: The satellite image is input into the contour instance segmentation model for instance segmentation processing to obtain a second segmentation result image including the planar contour; The second segmentation result image is subjected to binarization processing to obtain a second image; The second image is subjected to extraction processing to obtain the planar contour; The planar area is determined according to the planar contour.

5. The method of claim 1 to 3, wherein The generation of the alarm information if the height and / or the planar area meet the preset condition includes: The height and the planar area are stored in an index queue, and the index queue includes a plurality of heights and a plurality of planar areas arranged in chronological order according to acquisition time; A target height is determined in the plurality of heights as the first threshold value, and a target planar area is determined in the plurality of planar areas as the second threshold value; A first comparison result of the height and the first threshold value and / or a second comparison result of the planar area and the second threshold value are stored in a difference queue, the first comparison result is used to indicate that the height is equal to the first threshold value or a difference between the height and the first threshold value is less than the third threshold value, and the second comparison result is used to indicate that the planar area is equal to the second threshold value or a difference between the planar area and the second threshold value is less than the fourth threshold value; The alarm information is generated if it is determined that the difference queue meets the preset condition.

6. A device for monitoring progress of a construction project, characterized in that It includes: The first determination module is configured to determine a height of a target building included in a satellite image based on a pre-trained shadow instance segmentation model and meta information of the satellite image, the shadow instance segmentation model being configured to determine a shadow of the target building relative to the ground; The second determination module is configured to determine a planar area of the target building based on a pre-trained contour instance segmentation model and the satellite image, the contour instance segmentation model being configured to determine a planar contour of the target building relative to the ground; The alarm module is configured to generate an alarm information if it is determined that the height and / or the planar area meet preset conditions, the preset conditions including that the height is equal to a first threshold value and / or the planar area is equal to a second threshold value, or a difference between the height and the first threshold value is less than a third threshold value and / or a difference between the planar area and the second threshold value is less than a fourth threshold value, and the alarm information is used for alarming a project progress of the target building. The meta information includes a solar azimuth angle, a solar elevation angle, a satellite elevation angle, and same-side or different-side identification information of the satellite and the sun, and the same-side or different-side identification information is used for determining whether the satellite and the sun are located on the same side or different sides of the target building. The first determination module is specifically configured to: input the satellite image into the shadow instance segmentation model for instance segmentation processing to obtain a first segmentation result image including the shadow, determine a height projection direction of the shadow according to the solar azimuth angle, and perform rotation processing on the first segmentation result image according to the height projection direction to obtain an angle-corrected segmentation result image. perform binarization processing on the angle-corrected segmentation result image to obtain a first image. determine a minimum circumscribed rectangle of an angle-corrected shadow contour extracted from the first image, and determine a shadow length of the shadow according to the minimum circumscribed rectangle. determine the height based on the solar elevation angle, the satellite elevation angle, the same-side or different-side identification information, and the shadow length.

7. An electronic device, comprising: comprise: a processor, and a memory connected to the processor in communication; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory to implement the method for monitoring a construction project progress according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are executed by the processor to implement the method for monitoring a construction project progress according to any one of claims 1 to 5.

Images