Image-based great wall site wall disease identification method and electronic device

Abstract

The application provides a Great Wall site wall disease identification method and electronic equipment based on images, and relates to the technical fields of surveying and mapping and computer vision. The method comprises the following steps: acquiring at least one unmanned aerial vehicle oblique image of the Great Wall site; processing each unmanned aerial vehicle oblique image through a depth estimation model to obtain a plurality of depth images; determining a plurality of disease areas of the Great Wall site according to each depth image; determining the attributes such as the geographical position of each disease area according to the depth image, the unmanned aerial vehicle oblique image and the spatial position information corresponding to the disease area; and generating a Great Wall site monitoring report. The technical scheme provided by the application realizes the geographical positioning and automatic identification (which can identify wall diseases with a width greater than or equal to 5 times the image resolution) of wall collapse, holes and other diseases through the shape anomaly analysis of the Great Wall site wall in the unmanned aerial vehicle oblique image, and solves the problem that small-scale diseases cannot be identified by high-resolution satellite image analysis methods.

Description

Technical Field

[0001] This application belongs to the field of surveying and computer vision technology, and in particular relates to an image-based method and electronic device for identifying wall defects in Great Wall ruins. Background Technology

[0002] The Great Wall ruins are an important large-scale linear historical and cultural heritage site in my country. Accurate three-dimensional information extraction and damage identification of the Great Wall on a large scale can provide key data support for cultural heritage protection, structural risk early warning and restoration plan formulation.

[0003] Among related technologies, three-dimensional laser scanning, UAV multi-view image reconstruction (SfM+MVS), or interferometric synthetic aperture radar (InSAR) technology can be used to extract three-dimensional information and identify defects of the Great Wall ruins. This method acquires data through specialized equipment or multi-source images and directly relies on professional algorithms to complete structural modeling and defect analysis.

[0004] However, the Great Wall sites are mostly distributed in complex terrains such as mountains and ravines, and the need to simultaneously consider both "large-scale coverage" and "microscopic damage identification" is required. When using the above-mentioned existing technologies to monitor the Great Wall, problems such as "difficulty in balancing cost and efficiency" and "lack of microscopic damage identification" will occur, resulting in monitoring results that cannot fully meet the protection needs, which is not conducive to the efficient and comprehensive digital protection and structural safety management of the Great Wall sites. Summary of the Invention

[0005] This application provides an image-based method and electronic device for identifying wall defects in Great Wall ruins, which solves the problem that the monitoring results of Great Wall ruins in the prior art cannot fully meet the protection needs, and are not conducive to the efficient and comprehensive digital protection and structural safety management of Great Wall ruins.

[0006] To achieve the above objectives, this application adopts the following technical solution: In a first aspect, embodiments of this application provide a method for identifying wall defects in Great Wall ruins based on images, the method comprising: The initial model is trained to obtain the deep estimation model; Acquire at least one drone-based tilted image of each key structural region of the Great Wall ruins within a preset area; Each of the tilted images of the UAV is input into the depth estimation model to obtain multiple depth images; Analyze each of the depth images to determine the wall contour curve corresponding to the depth image; The contour curves of each wall are identified to determine multiple diseased areas of the Great Wall ruins; For each of the diseased areas, the geographical location, disease type, disease range, and degree of damage of the diseased area are determined based on the depth image, the UAV tilt image, and spatial location information corresponding to the diseased area. For each of the diseased areas, disease change information is obtained by continuously acquiring the geographical location, disease type, disease range, and degree of damage corresponding to the diseased area. The disease change information is used to indicate new and aggravated diseases of the Great Wall site. Based on the geographical location, disease type, disease range, and degree of damage corresponding to each diseased area, a Great Wall site monitoring report is generated. The Great Wall site monitoring report is used to present the disease of the Great Wall site in three dimensions.

[0007] Optionally, acquiring at least one UAV tilted image of each key structural region of the Great Wall ruins within the preset area includes: For each of the key structural regions of the Great Wall site, at least one initial image of the key structural region is acquired; Each of the initial images is preprocessed to obtain at least one tilted image of the UAV corresponding to each of the key structural regions.

[0008] Optionally, inputting each of the UAV tilted images into the depth estimation model to obtain multiple depth images includes: For each of the drone tilt images, the drone tilt image is input into the depth estimation model; The depth estimation model is used to extract features from the tilted image of the UAV, thereby obtaining the multi-scale spatial features of the tilted image of the UAV. The depth image corresponding to the tilted image of the UAV is obtained by mapping the multi-scale spatial features through the decoder of the depth estimation model.

[0009] Optionally, the step of analyzing each of the depth images to determine the wall contour curve corresponding to the depth image includes: Based on a pre-set depth difference threshold, the depth of each depth image is traversed to determine the target area and the background area; Based on the target area and the background area, the wall contour curve corresponding to the depth image is identified.

[0010] Optionally, the step of identifying the contour curves of each of the walls to determine multiple diseased areas of the Great Wall ruins includes: For each of the wall contour curves Y(x), the first derivative of the wall contour curve Y(x) is calculated to obtain the gradient function G(x); Based on the height of the Great Wall ruins, determine the threshold Tg for sudden changes in wall height; The wall contour curve is smoothed to obtain a baseline function B(x) that reflects the overall change shape of the wall. An adaptive thresholding method based on the absolute deviation of the median is used to estimate the offset threshold T. D ; When any point x in the wall contour curve satisfies the condition |G(x)| ≥ Tg or |Y(x) - B(x)| ≥ Tg D When this happens, the location of point x in the wall contour curve is determined as the location of the defect. The continuous disease points along the extension direction of the Great Wall ruins are merged to obtain at least one diseased area. The distance between any two disease points in any two diseased areas is greater than a preset width threshold.

[0011] Optionally, merging the spatially continuous disease points along the extension direction of the Great Wall ruins to obtain at least one diseased area includes: The vertical spatial range of the diseased area is determined based on the parameter value corresponding to the offset |Y(x)-B(x)| or the wall height abrupt change threshold Tg. The lateral spatial range of the diseased area is determined based on the number of consecutive diseased points in the space along the extension direction of the Great Wall ruins.

[0012] Optionally, determining the geographical location, disease type, disease extent, and degree of damage of the diseased area based on the depth image corresponding to the diseased area, the UAV tilt image, and spatial location information includes: From multiple tilted images of the drone, locate multiple disease images to which the diseased area belongs; Based on the depth image corresponding to the disease image, the UAV tilt image, and the spatial location information, the geographical location of the diseased area, the disease type, the disease range, and the degree of damage are determined.

[0013] Optionally, the step of obtaining disease change information by continuously acquiring the geographical location, disease type, disease range, and degree of damage corresponding to the diseased area includes: The geographical location, disease type, disease range, and degree of damage corresponding to the continuously acquired diseased areas are compared to obtain comparison results. Based on the comparison results, the disease change information is generated.

[0014] Optionally, the step of generating a Great Wall ruins monitoring report based on the geographical location, disease type, disease range, and degree of damage corresponding to each diseased area includes: The tilted images from the various UAVs are stitched together to obtain an image of the Great Wall ruins. Based on the diseased areas indicated in the Great Wall ruins monitoring report and the corresponding geographical locations, each of the diseased areas is marked in the Great Wall ruins image; A monitoring report for the Great Wall ruins is generated based on the labeled images of the ruins, each of the affected areas, and the extent of damage corresponding to each affected area.

[0015] In a second aspect, embodiments of this application provide an electronic device, including: a memory and a processor, wherein the memory is used to store a computer program; and the processor is used to execute the method described in the first aspect or any embodiment of the first aspect when the computer program is invoked.

[0016] Thirdly, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in the first aspect or any embodiment of the first aspect.

[0017] This application provides an image-based method for identifying wall defects at Great Wall ruins. By training an initial model to obtain a depth estimation model, and acquiring at least one UAV tilted image of each key structural area of ​​the Great Wall ruins, these images are input into the depth estimation model to obtain multiple depth images. Analysis of each depth image yields defect identification results, and finally, a Great Wall ruins monitoring report is generated based on these results. By training the initial model to obtain the depth estimation model, the computational load can be reduced while maintaining depth estimation accuracy (error ≤ 5%). This allows for efficient operation on low- to medium-power computing devices, meeting the needs of rapid on-site monitoring. Through depth anomaly detection and multi-feature analysis, automated identification of microscopic defects such as collapses and cracks is achieved (cracks with a width greater than or equal to 2 cm can be identified). This solves the problem of insufficient resolution and inability to capture small-scale defects in existing InSAR technology, improving the accuracy and reliability of identifying unstable defects at Great Wall ruins. Attached Figure Description

[0018] Figure 1 A schematic flowchart illustrating an image-based method for identifying wall defects at Great Wall ruins, provided as an embodiment of this application; Figure 2A This application provides an example of an unmanned aerial vehicle (UAV) tilted image of a first typical diseased section of the Xinguangwu Great Wall site in Shanxi Province. Figure 2BThis application provides a depth image generated from a drone tilted image of a first typical diseased section of the Xinguangwu Great Wall site in Shanxi Province, as an embodiment of the present application. Figure 2C This application provides a schematic diagram of extreme value detection of the wall outline curve in the first typical diseased section of the Xinguangwu Great Wall site in Shanxi Province, which is an embodiment of the present application. Figure 2D The wall outline curve and the labeled image of the diseased area in a depth image of a first typical diseased section of the Xinguangwu Great Wall site in Shanxi Province, provided for the embodiments of this application; Figure 3A This application provides an example of an unmanned aerial vehicle (UAV) oblique image of a second typical diseased section of the Xinguangwu Great Wall site in Shanxi Province. Figure 3B This application provides a depth image generated from a drone tilted image of a second typical diseased section of the Xinguangwu Great Wall site in Shanxi Province, as an embodiment of the present application. Figure 3C This application provides a schematic diagram of extreme value detection of the wall contour curve in a second typical diseased section of the Xinguangwu Great Wall site in Shanxi Province, as an embodiment of the present application. Figure 3D This application provides a depth image of the wall outline curve and the labeled image of the diseased area in a second typical diseased section of the Xinguangwu Great Wall site in Shanxi Province, which is provided as an embodiment of the present application. Figure 4A This application provides an example of an unmanned aerial vehicle (UAV) tilted image of a third typical diseased section of the Xinguangwu Great Wall site in Shanxi Province. Figure 4B This application provides a depth image generated from a drone tilted image of a third typical diseased section of the Xinguangwu Great Wall site in Shanxi Province, as an embodiment of the present application. Figure 4C This application provides a schematic diagram of extreme value detection of the wall contour curve in a third typical diseased section of the Xinguangwu Great Wall site in Shanxi Province, as an embodiment of the present application. Figure 4D The wall outline curve and the labeled image of the diseased area in a depth image of a third typical diseased section of the Xinguangwu Great Wall site in Shanxi Province, provided for the embodiments of this application; Figure 5A This application provides an example of an UAV tilted image of a fourth typical diseased section of the Xinguangwu Great Wall site in Shanxi Province. Figure 5B This application provides a depth image generated from a drone tilted image of a fourth typical diseased section of the Xinguangwu Great Wall site in Shanxi Province, as an embodiment of the present application. Figure 5C This application provides a schematic diagram of extreme value detection of the wall contour curve in the fourth typical diseased section of the Xinguangwu Great Wall site in Shanxi Province, which is an embodiment of the present application. Figure 5D The wall outline curve and the labeled image of the diseased area in a depth image of a fourth typical diseased section of the Xinguangwu Great Wall site in Shanxi Province, provided for the embodiments of this application; Figure 6 This application provides a visualization of the identification results of wall defects location in a given section of the Great Wall on a UAV real-view 3D model. Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0019] In the following description, specific details such as particular system architectures and technologies are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known image processing techniques, image recognition algorithms, and electronic devices are omitted so as not to obscure the description of this application with unnecessary detail.

[0020] The terminology used in the following embodiments is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. As used in the specification and appended claims of this application, the singular expressions “a,” “the,” “the,” and “the” are intended to also include expressions such as “one or more,” unless the context clearly indicates otherwise.

[0021] The Great Wall ruins are an important large-scale linear historical and cultural heritage site in my country. Accurate three-dimensional information extraction and damage identification of the Great Wall ruins on a large scale can provide crucial data support for cultural heritage protection, structural risk early warning, and restoration plan development. However, current monitoring schemes for Great Wall ruins in some areas suffer from high equipment costs, low operational efficiency, insufficient accuracy in identifying microscopic damage (such as cracks and localized voids), and low automation, making it difficult to quickly respond to sudden protection needs. Therefore, improving the efficiency and accuracy of Great Wall ruins monitoring, lowering equipment and operational barriers, and providing lightweight, automated three-dimensional information and damage identification results for cultural heritage protection and structural safety management are urgent problems that need to be solved in the field of Great Wall ruins protection.

[0022] Existing methods employ 3D laser scanning, UAV multi-view image reconstruction (SfM+MVS), or interferometric synthetic aperture radar (InSAR) technology to extract 3D information and identify defects in Great Wall ruins. These methods acquire data through specialized equipment or multi-source images, directly relying on professional algorithms to complete structural modeling and defect analysis. These methods do not depend on single ordinary images, require highly specialized data acquisition and processing, and have significant advantages in small-scale fine-grained modeling or large-scale macroscopic deformation monitoring, demonstrating high technological maturity.

[0023] In actual Great Wall site protection scenarios, Great Wall sites are mostly distributed in complex terrains such as mountains and ravines, and it is necessary to simultaneously consider the needs of "large-scale coverage" and "microscopic damage identification". However, different existing technologies have different characteristics in terms of cost, efficiency and accuracy. Three-dimensional laser scanning has high accuracy but expensive equipment and poor terrain adaptability. The SfM+MVS method has a long data processing cycle and is greatly affected by weather. InSAR is suitable for macroscopic deformation monitoring but has low spatial resolution and cannot capture small-scale damage. When using the above existing technologies to monitor Great Wall sites, the problems of "difficulty in balancing cost and efficiency" and "lack of microscopic damage identification" will occur, resulting in monitoring results that cannot fully meet protection needs, which is not conducive to the efficient and comprehensive digital protection and structural safety management of Great Wall sites.

[0024] Therefore, this application proposes an image-based method for identifying wall defects in Great Wall ruins. By training an initial model, a depth estimation model is obtained. At least one UAV tilt image of each key structural area of ​​the Great Wall ruins within a preset area is acquired. Each UAV tilt image is input into the depth estimation model to obtain multiple depth images. Each depth image is analyzed to determine the wall contour curve corresponding to the depth image. Each wall contour curve is identified to determine multiple defect areas of the Great Wall ruins. For each defect area, the geographical location of the defect area is determined based on the corresponding depth image and UAV tilt image. Based on each defect area and its corresponding geographical location, a Great Wall ruins monitoring report is generated.

[0025] The technical solution provided in this application, through training an initial model, obtains a depth estimation model. This reduces computational load while maintaining depth estimation accuracy (error ≤ 5%), enabling efficient operation on low-to-medium computing power devices and meeting the needs of rapid on-site monitoring. Through depth anomaly detection and multi-feature analysis, it achieves automated (and semi-intelligent) identification of microscopic defects such as collapses and cracks (identifying cracks with a width greater than or equal to 2 cm), achieving a recall rate of up to 88%. This solves the problems of insufficient resolution and inability to capture small-scale defects in existing InSAR technology, improving the accuracy and reliability of identifying unstable defects at Great Wall sites. It promotes the development of existing Great Wall wall defect identification technology towards intelligence, speed, and low cost, providing technical support for the protection and monitoring of the Great Wall linear cultural heritage.

[0026] Figure 1 This schematic flowchart illustrates an image-based method for identifying wall defects at Great Wall ruins, provided as an example and not a limitation. See also... Figure 1 The method includes: Step 101: Train the initial model to obtain the depth estimation model.

[0027] The initial model is a pre-set image processing model. For example, the initial model can be based on the DepthAnything model (a basic model for monocular depth estimation). This application does not specifically limit the initial model.

[0028] Before identifying the instability and damage of the Great Wall ruins based on images, an initial model can be trained first. After multiple iterations using a pre-set loss function, a depth estimation model can be obtained. In subsequent steps, images of the Great Wall ruins can be input into the depth estimation model, allowing analysis of the damage to the Great Wall ruins through the depth estimation model.

[0029] Accordingly, the initial model can be used as the "teacher model" (pre-trained on the publicly available depth estimation dataset (KITTI)) and then fine-tuned on a pre-set Great Wall ruins depth annotation dataset (containing 500+ sets of Great Wall RGB images and corresponding field-measured depth data). During training, KL divergence loss (measuring the difference in output distribution between the teacher and student models) and mean squared error loss (measuring the error between the student model and the true depth reference value) are used as the joint loss function. After multiple iterations (e.g., 120 iterations), training is stopped to obtain the depth estimation model.

[0030] Step 102: Obtain at least one UAV tilted image of each key structural area of ​​the Great Wall ruins within the preset area.

[0031] In identifying the extent of damage to Great Wall ruins, oblique images of the ruins can be acquired using drones. These images can then be analyzed in depth to determine if damage exists. Therefore, images of the Great Wall ruins can be acquired using various devices. The following section uses a drone as an example to illustrate the process of acquiring oblique drone images.

[0032] Optionally, for each key structural area of ​​the Great Wall site, at least one initial image of the key structural area (such as the wall facade, crenellations, gate towers or ramparts) can be acquired first, and then each initial image can be preprocessed to obtain at least one UAV tilted image corresponding to each key structural area.

[0033] Accordingly, for each key structural area of ​​the Great Wall site, at least one initial image of the key structural area can be acquired first, and then each initial image can be preprocessed to obtain at least one UAV tilted image of each key structural area of ​​the Great Wall site.

[0034] Specifically, images can be acquired using drones or high-resolution ground-based photography equipment in clear weather with an illumination intensity of 800-1200 lux (avoiding periods of overcast skies, backlighting, or strong shadows) to ensure that the image resolution is no less than 2K (2560×1440 pixels), the frame rate is no less than 30fps, and the overlap rate between adjacent images is controlled at 30%-50% (for subsequent multi-image fusion verification). Images of key structural areas of the Great Wall ruins can be acquired to obtain single or multiple RGB images as initial images.

[0035] After obtaining the initial image, image noise can be removed by first applying Gaussian filtering (convolution kernel size 3×3, standard deviation 0.8), then the initial image can be uniformly cropped to 512×512 pixels (to adapt to the model input size), and finally the pixel values ​​can be normalized, that is, the RGB channel values ​​can be mapped from 0-255 to 0-1.

[0036] Additionally, the RGB color space can be converted to the HSV color space, and histogram equalization can be performed on the luminance channel (V channel), thereby improving the contrast between the Great Wall structure and the background under complex lighting conditions.

[0037] It should be noted that RGB imagery has the advantages of conventional acquisition equipment and low data acquisition cost, but it needs to be preprocessed to eliminate noise and illumination interference in order to provide high-quality input for subsequent depth estimation. The selection of acquisition range covering key structures is to ensure that the depth information can fully reflect the core areas of the Great Wall that are prone to damage, and to avoid missing key monitoring points.

[0038] Step 103: Input the tilted images of each UAV into the depth estimation model to obtain multiple depth images.

[0039] The role of depth imaging is to transform the two-dimensional information of RGB images into three-dimensional depth information, thereby visually reflecting the geometric undulations of the Great Wall's surface. For example, collapsed areas will show a sharp drop in depth value, while cracked areas will show abrupt changes in depth value, thus providing core three-dimensional feature evidence for subsequent damage identification.

[0040] After obtaining multiple tilted images of the UAV, each tilted image can be input into the trained depth estimation model. The depth estimation model analyzes and processes each tilted image and outputs the depth image of each tilted image.

[0041] Optionally, for each UAV tilt image, the UAV tilt image can be input into a depth estimation model, and the depth estimation model can be used to extract features from the UAV tilt image to obtain multi-scale spatial features of the UAV tilt image. Then, the decoder of the depth estimation model can be used to map the multi-scale spatial features to obtain the depth image corresponding to the UAV tilt image.

[0042] Specifically, after inputting the tilted image of the UAV into the depth estimation model, the depth estimation model can extract multi-scale spatial features of the tilted image of the UAV through the encoder, and then map the multi-scale spatial features into pixel-level relative depth values ​​through the decoder, and finally output a depth image with the same size as the tilted image of the UAV.

[0043] Furthermore, the accuracy of the output depth map can be verified. For example, 100 actual depth points can be selected (the depth of key points on the surface of the Great Wall can be measured using a total station), and the depth value error of the corresponding points can be calculated to ensure that the error does not exceed 5% of the actual depth.

[0044] Step 104: Analyze each depth image to determine the wall contour curve corresponding to the depth image.

[0045] After obtaining multiple depth images, each depth image can be processed to analyze and identify the depth difference between the foreground and background corresponding to each depth image. Based on the multiple analysis results, the outline curve of the Great Wall ruins can be obtained.

[0046] Optionally, the depth of each depth image can be traversed first according to a pre-set depth difference threshold (e.g., 15%) to determine the target area and background area, and then the wall contour curve corresponding to the depth image can be identified based on the target area and background area.

[0047] Specifically, bilateral filtering (e.g., spatial standard deviation 1.2, grayscale standard deviation 0.1) can be used to filter the images at various depths to obtain denoised images. While removing noise, the detailed features of the Great Wall ruins' edges (e.g., the top of the wall, crack boundaries) can be preserved, avoiding edge blurring from affecting the location of the lesions.

[0048] Furthermore, based on a pre-set depth difference threshold, the depth of each denoised image can be traversed to determine whether the average depth of each region in the denoised image exceeds the depth difference threshold (e.g., 15%). If it does, the region can be identified as the target region, thus identifying the target regions and background regions in the denoised image.

[0049] For example, for each denoised image, the corresponding UAV tilted image can be input into a pre-set U-Net semantic segmentation model. The U-Net semantic segmentation model identifies the Great Wall ruins in the UAV tilted image and outputs the region mask of the main body of the Great Wall ruins (e.g., a mask value of 1 indicates the target area, and a mask value of 0 indicates the background area). The background area (such as vegetation, ground, or surrounding buildings) and the target area in the UAV tilted image are determined. Finally, features can be extracted from each target area and background area to obtain the wall contour curve.

[0050] Among them, the U-Net semantic segmentation model was trained on a large dataset (e.g., more than 3,000 sets) of Great Wall semantic annotation data, and its accuracy can reach 90%.

[0051] Furthermore, based on a pre-set depth difference threshold, it can be determined whether the average depth of each region in the denoised image exceeds the depth difference threshold (e.g., 15%). If it does not exceed the threshold, the region can be identified as the target area where the Great Wall ruins are located; otherwise, it is the background area, and the region mask of the main body of the Great Wall ruins is output (e.g., a mask value of 1 indicates the target area, and a mask value of 0 indicates the background area).

[0052] Step 105: Identify the contour curves of each wall to determine multiple diseased areas of the Great Wall ruins.

[0053] After obtaining the outline curves of each wall section of the Great Wall site, each outline curve can be identified. Based on the identification results, the damaged areas of the Great Wall site can be determined, so that a monitoring report of the Great Wall site can be generated in subsequent steps based on the identified damaged areas.

[0054] Optionally, for each wall profile curve Y(x), the first derivative of the wall profile curve Y(x) can be calculated to obtain the gradient function G(x). Based on the height of the Great Wall ruins, the threshold Tg for abrupt changes in wall height can be determined. Then, the wall profile curve can be smoothed to obtain the baseline function B(x) that reflects the overall change shape of the wall. At the same time, an adaptive thresholding method based on the absolute deviation of the median can be used to estimate the offset threshold T. D .

[0055] Correspondingly, when any point x in the wall contour curve satisfies the condition |G(x)| ≥ Tg or |Y(x) - B(x)| ≥ Tg, D If so, the location of point x in the wall outline curve can be determined as the location of the defect.

[0056] Then, the continuous disease points in the space along the extension direction of the Great Wall ruins can be merged to obtain at least one disease area. The distance between any two disease points in any two disease areas is greater than a preset width threshold.

[0057] Furthermore, in the process of merging to obtain at least one diseased area, the vertical spatial range of the diseased area can be determined first based on the parameter value corresponding to the offset |Y(x)-B(x)| or the threshold Tg for sudden change in wall height. Then, the horizontal spatial range of the diseased area can be determined based on the number of consecutive diseased points in the space along the extension direction of the Great Wall ruins.

[0058] For example, the threshold for sudden changes in wall height, Tg, can be 5 pixels, and the threshold for width, W, can be 5 pixels. If the length of the abnormal region is greater than or equal to 5 pixels and the width is greater than or equal to 5 pixels, calculated based on geometric features, then it can be determined that there is a structural instability-type disease in the extreme value region.

[0059] Furthermore, it can identify the spatial range and extent of changes in the horizontal and vertical coordinates of the Great Wall ruins facade in various structural instability and damage areas. By fusing depth information from different perspectives for damage identification, the accuracy of damage identification can be further improved, avoiding misjudgments caused by single features (such as the possibility of misjudgment or omission based solely on depth information anomalies from a single perspective).

[0060] Furthermore, after identifying the affected area, features can be extracted from other contour curves on the wall facade to obtain the morphological features, depth gradient features, and geometric features corresponding to the affected area. These morphological features can then be compared with a pre-set area threshold to determine whether any disease requiring attention exists within the affected area. The disease identification result is determined based on the extracted morphological features, depth gradient features, and geometric features.

[0061] Correspondingly, depth gradient features can be compared with a pre-set depth change rate threshold to determine whether cracks or cavities exist in the diseased area; then geometric features can be compared with a pre-set size threshold to determine whether crack-like diseases exist in the diseased area. In subsequent steps, disease identification results can be generated based on the comparison results corresponding to each diseased area. Disease identification through multi-feature fusion can further improve the accuracy of disease identification and avoid misjudgments caused by a single feature (such as misjudging wall stains as diseases based solely on depth anomalies).

[0062] For example, with an area threshold of 0.5 square meters, the area and perimeter of the abnormal region can be determined based on morphological characteristics, and the area of ​​the abnormal region can be calculated. If the area of ​​the abnormal region is greater than or equal to 0.5 square meters, it can be determined that the Great Wall site corresponding to the abnormal region has a disease that needs attention. The depth change rate threshold is 0.3 per millimeter. If the edge depth change rate of the abnormal region is calculated to be greater than or equal to 0.3 per millimeter based on depth gradient characteristics, it can be determined that the abnormal region has cracks or cavities. The size threshold includes a length threshold and a width threshold. The length threshold can be 50 centimeters, and the width threshold can be 2 centimeters. If the length of the abnormal region is greater than or equal to 50 centimeters and the width is greater than or equal to 2 centimeters based on geometric characteristics, it can be determined that the abnormal region has crack-like diseases.

[0063] Step 106: For each diseased area, determine the geographical location, disease type, disease range, and degree of damage based on the depth image, UAV tilt image, and spatial location information corresponding to the diseased area.

[0064] The geographic location is used to represent the coordinates of the affected area. For example, it can be the latitude and longitude and ellipsoidal elevation geographic coordinates (B, L, H) in the WGS-84 coordinate system (World Geodetic System-1984 Coordinate System) (a geocentric coordinate system established in 1984), or it can be the three-dimensional spatial coordinates (X, Y, Z) in the geocentric rectangular coordinate system of the WGS-84 coordinate system.

[0065] Furthermore, the disease type is used to indicate the type of disease; the disease range is used to indicate the area covered by the disease on the Great Wall site; and the degree of damage is used to indicate the degree of erosion of the Great Wall site by the disease.

[0066] After identifying each affected area, the location of the Great Wall ruins within a preset area can be determined using depth images, UAV tilt images, and spatial location information for each affected area, thus obtaining the geographical location corresponding to each affected area. Similarly, the affected areas can be further identified to determine the type, extent, and degree of damage.

[0067] Optionally, multiple disease images belonging to the diseased area can be found from multiple drone tilt images. Based on the depth image corresponding to the disease image, the drone tilt image, and the spatial location information, the geographical location, disease type, disease range, and degree of damage of the diseased area can be determined.

[0068] Specifically, for each affected area, the corresponding affected image can be found from multiple UAV tilt images based on the depth image of the affected area. Then, by combining the location of the affected image in the Great Wall ruins and the location of the affected area in the affected image, the location of the affected area in the Great Wall ruins can be determined, thus obtaining the geographical location of each affected area.

[0069] For example, in the process of retrieving UAV tilt images corresponding to the diseased area, both the depth image and the UAV tilt image are two-dimensional raster images. Their mapping relationship means that they are used as the input and output of the depth estimation model, respectively, and there is a non-linear function transformation. Based on this relationship, algorithms such as KD-tree can be used to automatically retrieve at least two (3-5 images are recommended to balance accuracy and efficiency) UAV tilt images containing the diseased area.

[0070] Furthermore, during the process of calculating the geographic coordinates of the diseased area using SfM, the geographic coordinates corresponding to each corresponding feature point in the diseased area (such as latitude and longitude and ellipsoidal height geographic coordinates (B, L, H) under the WGS-84 coordinate system (World Geodetic System-1984 Coordinate System) can be calculated based on the principle of image homonymous feature matching and object coordinate calculation of the structure of motion restoration (SfM).

[0071] Step 107: For each diseased area, obtain disease change information by continuously acquiring the corresponding geographical location, disease type, disease range, and degree of damage.

[0072] Among them, the disease change information is used to indicate new and aggravated diseases of the Great Wall site.

[0073] Corresponding to step 106, after determining the geographical location, disease type, disease range and degree of damage of each diseased area, drone tilt images of the Great Wall ruins can be continuously acquired, and steps 103 to 106 can be executed in a loop to obtain the geographical location, disease type, disease range and degree of damage of each diseased area at different times.

[0074] Optionally, the geographical location, disease type, disease range, and degree of damage corresponding to the continuously acquired diseased areas can be compared to obtain comparison results, and then disease change information can be generated based on the comparison results.

[0075] Specifically, for the same diseased area at different times, the geographical location, disease type, disease range, and degree of damage can be compared to obtain location differences; disease type can be compared to obtain type differences; disease range can be compared to obtain range differences; and degree of damage can be compared to obtain disease change information. Thus, based on the comparison results composed of various differences, disease change information can be generated, and the changes of the Great Wall site can be determined based on the disease change information.

[0076] Step 108: Generate a monitoring report for the Great Wall ruins based on the geographical location, disease type, disease range, and degree of damage corresponding to each diseased area.

[0077] Among them, the Great Wall site monitoring report is used to present the damage to the Great Wall site in three dimensions.

[0078] After determining the geographical location, disease type, disease range, and degree of damage for each diseased area, a Great Wall ruins monitoring report can be generated for each diseased area. This allows users to identify which locations on the Great Wall ruins have structural instability and the extent of the damage, and to maintain the Great Wall ruins based on the recommendations given in the Great Wall ruins disease identification report.

[0079] Moreover, the verification method can use the results of visual disease surveys from UAV images collected on-site as the real results, and compare the identification results obtained in the embodiments of this application with the real results. The results can be verified and visualized in three dimensions using the indicators of precision, recall, and F1 score. The correctly identified disease areas can also be stored in the database.

[0080] Optionally, the various UAV tilt images can be stitched together to obtain images of the Great Wall ruins. Based on the diseased areas indicated in the Great Wall ruins monitoring report and their corresponding geographical locations, each diseased area can be marked in the Great Wall ruins images. Based on the marked Great Wall ruins images, each diseased area, and the diseased range corresponding to each diseased area, a Great Wall ruins monitoring report can be generated.

[0081] Specifically, each drone tilt image can be pixel-level fused with its corresponding depth image, and the fused images can be stitched together to generate Great Wall ruins images with depth information using pseudo-color encoding (depth values ​​from light to dark correspond to blue to red).

[0082] At the same time, based on the geographical location corresponding to each diseased area, the diseased areas can be marked on the Great Wall ruins image (e.g., marked with a red rectangle, with the marking content including disease type, location coordinates (latitude and longitude), and size parameters).

[0083] Afterwards, the marked diseased areas in the Great Wall ruins image and the corresponding information for each marked diseased area can be organized and the disease level can be supplemented (e.g., minor disease: crack length < 1 meter, width < 5 cm; moderate disease: crack length 1 to 3 meters, width 5 to 10 cm or collapse area 0.5 to 2 square meters; severe disease: crack length > 3 meters, width > 10 cm or collapse area > 2 square meters). Repair suggestions can be proposed for different levels of disease (e.g., minor cracks can be repaired with grouting, and severe collapses require structural reinforcement). Finally, a complete monitoring report containing "three-dimensional depth information - disease distribution - level assessment - repair suggestions" can be output.

[0084] For example, for each disease identified in the tilted images of various drones, attributes such as disease type, geographic coordinates, disease range, and degree of damage can be stored in a pre-set disease monitoring database through images, text, numbers, and hyperlinks, and a disease monitoring report document can be generated.

[0085] Simultaneously, multi-view image reconstruction can be performed on various UAV images to obtain images of the Great Wall ruins, which can then be displayed using a realistic 3D model. Correspondingly, based on the type of damage indicated in the damage monitoring report and the corresponding attributes such as geographical location, damage type, damage range, and degree of damage, each damaged area can be visualized within the 3D realistic display of the Great Wall ruins.

[0086] It should be noted that, for example, such as Figures 2A to 2D , Figures 3A to 3D , Figures 4A to 4D and Figures 5A to 5D As shown, the images presented are UAV tilt images of four typical diseased sections of the Xinguangwu Great Wall site in Shanxi Province, depth images generated from the UAV tilt images, a schematic diagram of extreme value detection of the wall contour curve, and labeled images of the wall contour curve and diseased areas in the depth images.

[0087] like Figures 2A to 2B , Figures 3A to 3B , Figures 4A to 4B and Figures 5A to 5B As shown, UAV oblique and depth images of the Great Wall ruins in four different diseased areas are presented respectively. The method proposed in this application is applied to a section of the Xinguangwu Great Wall in Shuozhou, Shanxi Province, as the study area, as shown in Table 1 and... Figure 6 As shown, the method indicated in the embodiments of this application achieves an 88% recall rate for identifying structural defects in the Great Wall site, a depth prediction time of 12.7 seconds per image, and a processing time of ≤5 minutes per image (running on a regular laptop), fully meeting the needs of "low cost, high efficiency, and high accuracy" for investigation and monitoring in the field of Great Wall site protection.

[0088] Table 1. Accuracy Evaluation Based on DepthAnything V3 Recognition Results

[0089] In summary, the image-based method for identifying wall defects at Great Wall ruins proposed in this application involves training an initial model to obtain a depth estimation model, acquiring at least one UAV tilt image of each key structural area of ​​the Great Wall ruins within a preset area, inputting each UAV tilt image into the depth estimation model to obtain multiple depth images, analyzing each depth image to determine the wall contour curve corresponding to the depth image, identifying each wall contour curve to determine multiple defect areas of the Great Wall ruins, determining the geographical location of each defect area based on the corresponding depth image and UAV tilt image, and generating a Great Wall ruins monitoring report based on each defect area and its corresponding geographical location.

[0090] The technical solution provided in this application, through training an initial model, obtains a depth estimation model. This reduces computational load while maintaining depth estimation accuracy (error ≤ 5%), enabling efficient operation on low-to-medium computing power devices and meeting the needs of rapid on-site monitoring. Through depth anomaly detection and multi-feature analysis, it achieves automated (and semi-intelligent) identification of microscopic defects such as collapses and cracks (identifying cracks with a width greater than or equal to 2 cm), achieving a recall rate of up to 88%. This solves the problems of insufficient resolution and inability to capture small-scale defects in existing InSAR technology, improving the accuracy and reliability of identifying unstable defects at Great Wall sites. It promotes the development of existing Great Wall wall defect identification technology towards intelligence, speed, and low cost, providing technical support for the protection and monitoring of the Great Wall linear cultural heritage.

[0091] Moreover, during the process of acquiring tilted images from drones, a single RGB image of the Great Wall ruins can be captured without the need for dedicated laser scanning equipment or multi-view images, which greatly reduces equipment costs and data acquisition difficulty, and can improve the convenience and flexibility of acquiring tilted images from drones.

[0092] In addition, combining the U-Net semantic segmentation model to accurately extract the main area of ​​the Great Wall site can avoid background interference and improve the accuracy of identifying instability and defects of the Great Wall site.

[0093] This application integrates lightweight depth estimation, semantic segmentation, and disease analysis functions. It can quickly acquire three-dimensional depth information of the Great Wall from a single image and achieve high-accuracy identification of microscopic diseases. The entire process is highly automated (no manual intervention required), improving monitoring efficiency by more than 60%. It can provide technical support for the low-cost, high-efficiency, and comprehensive protection of the Great Wall cultural heritage.

[0094] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0095] Based on the same inventive concept, embodiments of this application also provide an electronic device. Figure 7 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application, such as... Figure 7 As shown, the electronic device provided in this embodiment includes a memory 71 and a processor 72. The memory 71 is used to store a computer program 73; the processor 72 is used to execute the method described in the above method embodiment when the computer program 73 is invoked.

[0096] The electronic device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, so they will not be described again here.

[0097] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the methods described in the above-described method embodiments.

[0098] This application also provides a computer program product that, when run on an electronic device, causes the electronic device to implement the method described in the above-described method embodiments.

[0099] If the integrated units described above are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable storage medium can include at least: any entity or device capable of carrying computer program code to a photographing device / terminal device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0100] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0101] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0102] In the embodiments provided in this application, it should be understood that the disclosed apparatus / devices and methods can be implemented in other ways. For example, the apparatus / device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0103] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0104] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0105] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0106] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0107] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0108] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A method for identifying wall defects in Great Wall ruins based on image analysis, characterized in that, The method includes: The initial model is trained to obtain the deep estimation model; Acquire at least one drone-based tilted image of each key structural region of the Great Wall ruins within a preset area; Each of the tilted images of the UAV is input into the depth estimation model to obtain multiple depth images; Analyze each of the depth images to determine the wall contour curve corresponding to the depth image; The contour curves of each wall are identified to determine multiple diseased areas of the Great Wall ruins; For each of the diseased areas, the geographical location, disease type, disease range, and degree of damage of the diseased area are determined based on the depth image, the UAV tilt image, and the spatial location information corresponding to the diseased area. For each of the diseased areas, disease change information is obtained by continuously acquiring the geographical location, disease type, disease range, and degree of damage corresponding to the diseased area. The disease change information is used to indicate new and aggravated diseases of the Great Wall site. Based on the geographical location, disease type, disease range, and degree of damage corresponding to each diseased area, a Great Wall site monitoring report is generated. The Great Wall site monitoring report is used to present the disease of the Great Wall site in three dimensions.

2. The method according to claim 1, characterized in that, The acquisition of at least one UAV tilted image of each key structural region of the Great Wall ruins within the preset area includes: For each of the key structural regions of the Great Wall site, at least one initial image of the key structural region is acquired; Each of the initial images is preprocessed to obtain at least one tilted image of the UAV corresponding to each of the key structural regions.

3. The method according to claim 1, characterized in that, The process involves inputting each of the UAV tilt images into the depth estimation model to obtain multiple depth images, including: For each of the drone tilt images, the drone tilt image is input into the depth estimation model; The depth estimation model is used to extract features from the tilted image of the UAV, thereby obtaining the multi-scale spatial features of the tilted image of the UAV. The depth image corresponding to the tilted image of the UAV is obtained by mapping the multi-scale spatial features through the decoder of the depth estimation model.

4. The method according to claim 1, characterized in that, The step of analyzing each of the depth images to determine the wall contour curve corresponding to the depth image includes: Based on a pre-set depth difference threshold, the depth of each depth image is traversed to determine the target area and the background area; Based on the target area and the background area, the wall contour curve corresponding to the depth image is identified.

5. The method according to claim 1, characterized in that, The process of identifying the contour curves of each wall section to determine multiple diseased areas of the Great Wall ruins includes: For each of the wall contour curves Y(x), the first derivative of the wall contour curve Y(x) is calculated to obtain the gradient function G(x); Based on the height of the Great Wall ruins, determine the threshold Tg for sudden changes in wall height; The wall contour curve is smoothed to obtain a baseline function B(x) that reflects the overall change shape of the wall. An adaptive thresholding method based on the absolute deviation of the median is used to estimate the offset threshold T. D ; When any point x in the wall contour curve satisfies the condition |G(x)| ≥ Tg or |Y(x) - B(x)| ≥ Tg D When this happens, the location of point x in the wall contour curve is determined as the location of the defect. The continuous disease points along the extension direction of the Great Wall ruins are merged to obtain at least one diseased area. The distance between any two disease points in any two diseased areas is greater than a preset width threshold.

6. The method according to claim 5, characterized in that, The process of merging the various diseased points that are spatially continuous along the extension direction of the Great Wall ruins to obtain at least one diseased area includes: The vertical spatial range of the diseased area is determined based on the parameter value corresponding to the offset |Y(x)-B(x)| or the wall height abrupt change threshold Tg. The lateral spatial range of the diseased area is determined based on the number of consecutive diseased points in the space along the extension direction of the Great Wall ruins.

7. The method according to claim 1, characterized in that, The step of determining the geographical location, disease type, disease extent, and degree of damage of the diseased area based on the depth image corresponding to the diseased area, the UAV tilt image, and spatial location information includes: From multiple tilted images of the drone, locate multiple disease images to which the diseased area belongs; Based on the depth image corresponding to the disease image, the UAV tilt image, and the spatial location information, the geographical location of the diseased area, the disease type, the disease range, and the degree of damage are determined.

8. The method according to claim 1, characterized in that, The process of acquiring disease change information by continuously obtaining the geographical location, disease type, disease range, and degree of damage corresponding to the diseased area includes: The geographical location, disease type, disease range, and degree of damage corresponding to the continuously acquired diseased areas are compared to obtain comparison results. Based on the comparison results, the disease change information is generated.

9. The method according to claim 1, characterized in that, The process generates a Great Wall ruins monitoring report based on the geographical location, disease type, disease range, and degree of damage corresponding to each diseased area, including: The tilted images from the various UAVs are stitched together to obtain an image of the Great Wall ruins. Based on the diseased areas indicated in the Great Wall ruins monitoring report and the corresponding geographical locations, each of the diseased areas is marked in the Great Wall ruins image; A monitoring report for the Great Wall ruins is generated based on the labeled images of the ruins, each of the affected areas, and the extent of damage corresponding to each affected area.

10. An electronic device, characterized in that, include: A memory and a processor, the memory being used to store a computer program; the processor being used to execute the method as described in any one of claims 1 to 9 when the computer program is invoked.

Images