Method for determining damage state of building in post-earthquake inclined image and latitude and longitude inversion method

Abstract

The application provides a post-earthquake building damage state judgment and latitude and longitude inversion method based on oblique images, comprising: obtaining post-earthquake oblique photography images, obtaining a normalized digital surface model (nDSM) based on the oblique photography images; performing multi-threshold segmentation on the normalized digital surface model (nDSM) to preliminarily identify damaged buildings and obtain a damage rate of regional buildings; extracting building multi-side images from the oblique photography images, identifying earthquake damage information, and extracting refined earthquake damage features including cracks and wall peeling; comprehensively judging the damage grade of a target building according to the damage rate of regional buildings and the refined earthquake damage features; and based on the earthquake damage information identified by the earthquake damage information, the geographical latitude and longitude coordinates corresponding to the earthquake damage information are calculated through color spot positioning and coordinate conversion. The application realizes full-process automation from data collection to position output, has high practical value and emergency response capability.

Description

Technical Field

[0001] This application belongs to the field of earthquake engineering and image processing technology, and in particular relates to the method for determining the damage status of buildings in post-earthquake tilted images and the method for latitude and longitude inversion. Background Technology

[0002] Building earthquake damage information is a crucial indicator for assessing the severity of building damage. Accurate identification and quantitative extraction of this information are key priorities. Oblique photography can capture surface information from the side and top of buildings, obtaining rich textural and detailed information to compensate for the limitations of orthophotos.

[0003] Currently, oblique photography is used to extract earthquake damage information from the sides of buildings. However, the identification and extraction process requires a large number of manual experiments to select the optimal feature combination and to extract target features using a series of threshold conditions. This results in inaccurate identification and classification of target features, affecting classification accuracy and efficiency.

[0004] Furthermore, rapidly and accurately obtaining the location information of damaged buildings after an earthquake is crucial for emergency rescue and disaster assessment. Traditional methods of obtaining building damage information have many drawbacks: on-site surveys are dangerous and time-consuming, and satellite remote sensing imagery is limited by various factors and cannot meet the needs. While UAV oblique photography technology has advantages, the extracted building damage results do not carry location information, making it impossible to determine the specific location of damaged buildings.

[0005] The foregoing statements are for informational purposes only and are not intended to provide background information in connection with this application. Unless otherwise stated herein, the content described in this section is not prior art to the rest of this application. Summary of the Invention

[0006] This invention proposes a method for determining the damage state of buildings in post-earthquake tilted images and for inverting latitude and longitude. By performing coordinate inversion on tilted images of buildings based on deep learning, it solves the core pain points of inaccurate extraction of post-earthquake building damage information and lack of location information. It realizes full-process automation from data acquisition to location output, and has high practical value and emergency response capability.

[0007] According to a first aspect of the embodiments of this application, a method for determining the damage state of buildings based on post-earthquake tilted images and for retrieving latitude and longitude is provided, comprising the following steps:

[0008] Oblique photographic images of the post-earthquake area are acquired, and a digital surface model (DSM) and a digital elevation model (DEM) are generated based on the oblique photographic images. The difference between the DSM and the DEM is calculated to obtain a normalized digital surface model (nDSM).

[0009] Multi-threshold segmentation was performed on the normalized digital surface model (nDSM) to initially identify damaged buildings and obtain the damage rate of buildings in the region. Multi-side images of buildings were extracted from oblique photogrammetry images, and seismic damage information and refined seismic damage features, including cracks and peeling of wall plaster, were identified. The damage level of the target buildings was determined comprehensively based on the damage rate of buildings in the region and the refined seismic damage features.

[0010] Based on the earthquake damage information identification, the geographical latitude and longitude coordinates corresponding to the earthquake damage information are calculated by color spot positioning and coordinate transformation.

[0011] In some embodiments of this application, the damage level of a target building is determined comprehensively based on the damage rate of buildings in the region and refined earthquake damage characteristics, including:

[0012] Multi-view images of buildings are extracted from oblique photographic images. A trained deep learning model is used to perform multi-scale segmentation and seismic damage information identification on the multi-view images of buildings, and to extract refined seismic damage features, including cracks and peeling paint.

[0013] Based on the refined earthquake damage characteristics, the comprehensive damage index D of the target building is obtained;

[0014] Based on the comprehensive damage index D of the target building and the damage rate of buildings in the area The weighted average score is S.

[0015] Then, based on the comprehensive score S, the damage level of the target building is comprehensively determined according to the preset damage level rules.

[0016] In some embodiments of this application, the formula for calculating the comprehensive damage index D of the target building is as follows:

[0017] ;

[0018] Among them, F crack This is a crack characteristic factor (e.g., the ratio of total crack length to wall length or height); F spalling It is a characteristic factor of wall peeling (e.g., the ratio of peeling area to wall area); F collapse This is a roof collapse characteristic factor (e.g., the ratio of collapsed area to building footprint, which can be calculated from nDSM); F tilting It is the building tilt factor (e.g., the tilt angle extracted from point cloud data);

[0019] , , , , The weights of each factor satisfy the following conditions: ;

[0020] Regional building damage rate The calculation formula is:

[0021] ;

[0022] in, The number of buildings that are basically intact. This represents the total number of buildings in the experimental area.

[0023] The formula for calculating the overall score S is:

[0024] ;

[0025] Among them, α, β, It is a weighting coefficient, and ;

[0026] in, λ is a correction coefficient used to adjust the degree of influence of the regional damage rate on the damage level of a single building.

[0027] In some embodiments of this application, multi-view images of buildings are extracted from oblique photographic images. A trained deep learning model is used to perform multi-scale segmentation and seismic damage information identification on the multi-view images of buildings, and refined seismic damage features, including cracks and peeling paint, are extracted, including:

[0028] A sample dataset containing various earthquake damage information was constructed, and a VGG-16 convolutional neural network was used to train the model to obtain a target model that can identify earthquake damage characteristics.

[0029] Multi-scale dynamic segmentation of building top and side images is performed to obtain the best segmentation effect for target features;

[0030] The target model is used to identify and classify earthquake damage information from the segmented images, resulting in earthquake damage information and refined earthquake damage features, including cracks and peeling paint.

[0031] In some embodiments of this application, multi-threshold segmentation employs the OTSU adaptive thresholding algorithm, which finds a threshold k such that the inter-class variance σ 2 (k) is the largest, and the inter-class variance σ is the largest. 2 (k) is defined as:

[0032] ;

[0033] in, It represents the proportion of pixels below the threshold k; It represents the proportion of pixels that are above the threshold k; It is the average grayscale value of pixels below the threshold k; It is the average grayscale value of pixels that are above the threshold k;

[0034] The optimal threshold k* is:

[0035] ;

[0036] Where L is the number of gray levels.

[0037] In some embodiments of this application, earthquake damage information identified based on earthquake damage information is used to calculate the corresponding geographical latitude and longitude coordinates of the earthquake damage information through color spot positioning and coordinate transformation, including:

[0038] The earthquake damage information is binarized to generate a single-channel binary image; the binary image is then divided into grids, and the grid center points containing the earthquake damage information are extracted. The planar coordinates of the grid center points are then calculated.

[0039] Select at least one control point and obtain its plane coordinates and latitude and longitude coordinates; based on the coordinates of the control point and the plane coordinates of the center point, derive the geographical latitude and longitude coordinates corresponding to the earthquake damage information through geometric calculations.

[0040] In some embodiments of this application, the geographic latitude and longitude coordinates corresponding to the earthquake damage information are calculated by inversion, including:

[0041] Based on the planar coordinates of the grid center point , Based on the affine transformation model, the latitude and longitude coordinates corresponding to the earthquake damage information were calculated. , The formula is:

[0042] ;

[0043] in, , , , , , These are the transformation coefficients;

[0044] Then, using the plane coordinates and latitude and longitude coordinates of at least three control points, the transformation coefficients are solved by the least squares method to complete the coordinate transformation.

[0045] According to a third aspect of the embodiments of this application, a system for determining the damage state of buildings based on post-earthquake tilted images and for retrieving latitude and longitude is provided, comprising:

[0046] The data preprocessing module is used to acquire oblique photographic images of the post-earthquake area, generate a digital surface model (DSM) and a digital elevation model (DEM) based on the oblique photographic images, and perform a difference operation between the DSM and the DEM to obtain a normalized digital surface model (nDSM).

[0047] The building damage status determination module is used to perform multi-threshold segmentation on the normalized digital surface model nDSM, initially identify damaged buildings, and obtain the damage rate of buildings in the area; extract multi-side images of buildings from oblique photogrammetry images, identify seismic damage information, and extract refined seismic damage features including cracks and peeling paint; and comprehensively determine the damage level of the target building based on the damage rate of buildings in the area and the refined seismic damage features.

[0048] The latitude and longitude inversion module is used to invert and calculate the geographic latitude and longitude coordinates corresponding to the earthquake damage information based on the earthquake damage information identification through color spot positioning and coordinate transformation.

[0049] According to a third aspect of the embodiments of this application, a device for determining the damage state of buildings in post-earthquake tilted images and retrieving latitude and longitude is provided, comprising: a storage unit for storing executable instructions; and a processing unit for connecting to the storage unit to execute the executable instructions to complete the method for determining the damage state of buildings in post-earthquake tilted images and retrieving latitude and longitude.

[0050] According to a fourth aspect of the embodiments of this application, a computer-readable storage medium is provided, on which a computer program is stored; the computer program is executed by a processor to implement a method for determining the damage state of buildings based on post-earthquake tilted images and for latitude and longitude inversion.

[0051] The method and system for determining the damage state of buildings based on post-earthquake oblique imagery and for inverting latitude and longitude coordinates, as described in this application, include: acquiring oblique photographic images of the post-earthquake region; obtaining a normalized digital surface model (nDSM) based on the oblique photographic images; performing multi-threshold segmentation on the nDSM to initially identify damaged buildings and obtain the damage rate of buildings in the region; extracting multi-side images of buildings from the oblique photographic images and identifying earthquake damage information and extracting refined earthquake damage features, including cracks and peeling paint; comprehensively determining the damage level of target buildings based on the damage rate of buildings in the region and the refined earthquake damage features; and inverting and calculating the geographic latitude and longitude coordinates corresponding to the earthquake damage information based on the earthquake damage information identified through color spot localization and coordinate transformation.

[0052] By using deep learning to perform coordinate inversion on tilted images of buildings, the core pain points of inaccurate extraction of post-earthquake building damage information and lack of location information are solved. The entire process from data acquisition to location output is automated, which has high practical value and emergency response capability. Attached Figure Description

[0053] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0054] Figure 1 The diagram illustrates the steps of the method for determining the damage state of buildings based on post-earthquake tilted images and retrieving latitude and longitude according to an embodiment of this application.

[0055] Figure 2 The diagram illustrates the steps for determining the damage state of a building according to an embodiment of this application.

[0056] Figure 3 The diagram illustrates the steps of the latitude and longitude inversion process according to an embodiment of this application.

[0057] Figure 4 The diagram illustrates the principle of normalized DSM imagery according to an embodiment of this application.

[0058] Figure 5 The diagram illustrates the principle of multi-scale dynamic segmentation according to an embodiment of this application;

[0059] Figure 6 The diagram shows a structural schematic of a system for determining the damage status of buildings based on post-earthquake tilted images and for inverting latitude and longitude according to an embodiment of this application.

[0060] Figure 7 The diagram shows a structural schematic of a device for determining the damage status of buildings based on post-earthquake tilt images and for inverting latitude and longitude according to an embodiment of this application. Detailed Implementation

[0061] Regarding this application, when acquiring and assessing the post-earthquake damage status of buildings, the following technical shortcomings are generally considered in common techniques:

[0062] 1. Inaccurate and incomplete extraction of earthquake damage information: Existing methods rely on manually selecting features and threshold conditions, resulting in low accuracy and poor efficiency in identifying and classifying earthquake damage information of buildings (such as cracks and peeling paint).

[0063] 2. Lack of location information: Traditional remote sensing technologies (such as satellite imagery) are limited by resolution and weather conditions, while UAV oblique photography can obtain earthquake damage information, but the results do not carry geographic location coordinates, making it impossible to quickly locate damaged buildings.

[0064] 3. Post-earthquake emergency response delays: On-site investigations are dangerous and time-consuming, making it difficult to obtain the specific locations of damaged buildings in the immediate aftermath of a disaster, which affects rescue efficiency and disaster assessment.

[0065] This invention uses acquired oblique 3D imagery as data for earthquake damage identification. It employs a method combining object-oriented multi-scale segmentation algorithms with deep learning convolutional neural network training models to extract earthquake damage information from building roof and wall images. This overcomes the difficulties of segmentation and limited sample size, addressing the problems of incomplete damage information extraction and inaccurate identification. Furthermore, it creatively combines grid segmentation, center point color spot localization, and coordinate transformation to propose a novel method for inverting the coordinates of damaged buildings, providing a new technical means for post-earthquake rescue.

[0066] This invention has high technical precision and can quickly obtain the location information of damaged buildings, effectively solving the problem of not being able to determine the specific location of damaged buildings in a short time after an earthquake.

[0067] The main innovation of this invention compared to existing technologies lies in:

[0068] 1) Multi-source data fusion and preprocessing:

[0069] A digital surface model (DSM) is generated using oblique photogrammetry. The difference between the DSM and the digital elevation model (DEM) is used to obtain a normalized DSM (nDSM), which removes the influence of terrain and extracts the true height information of buildings.

[0070] Then, multi-threshold segmentation is performed based on nDSM to quickly identify damaged and undamaged buildings.

[0071] 2) Combining deep learning with multi-scale segmentation:

[0072] An object-oriented multi-scale dynamic segmentation algorithm is used, combined with a convolutional neural network (such as VGG-16) for sample training, to automatically extract refined seismic damage information (such as crack size, shape, and location) from the top and sides of buildings.

[0073] Then, based on the 5-level damage status (basically intact to destroyed) rule, the damage level of the building is comprehensively determined.

[0074] 3) Coordinate inversion innovation method:

[0075] By segmenting the grid, locating the center point color spots, and transforming the geometric coordinates, the binarized earthquake damage information is mapped to the actual geographic coordinates (latitude and longitude).

[0076] Then, by using the control point coordinates and 3D model information, a rapid inversion of "earthquake damage extraction results → actual location" is achieved.

[0077] It has the following technical effects:

[0078] 1. Improve the accuracy and efficiency of earthquake damage identification: Oblique imagery makes up for the shortcomings of traditional remote sensing, which can only extract information from the top surface. Combined with deep learning models, it significantly improves the automatic identification and classification accuracy of side earthquake damage such as cracks and spalling.

[0079] 2. Achieve automated location information association: Innovatively bind earthquake damage information with latitude and longitude coordinates to quickly output the specific location of damaged buildings, providing key data support for post-disaster relief.

[0080] 3. Enhance emergency rescue capabilities: The method is fast and highly automated, and can provide accurate information on the damage status and location of buildings in a short time after an earthquake, assisting in rescue decision-making and reducing casualties and property losses.

[0081] To make the technical solutions and advantages of the embodiments of this application clearer, the exemplary embodiments of this application will be described in further detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not an exhaustive list of all embodiments. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of this application can be combined with each other.

[0082] Example 1

[0083] Figure 1 The diagram illustrates the steps of the method for determining the damage state of buildings from post-earthquake tilted images and retrieving latitude and longitude according to an embodiment of this application.

[0084] like Figure 1 As shown, a method for determining the damage state of buildings based on post-earthquake tilted images and for retrieving latitude and longitude is provided, including the following steps:

[0085] S1. Data preprocessing steps: Obtain oblique photogrammetry images of the post-earthquake area, generate digital surface model (DSM) and digital elevation model (DEM) based on the oblique photogrammetry images, and perform difference calculation between DSM and DEM to obtain normalized digital surface model nDSM.

[0086] S2. Building damage status determination steps: Perform multi-threshold segmentation on the normalized digital surface model nDSM to initially identify damaged buildings and obtain the damage rate of buildings in the area; extract multi-side images of buildings from oblique photogrammetry images, identify seismic damage information, and extract refined seismic damage features including cracks and peeling paint; comprehensively determine the damage level of the target building based on the damage rate of buildings in the area and the refined seismic damage features.

[0087] S3. Latitude and longitude inversion steps: Based on the earthquake damage information identified by earthquake damage information, the geographical latitude and longitude coordinates corresponding to the earthquake damage information are calculated by color spot positioning and coordinate transformation.

[0088] Figure 2 The diagram illustrates the steps for determining the damage status of a building according to an embodiment of this application.

[0089] like Figure 2 As shown in S2, the damage level of the target building is determined comprehensively based on the damage state of buildings in the area and refined earthquake damage characteristics, including:

[0090] S21: Extract multi-sided images of buildings from oblique photography images, use a trained deep learning model to perform multi-scale segmentation and seismic damage information identification on the multi-sided images of buildings, and extract refined seismic damage features including cracks and peeling paint.

[0091] S22: Based on refined earthquake damage characteristics, the comprehensive damage index D of the target building is obtained;

[0092] S23: Based on the comprehensive damage index D of the target building and the damage rate of buildings in the area. The weighted average score is S.

[0093] S24: Then, based on the comprehensive score S, the damage level of the target building is comprehensively determined according to the preset damage level rules.

[0094] In the preferred embodiment, the formula for calculating the comprehensive damage index D of the target building is:

[0095] ;

[0096] Among them, F crack This is a crack characteristic factor (e.g., the ratio of the total crack length to the wall length or width); F spalling It is a characteristic factor of wall peeling (e.g., the ratio of peeling area to wall area); F collapse This is a roof collapse characteristic factor (e.g., the ratio of collapsed area to building footprint, which can be calculated from nDSM); F tilting It is the building tilt factor (e.g., the tilt angle extracted from point cloud data);

[0097] , , , , The weights of each factor satisfy the following conditions: ;

[0098] Regional building damage rate The calculation formula is:

[0099] ;

[0100] in, The number of buildings that are basically intact. This represents the total number of buildings in the experimental area.

[0101] The formula for calculating the overall score S is:

[0102] ;

[0103] Among them, α, β, It is a weighting coefficient, and ;

[0104] in, λ is a correction coefficient used to adjust the degree of influence of the regional damage rate on the damage level of a single building.

[0105] During further implementation, when S24 comprehensively determines the damage level of the target building, it makes the determination based on the following preset damage level rules:

[0106] ;

[0107] Among them, threshold The adaptive adjustment formula is:

[0108] ;

[0109] The baseline threshold; This represents the regional influence coefficient. The damage rate of buildings in the area.

[0110] Finally, based on the preset threshold range of the comprehensive score S, the damage status is divided into five levels: basically intact, slightly damaged, moderately damaged, severely damaged, and destroyed.

[0111] In other preferred embodiments, S21 extracts multi-sided images of the building from the oblique photographic images, uses a trained deep learning model to perform multi-scale segmentation and seismic damage information identification on the multi-sided images of the building, and extracts refined seismic damage features, including cracks and peeling paint, including:

[0112] First, a sample dataset containing various earthquake damage information was constructed, and a convolutional neural network VGG-16 was used to train the model to obtain a target model that can identify earthquake damage characteristics.

[0113] Then, multi-scale dynamic segmentation is performed on the images of the building's top and side surfaces to obtain the best segmentation effect for the target features;

[0114] Finally, the target model is used to identify and classify the earthquake damage information from the segmented images, resulting in earthquake damage information and refined earthquake damage features, including cracks and peeling paint.

[0115] In other preferred embodiments, when S2 performs multi-threshold segmentation on the normalized digital surface model nDSM, the multi-threshold segmentation is calculated using the OTSU adaptive threshold algorithm.

[0116] By finding a threshold k such that the inter-class variance σ 2 (k) is the largest, and the inter-class variance σ is the largest. 2 (k) is defined as:

[0117] ;

[0118] in, It represents the proportion of pixels below the threshold k; It represents the proportion of pixels that are above the threshold k; It is the average grayscale value of pixels below the threshold k; It is the average grayscale value of pixels that are above the threshold k;

[0119] The optimal threshold k* is:

[0120] ;

[0121] Where L is the number of gray levels.

[0122] Figure 3 The diagram illustrates the steps of the latitude and longitude inversion process according to an embodiment of this application.

[0123] like Figure 3 As shown, in S3, the earthquake damage information identified based on earthquake damage information is used to calculate the corresponding geographical latitude and longitude coordinates of the earthquake damage information through color spot positioning and coordinate transformation, including:

[0124] S31, Color Spot Location Sub-step: Binarize the earthquake damage information to generate a single-channel binary image; perform grid segmentation on the binary image, extract the grid center point containing the earthquake damage information, and calculate the planar coordinates of the grid center point;

[0125] S32, Coordinate Transformation Sub-step: Select at least one control point and obtain its plane coordinates and latitude and longitude coordinates; based on the coordinates of the control point and the plane coordinates of the center point, calculate and derive the geographical latitude and longitude coordinates corresponding to the earthquake damage information through geometric relationships.

[0126] In the preferred implementation, the geographic latitude and longitude coordinates corresponding to the earthquake damage information are calculated through inversion, specifically including the following calculation process:

[0127] 1) Planar coordinates based on the grid center point Based on the affine transformation model, the latitude and longitude coordinates corresponding to the earthquake damage information were calculated. The formula is:

[0128] ;

[0129] in, These are the transformation coefficients;

[0130] 2) Then, using the plane coordinates and latitude and longitude coordinates of at least three control points, the transformation coefficients are solved by the least squares method to complete the coordinate transformation.

[0131] When using the least squares method to solve for the transformation coefficients, assume there are n control points (n≥3), each control point provides planar coordinates ( , ) and the corresponding latitude and longitude ( , ).

[0132] Construct the matrix equation: L = X·A; where:

[0133] ;

[0134] The least squares solution of the coefficient matrix A is:

[0135] .

[0136] To further illustrate the solution of this embodiment in detail, the following is a further explanation through a specific implementation process.

[0137] 1. Extract damaged buildings from oblique images.

[0138] (1) The difference between the digital surface image and the digital elevation image is used to obtain the normalized DSM image, i.e., nDSM.

[0139] The formula for calculating nDSM is shown below:

[0140] ;

[0141] in, This represents the elevation value in the i-th row and j-th column of an nDSM image; This represents the elevation value in the i-th row and j-th column of a DSM image; This represents the elevation value in the i-th row and j-th column of the DEM image.

[0142] like Figure 4 The image shown is a schematic diagram illustrating the principle of normalized DSM imagery according to an embodiment of this application.

[0143] (2) The nDSM is segmented by multiple thresholds to obtain damaged buildings. Damaged buildings are extracted by setting elevation thresholds, and the damage status of buildings is obtained according to the damage rate formula.

[0144] Multiple thresholds can be used to classify buildings into several categories such as "undamaged", "partially damaged", and "completely damaged" based on different elevation thresholds.

[0145] The damage rate formula is as follows:

[0146] ;

[0147] in, For the rate of destruction, The number of buildings that are basically intact. This represents the total number of buildings in the experimental area.

[0148] II. Extract detailed earthquake damage information from buildings.

[0149] Building earthquake damage extraction mainly involves extracting earthquake damage information from the top and sides of buildings, such as cracks and peeling paint.

[0150] The identification and extraction can be performed mainly through deep learning. The specific steps are as follows:

[0151] (1) Construction of training samples: Select photos containing various earthquake damage information from the top and sides of buildings as sample areas, and use the VGG-16 network model to train sample features, automatically identify and learn the shape, area, color, texture and positional relationship of different ground objects, and describe the features of ground objects more accurately.

[0152] (2) Perform multi-scale dynamic segmentation on the top and side images of buildings to obtain the best segmentation effect of the target features. That is, select multiple scale thresholds to segment each target feature on the remote sensing image separately, so as to obtain the best segmentation effect of the target features. In other words, when the scale of a target feature is changed, the segmentation scale of other adjacent features remains unchanged, ensuring the segmentation effect of the extracted features. It has a good effect on the classification and segmentation of buildings and roads with the same size and color.

[0153] like Figure 5 The diagram shown is a schematic representation of the principle of multi-scale dynamic segmentation according to an embodiment of this application.

[0154] (3) Use training samples to identify and classify earthquake damage information on the top and sides of buildings in the disaster area.

[0155] III. Method for inverting the latitude and longitude of damaged buildings.

[0156] (1) Spot localization: ① The extracted earthquake damage information is binarized into a single-channel binary image, while retaining all contour information of the target image for subsequent identification and localization. ② To ensure the accuracy of subsequent localization, a custom coordinate system needs to be established for the localization image, and control points are selected to obtain the coordinates (x1, y1) and latitude and longitude (lat1, lon1) of the control points. ③ The binary image is divided into grids, the center points of the grids containing damage information are extracted, and the plane coordinates (x2, y2) of the center points are calculated. Small-sized earthquake damage information can be ignored to improve localization efficiency and accuracy.

[0157] (2) Coordinate Transformation: Utilizing the latitude and longitude information and earthquake damage information coordinates carried by the 3D model, based on the coordinates of the control points (x1, y1), latitude and longitude (lat1, lon1), and the target point coordinates (x2, y2), the distance Δd and azimuth α between each grid point are calculated, and the latitude and longitude (lat2, lon2) of the building points with earthquake damage information are derived. This achieves the transformation between planar coordinates and latitude and longitude, completing the coordinate inversion. Partial code is as follows:

[0158] # Read Image

[0159] frame=cv2.imread(r"...",cv2.IMREAD_UNCHANGED)

[0160] framel = frame

[0161] If len(frame.shape) == 2:

[0162] print(frame.shape)

[0163] print(np.max(frame))

[0164] channel13_3 = np.zeros((frame.shape[0], frame.shape[1], 3), dtype=np.uint8)

[0165] print(channe13_3.shape)

[0166] channel13_3[frame>= 1] = np.array([0, 0, 255])

[0167] frame = channe13 3

[0168] print(np.max(frame))

[0169] # Gaussian Blur

[0170] gs_frame = cv2.GaussianBlur(frame, (5, 5), 0)

[0171] # Convert to HSV image

[0172] hsv = cv2.cvtColor(gs_frame, cv2.COLOR_BGR2HSV)

[0173] # Corrosion from coarse to fine

[0174] erode_hsv = cv2.erode(hsv, None, iterations=2)

[0175] # The range of HSV values ​​in red

[0176] inRange_hsv = cv2.inRange(erode_hsv, color_dist[ball_color]['Lower'],color_dist[ball_color]['Upper'])

[0177] # Get each red block as a list data type

[0178] cnts = cv2.findContours(inRange_hsv.copy(), CV2.RETR_EXTERNAL,cv2.CHAIN_APPROX_SIMPLE)[-2]

[0179] # Latitude and Longitude

[0180] # lat lon

[0181] right_t_x =

[0182] right_t_y = .

[0183] In summary, the method for determining the damage state of buildings based on post-earthquake oblique imagery and for inverting latitude and longitude coordinates using the proposed method includes: acquiring oblique photographic images of the post-earthquake region; obtaining a normalized digital surface model (nDSM) based on the oblique photographic images; performing multi-threshold segmentation on the nDSM to initially identify damaged buildings and obtain the damage rate of buildings in the region; extracting multi-side images of buildings from the oblique photographic images and identifying earthquake damage information and extracting refined earthquake damage features, including cracks and peeling paint; comprehensively determining the damage level of target buildings based on the damage rate of buildings in the region and the refined earthquake damage features; and inverting and calculating the corresponding geographic latitude and longitude coordinates of the earthquake damage information through color spot localization and coordinate transformation based on the earthquake damage information identified. This application solves the core pain points of inaccurate extraction of post-earthquake building damage information and lack of location information by using deep learning to invert the coordinates of buildings in oblique images, achieving full automation from data acquisition to location output, and possessing high practical value and emergency response capabilities.

[0184] The method for determining the damage state of buildings based on post-earthquake tilted images and retrieving latitude and longitude in the embodiments of this application achieves the following beneficial technical effects:

[0185] 1. Based on oblique photogrammetry modeling technology, a digital surface model image (DSM) with elevation information is generated. The difference between the DSM image and the digital elevation image (DEM) is calculated to obtain a normalized digital surface model. The terrain influence is removed to obtain the true height of ground features. A multi-threshold segmentation algorithm is used to extract damaged and undamaged buildings.

[0186] 2. Combining the advantages of multi-scale segmentation technology and convolutional neural network model, a combined approach is used to extract detailed information on earthquake damage to the top and sides of buildings in oblique images, including cracks and their size, shape, number, and location. The damage level of the buildings is comprehensively determined according to the five damage states: basically intact, slightly damaged, moderately damaged, severely damaged, and destroyed.

[0187] 3. By creatively combining grid segmentation, center point color spot positioning, and coordinate transformation, a novel method for inverting the coordinates of damaged buildings is proposed. This method addresses the deficiency of extracting earthquake damage information without location information and enables rapid mapping from "earthquake damage extraction results to actual coordinates".

[0188] The oblique photogrammetry technique proposed in this application has significant advantages in extracting earthquake damage information from the sides of buildings, such as those with large areas of peeling paint and obvious cracks, overcoming the limitation of traditional remote sensing, which can only extract earthquake damage information from the roof of buildings. Furthermore, quantitative extraction and accuracy evaluation of the classification results facilitate further improvement of the classification model, enhancing classification accuracy and efficiency. Additionally, the application implements automated correlation between "earthquake damage information and actual coordinates" for the extracted earthquake damage information, rapidly obtaining the location information of damaged buildings. This provides strong data support for immediate post-earthquake rescue response, helping to improve rescue efficiency and reduce casualties and property damage.

[0189] Example 2

[0190] This embodiment provides a system for determining the damage status of buildings in post-earthquake tilted images and for latitude and longitude inversion. For details not disclosed in this embodiment, please refer to the specific implementation details of the post-earthquake tilted image building damage status determination and latitude and longitude inversion schemes in other embodiments.

[0191] Figure 6 The diagram shows a structural schematic of a system for determining the damage status of buildings based on post-earthquake tilted images and for inverting latitude and longitude according to an embodiment of this application.

[0192] like Figure 6 As shown, the system for determining the damage status of buildings based on post-earthquake tilted images and for retrieving latitude and longitude includes:

[0193] Data preprocessing module 10 is used to acquire oblique photographic images of the post-earthquake area, generate a digital surface model (DSM) and a digital elevation model (DEM) based on the oblique photographic images, and perform a difference operation between the DSM and the DEM to obtain a normalized digital surface model (nDSM).

[0194] The building damage status determination module 20 is used to perform multi-threshold segmentation on the normalized digital surface model nDSM, initially identify damaged buildings, and obtain the damage rate of buildings in the area; extract multi-side images of buildings from oblique photogrammetry images, identify seismic damage information, and extract refined seismic damage features including cracks and peeling paint; and comprehensively determine the damage level of the target building based on the damage rate of buildings in the area and the refined seismic damage features.

[0195] The latitude and longitude inversion module 30 is used to invert and calculate the geographical latitude and longitude coordinates corresponding to the earthquake damage information based on the earthquake damage information identification through color spot positioning and coordinate transformation.

[0196] The building damage condition determination module 20 specifically includes:

[0197] The initial judgment unit is used to perform multi-threshold segmentation on the normalized digital surface model nDSM, initially identify damaged buildings, and calculate the damage status of buildings in the area based on the damage rate formula.

[0198] The fine extraction unit is used to extract multi-side images of buildings from oblique photography images. It uses a trained deep learning model to perform multi-scale segmentation and seismic damage information identification on the multi-side images of buildings, and extracts fine seismic damage features including cracks and peeling paint.

[0199] The comprehensive assessment unit is used to comprehensively assess the damage level of the target building based on refined earthquake damage characteristics and according to preset damage level rules.

[0200] The post-earthquake oblique image building damage state determination and latitude-longitude inversion system of this application is adopted. The oblique photogrammetric images of the post-earthquake area are acquired by the data preprocessing module 10, and a normalized digital surface model (nDSM) is obtained based on the oblique photogrammetric images. The nDSM is segmented by multiple thresholds to initially identify damaged buildings and obtain the damage rate of buildings in the area. The building damage state determination module 20 extracts multi-side images of buildings from the oblique photogrammetric images, identifies earthquake damage information, and extracts refined earthquake damage features, including cracks and peeling paint. The damage level of the target buildings is determined comprehensively based on the damage rate of buildings in the area and the refined earthquake damage features. The latitude-longitude inversion module 30 calculates the geographic latitude and longitude coordinates corresponding to the earthquake damage information by inverting the earthquake damage information based on the earthquake damage information identified by the earthquake damage information through color spot positioning and coordinate transformation.

[0201] This application embodiment solves the core pain points of inaccurate extraction of post-earthquake building damage information and lack of location information by performing coordinate inversion on tilted image buildings based on deep learning. It realizes full-process automation from data acquisition to location output, and has high practical value and emergency response capability.

[0202] Example 3

[0203] This embodiment provides a device for determining the damage state of buildings from post-earthquake tilted images and for inverting latitude and longitude. For details not disclosed in this embodiment, please refer to the specific implementation details of the methods or systems for determining the damage state of buildings from post-earthquake tilted images in other embodiments.

[0204] Figure 7 The diagram shows a structural schematic of a device 400 for determining the damage status of buildings based on post-earthquake tilt images and for inverting latitude and longitude according to an embodiment of this application.

[0205] like Figure 7As shown, the device 400 for determining the damage status of buildings in post-earthquake tilted images and retrieving latitude and longitude includes: a storage unit 402 for storing executable instructions; and a processing unit 401 for connecting to the storage unit 402 to execute the executable instructions to complete the method for determining the damage status of buildings in post-earthquake tilted images and retrieving latitude and longitude.

[0206] Those skilled in the art will understand that the illustration Figure 7 This is merely an example of the post-earthquake tilt image building damage state determination and latitude and longitude inversion device 400, and does not constitute a limitation on the post-earthquake tilt image building damage state determination and latitude and longitude inversion device 400. It may include more or fewer components than shown in the figure, or combine certain components, or different components. For example, the post-earthquake tilt image building damage state determination and latitude and longitude inversion device 400 may also include input and output devices, network access devices, buses, etc.

[0207] The processing unit 401 (Central Processing Unit, CPU) can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor, or the processing unit 401 can be any conventional processor. The processing unit 401 is the control center of the post-earthquake tilt image building damage state determination and latitude / longitude inversion device 400, connecting all parts of the device through various interfaces and lines.

[0208] Storage unit 402 can be used to store computer-readable instructions. Processing unit 401 realizes various functions of the post-earthquake tilt image building damage state determination and latitude-longitude inversion device 400 by running or executing the computer-readable instructions or modules stored in storage unit 402 and calling the data stored in storage unit 402. Storage unit 402 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created based on the post-earthquake tilt image building damage state determination and latitude-longitude inversion device 400, etc. In addition, storage unit 402 may include hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, read-only memory (ROM), random access memory (RAM), or other non-volatile / volatile storage devices.

[0209] If the module integrating the post-earthquake tilt image building damage state determination and latitude / longitude inversion device 400 is implemented as a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the above-described embodiments of the present invention can also be implemented by instructing related hardware through computer-readable instructions. The computer-readable instructions can be stored in a computer-readable storage medium, and when executed by a processor, they can implement the steps of the various method embodiments described above.

[0210] Example 4

[0211] This embodiment provides a computer-readable storage medium storing a computer program thereon; the computer program is executed by a processor to implement the method for determining the damage state of buildings in post-earthquake tilted images and retrieving latitude and longitude in other embodiments.

[0212] The device and storage medium for determining the damage status of buildings based on post-earthquake oblique imagery and for inverting latitude and longitude using embodiments of this application include: acquiring oblique photographic images of the post-earthquake region; obtaining a normalized digital surface model (nDSM) based on the oblique photographic images; performing multi-threshold segmentation on the nDSM to initially identify damaged buildings and obtain the damage rate of buildings in the region; extracting multi-side images of buildings from the oblique photographic images and identifying earthquake damage information and extracting refined earthquake damage features, including cracks and peeling paint; comprehensively determining the damage level of target buildings based on the damage rate of buildings in the region and the refined earthquake damage features; and inverting and calculating the geographic latitude and longitude coordinates corresponding to the earthquake damage information based on the earthquake damage information identified through color spot localization and coordinate transformation.

[0213] This application solves the core problems of inaccurate extraction of post-earthquake building damage information and lack of location information by performing coordinate inversion on tilted images of buildings based on deep learning. It realizes full-process automation from data acquisition to location output and has high practical value and emergency response capability.

[0214] Those skilled in the art will understand that the terminology used in this invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The singular forms “a,” “the,” and “the” as used in this invention and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

[0215] It should be understood that although the terms first, second, third, etc., may be used in this invention to describe various information, this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first information may also be referred to as second information without departing from the scope of this invention, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0216] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0217] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for determining the damage state of buildings from post-earthquake tilted images and for retrieving latitude and longitude, characterized in that, Includes the following steps: Oblique photographic images of the post-earthquake area are acquired, and a digital surface model (DSM) and a digital elevation model (DEM) are generated based on the oblique photographic images. The difference between the DSM and the DEM is calculated to obtain a normalized digital surface model (nDSM). The normalized digital surface model (nDSM) is segmented using multiple thresholds to initially identify damaged buildings and obtain the damage rate of buildings in the area. Multi-side images of buildings are extracted from the oblique photographic images, and earthquake damage information is identified and refined earthquake damage features, including cracks and peeling paint, are extracted. The damage level of the target building is determined comprehensively based on the damage rate of buildings in the area and the refined earthquake damage characteristics. Based on the earthquake damage information identified by the earthquake damage information, the geographical latitude and longitude coordinates corresponding to the earthquake damage information are calculated by color spot positioning and coordinate transformation. The step of comprehensively determining the damage level of the target building based on the damage rate of buildings in the area and refined earthquake damage characteristics includes: Multi-view images of buildings are extracted from the oblique photographic images. A trained deep learning model is used to perform multi-scale segmentation and seismic damage information identification on the multi-view images of buildings, and refined seismic damage features, including cracks and peeling paint, are extracted. Based on the refined earthquake damage characteristics, the comprehensive damage index D of the target building is obtained; Based on the comprehensive damage index D of the target building and the damage rate of buildings in the area The weighted average score is S. Then, based on the comprehensive score S, the damage level of the target building is comprehensively determined according to the preset damage level rules; The formula for calculating the comprehensive damage index D of the target building is as follows: ； Among them, F crack It is the crack characteristic factor; F spalling It is a characteristic factor of wall peeling; F collapse It is a characteristic factor of roof collapse; F tilting It is the building tilt factor; The weights of each factor satisfy the following conditions: ; Regional building damage rate The calculation formula is: ； in, The number of buildings that are basically intact. This represents the total number of buildings in the experimental area. The formula for calculating the overall score S is: ； Among them, α, β, It is a weighting coefficient, and ; in, λ is a correction coefficient used to adjust the degree of influence of the regional damage rate on the damage level of a single building.

2. The method for determining the damage state of buildings from post-earthquake tilted images and retrieving latitude and longitude according to claim 1, characterized in that, The process involves extracting multi-sided images of the building from the oblique photographic images, using a trained deep learning model to perform multi-scale segmentation and seismic damage information identification on the multi-sided images, and extracting refined seismic damage features, including cracks and peeling paint. A sample dataset containing various earthquake damage information was constructed, and a VGG-16 convolutional neural network was used to train the model to obtain a target model that can identify earthquake damage characteristics. Multi-scale dynamic segmentation is performed on the top and side images of the building to obtain the best segmentation effect for the target feature; The target model is used to identify and classify earthquake damage information from the segmented images, resulting in earthquake damage information and refined earthquake damage features, including cracks and peeling paint.

3. The method for determining the damage state of buildings from post-earthquake tilted images and retrieving latitude and longitude according to claim 1, characterized in that, The multi-threshold segmentation employs the OTSU adaptive thresholding algorithm, which finds a threshold k such that the inter-class variance σ 2 (k) is the largest, and the inter-class variance σ is the largest. 2 (k) is defined as: ； in, It represents the proportion of pixels below the threshold k; It represents the proportion of pixels that are above the threshold k; It is the average grayscale value of pixels below the threshold k; It is the average grayscale value of pixels that are above the threshold k; The optimal threshold k* is: ； Where L is the number of gray levels.

4. The method for determining the damage state of buildings from post-earthquake tilted images and retrieving latitude and longitude according to claim 1, characterized in that, The earthquake damage information identified based on the earthquake damage information is used to calculate the corresponding geographical latitude and longitude coordinates of the earthquake damage information through color spot positioning and coordinate transformation, including: The earthquake damage information is binarized to generate a single-channel binary image; the binary image is then segmented into a grid, and the center point of the grid containing the earthquake damage information is extracted. The planar coordinates of the center point of the grid are then calculated. Select at least one control point and obtain the plane coordinates and latitude and longitude coordinates of the control point; based on the coordinates of the control point and the plane coordinates of the center point, derive the geographical latitude and longitude coordinates corresponding to the earthquake damage information through geometric calculations.

5. The method for determining the damage state of buildings from post-earthquake tilted images and retrieving latitude and longitude according to claim 4, characterized in that, The inversion calculation of the geographic latitude and longitude coordinates corresponding to the earthquake damage information includes: Based on the planar coordinates of the grid center point ( , Based on the affine transformation model, the latitude and longitude coordinates corresponding to the earthquake damage information were calculated. , The formula is: ； in, , , , , , These are the transformation coefficients; Then, using the plane coordinates and latitude and longitude coordinates of at least three control points, the transformation coefficients are solved by the least squares method to complete the coordinate transformation.

6. A system for determining the damage state of buildings from post-earthquake tilted images and for retrieving latitude and longitude coordinates, characterized in that, include: The data preprocessing module is used to acquire oblique photographic images of the post-earthquake area, generate a digital surface model (DSM) and a digital elevation model (DEM) based on the oblique photographic images, and perform a difference operation between the DSM and the DEM to obtain a normalized digital surface model (nDSM). The building damage status determination module is used to perform multi-threshold segmentation on the normalized digital surface model (nDSM) to initially identify damaged buildings and obtain the damage rate of buildings in the area; extract multi-side images of buildings from the oblique photographic images, identify seismic damage information, and extract refined seismic damage features including cracks and peeling paint; and comprehensively determine the damage level of the target building based on the damage rate of buildings in the area and the refined seismic damage features; wherein, the comprehensive determination of the damage level of the target building based on the damage rate of buildings in the area and the refined seismic damage features includes: Multi-view images of buildings are extracted from the oblique photographic images. A trained deep learning model is used to perform multi-scale segmentation and seismic damage information identification on the multi-view images of buildings, and refined seismic damage features, including cracks and peeling paint, are extracted. Based on the refined earthquake damage characteristics, the comprehensive damage index D of the target building is obtained; Based on the comprehensive damage index D of the target building and the damage rate of buildings in the area The weighted average score is S. Then, based on the comprehensive score S, the damage level of the target building is comprehensively determined according to the preset damage level rules; The formula for calculating the comprehensive damage index D of the target building is as follows: ； Among them, F crack It is the crack characteristic factor; F spalling It is a characteristic factor of wall peeling; F collapse It is a characteristic factor of roof collapse; F tilting It is the building tilt factor; The weights of each factor satisfy the following conditions: ; Regional building damage rate The calculation formula is: ； in, The number of buildings that are basically intact. This represents the total number of buildings in the experimental area. The formula for calculating the overall score S is: ； Among them, α, β, It is a weighting coefficient, and ; in, λ is a correction coefficient used to adjust the degree of influence of the regional damage rate on the damage level of a single building. Based on the earthquake damage information identified, the geographical latitude and longitude coordinates corresponding to the earthquake damage information are calculated by using color spot positioning and coordinate transformation.

7. A device for determining the damage state of buildings from post-earthquake tilted images and for retrieving latitude and longitude, characterized in that, include: Storage unit, used to store executable instructions; And a processing unit, for connecting to a memory to execute executable instructions to complete the method for determining the damage status of buildings in post-earthquake tilted images and retrieving latitude and longitude as described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, It stores a computer program thereon; the computer program is executed by a processor to implement the method as described in any one of claims 1-5.

Images