A method for quickly extracting damaged bridges of a zero-shot visual large model in an emergency scenario

Abstract

This application provides a method for rapid extraction of damaged bridges from a large visual model with zero samples in emergency scenarios. It relates to the field of disaster-damaged bridge extraction technology. The method includes: extracting bridge linear features from road data; merging two bridge linear features representing two lanes into a single-lane feature to obtain bridge linear data; and using auxiliary data for spatial location correction and attribute fusion to obtain a disaster-time bridge linear database; extracting bridge images; inputting the bridge images and preset prompts into a large model; outputting bridge status and bridge features; determining the bridge damage based on the output bridge status; and associating the disaster-time bridge linear database to match corresponding bridge geographic information to obtain the geographic information of the damaged bridge. This application enables rapid association and accurate acquisition of geographic information of damaged bridges in emergency scenarios, significantly shortening the disaster damage assessment cycle and providing efficient data support for disaster relief decision-making.

Description

Technical Field

[0001] This application relates to the field of disaster-damaged bridge extraction technology, and in particular to a method for rapid extraction of damaged bridges using a large visual model with zero samples in emergency scenarios. Background Technology

[0002] Earthquake damage to infrastructure, especially bridges, severely impacts post-disaster relief and recovery efforts. As critical nodes in transportation networks, the extent of bridge damage directly affects rescue efficiency and reconstruction progress. Traditional disaster damage assessment methods rely heavily on manual inspections and visual checks, which are not only time-consuming but also limited by personnel and time constraints, making it difficult to provide timely and accurate information during the critical post-disaster relief period.

[0003] In recent years, the synergistic development of remote sensing technology and deep learning methods has provided a new technological paradigm for post-disaster damage assessment. Existing research largely focuses on road and building damage extraction; in contrast, research on damage detection methods for bridge structures is still in the exploratory stage, with current research mainly concentrating on the automated extraction of intact bridges. In the field of bridge damage identification, existing technical approaches can be summarized into two categories: one is based on constructing object-oriented classification models using multispectral features of optical images, and inferring damage through water-bridge segmentation combined with domain knowledge; the other utilizes synthetic aperture radar interferometry (InSAR) technology to obtain pre- and post-disaster deformation coherence coefficient changes (CCD) for state determination.

[0004] In general, both paradigms face significant technical bottlenecks in emergency response scenarios: (1) Time constraints: There is a dual pressure of limited data acquisition and time requirements in the post-disaster emergency phase. Existing methods are difficult to meet the decision-making needs of the "golden 72 hours" rescue window in terms of time cost in data preprocessing, feature engineering, etc.; (2) Sample scarcity: Bridge damage has significant spatiotemporal heterogeneity. Its damage mode is different from the continuous collapse of buildings and the linear damage of roads, which leads to an exponential increase in the difficulty of collecting effective samples. Existing deep learning frameworks are at risk of model degradation due to insufficient training data; (3) Technical applicability limitations: Pixel-level classification methods are easily affected by nearby objects in complex background separation, while InSAR technology is limited by spatial resolution (usually >5 meters) and has insufficient sensitivity for monitoring damage of small and medium-sized bridges with spans of less than 50 meters. Summary of the Invention

[0005] This application provides a method for rapid extraction of damaged bridges using a large visual model with zero samples in emergency scenarios, in order to address the shortcomings of the prior art mentioned in the background section.

[0006] Firstly, this application provides a method for rapid extraction of damaged bridges using a large visual model with zero samples in emergency scenarios, including:

[0007] Obtain road data and auxiliary data, extract bridge linear elements from the road data, merge two bridge linear elements representing two lanes into a single lane element to obtain bridge linear data, and use the place name and address data in the auxiliary data to perform spatial location correction and attribute fusion on the geospatial location information of the bridge linear data to obtain a disaster-time bridge linear database.

[0008] Based on the aforementioned disaster-time bridge linear database, bridge images are extracted;

[0009] The bridge images and preset prompts are input into a large model, which outputs the bridge status and bridge features. Based on the output bridge status, the damage to the bridge is determined. The model is then associated with the disaster-time bridge linear database to match the corresponding bridge geographic information and obtain the geographic information of the damaged bridge.

[0010] In one possible design, the road data includes OSM road vector line feature data, and bridge linear features are extracted from the road data, including:

[0011] Based on the bridge attribute field in the OSM road vector line feature data, bridge features are extracted, and the topology of the bridge features is calculated using the following formula:

[0012]

[0013] In the formula, ρ represents the topology of the bridge element, and P end and P start Let |P| represent the coordinates of the endpoints of the line segment, respectively. end -P start || represents P end and P start The distance, L total Indicates the actual length of the line element;

[0014] If the topology of the bridge element is greater than a set element threshold, then the bridge element is determined to be a bridge linear element.

[0015] In one possible design, the two linear bridge features representing two lanes are merged into a single-lane feature to obtain the bridge linear data, including:

[0016] The bridge linear features are transformed into point features, and the midpoint set of the bridge linear features is calculated using the following formula:

[0017]

[0018] In the formula, M represents the midpoint set, avg represents the average value of the point set, and P... k Let n represent the k-th point set that constitutes a certain linear element, where n represents the number of point sets and k represents the point set index.

[0019] Based on the midpoint set of the bridge's linear features, the set of all nearest neighbor pairs is determined using the following formula:

[0020] Γ={(i,j)|||M i -M j ||≤l}

[0021] In the formula, Γ represents the set of all neighboring pairs, and M i M represents the midpoint of the i-th linear bridge feature. j Let represent the midpoint of the j-th bridge linear feature, l represent the nearest distance threshold, i,j represent the index of the bridge linear feature, and ||M i -M j || represents M i and M j The distance;

[0022] Determine whether the linear neighbor pairs in the neighbor pair set are parallel to determine whether the linear neighbor pairs represent two lanes;

[0023] When it is determined that the linear adjacent pair represents a two-lane road, the two bridge linear elements in the linear adjacent pair are merged into a single-lane element, and the single-lane element is represented by the centerline of the linear adjacent pair.

[0024] The bridge linear data is formed by combining unmerged bridge linear features and single-lane features.

[0025] In one possible design, whether linear neighbor pairs in the neighbor pair set are parallel is determined as follows:

[0026] The direction vector of the bridge linear elements is calculated using the following formula:

[0027]

[0028] In the formula, This represents the direction vector of the i-th linear bridge element. and The x and y coordinates represent the endpoint of the i-th linear element of the bridge. The x and y coordinates represent the starting point of the i-th bridge linear element;

[0029] Based on the direction vector of the bridge linear elements, the included angle between adjacent linear pairs is calculated using the following formula:

[0030]

[0031] In the formula, α represents the included angle between linear adjacent pairs. This represents the direction vector of the j-th bridge linear element. and They represent and Length, express and The absolute value of the dot product;

[0032] When α < 1° or α > 179°, the two bridge linear elements in the linear adjacent pair are judged to be parallel.

[0033] In one possible design, the geospatial location information of the bridge linear data is spatially corrected and attribute-fused using the place name and address data in the auxiliary data to obtain a disaster-time bridge linear database, including:

[0034] The spatial similarity between the auxiliary data and the linear bridge data is calculated using the following formula:

[0035]

[0036] In the formula, d spatial This represents the spatial similarity between the auxiliary data and the bridge linear data. min indicates taking the minimum value. L1 represents the bridge linear features in the bridge linear data. P represents the coordinates of a point in the point set that constitutes the bridge linear features. Q represents the coordinates of the point features in the auxiliary data.

[0037] The similarity in names between auxiliary data and bridge linear data is calculated using the following formula:

[0038]

[0039] In the formula, s name Indicates the similarity between the names of auxiliary data and bridge linear data, name point The name field attribute represents the name of a point feature. line The name field attribute represents the linear feature; max indicates the maximum value; ED indicates the edit distance between the auxiliary data name and the road data name.

[0040] Based on the spatial similarity and name similarity, the matching score is calculated using the following formula:

[0041] match score =λ·d spatial +(1-λ)·s name

[0042] In the formula, match score λ represents the matching score, and λ represents the weight of spatial similarity.

[0043] Based on a set score threshold, if the matching score is greater than the score threshold, it is determined that the bridge linear feature and the point feature belong to the same bridge.

[0044] Using the midpoint coordinates of the bridge's linear and point features representing the same bridge as a reference, the final geospatial location information of the linear bridge is obtained, thus creating a disaster-time bridge linear database.

[0045] In one possible design, bridge images are extracted based on the disaster-time bridge linear database, including:

[0046] The bounding rectangle is extracted based on the feature vector information, and the corresponding buffer is dynamically expanded outward according to the bridge dimensions. The buffer is then used to mask the post-disaster satellite imagery. The calculation formula for the buffer is as follows:

[0047]

[0048] In the formula, B represents the value of the buffer. min This represents the minimum threshold, set to 10m, B max α represents the maximum threshold, α represents the length scaling factor (a constant), and L represents the length of the bridge.

[0049] In one possible design, the preset prompt words are determined as follows:

[0050] Determine the state determination instruction: limit the model to output only one of four state determination results, namely damaged, intact, uncertain or non-existent;

[0051] Determine feature description instructions, the feature description instructions including:

[0052] The model is required to list specific damage features only when the condition is determined to be damaged. The damage features include one or a combination of structural fracture, bridge deck collapse, bridge pier displacement, and building material scattering.

[0053] Rust, vegetation cover, shadows, and blurry images were excluded as criteria for damage assessment.

[0054] If any bridge exhibits the damage characteristics defined above, the overall condition of the bridge is determined to be damaged;

[0055] Output format specification: The model output should conform to the preset key-value pair JSON format, including a bridge status field to store the status determination result, and a damage feature field to store the damage feature list.

[0056] Secondly, this application provides a device for rapid extraction of damaged bridges using a large visual model with zero samples in emergency scenarios, the device comprising:

[0057] The road data fusion module is configured to acquire road data and auxiliary data, extract bridge linear elements from the road data, merge two bridge linear elements representing two lanes into a single lane element to obtain bridge linear data, and use place name and address data in the auxiliary data to perform spatial location correction and attribute fusion on the geospatial location information of the bridge linear data to obtain a disaster-time bridge linear database.

[0058] The bridge image extraction module is configured to extract bridge images based on the disaster-time bridge linear database;

[0059] The damaged bridge extraction module is configured to input the bridge image and preset prompt words into a large model, output the bridge status and bridge features, determine the damage of the bridge based on the output bridge status, associate with the disaster-time bridge linear database to match the corresponding bridge geographic information, and obtain the geographic information of the damaged bridge.

[0060] Thirdly, embodiments of this application provide an electronic device, including: at least one processor and a memory; the memory stores computer execution instructions; the at least one processor executes the computer execution instructions stored in the memory, causing the at least one processor to perform the method for rapid extraction of damaged bridges with zero-sample visual large models in emergency scenarios as described in the first aspect and various possible designs of the first aspect.

[0061] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions. When a processor executes the computer-executable instructions, it implements the method for rapid extraction of damaged bridges with zero-sample visual large models in emergency scenarios as described in the first aspect and various possible designs of the first aspect.

[0062] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the method for rapid extraction of damaged bridges with zero-sample visual large models in emergency scenarios as described in the first aspect and various possible designs of the first aspect.

[0063] The method for rapid extraction of damaged bridges using zero-sample visual large-scale models in emergency scenarios provided in this application has at least the following beneficial effects:

[0064] This application effectively improves the integrity and consistency of bridge basic data by spatial-attribute fusion of multi-source road data and place name and address auxiliary data, as well as merging of two-lane linear elements, laying a precise vector benchmark for bridge image extraction during disasters. A dynamic buffer strategy based on bridge vectors is used to selectively extract bridge images, reducing background interference and optimizing computational resource allocation. A zero-sample visual large model combined with preset prompts overcomes the dependence on training data, quickly adapting to emergency scenarios. Simultaneously, bridge status determination and feature constraints improve the accuracy and standardization of bridge damage identification. Ultimately, this achieves rapid association and accurate acquisition of geographic information of damaged bridges in emergency scenarios, significantly shortening the disaster damage assessment cycle and providing efficient data support for disaster relief decision-making. Attached Figure Description

[0065] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0066] Figure 1 The flowchart of a method for rapid extraction of damaged bridges in emergency scenarios using a large visual model with zero samples provided in this application embodiment. Figure 1 ;

[0067] Figure 2 The flowchart of a method for rapid extraction of damaged bridges in emergency scenarios using a large visual model with zero samples provided in this application embodiment. Figure 2 ;

[0068] Figure 3 The study area overview map provided for the embodiments of this application includes (a) the location of the epicenter of the study area and the extracted road database, and (b) the location of the study area.

[0069] Figure 4 This is a schematic diagram of a damaged bridge extracted using a rapid bridge extraction method based on a zero-sample visual large model in emergency scenarios, provided in an embodiment of this application; where (a)-(f) represent different bridges respectively.

[0070] Figure 5 This is a structural diagram of the device for rapid extraction of damaged bridges using a large visual model in an emergency scenario, provided in an embodiment of this application.

[0071] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0072] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0073] The collection, storage, use, processing, transmission, provision, and disclosure of financial data or user data involved in the technical solution of this application all comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0074] It should be noted that in the embodiments of this application, certain software, components, models and other existing solutions in the industry may be mentioned. These should be regarded as exemplary and are only intended to illustrate the feasibility of implementing the technical solution of this application. However, it does not mean that the applicant has used or necessarily used the solution.

[0075] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0076] This application provides a method for rapid extraction of damaged bridges using a large visual model with zero samples in emergency scenarios. For example... Figure 1 and Figure 2 The diagram shows the flowchart of the method for rapid extraction of damaged bridges in emergency scenarios using a large visual model with zero samples, provided in this application embodiment. Figure 1 and two This embodiment takes the area within 30km of the epicenter of a 7.9 magnitude earthquake in a certain region in 2025 as an example. This study area is as follows: Figure 3 As shown, the method for rapid extraction of damaged bridges using a zero-sample visual large model in this emergency scenario includes the following steps S100-S300.

[0077] S100: Acquire road data and auxiliary data, extract bridge linear features from the road data, merge the two bridge linear features representing two lanes into a single lane feature to obtain bridge linear data, and use the place name and address data in the auxiliary data to perform spatial location correction and attribute fusion on the geospatial location information of the bridge linear data to obtain a disaster-time bridge linear database.

[0078] In this embodiment, linear bridge geographic information data and point-based auxiliary bridge geographic information data are obtained by collecting OpenStreetMap linear road data, GeographicNames Server (GNS) data, and GeoNames data within the study area. The OSM linear road data and auxiliary data are then fused to obtain a disaster-prone linear road database. Specifically, OpenStreetMap linear road data serves as the road data, while GNS and GeoNames data serve as auxiliary data.

[0079] In some embodiments, such as Figure 2 As shown, in step S100, disaster-time road data can be constructed in the following way: relying on open-source datasets (OSM, GNS, GeoNames), bridge-related data is first extracted through the name.bridge field; linear bridges are filtered in the OSM data, two-way bridge elements are merged, spatial / name similarity between GNS / GeoNames auxiliary data and OSM data is calculated, and finally a linear road database is constructed to achieve disaster-time location.

[0080] In some embodiments, based on the bridge attribute field in the OSM road vector line feature data, linear bridge features are extracted, and non-linear bridge features such as viaducts are excluded by calculating the topological straightness of the features. The specific formula is as follows:

[0081]

[0082] In the formula, ρ represents the topology of the bridge element, and P end and P start Let |P| represent the coordinates of the endpoints of the line segment, respectively. end -P start || represents P end and P start The distance, L total Indicates the actual length of the line element.

[0083] Set threshold θ ρ =0.95, when ρ>θ ρ The bridge was determined to be a straight bridge.

[0084] The process involves merging duplicate features representing two lanes. Specifically, the midpoint of the bridge is first calculated, converting the bridge's linear features into point features. Then, neighboring point pairs are created within a 20m radius of each point, assuming they represent two lanes. The calculation formula is as follows:

[0085]

[0086] In the formula, M represents the midpoint set, avg represents the average value of the point set, and P... kLet n represent the k-th point set that constitutes a certain linear element, where n represents the number of point sets and k represents the point set index.

[0087] Based on the midpoint set of the bridge's linear features, the set of all nearest neighbor pairs is determined using the following formula:

[0088] Γ={(i,j)|||M i -M j ||≤l}

[0089] In the formula, Γ represents the set of all neighboring pairs, and M i M represents the midpoint of the i-th linear bridge feature. j Let represent the midpoint of the j-th bridge linear feature, l represent the nearest distance threshold (20m in this embodiment), and i,j represent the sequence number of the bridge linear feature. ||M i -M j || represents M i and M j The distance.

[0090] By matching the center point features with the corresponding bridge linear features, the bridge's linear proximity pairs can be reconstructed. Whether a linear proximity pair represents a two-lane road can be determined by judging whether the bridge linear features contained within it are parallel. The formula for determining parallelism is shown below:

[0091]

[0092] In the formula, This represents the direction vector of the i-th linear bridge element. and The x and y coordinates represent the endpoint of the i-th linear element of the bridge. The x and y coordinates represent the starting point of the i-th bridge linear element.

[0093]

[0094] In the formula, α represents the included angle between linear adjacent pairs. This represents the direction vector of the j-th bridge linear element. and They represent and Length, express and The absolute value of the dot product.

[0095] When α < 1° or α > 179°, the two straight lines (two bridge linear elements) are considered to be parallel.

[0096] The original linear proximity pairs are removed, and the centerline of each linear proximity pair is used to represent the original single-lane feature. Based on the GeoNames and GNS data name fields, the bridge field containing the bridge name attribute is separated into bridge auxiliary data. The spatial similarity and name similarity between the auxiliary data and the OSM linear data are calculated, and the corresponding road data is matched. The formula for calculating spatial similarity is shown below:

[0097]

[0098] In the formula, d spatial This represents the spatial similarity between the auxiliary data and the bridge linear data. min indicates taking the minimum value. L1 represents the bridge linear features in the bridge linear data. P represents the coordinates of a point in the point set that constitutes the bridge linear features. Q represents the coordinates of the point features in the auxiliary data.

[0099] The formula for name similarity is as follows:

[0100]

[0101] In the formula, s name Indicates the similarity between the names of auxiliary data and bridge linear data, name point The name field attribute represents the name of a point feature. line The name field attribute represents the linear feature. max indicates the maximum value, and ED indicates the edit distance between the auxiliary data name and the road data name.

[0102] Based on the spatial similarity and name similarity, the matching score is calculated using the following formula:

[0103] match score =λ·d spatial +(1-λ)·s name

[0104] In the formula, match score λ represents the matching score, and λ represents the weight of spatial similarity, which is 0.5 in this embodiment.

[0105] In this embodiment, if match score A value greater than 0.6 indicates that both represent the same bridge. The midpoint coordinates of the bridge's linear and point features representing the same bridge are used as the baseline to obtain the final geospatial location information for the linear bridge. Spatial location correction and attribute fusion are performed on the OSM linear data using place name and address data from auxiliary data to obtain a disaster-time bridge linear database.

[0106] By fusing OSM, GNS, and GeoNames data using the above process, a total of 733 linear bridge features were obtained within the study area. To improve the mapping effect, the linear features were converted into point features for mapping, as shown below. Figure 3 As shown in (a).

[0107] S200: Based on a disaster-time bridge linear database, extract bridge images.

[0108] In this embodiment, the circumscribed rectangle is extracted based on the vector information of the linear bridge obtained in step S100, and the corresponding buffer is dynamically expanded outward according to the size of the bridge. The buffer is used to perform masking processing on the post-disaster orthophoto image to extract the bridge image.

[0109] In some embodiments, such as Figure 2 As shown, in step S200, image extraction can be achieved in the following way: based on the linear road database, bridge images are extracted from the post-disaster orthophotos using the image extraction strategy of the bridge dynamic buffer.

[0110] In some embodiments, the bounding rectangle is extracted based on the feature vector information, and the corresponding buffer is dynamically expanded outward according to the size of the bridge. The buffer is then used to mask the post-disaster satellite imagery. The specific calculation formula is as follows:

[0111]

[0112] In the formula, B represents the value of the buffer. min This represents the minimum threshold, which can be set to 10m, B. max The maximum threshold is represented by α, which can be set to 100m. α represents the length scaling factor, a constant that can be set to 0.1. L represents the length of the bridge.

[0113] For bridges exceeding 1000m in length, the image is segmented into multiple segments of hundreds of meters along the bridge vector to enable model-based detailed detection of structural components. Damage assessment integrates the local states of all subdivided regions to derive the overall structural integrity diagnosis of the bridge.

[0114] S300: Input bridge images and preset prompts into the large model, output bridge status and bridge features, determine the damage of the bridge based on the output bridge status, and associate with the disaster-time bridge linear database to match the corresponding bridge geographic information and obtain the geographic information of the damaged bridge.

[0115] In this embodiment, the extracted bridge images and standardized design prompts are input into the Qwen2.5-VL large model, which outputs the bridge's state and features. Based on the output bridge state, the bridge's damage status is determined. The corresponding bridge geographic information is then matched against the road database obtained in step S100 to acquire the geographic information of the damaged bridge. It should be noted that the Qwen2.5-VL large model is merely an example of a large model that can be used in this application and does not constitute a limitation on this application.

[0116] In some embodiments, such as Figure 2 As shown, step S200 obtains the geographical information of the damaged bridge in the following way: Standardized prompts are generated, an example of which is: "As a remote sensing image analysis expert, please strictly adhere to the following bridge condition assessment requirements: ...", serving as rule constraints for model inference. The bridge image and the standardized prompts are input into the visual encoder of the Qwen2.5-VL large model; the visual encoder output is parsed by the Qwen2.5-LM decoder to generate the bridge status and features; based on the model results, the geographical location of the damaged bridge is output, completing the disaster damage assessment.

[0117] The extracted bridge images and standardized design prompts were input into the Qwen2.5-VL large model. The method for designing the standardized prompts is as follows:

[0118] (a) Status determination instruction: The model is limited to outputting only one of four status determination results, namely [damaged], [intact], [uncertain], or [not present];

[0119] (b) Feature description instructions: (1) If and only if the state is determined to be [damaged], the model must list the specific damage features (visual features) in English. The damage features are limited to: structural fracture, bridge deck collapse, bridge pier displacement, and scattered building materials; (2) Rust, vegetation cover, shadow occlusion and image blur are explicitly excluded as damage determination criteria; (3) Define the determination rules for multi-bridge scenarios: if any bridge has the damage features defined in (b)(1) above, the overall state is determined to be [damaged];

[0120] (c) Output format instructions: The model output must strictly conform to the preset key-value pair JSON format, including the "Bridge Status" field to store the status determination result, and the "Damage Features" field to store the list of damage features (if the status is not [Damaged], it will be an empty list).

[0121] Based on the above method, the model prompt words used in this invention are:

[0122] As a remote sensing image analysis expert, please strictly adhere to the following bridge condition assessment requirements:

[0123] The bridge status is determined using only four predefined options: [damaged / not damaged / uncertain / does not exist].

[0124] When identifying damage, specific characteristics must be clearly stated using Chinese descriptive terms, in the format of ["Characteristic 1", "..."...]

[0125] The output must strictly adhere to the JSON format: {"Bridge ID":"[]","Bridge Status":"[Damaged / Undamaged / Uncertain / None Existing]","Damage Characteristics":[]}

[0126] Evaluation criteria:

[0127] Effective damage indicators: structural cracks, bridge deck collapse, pier displacement, and building material debris (damage to a single span in a multi-span bridge is considered "damaged");

[0128] Non-damage indicators: Image features such as coverings / shades, rust / vegetation, etc., shall not be used as evidence of damage.

[0129] The system outputs the bridge's status and characteristics. Based on the output bridge status, it determines the bridge's damage level. It then uses the bridge ID attribute in the output format to match the corresponding bridge geographic information obtained in step 1 from the road database, thereby acquiring the geographic information of the damaged bridges. The final extracted damaged bridges are shown below. Figure 4 As shown.

[0130] This application also provides a device for rapid extraction of damaged bridges using a large visual model with zero samples in emergency scenarios, such as... Figure 5 As shown, the zero-sample visual large model rapid extraction device for damaged bridges in this emergency scenario includes:

[0131] The road data fusion module 501 is configured to acquire road data and auxiliary data, extract bridge linear elements from the road data, merge two bridge linear elements representing two lanes into a single lane element to obtain bridge linear data, and use the place name and address data in the auxiliary data to perform spatial location correction and attribute fusion on the geospatial location information of the bridge linear data to obtain a disaster-time bridge linear database.

[0132] The bridge image extraction module 502 is configured to extract bridge images based on the disaster-time bridge linear database.

[0133] The damaged bridge extraction module 503 is configured to input the bridge image and preset prompt words into a large model, output the bridge status and bridge features, determine the damage of the bridge based on the output bridge status, associate with the disaster-time bridge linear database to match the corresponding bridge geographic information, and obtain the geographic information of the damaged bridge.

[0134] In some embodiments, the road data fusion module is further configured to:

[0135] Based on the bridge attribute field in the OSM road vector line feature data, bridge features are extracted, and the topology of the bridge features is calculated using the following formula:

[0136]

[0137] In the formula, ρ represents the topology of the bridge element, and P end and P start Let |P| represent the coordinates of the endpoints of the line segment, respectively. end -P start || represents P end and P start The distance, L total Indicates the actual length of the line element;

[0138] If the topology of the bridge element is greater than a set element threshold, then the bridge element is determined to be a bridge linear element.

[0139] In some embodiments, the road data fusion module is further configured to:

[0140] The bridge linear features are transformed into point features, and the midpoint set of the bridge linear features is calculated using the following formula:

[0141]

[0142] In the formula, M represents the midpoint set, avg represents the average value of the point set, and P... k Let n represent the k-th point set that constitutes a certain linear element, where n represents the number of point sets and k represents the point set index.

[0143] Based on the midpoint set of the bridge's linear features, the set of all nearest neighbor pairs is determined using the following formula:

[0144] Γ={(i,j)|||M i -M j ||≤l}

[0145] In the formula, Γ represents the set of all neighboring pairs, and M i M represents the midpoint of the i-th linear bridge feature. j Let represent the midpoint of the j-th bridge linear feature, l represent the nearest distance threshold, i,j represent the index of the bridge linear feature, and ||M i -M j || represents M i and M j The distance;

[0146] Determine whether the linear neighbor pairs in the neighbor pair set are parallel to determine whether the linear neighbor pairs represent two lanes;

[0147] When it is determined that the linear adjacent pair represents a two-lane road, the two bridge linear elements in the linear adjacent pair are merged into a single-lane element, and the single-lane element is represented by the centerline of the linear adjacent pair.

[0148] The bridge linear data is formed by combining unmerged bridge linear features and single-lane features.

[0149] In some embodiments, the road data fusion module is further configured to determine whether linear neighbor pairs in the neighbor pair set are parallel by:

[0150] The direction vector of the bridge linear elements is calculated using the following formula:

[0151]

[0152] In the formula, This represents the direction vector of the i-th linear bridge element. and The x and y coordinates represent the endpoint of the i-th linear element of the bridge. The x and y coordinates represent the starting point of the i-th bridge linear element;

[0153] Based on the direction vector of the bridge linear elements, the included angle between adjacent linear pairs is calculated using the following formula:

[0154]

[0155] In the formula, α represents the included angle between linear adjacent pairs. This represents the direction vector of the j-th bridge linear element. and They represent and Length, express and The absolute value of the dot product;

[0156] When α < 1° or α > 179°, the two bridge linear elements in the linear adjacent pair are judged to be parallel.

[0157] In some embodiments, the road data fusion module is further configured to:

[0158] The spatial similarity between the auxiliary data and the linear bridge data is calculated using the following formula:

[0159]

[0160] In the formula, d spatial This represents the spatial similarity between the auxiliary data and the bridge linear data. min indicates taking the minimum value. L1 represents the bridge linear features in the bridge linear data. P represents the coordinates of a point in the point set that constitutes the bridge linear features. Q represents the coordinates of the point features in the auxiliary data.

[0161] The similarity in names between auxiliary data and bridge linear data is calculated using the following formula:

[0162]

[0163] In the formula, S name Indicates the similarity between the names of auxiliary data and bridge linear data, name point The name field attribute represents the name of a point feature. line The name field attribute represents the linear feature; max indicates the maximum value; ED indicates the edit distance between the auxiliary data name and the road data name.

[0164] Based on the spatial similarity and name similarity, the matching score is calculated using the following formula:

[0165] match score =λ·d spatial +(1-λ)·s name

[0166] In the formula, match score λ represents the matching score, and λ represents the weight of spatial similarity.

[0167] Based on a set score threshold, if the matching score is greater than the score threshold, it is determined that the bridge linear feature and the point feature belong to the same bridge.

[0168] Using the midpoint coordinates of the bridge's linear and point features representing the same bridge as a reference, the final geospatial location information of the linear bridge is obtained, thus creating a disaster-time bridge linear database.

[0169] In some embodiments, the bridge image extraction module is further configured to:

[0170] The bounding rectangle is extracted based on the feature vector information, and the corresponding buffer is dynamically expanded outward according to the bridge dimensions. The buffer is then used to mask the post-disaster satellite imagery. The calculation formula for the buffer is as follows:

[0171]

[0172] In the formula, B represents the value of the buffer. min This represents the minimum threshold, set to 10m, B maxα represents the maximum threshold, α represents the length scaling factor (a constant), and L represents the length of the bridge.

[0173] In some embodiments, the damaged bridge extraction module is further configured to determine preset prompt words in the following manner:

[0174] Determine the state determination instruction: limit the model to output only one of four state determination results, namely damaged, intact, uncertain or non-existent;

[0175] Determine feature description instructions, the feature description instructions including:

[0176] The model is required to list specific damage features only when the condition is determined to be damaged. The damage features include one or a combination of structural fracture, bridge deck collapse, bridge pier displacement, and building material scattering.

[0177] Rust, vegetation cover, shadows, and blurry images were excluded as criteria for damage assessment.

[0178] If any bridge exhibits the damage characteristics defined above, the overall condition of the bridge is determined to be damaged;

[0179] Output format specification: The model output should conform to the preset key-value pair JSON format, including a bridge status field to store the status determination result, and a damage feature field to store the damage feature list.

[0180] This application provides an electronic device. The electronic device may include a processor and a memory, wherein the processor and the memory can communicate; exemplarily, the processor and the memory communicate via a communication bus.

[0181] The processor executes computer execution instructions stored in memory, causing the processor to perform the schemes in the above embodiments. The processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0182] The communication bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The system bus can be divided into address bus, data bus, control bus, etc. Transceivers are used to enable communication between database access devices and other computers (e.g., clients, read-write libraries, and read-only libraries). Memory may include random access memory (RAM) and may also include non-volatile memory.

[0183] The electronic device provided in this application embodiment can be the terminal device described in the above embodiments.

[0184] This application also provides a computer-readable storage medium storing computer instructions. When the computer instructions are executed on a computer, the computer performs the technical solution of the zero-sample visual large model rapid extraction method for damaged bridges in emergency scenarios described in the above embodiments.

[0185] This application also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. At least one processor can read the computer program from the computer-readable storage medium. When the at least one processor executes the computer program, it can implement the technical solution of the method for rapid extraction of damaged bridges with zero-sample visual large model in emergency scenarios described in the above embodiments.

[0186] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or modules, and may be electrical, mechanical, or other forms.

[0187] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to implement the solution of this embodiment according to actual needs.

[0188] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing unit, or each module can exist physically separately, or two or more modules can be integrated into one unit. The unit composed of the above modules can be implemented in hardware or in the form of hardware plus software functional units.

[0189] The integrated modules described above, implemented as software functional modules, can be stored in a computer-readable storage medium. These software functional modules, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute some steps of the methods of the various embodiments of this application.

[0190] It should be understood that the aforementioned processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.

[0191] The memory may include high-speed RAM, and may also include non-volatile storage (NVM), such as at least one disk storage device, and may also be a USB flash drive, external hard drive, read-only memory, disk or optical disc, etc.

[0192] Buses can be Industry Standard Architecture (ISA) buses, Peripheral Component Interconnect (PCI) buses, or Extended Industry Standard Architecture (EISA) buses, etc. Buses can be categorized into address buses, data buses, control buses, etc.

[0193] The aforementioned storage medium can be implemented from any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The storage medium can be any available medium accessible to general-purpose or special-purpose computers.

[0194] An exemplary storage medium is coupled to a processor, enabling the processor to read information from and write information to the storage medium. Alternatively, the storage medium can be an integral part of the processor. The processor and storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and storage medium can exist as discrete components in an electronic control unit or main control device.

[0195] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0196] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A method for quickly extracting damaged bridges of a large visual model in a zero-shot emergency scenario, characterized in that, The method comprises: obtaining road data and auxiliary data, extracting bridge linear elements from the road data, merging two bridge linear element lines representing double lanes into single lane elements to obtain bridge linear data, and correcting the spatial position information of the bridge linear data and fusing attributes using the place name address data in the auxiliary data to obtain a disaster-time bridge linear database; based on the disaster-time bridge linear database, extracting bridge images; inputting the bridge images and preset prompt words into a large model to output bridge states and bridge features, determining the damage of the bridge based on the output bridge states, matching corresponding bridge geographic information by associating the disaster-time bridge linear database to obtain the geographic information of the damaged bridge; the road data comprises OSM road vector line element data, and the bridge linear elements are extracted from the road data, comprising: based on the bridge attribute field in the OSM road vector line element data, extracting bridge elements, and calculating the topology of the bridge elements by the following formula: where p denotes the topology of the bridge element, P end and P start denote the coordinates of the end points of the line segment, ||P end -P start || denotes the distance between P end and P start , and L total denotes the actual length of the line element. if the topology of the bridge element is greater than a set element threshold, the bridge element is determined as a bridge linear element; merging two bridge linear element lines representing double lanes into single lane elements to obtain bridge linear data, comprising: converting the bridge linear elements into point elements, and calculating the midpoint set of the bridge linear elements by the following formula: In the formula, M represents a midpoint set, avg represents an average value of the point set, P k represents the kth point set constituting a certain linear element, n represents the number of point sets, and k represents the sequence number of the point set. based on the midpoint set of the bridge linear elements, determining all adjacent pairs by the following formula: Γ = {(i, j) || |M i - M j || ≤ l} where Γ denotes all sets of neighboring pairs, M i denotes the midpoint of the i-th bridge linear feature, M j denotes the midpoint of the j-th bridge linear feature, l represents the proximity distance threshold, i, j represent the serial number of the bridge linear feature, ||M i -M j || denotes the distance between M i and M j . determining whether the linear adjacent pairs in the adjacent pair set are parallel to determine whether the linear adjacent pairs represent double lanes; in the case of determining that the linear adjacent pairs represent double lanes, merging the two bridge linear elements in the linear adjacent pairs into single lane elements, and representing the single lane elements by the center line of the linear adjacent pairs; combining the unmerged bridge linear elements and single lane elements to form the bridge linear data.

2. The method according to claim 1, wherein, determine whether the linear adjacent pairs in the adjacent pair set are parallel by the following method: calculating the direction vector of the bridge linear element by the following formula: wherein denotes the direction vector of the i-th bridge line element, and denotes the end point x and y coordinates of the i-th bridge line element, denotes the start point x and y coordinates of the i-th bridge line element; calculating the included angle of the linear adjacent pairs according to the direction vector of the bridge linear element by the following formula: wherein α represents an included angle of linearly adjacent pairs, denotes a direction vector of the jth bridge linear element, and respectively denote and lengths of denotes and absolute values of dot products; when α < 1° or α > 179°, it is determined that the two bridge linear elements in the linear adjacent pairs are parallel.

3. The method according to claim 1, wherein, correcting the spatial position information of the bridge linear data and fusing attributes using the place name address data in the auxiliary data to obtain a disaster-time bridge linear database, comprising: calculating the spatial similarity of the auxiliary data and the bridge linear data by the following formula: In the formula, d spatial represents the spatial similarity of the auxiliary data and the bridge linear data, min represents taking the minimum value, L1 represents a bridge linear element in the bridge linear data, P represents the coordinates of a certain point in the point set constituting the bridge linear element, and Q represents the coordinates of a point element in the auxiliary data; calculating the name similarity of the auxiliary data and the bridge linear data by the following formula: where s name represents the name similarity between the auxiliary data and the bridge linear data, name point represents the name field attribute of the point-like element, name line represents the name field attribute of the linear element, max represents taking the maximum value, and ED represents the edit distance of the auxiliary data name and the road data name based on the spatial similarity and the name similarity, calculating the matching score by the following formula: match score = λ · d spatial + (1 - λ) · s name In the formula, match score represents the matching score, and λ represents the weight of spatial similarity. based on the set score threshold, if the matching score is greater than the score threshold, it is determined that the bridge linear element and the point element belong to the same bridge; taking the midpoint coordinates of the bridge linear element and the point element representing the same bridge as the reference as the geographic spatial position information of the final linear bridge, and further obtaining the disaster-time bridge linear database.

4. The method according to claim 1, wherein, based on the disaster-time bridge linear database, extracting bridge images, comprising: According to the element vector information, an external rectangle is extracted, a corresponding buffer zone is dynamically expanded outward according to the size of the bridge, and the buffer zone is used to mask the post-disaster satellite image; wherein the calculation formula of the buffer zone is: where B represents the value of the buffer, B min represents the minimum threshold, set to 10 m, B max represents the maximum threshold, a represents a constant length proportion factor, L represents the length of the bridge.

5. The method of claim 1-4, wherein, The preset prompt word is determined in the following manner: Determine the state determination instruction: limit the model to output a four-choice state determination result, which is damage, intact, uncertain or non-existent; Determine the feature description instruction, which includes: When and only when the determined state is damage, the model is required to list specific damage features, including one of structural fracture, bridge deck collapse, pier displacement and building material scattering and combinations thereof; Rust, vegetation coverage, shadow shielding and image blur are excluded as damage determination basis; If any bridge has the damage features defined above, it is determined that the overall state of the bridge is damaged; Determine the output format instruction: require the model to output the JSON format conforming to the preset key-value pair, including the bridge state field storing the state determination result and the damage feature field storing the damage feature list.

6. An emergency scene zero-shot visual large model bridge damage rapid extraction device, characterized in that, The device comprises: A road data fusion module configured to obtain road data and auxiliary data, extract bridge linear elements from the road data, combine two bridge linear elements representing double lanes into a single lane element to obtain bridge linear data, and correct the geospatial position information of the bridge linear data using the place name and address data in the auxiliary data to obtain a disaster-time bridge linear database; A bridge image extraction module configured to extract bridge images based on the disaster-time bridge linear database; A damaged bridge extraction module configured to input the bridge images and preset prompt words into a large model to output bridge state and bridge features, determine the damage of the bridge based on the output bridge state, associate the disaster-time bridge linear database to match corresponding bridge geographic information, and obtain the geographic information of the damaged bridge; The road data fusion module is further configured to: Extract bridge elements based on the bridge attribute field in the OSM road vector line element data, and calculate the topology of the bridge elements by the following formula: where p denotes the topology of the bridge element, P end and P start denote the coordinates of the end points of the line segment, ||P end -P start || denotes the distance of P end and P start , and L total denotes the actual length of the line element; If the topology of the bridge element is greater than a set element threshold, the bridge element is determined to be a bridge linear element; The road data fusion module is further configured to: Convert the bridge linear element into a point element, and calculate the midpoint set of the bridge linear element by the following formula: In the formula, M represents a midpoint set, avg represents an average value of the point set, P k represents the kth point set constituting a certain linear element, n represents the number of point sets, and k represents the point set serial number. Determine all adjacent pair sets based on the midpoint set of the bridge linear element by the following formula: Γ = {(i, j) || |M i - M j || ≤ l} where Γ denotes all sets of neighboring pairs, M i denotes the midpoint of the i-th bridge linear feature, M j denotes the midpoint of the j-th bridge linear feature, l represents the proximity distance threshold, i, j represent the serial number of the bridge linear feature, ||M i -M j || denotes the distance between M i and M j ; Determine whether the linear adjacent pairs in the adjacent pair set are parallel to determine whether the linear adjacent pairs represent double lanes; In the case where it is determined that the linear adjacent pairs represent double lanes, combine the two bridge linear elements in the linear adjacent pairs into a single lane element, and represent the single lane element with the center line of the linear adjacent pairs; Combine the uncombined bridge linear elements and single lane elements to form the bridge linear data.

7. An electronic device, comprising: Comprise: A processor, and a memory connected in communication with the processor; The memory stores computer execution instructions; The processor executes computer-executed instructions stored in the memory to implement the method for quickly extracting damaged bridges of a large visual model in an emergency scene with zero samples according to any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executed instructions, and the computer-executed instructions are executed by the processor to implement the method for quickly extracting damaged bridges of a large visual model in an emergency scene with zero samples according to any one of claims 1-5.

Images