Method for rapidly extracting zero-sample visual large model damaged bridge in emergency scene

Abstract

The invention provides a method for rapidly extracting a bridge damaged by a zero-sample visual large model in an emergency scene. Relates to the technical field of disaster damage bridge extraction. The method comprises the following steps: extracting bridge linear elements from road data, combining two bridge linear element lines representing double lanes into single lane elements to obtain bridge linear data, and performing spatial position correction and attribute fusion by using auxiliary data to obtain a disaster bridge linear database; extracting a bridge image; inputting a bridge image and a preset cue word into the large model, outputting a bridge state and a bridge feature, determining the damage condition of the bridge based on the output bridge state, associating the bridge linear database in disaster to match the corresponding bridge geographic information, and obtaining the geographic information of the damaged bridge. According to the method, the geographic information of the damaged bridge in the emergency scene can be quickly associated and accurately acquired, the disaster damage assessment period is greatly shortened, and efficient data support is provided for rescue and relief decision.

Description

Technical Field

[0001] The present application relates to the technical field of disaster-damaged bridge extraction, and in particular to a method for rapidly extracting damaged bridges using a large zero-sample visual model in emergency scenarios. Background Art

[0002] Earthquake damage to infrastructure, especially bridges, severely impacts post-disaster rescue and recovery efforts. As critical nodes in transportation networks, the extent of damage to bridges directly impacts rescue efficiency and reconstruction progress. Traditional damage assessment methods rely heavily on manual inspections and visual checks, which are time-consuming and limited by both personnel and time constraints, making it difficult to provide timely and accurate information during the crucial post-disaster rescue period.

[0003] In recent years, the coordinated development of remote sensing technology and deep learning methods has provided a new technical paradigm for post-disaster damage assessment. Existing research has primarily focused on extracting damage to roads and buildings. In contrast, damage detection methods for bridge structures are still in the exploratory stage, with current research primarily focused on the automated extraction of intact bridges. In the field of bridge damage identification, existing technical approaches can be categorized into two types: one is to construct object-oriented classification models based on multispectral features of optical images, inferring damage through water-bridge segmentation combined with domain knowledge; the other is to utilize synthetic aperture radar interferometry (InSAR) technology to obtain the change in coherence coefficient (CCD) of deformation before and after a disaster for state identification.

[0004] In general, both paradigms face significant technical bottlenecks in emergency response scenarios: (1) Time constraints: The post-disaster emergency phase is under the dual pressure of limited data acquisition and processing time requirements. The time cost of existing methods in data preprocessing, feature engineering, etc. is difficult to meet the decision-making needs of the "golden 72 hours" rescue window period; (2) Sample scarcity: Bridge damage has significant spatiotemporal heterogeneity. Its damage pattern is different from the continuous collapse of buildings and the linear damage of roads, resulting in an exponential increase in the difficulty of collecting effective samples. The existing deep learning framework has the risk of model degradation due to insufficient training data; (3) Technical applicability limitations: Pixel-level classification-based methods are easily interfered with by neighboring objects in complex background separation, while InSAR technology is limited by spatial resolution (usually >5 meters) and is not sensitive enough to monitor damage to small and medium-sized bridges with a span of less than 50 meters. Summary of the Invention

[0005] The present application provides a method for rapidly extracting damaged bridges using a large zero-sample visual model in an emergency scenario to solve the existing technology proposed in the background art.

[0006] In a first aspect, the present application provides a method for rapidly extracting damaged bridges using a large zero-sample visual model in an emergency scenario, comprising:

[0007] Acquire road data and auxiliary data, extract bridge linear features from the road data, merge two bridge linear features representing dual lanes into a single lane feature to obtain bridge linear data, and use place name and address data in the auxiliary data to perform spatial position correction and attribute fusion on the geographic spatial location information of the bridge linear data to obtain a disaster bridge linear database;

[0008] Extracting bridge images based on the disaster bridge linear database;

[0009] The bridge image and preset prompt words are input into the large model, the bridge status and bridge characteristics are output, the damage of the bridge is determined based on the output bridge status, the disaster bridge linear database is associated with the corresponding bridge geographic information, and the geographic information of the damaged bridge is obtained.

[0010] In a possible design, the road data includes OSM road vector line feature data, and extracting bridge linear features from the road data includes:

[0011] Based on the bridge attribute field in the OSM road vector line feature data, bridge features are extracted and the topology of the bridge features is calculated using the following formula:

[0012]

[0013] Where ρ represents the topology of the bridge element, P end and P start Represent the coordinates of the endpoints of the line segment, ||P end -P start || indicates P end and P start The distance, L total Indicates the actual length of the line feature;

[0014] If the topology of the bridge element is greater than a set element threshold, the bridge element is determined to be a bridge linear element.

[0015] In one possible design, two bridge linear features representing dual lanes are merged into a single lane feature, resulting in bridge linear data including:

[0016] Convert the bridge linear feature into a point feature, and calculate the midpoint set of the bridge linear feature using the following formula:

[0017]

[0018] In the formula, M represents the midpoint set, avg represents the average value of the point set, and P k It represents the kth point set that constitutes a linear feature, n represents the number of point sets, and k represents the sequence number of the point set;

[0019] Based on the midpoint set of the bridge linear feature, all adjacent pair sets are determined using the following formula:

[0020] Γ={(i,j)||M i -M j ||≤l}

[0021] In the formula, Γ represents the set of all adjacent pairs, M i Indicates the midpoint of the i-th bridge linear element, M j represents the midpoint of the jth bridge linear feature, l represents the proximity distance threshold, i, j represents the serial number of the bridge linear feature, ||M i -M j || indicates M i and M j distance;

[0022] Determining whether linear adjacent pairs in the adjacent pair set are parallel to determine whether the linear adjacent pairs represent a dual lane;

[0023] When it is determined that the linear proximity pair represents a dual lane, the two bridge linear features in the linear proximity pair are merged into a single lane feature, and the center line of the linear proximity pair is used to represent the single lane feature;

[0024] The unmerged bridge linear features and single lane features are combined to form the bridge linear data.

[0025] In one possible design, whether the linear adjacent pairs in the adjacent pair set are parallel is determined by:

[0026] The direction vector of the bridge linear feature is calculated using the following formula:

[0027]

[0028] Where, Represents the direction vector of the i-th bridge linear feature, and Represents the x-coordinate and y-coordinate of the end point of the i-th bridge linear feature, Represents the x-coordinate and y-coordinate of the starting point of the i-th bridge linear feature;

[0029] According to the direction vector of the bridge linear element, the angle between adjacent linear pairs is calculated using the following formula:

[0030]

[0031] Where α represents the angle between adjacent linear pairs, Represents the direction vector of the j-th bridge linear feature, and Respectively and length, express and The absolute value of the dot product;

[0032] When α<1° or α>179°, the two bridge linear features in the adjacent linear pair are judged to be parallel.

[0033] In one possible design, the geographical location information of the bridge linear data is spatially corrected and attribute-fused using the place name and address data in the auxiliary data to obtain a disaster bridge linear database, including:

[0034] The spatial similarity between the auxiliary data and the bridge linear data is calculated using the following formula:

[0035]

[0036] Where, d spatial Indicates the spatial similarity between the auxiliary data and the bridge linear data, min represents the minimum value, L1 represents the bridge linear feature in the bridge linear data, P represents the coordinates of a point in the point set that constitutes the bridge linear feature, and Q represents the coordinates of the point feature in the auxiliary data;

[0037] The similarity between the names of the auxiliary data and the bridge linear data is calculated using the following formula:

[0038]

[0039] Where s name Indicates the similarity between the name of the auxiliary data and the bridge linear data, name point Indicates the name field attribute of the point feature, name line Represents the name field attribute of the linear feature, max indicates the maximum value, and ED indicates the edit distance between the auxiliary data name and the road data name;

[0040] Based on the spatial similarity and name similarity, the matching score is calculated using the following formula:

[0041] match score =λ·d spatial +(1-λ)·s name

[0042] In the formula, match score represents the matching score, and λ represents the weight of spatial similarity;

[0043] Based on a set score threshold, if the matching score is greater than the score threshold, it is determined that the bridge linear feature and the point feature belong to the same bridge;

[0044] Taking the midpoint coordinates of the bridge linear feature and the point feature representing the same bridge as the reference, the geographic spatial position information of the final linear bridge is obtained, and a disaster-time bridge linear database is further obtained.

[0045] In a possible design, based on the disaster-time bridge linear database, a bridge image is extracted, including:

[0046] According to the feature vector information, an inscribed rectangle is extracted, a corresponding buffer area is dynamically expanded outward according to the size of the bridge, and the buffer area is used to mask the post-disaster satellite image; wherein the calculation formula of the buffer area is:

[0047]

[0048] In the formula, B represents the value of the buffer area, B min represents the minimum threshold, which is set to 10 m, B max represents the maximum threshold, a represents a length proportion factor constant, and L represents the length of the bridge.

[0049] In a possible design, the preset prompt word is determined in the following manner:

[0050] A state determination instruction is determined: the model is limited to output a one-in-four state determination result, which is damage, intact, uncertain or non-existent;

[0051] A feature description instruction is determined, and the feature description instruction includes:

[0052] When and only when the determined state is damage, the model is required to list specific damage features, and the damage features include one or a combination of structural fracture, bridge deck collapse, pier displacement and building material scattering;

[0053] Rust, vegetation coverage, shadow shielding and image blur are excluded as damage determination basis;

[0054] If any bridge has the damage features defined above, it is determined that the overall state of the bridge is damage;

[0055] An output format instruction is determined: the model is required to output a JSON format conforming to a preset key-value pair, including a bridge state field storing the state determination result and a damage feature field storing a damage feature list.

[0056] In a second aspect, the present application provides a zero-sample visual large model damage bridge rapid extraction device in an emergency scene, and the device includes:

[0057] a road data fusion module configured to acquire road data and auxiliary data, extract bridge linear features from the road data, merge two bridge linear features representing a dual-lane into a single-lane feature to obtain bridge linear data, and perform spatial position correction and attribute fusion on the geographic spatial location information of the bridge linear data using place name and address data in the auxiliary data to obtain a disaster bridge linear database;

[0058] a bridge image extraction module configured to extract bridge images based on the disaster bridge linear database;

[0059] The damaged bridge extraction module is configured to input the bridge image and preset prompt words into the large model, output the bridge status and bridge characteristics, determine the damage of the bridge based on the output bridge status, associate the disaster-time bridge linear database to match the corresponding bridge geographic information, and obtain the geographic information of the damaged bridge.

[0060] In a third aspect, an embodiment of the present application provides an electronic device comprising: at least one processor and a memory; the memory stores computer-executable instructions; the at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the method for rapidly extracting damaged bridges from a large zero-sample visual model in an emergency scenario as described in the first aspect and various possible designs of the first aspect.

[0061] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores computer-executable instructions. When a processor executes the computer-executable instructions, it implements the method for quickly extracting damaged bridges using a zero-sample visual large model in an emergency scenario as described in the first aspect and various possible designs of the first aspect.

[0062] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program. When the computer program is executed by a processor, it implements the method for quickly extracting damaged bridges using a zero-sample visual large model in an emergency scenario as described in the first aspect and various possible designs of the first aspect.

[0063] The method for rapidly extracting damaged bridges using a large zero-sample visual model in emergency scenarios provided by this application has at least the following beneficial effects:

[0064] This application effectively improves the integrity and consistency of bridge basic data through the spatial-attribute fusion of multi-source road data and place name and address auxiliary data, as well as the merging of two-lane linear elements, laying a precise vector benchmark for bridge image extraction during disasters; based on the dynamic buffer strategy of bridge vectors, bridge images are extracted in a targeted manner, reducing background interference and optimizing computing resource investment; the zero-sample visual large model is combined with preset prompt words to break through the dependence on training data and quickly adapt to emergency scenarios. At the same time, through bridge status judgment and feature constraints, the accuracy and output standardization of bridge damage identification are improved; ultimately, the rapid association and accurate acquisition of geographic information of damaged bridges in emergency scenarios are achieved, which significantly shortens the disaster damage assessment cycle and provides efficient data support for emergency rescue decisions. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0066] Figure 1 The process of a method for quickly extracting damaged bridges using a zero-sample visual large model in an emergency scenario provided by the embodiment of this application Figure 1 ;

[0067] Figure 2 The process of a method for quickly extracting damaged bridges using a zero-sample visual large model in an emergency scenario provided by the embodiment of this application Figure 2 ;

[0068] Figure 3 An overview of the study area provided in an embodiment of the present application, wherein (a) is the location of the epicenter of the study area and the extracted road database, and (b) is the location of the study area;

[0069] Figure 4 Schematic diagram of damaged bridges extracted using the zero-shot visual large model method for rapid extraction of damaged bridges in emergency scenarios provided by an embodiment of the present application; (a)-(f) represent different bridges respectively;

[0070] Figure 5 This is a structural diagram of the device for quickly extracting damaged bridges using a zero-sample visual large model in emergency scenarios provided in an embodiment of the present application.

[0071] The above drawings illustrate specific embodiments of the present application, which will be described in more detail below. These drawings and the textual description are not intended to limit the scope of the present application in any way, but rather to illustrate the concepts of the present application to those skilled in the art by reference to specific embodiments. DETAILED DESCRIPTION

[0072] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

[0073] In the technical solution of this application, the collection, storage, use, processing, transmission, provision and disclosure of information such as financial data or user data involved comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0074] It should be noted that in the embodiments of the present application, certain software, components, models and other existing solutions in the industry may be mentioned. They should be regarded as exemplary. Their purpose is only to illustrate the feasibility of implementing the technical solution of the present application, but it does not mean that the applicant has or will necessarily use the solution.

[0075] The following specific embodiments describe in detail the technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0076] The embodiment of the present application provides a method for quickly extracting damaged bridges with a zero-sample visual large model in an emergency scenario. Figure 1 and Figure 2 As shown in the figure, the process of the method for quickly extracting damaged bridges using a zero-sample visual large model in an emergency scenario provided by the embodiment of the present application is as follows: Figure 1 and two In this example, the study area is 30 km around the epicenter of a magnitude 7.9 earthquake in a certain area in 2025. Figure 3 As shown, the method for rapidly extracting damaged bridges using a zero-sample visual large model in an emergency scenario includes the following steps S100-S300.

[0077] S100: Obtain road data and auxiliary data, extract bridge linear features from the road data, merge the two bridge linear feature lines representing the two-lane road into a single-lane feature to obtain bridge linear data, and use the place name and address data in the auxiliary data to perform spatial position correction and attribute fusion on the geographic spatial location information of the bridge linear data to obtain a disaster bridge linear database.

[0078] In this example, OpenStreetMap linear road data, Geographic Names Server (GNS) data, and GeoNames data were collected within the study area to generate linear bridge geographic information data and point-based auxiliary bridge geographic information data. The OSM linear road data and auxiliary data were then integrated to create a disaster linear road database. The OpenStreetMap linear road data served as the road data, while the GNS and GeoNames data served as the auxiliary data.

[0079] In some embodiments, as Figure 2 As shown, in step S100, disaster road data construction can be achieved in the following manner: relying on open source datasets (OSM, GNS, GeoNames), first extract bridge-related data through the name.bridge field; filter linear bridges in the OSM data, merge two-way bridge elements, calculate the spatial / name similarity between GNS / GeoNames auxiliary data and OSM data, and finally build a linear road database to achieve disaster positioning.

[0080] In some embodiments, bridge linear features are extracted based on the bridge attribute field in the OSM road vector line feature data, and non-linear bridge features such as viaducts are excluded by calculating the topological straightness of the features. The specific formula is as follows:

[0081]

[0082] Where ρ represents the topology of the bridge element, P end and P start Represent the coordinates of the endpoints of the line segment, ||P end -P start || indicates P end and P start The distance, L total Indicates the actual length of the line feature.

[0083] Set the threshold θ ρ =0.95, when ρ>θ ρ The feature is determined to be a straight bridge.

[0084] The repeated features representing dual lanes are merged. The specific process is to first calculate the midpoint of the bridge, convert the bridge linear features into point features, and search for point features 20 meters around each point to establish a neighboring pair, assuming that it represents a dual lane. The calculation formula is as follows:

[0085]

[0086] In the formula, M represents the midpoint set, avg represents the average value of the point set, and P kIt represents the kth point set that constitutes a linear feature, n represents the number of point sets, and k represents the sequence number of the point set.

[0087] Based on the midpoint set of the bridge linear feature, all adjacent pair sets are determined using the following formula:

[0088] Γ={(i,j)||M i -M j ||≤l}

[0089] In the formula, Γ represents the set of all adjacent pairs, M i Indicates the midpoint of the i-th bridge linear element, M j represents the midpoint of the jth bridge linear element, l represents the proximity distance threshold, which is 20m in this embodiment, i, j represents the serial number of the bridge linear element, || M i -M j || indicates M i and M j distance.

[0090] By matching the center point feature with the corresponding bridge linear feature, the bridge linear proximity pair can be restored. By determining whether the bridge linear features included in the linear proximity pair are parallel, it can be determined whether the linear proximity pair represents a two-lane road. The formula for determining parallelism is as follows:

[0091]

[0092] Where, Represents the direction vector of the i-th bridge linear feature, and Represents the x-coordinate and y-coordinate of the end point of the i-th bridge linear feature, Represents the x-coordinate and y-coordinate of the starting point of the i-th bridge linear feature.

[0093]

[0094] Where α represents the angle between adjacent linear pairs, Represents the direction vector of the j-th bridge linear feature, and Respectively and length, express and The absolute value of the dot product.

[0095] When α<1° or α>179°, the two straight lines (two bridge linear elements) are considered parallel.

[0096] The original linear proximity pairs were removed, and the center lines of the linear proximity pairs were used to represent the original single lane features. Based on the GeoNames and GNS data name fields, the bridge linear data name attributes containing the bridge field were separated as auxiliary bridge data. The spatial similarity and name similarity between the auxiliary data and the OSM linear data were calculated, and the corresponding road data were matched. The spatial similarity calculation formula is as follows:

[0097]

[0098] Where, d spatial It represents the spatial similarity between the auxiliary data and the bridge linear data, min represents the minimum value, L1 represents the bridge linear feature in the bridge linear data, P represents the coordinates of a point in the point set that constitutes the bridge linear feature, and Q represents the coordinates of the point feature in the auxiliary data.

[0099] The formula for name similarity is as follows:

[0100]

[0101] Where s name Indicates the similarity between the name of the auxiliary data and the bridge linear data, name point Indicates the name field attribute of the point feature, name line Represents the name field attribute of the linear feature, max indicates the maximum value, and ED indicates the edit distance between the auxiliary data name and the road data name.

[0102] Based on the spatial similarity and name similarity, the matching score is calculated using the following formula:

[0103] match score =λ·d spatial +(1-λ)·s name

[0104] In the formula, match score represents the matching score, and λ represents the weight of spatial similarity, which is 0.5 in this embodiment.

[0105] In this embodiment, if match score If the value is greater than 0.6, the two bridges are considered to represent the same bridge. The midpoint coordinates of the bridge linear feature and point feature representing the same bridge are used as the reference for the final linear bridge geospatial location information. Using the place name and address data from the auxiliary data, the OSM linear data is spatially corrected and attribute fused to obtain the disaster bridge linear database.

[0106] Using the above process to fuse OSM, GNS, and GeoNames data, a total of 733 linear bridge features in the study area were obtained. In order to beautify the mapping effect, the linear features were converted into point features for mapping. The results are as follows Figure 3 As shown in (a).

[0107] S200: Extract bridge images based on the disaster bridge linear database.

[0108] In this embodiment, a bounding rectangle is extracted based on the vector information of the linear bridge obtained in step S100, and a corresponding buffer zone is dynamically expanded outward according to the size of the bridge. The buffer zone is used to perform mask processing on the post-disaster orthophoto to extract the bridge image.

[0109] In some embodiments, as Figure 2 As shown, in step S200 , image extraction can be achieved in the following manner: based on the linear road database, the bridge image is extracted from the post-disaster orthophoto using the image extraction strategy of the bridge dynamic buffer zone.

[0110] In some embodiments, a bounding rectangle is extracted based on the element vector information, and a corresponding buffer zone is dynamically expanded outward according to the size of the bridge. The buffer zone is used to mask the post-disaster satellite image. The specific calculation formula is:

[0111]

[0112] Where B represents the value of the buffer, B min Indicates the minimum threshold, which can be set to 10m, B max It represents the highest threshold value, which can be set to 100m, α represents a constant length scaling factor, which can be set to 0.1, and L represents the length of the bridge.

[0113] For kilometer-long bridges (greater than 1000m in length), images are segmented along the bridge vector into multiple 100-meter segments to enable model-based inspection of structural component details. Damage assessment integrates the local conditions of all subdivided areas to derive a diagnosis of the bridge's overall structural integrity.

[0114] S300: Input the bridge image and preset prompt words into the large model, output the bridge status and bridge characteristics, determine the damage of the bridge based on the output bridge status, associate the disaster bridge linear database to match the corresponding bridge geographic information, and obtain the geographic information of the damaged bridge.

[0115] In this embodiment, the extracted bridge image and the designed standardized prompt words are input into the Qwen2.5-VL large model, which outputs the bridge's status and characteristics. Based on the output bridge status, the bridge damage is determined. The road database obtained in step S100 is then linked to the corresponding bridge geographic information to obtain the geographic information of the damaged bridge. It should be noted that the Qwen2.5-VL large model is merely an example of a large model that can be used in this application and does not constitute a limitation of this application.

[0116] In some embodiments, as Figure 2 As shown, step S200 obtains the geographic information of damaged bridges by generating standardized prompts, such as "As a remote sensing image analysis expert, please strictly adhere to the following bridge condition assessment requirements:...", as rule constraints for model inference. The bridge image and the standardized prompts are input into the visual encoder of the Qwen2.5-VL large model. The output of the visual encoder is parsed by the Qwen2.5-LM decoder to generate bridge status and features. Based on the model results, the geographic location of the damaged bridge is output, completing the damage assessment.

[0117] The extracted bridge images and the designed standardized prompt words are input into the Qwen2.5-VL large model. The method for designing the standardized prompt words is as follows:

[0118] (a) Status determination instruction: The model is limited to outputting only one of four status determination results, namely [damaged], [intact], [uncertain], or [non-existent];

[0119] (b) Feature description instructions: (1) When and only when the state is determined to be [damaged], the model must list specific damage features (visual features) in English. The damage features are limited to: structural fracture, bridge deck collapse, pier displacement, and scattered building materials; (2) Rust, vegetation cover, shadow occlusion, and image blur are clearly excluded as damage determination criteria; (3) Define the judgment rules for multiple bridge scenarios: If any bridge has the damage features defined in (b)(1) above, the overall state is determined to be [damaged];

[0120] (c) Output format directive: forces the model to output a strict JSON format that conforms to the preset key-value pairs, including a "bridge status" field to store the status determination result, and a "damage feature" field to store the damage feature list (an empty list if the status is not [damage]).

[0121] Based on the above method, the model prompt words used in the present invention are:

[0122] As a remote sensing image analysis expert, please strictly follow the following bridge condition assessment requirements:

[0123] Bridge status is determined using only four predefined options: [Damaged / Not Damaged / Uncertain / Not Present]

[0124] When identifying damage, you must use Chinese descriptors to clearly identify specific features, in the format ["Feature 1","..."...]

[0125] The output must be in strict JSON format: {"Bridge ID":"[]","Bridge Status":"[Damaged / Not Damaged / Uncertain / Not Existing]","Damage Characteristics":[]}

[0126] Evaluation criteria:

[0127] Effective damage indicators: structural cracks, bridge deck collapse, pier displacement, and building material debris (damage to a single span of a multi-span bridge is considered "damaged");

[0128] Non-damage indicators: Image features such as cover / shadow, rust / vegetation shall not be used as evidence of damage.

[0129] Output the status and characteristics of the bridge, determine the damage of the bridge based on the output bridge status, use the bridge ID attribute in the output format to associate the road database obtained in step 1 with the corresponding bridge geographic information, and thus obtain the geographic information of the damaged bridge. The damaged bridge is finally extracted as follows: Figure 4 shown.

[0130] The embodiment of the present application also provides a device for quickly extracting damaged bridges using a zero-sample visual large model in an emergency scenario, such as Figure 5 As shown in FIG, the zero-sample visual large model damaged bridge rapid extraction device in the emergency scenario includes:

[0131] The road data fusion module 501 is configured to obtain road data and auxiliary data, extract bridge linear features from the road data, merge two bridge linear features representing two lanes into a single lane feature to obtain bridge linear data, and use the place name and address data in the auxiliary data to perform spatial position correction and attribute fusion on the geographic spatial location information of the bridge linear data to obtain a disaster bridge linear database;

[0132] The bridge image extraction module 502 is configured to extract bridge images based on the disaster bridge linear database;

[0133] The damaged bridge extraction module 503 is configured to input the bridge image and preset prompt words into the large model, output the bridge status and bridge characteristics, determine the damage of the bridge based on the output bridge status, associate the disaster bridge linear database to match the corresponding bridge geographic information, and obtain the geographic information of the damaged bridge.

[0134] In some embodiments, the road data fusion module is further configured to:

[0135] Based on the bridge attribute field in the OSM road vector line feature data, bridge features are extracted and the topology of the bridge features is calculated using the following formula:

[0136]

[0137] Where ρ represents the topology of the bridge element, P end and P start Represent the coordinates of the endpoints of the line segment, ||P end -P start || means p end and p start The distance, L total Indicates the actual length of the line feature;

[0138] If the topology of the bridge element is greater than a set element threshold, the bridge element is determined to be a bridge linear element.

[0139] In some embodiments, the road data fusion module is further configured to:

[0140] Convert the bridge linear feature into a point feature, and calculate the midpoint set of the bridge linear feature using the following formula:

[0141]

[0142] In the formula, M represents the midpoint set, avg represents the average value of the point set, and P k It represents the kth point set that constitutes a linear feature, n represents the number of point sets, and k represents the sequence number of the point set;

[0143] Based on the midpoint set of the bridge linear feature, all adjacent pair sets are determined using the following formula:

[0144] Γ={(i,j)||M i -M j ||≤l}

[0145] In the formula, Γ represents the set of all adjacent pairs, M i Indicates the midpoint of the i-th bridge linear element, M j represents the midpoint of the jth bridge linear feature, l represents the proximity distance threshold, i, j represents the serial number of the bridge linear feature, ||M i -M j || indicates M i and M j distance;

[0146] Determining whether linear adjacent pairs in the adjacent pair set are parallel to determine whether the linear adjacent pairs represent a dual lane;

[0147] When it is determined that the linear proximity pair represents a dual lane, the two bridge linear features in the linear proximity pair are merged into a single lane feature, and the center line of the linear proximity pair is used to represent the single lane feature;

[0148] The unmerged bridge linear features and single lane features are combined to form the bridge linear data.

[0149] In some embodiments, the road data fusion module is further configured to determine whether the linear adjacent pairs in the adjacent pair set are parallel by:

[0150] The direction vector of the bridge linear feature is calculated using the following formula:

[0151]

[0152] Where, Represents the direction vector of the i-th bridge linear feature, and Represents the x-coordinate and y-coordinate of the end point of the i-th bridge linear feature, Represents the x-coordinate and y-coordinate of the starting point of the i-th bridge linear feature;

[0153] According to the direction vector of the bridge linear element, the angle between adjacent linear pairs is calculated using the following formula:

[0154]

[0155] Where α represents the angle between adjacent linear pairs, Represents the direction vector of the j-th bridge linear feature, and Respectively and length, express and The absolute value of the dot product;

[0156] When α<1° or α>179°, the two bridge linear features in the adjacent linear pair are judged to be parallel.

[0157] In some embodiments, the road data fusion module is further configured to:

[0158] The spatial similarity between the auxiliary data and the bridge linear data is calculated using the following formula:

[0159]

[0160] wherein d spatial represents the spatial similarity of the auxiliary data and the bridge linear data, min represents taking the minimum value, L1 represents a bridge linear element in the bridge linear data, P represents the coordinates of a certain point in the point set constituting the bridge linear element, and Q represents the coordinates of a point element in the auxiliary data;

[0161] The name similarity of the auxiliary data and the bridge linear data is calculated by the following formula:

[0162]

[0163] wherein s name represents the name similarity of the auxiliary data and the bridge linear data, name point represents the name field attribute of the point element, name line represents the name field attribute of the linear element, max represents taking the maximum value, and ED represents the edit distance of the auxiliary data name and the road data name;

[0164] Based on the spatial similarity and the name similarity, a matching score is calculated by the following formula:

[0165] match score = λ · d spatial + (1- λ) · s name

[0166] wherein match score represents the matching score, and λ represents the weight of the spatial similarity;

[0167] Based on a set score threshold, if the matching score is greater than the score threshold, it is determined that the bridge linear element and the point element belong to the same bridge;

[0168] The midpoint coordinates of the bridge linear element and the point element representing the same bridge are taken as the reference to obtain the geographic spatial position information of the final linear bridge, and further to obtain the disaster-time bridge linear database.

[0169] In some embodiments, the bridge image extraction module is further configured to:

[0170] An external rectangle is extracted according to the element vector information, a corresponding buffer zone is dynamically expanded outward according to the size of the bridge, and the buffer zone is used to mask the post-disaster satellite image; wherein the calculation formula of the buffer zone is:

[0171]

[0172] wherein B represents the value of the buffer zone, B min represents the minimum threshold value, which is set to 10 m, and B maxrepresents the highest threshold, α represents a constant length scaling factor, and L represents the length of the bridge.

[0173] In some embodiments, the damaged bridge extraction module is further configured to determine the preset prompt word by:

[0174] Determine the state judgment instruction: limit the model to output only one of four state judgment results: damaged, intact, uncertain or non-existent;

[0175] Determine a feature description instruction, wherein the feature description instruction includes:

[0176] If and only if the state is determined to be damaged, the model is required to list specific damage characteristics, including one or a combination of structural fracture, bridge deck collapse, pier displacement, and scattered building materials;

[0177] Exclude rust, vegetation cover, shadows and image blur as the basis for damage determination;

[0178] If any bridge has the damage characteristics defined above, the overall status of the bridge is determined to be damaged;

[0179] Determine the output format instruction: Require the model to output a JSON format that conforms to the preset key-value pairs, including a bridge status field to store the status determination result, and a damage feature field to store a list of damage features.

[0180] An embodiment of the present application provides an electronic device, which may include a processor and a memory, wherein the processor and the memory can communicate with each other; illustratively, the processor and the memory communicate with each other via a communication bus.

[0181] The processor executes the computer-executable instructions stored in the memory, so that the processor implements the solutions in the above embodiments. The processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0182] The communication bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. System buses can be categorized as address buses, data buses, and control buses. Transceivers enable communication between the database access device and other computers (e.g., clients, read-write libraries, and read-only libraries). Memory may include random access memory (RAM) or non-volatile memory.

[0183] The electronic device provided in the embodiment of the present application may be the terminal device of the above embodiment.

[0184] An embodiment of the present application also provides a computer-readable storage medium, which stores computer instructions. When the computer instructions are executed on a computer, the computer executes the technical solution of the method for quickly extracting damaged bridges with a zero-sample visual large model in an emergency scenario in the above-mentioned embodiment.

[0185] An embodiment of the present application also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. At least one processor can read the computer program from the computer-readable storage medium. When at least one processor executes the computer program, it can implement the technical solution of the method for quickly extracting damaged bridges with a zero-sample visual large model in an emergency scenario in the above-mentioned embodiment.

[0186] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of modules is only a logical function division. In actual implementation, there may be other division methods, such as multiple modules can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or module, which can be electrical, mechanical or other forms.

[0187] Modules described as separate components may or may not be physically separate, and components shown as modules may or may not be physical units, that is, they may be located in one place or distributed across multiple network elements. Some or all of these modules may be selected to implement the solution of this embodiment based on actual needs.

[0188] In addition, the functional modules in the various embodiments of the present application may be integrated into a single processing unit, or each module may exist physically separately, or two or more modules may be integrated into a single unit. The above-mentioned modules may be implemented in the form of hardware or hardware plus software functional units.

[0189] The above-mentioned integrated module implemented in the form of a software functional module can be stored in a computer-readable storage medium. The above-mentioned software functional module is stored in a storage medium and includes a number of instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) or a processor to perform some steps of the methods of various embodiments of the present application.

[0190] It should be understood that the processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), or application-specific integrated circuits (ASICs). A general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the present invention may be directly implemented by a hardware processor or implemented by a combination of hardware and software modules in the processor.

[0191] The memory may include a high-speed RAM memory, and may also include non-volatile storage NVM, such as at least one disk memory, and may also be a USB flash drive, a mobile hard disk, a read-only memory, a magnetic disk or an optical disk.

[0192] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus. Buses can be divided into address buses, data buses, and control buses.

[0193] The storage medium may be implemented by any type of volatile or non-volatile memory device, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The storage medium may be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0194] An exemplary storage medium is coupled to a processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be an integral part of the processor. The processor and the storage medium can be located in an application specific integrated circuit (ASIC). Of course, the processor and the storage medium can also exist as discrete components in an electronic control unit or a main control device.

[0195] Those skilled in the art will appreciate that all or part of the steps in the above-described method embodiments can be implemented using hardware associated with program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0196] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some or all of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for rapidly extracting damaged bridges using a zero-shot visual large model in emergency scenarios, characterized by: The method comprises: Acquire road data and auxiliary data, extract bridge linear features from the road data, merge two bridge linear features representing dual lanes into a single lane feature to obtain bridge linear data, and use place name and address data in the auxiliary data to perform spatial position correction and attribute fusion on the geographic spatial location information of the bridge linear data to obtain a disaster bridge linear database; Extracting bridge images based on the disaster bridge linear database; The bridge image and preset prompt words are input into the large model, the bridge status and bridge characteristics are output, the damage of the bridge is determined based on the output bridge status, the disaster bridge linear database is associated with the corresponding bridge geographic information, and the geographic information of the damaged bridge is obtained.

2. The method for rapid extraction of damaged bridges using a zero-shot visual large model in emergency scenarios according to claim 1 is characterized in that: The road data includes OSM road vector line feature data, and extracting bridge linear features from the road data includes: Based on the bridge attribute field in the OSM road vector line feature data, bridge features are extracted and the topology of the bridge features is calculated using the following formula: Where ρ represents the topology of the bridge element, P end and P start Represent the coordinates of the endpoints of the line segment, ||P end -P start || indicates P end and P start The distance, L total Indicates the actual length of the line feature; If the topology of the bridge element is greater than a set element threshold, the bridge element is determined to be a bridge linear element.

3. The method for rapid extraction of damaged bridges using a zero-shot visual large model in emergency scenarios according to claim 2 is characterized in that: Merge the two bridge linear features representing two lanes into a single lane feature to obtain the bridge linear data, including: Convert the bridge linear feature into a point feature, and calculate the midpoint set of the bridge linear feature using the following formula: In the formula, M represents the midpoint set, avg represents the average value of the point set, and P k It represents the kth point set that constitutes a linear feature, n represents the number of point sets, and k represents the sequence number of the point set; Based on the midpoint set of the bridge linear feature, all adjacent pair sets are determined using the following formula: Γ={(i,j)∣||M i -M j ||≤l} In the formula, Γ represents the set of all adjacent pairs, M i Indicates the midpoint of the i-th bridge linear element, M j represents the midpoint of the jth bridge linear feature, l represents the proximity distance threshold, i, j represents the serial number of the bridge linear feature, ||M i -M j || indicates M i and M j distance; Determining whether linear adjacent pairs in the adjacent pair set are parallel to determine whether the linear adjacent pairs represent a dual lane; When it is determined that the linear proximity pair represents a dual lane, the two bridge linear features in the linear proximity pair are merged into a single lane feature, and the center line of the linear proximity pair is used to represent the single lane feature; The unmerged bridge linear features and single lane features are combined to form the bridge linear data.

4. The method for rapid extraction of damaged bridges using a zero-shot visual large model in emergency scenarios according to claim 3 is characterized in that: Whether the linear adjacent pairs in the adjacent pair set are parallel is determined by: The direction vector of the bridge linear feature is calculated using the following formula: Where, Represents the direction vector of the i-th bridge linear feature, and Represents the x-coordinate and y-coordinate of the end point of the i-th bridge linear feature, Represents the x-coordinate and y-coordinate of the starting point of the i-th bridge linear feature; According to the direction vector of the bridge linear element, the angle between adjacent linear pairs is calculated using the following formula: Where α represents the angle between adjacent linear pairs, Represents the direction vector of the j-th bridge linear feature, and Respectively and length, express and The absolute value of the dot product; When α<1° or α>179°, the two bridge linear features in the adjacent linear pair are judged to be parallel.

5. The method for rapid extraction of damaged bridges using a zero-shot visual large model in emergency scenarios according to claim 1 is characterized in that: Using the place name and address data in the auxiliary data to perform spatial position correction and attribute fusion on the geographic spatial position information of the bridge linear data, a disaster bridge linear database is obtained, including: The spatial similarity between the auxiliary data and the bridge linear data is calculated using the following formula: Where, d spatial Indicates the spatial similarity between the auxiliary data and the bridge linear data, min represents the minimum value, L1 represents the bridge linear feature in the bridge linear data, P represents the coordinates of a point in the point set that constitutes the bridge linear feature, and Q represents the coordinates of the point feature in the auxiliary data; The similarity between the names of the auxiliary data and the bridge linear data is calculated using the following formula: Where s name Indicates the similarity between the name of the auxiliary data and the bridge linear data, name point Indicates the name field attribute of the point feature, name line Represents the name field attribute of the linear feature, max indicates the maximum value, and ED indicates the edit distance between the auxiliary data name and the road data name; Based on the spatial similarity and name similarity, the matching score is calculated using the following formula: match score =λ·d spatial +(1-λ)·s name In the formula, match score represents the matching score, and λ represents the weight of spatial similarity; Based on a set score threshold, if the matching score is greater than the score threshold, it is determined that the bridge linear element and the point element belong to the same bridge; The midpoint coordinates of the bridge linear elements and point elements representing the same bridge are used as the benchmark and the geographic spatial location information of the final linear bridge, thereby obtaining the disaster bridge linear database.

6. The method for rapid extraction of damaged bridges using a zero-shot visual large model in emergency scenarios according to claim 1 is characterized in that: Based on the disaster bridge linear database, bridge images are extracted, including: The bounding rectangle is extracted based on the feature vector information, and the corresponding buffer zone is dynamically expanded according to the size of the bridge. The buffer zone is used to mask the post-disaster satellite image. The calculation formula of the buffer zone is: Where B represents the value of the buffer, B min Indicates the lowest threshold, set to 10m, B max represents the highest threshold, α represents a constant length scaling factor, and L represents the length of the bridge.

7. The method for rapidly extracting damaged bridges using a large zero-sample visual model in an emergency scenario according to any one of claims 1 to 6, characterized in that: Determine the preset prompt words in the following ways: Determine the state judgment instruction: limit the model to output only one of four state judgment results: damaged, intact, uncertain or non-existent; Determine a feature description instruction, wherein the feature description instruction includes: If and only if the state is determined to be damaged, the model is required to list specific damage characteristics, including one or a combination of structural fracture, bridge deck collapse, pier displacement, and scattered building materials; Exclude rust, vegetation cover, shadows and image blur as the basis for damage determination; If any bridge has the damage characteristics defined above, the overall status of the bridge is determined to be damaged; Determine the output format instruction: Require the model to output a JSON format that conforms to the preset key-value pairs, including a bridge status field to store the status determination result, and a damage feature field to store a list of damage features.

8. A device for quickly extracting damaged bridges using a zero-sample visual large model in emergency scenarios, characterized by: The device comprises: a road data fusion module configured to acquire road data and auxiliary data, extract bridge linear features from the road data, merge two bridge linear features representing a dual-lane into a single-lane feature to obtain bridge linear data, and perform spatial position correction and attribute fusion on the geographic spatial location information of the bridge linear data using place name and address data in the auxiliary data to obtain a disaster bridge linear database; a bridge image extraction module configured to extract bridge images based on the disaster bridge linear database; The damaged bridge extraction module is configured to input the bridge image and preset prompt words into the large model, output the bridge status and bridge characteristics, determine the damage of the bridge based on the output bridge status, associate the disaster-time bridge linear database to match the corresponding bridge geographic information, and obtain the geographic information of the damaged bridge.

9. An electronic device, characterized in that: include: a processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the zero-sample visual large-model damaged bridge rapid extraction method in an emergency scenario according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method for rapidly extracting damaged bridges using a zero-sample visual large model in an emergency scenario according to any one of claims 1 to 7.

Images