Bridge health diagnosis method and device based on dynamic lightweight visual language model

Abstract

The invention provides a bridge health diagnosis method and device based on a dynamic lightweight visual language model, and the method comprises the steps: obtaining an original bridge image with super-definition resolution, and executing overlapped image partitioning; carrying out coding and cosine similarity calculation on the original image, generating differential scores, and implementing TopK screening to obtain a dynamic multi-scale feature candidate set; fusing the multi-scale feature candidate set, solving an optimal single-scale feature, and outputting a preferential semantic feature vector; receiving a natural language text cue word, inputting the preferential semantic feature vector into the nonlinear projection network, aligning the text cue word with the visual feature, and outputting a multi-dimensional feature vector; and packaging the generated content according to a standard protocol, and generating a diagnosis report of the bridge health state. According to the method, aiming at the structural damage detection requirement of a high-resolution image in a bridge engineering scene, the accuracy and reliability of bridge health diagnosis are remarkably improved through cross-modal depth alignment of visual features and text semantics.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of bridge health diagnosis, and more particularly to a bridge health diagnosis method and device based on a dynamic lightweight visual language model. BACKGROUND

[0002] Bridge health diagnosis, as a core technology for safe operation of infrastructure, its core task is to achieve early detection and accurate assessment of structural damage such as cracks and corrosion. The traditional operation method generally follows the following process: first, the bridge structure surface 4096x2160 pixel RGB image or three-dimensional point cloud data is obtained by high-resolution industrial camera or unmanned aerial vehicle aerial photography, then artificial method or Otsu algorithm or Canny edge detector based on fixed threshold is used for preliminary feature extraction, the original image is converted into binary representation containing damage features such as cracks and corrosion, and finally these features are input into the preset rule health assessment model for damage level determination.

[0003] However, in engineering practice, there are serious technical bottlenecks: first, when facing 4096x2160 high-resolution engineering images, the existing language model will reduce the dimension embedding space (such as 50x768), and then use the Q-Former fixed query mechanism (usually 50 query vectors), thereby forming double compression. Q-Former has a fixed query capacity limit in engineering image processing, which further leads to the excessive loss of fine-grained structural information in high-resolution images, making it easy to ignore the crack area and other image blocks.

[0004] Secondly, the mainstream token compression method (such as the cluster merging of LLaVA-PruMerge and the similarity threshold screening of TextMonkey) relies on artificial setting of merging rules, and presents significant performance fluctuations (±6.3%) in cross-task multi-modal evaluation. Especially for bridge component identification tasks, the fixed similarity threshold leads to a high deletion rate of 18.7% in the reinforcement connection area token, making it difficult to adaptively adjust the compression granularity. The recognition error of traditional methods (such as fixed threshold segmentation) for sub-millimeter cracks is as high as ±0.2mm, and is easily disturbed by light and shadow. Structural damage such as bolt loosening and steel displacement is missed at low resolution.

[0005] Finally, existing lightweight models generally use static computation graphs, which have fixed computation budgets and cannot adapt to the multi-scale characteristics of engineering images. For example, Xmodel-VLM uniformly allocates 50 ms of computation time to each image, resulting in insufficient processing of high-resolution regions (such as bolt arrays) and a 14.8% increase in the missed detection rate. Moreover, traditional methods lack adaptive mechanisms and cannot dynamically adjust the model depth / width according to the image content, resulting in high memory usage of 6.4 GB (measured on a Jetson AGX Xavier) for 4K image processing, and the domain representation unit is incorrectly merged. SUMMARY

[0006] The present application aims to solve the technical problems existing in the prior art and provides a bridge health diagnosis method and device based on a dynamic lightweight visual language model. For the structural damage detection requirements of high-resolution images in the bridge engineering scene, the present application significantly improves the accuracy and reliability of bridge health diagnosis through cross-modal deep alignment of visual features and text semantics.

[0007] According to a first aspect of the present application, a bridge health diagnosis method based on a dynamic lightweight visual language model is provided, comprising the following steps: Obtain a super-resolution bridge original image, and perform overlapping image partition processing on the original image; Encode the original image after partition processing, calculate the cosine similarity, generate a differentiated score, and implement TopK filtering to obtain a dynamic multi-scale feature candidate set; Fuse the multi-scale feature candidate set, balance the details and global information, find the best single-scale feature, and output the optimized semantic feature vector; Receive a natural language text prompt word, complete text encoding, input the optimized semantic feature vector into a nonlinear projection network, align the text prompt word with the visual feature, and output a multi-dimensional feature vector; Encapsulate the generated content according to a standard protocol to generate a bridge health status diagnosis report.

[0008] Based on the above technical solution, the present application can also be improved as follows.

[0009] Optionally, the obtaining of the super-resolution bridge original image and the performing of the overlapping image partition processing on the original image comprise: Collect the super-pixel bridge original image through a drone or a fixed monitoring device, divide the super-pixel bridge image into a globally down-sampled image and a plurality of overlapping local image blocks through a sliding window mechanism, maintain geometric consistency by using bilinear interpolation, and output an image block sequence with spatial encoding.

[0010] Optionally, the process of encoding the original image after partitioning, calculating cosine similarity, generating a differential score, and performing TopK filtering to obtain a dynamic multi-scale feature candidate set includes: Local features of each image patch are extracted using a ViT-B / 16 encoder with shared weights; Calculate the cosine similarity matrix between representation units within the block, and dynamically select the most critical representation units for the bridge detection task from high-density visual features; Finally, the multi-scale feature candidate set is output.

[0011] Optionally, the most critical representation units for bridge detection are dynamically selected from high-density visual features, as expressed by the formula:

[0012] In the formula, DS[s]( ( ) represents the feature matrix generated through dynamic sampling at resolution scale s. Let be the visual feature sequence of the i-th image patch, and D be the importance score of the representation unit. The number of representation units reserved for the s-th scale ∈ .

[0013] Optionally, the fusion of multi-scale feature candidate sets, preserving a balance between detail and global information, finding the optimal single-scale feature, and outputting the optimized semantic feature vector includes: During the training phase, weights are calculated based on the features at each scale of the candidate set and the semantic relevance of the whole-image encoding; then the obtained multi-scale features are fused to preserve the balance between details and global information. Design a balanced loss function to constrain the multi-scale selection distribution; During the inference phase, the single-scale feature with the highest probability is selected, the index of the optimal scale is selected, and the optimized semantic feature vector is output.

[0014] Optionally, the balance loss function prevents the dynamic scale selector from getting trapped in local optima during training by using probability distribution constraints, always selecting a fixed scale ratio; the loss function is expressed as follows:

[0015] in, To select the balanced loss for multi-scale selection, α is the weight coefficient of the balanced loss term, n is the number of candidate sets, and N is the total number of samples in the current training batch. The frequency at which scale i is selected. When processing the j-th sample, this sample belongs to The sampling probability; this loss function ensures the stability of feature ratio selection at different scales.

[0016] Optionally, the natural language text prompts include: semantic text that guides the health monitoring process, wherein the semantic text is used to combine the semantic guidance of the natural language prompts with visual focusing through dynamic multi-scale feature compression, i.e., text-image feature fusion / alignment; the nonlinear projection network is constructed using the Mish activation function.

[0017] Optionally, the optimized semantic feature vector is input into a nonlinear projection network to align text prompts with visual features, and the output multidimensional feature vector includes: It accepts natural language text prompts and aligns health monitoring guidance text with visual features. Through a multi-layer Mish-activated MLP network, the bridge semantic feature vector is projected onto the language model space to complete the cross-modal alignment of visual features and text instructions. The system utilizes a knowledge graph to retrieve and dynamically expand knowledge related to defects, ultimately automatically generating a structured JSON report containing defect types, location coordinates, and repair plans according to engineering specification templates.

[0018] Optionally, the step of encapsulating the generated content according to a standard protocol to generate a diagnostic report on the bridge's health status includes: Based on cross-modal feature parallel generation, a structured knowledge graph containing defect reports, component status descriptions, and maintenance suggestions is generated. The output is encapsulated through standard protocols, forming an end-to-end processing pipeline from feature input to report output. When deployed on edge devices, memory optimization strategies ensure real-time requirements.

[0019] According to a second aspect of the present invention, a bridge health diagnosis device based on a dynamic lightweight visual language model is provided, comprising: A visual encoder is used to acquire the original image of the bridge in ultra-high resolution, perform overlapping image partitioning on the original image, and encode the original image after partitioning, calculate cosine similarity, generate a differential score and implement TopK screening to obtain a dynamic multi-scale feature candidate set. The multi-scale downsampling layer is used to fuse multi-scale feature candidate sets, preserve the balance between details and global information, find the best single-scale feature, and output the optimized semantic feature vector. A text encoder is used to receive natural language text prompts and complete text encoding. The nonlinear projection layer is used to input the optimized semantic feature vector into the nonlinear projection network, align the text prompt words with the visual features, and output a multidimensional feature vector. A report generator is used to encapsulate the generated content according to standard protocols and generate diagnostic reports on the health status of bridges.

[0020] The technical effects and advantages of this invention are as follows: This invention provides a bridge health diagnosis method and apparatus based on a dynamic lightweight visual language model. It takes an ultra-high-resolution original image of the bridge as input, performs region segmentation and information density assessment on the high-resolution image through a dynamic spatial feature downsampling mechanism, and employs a global semantic-guided multi-scale fusion strategy to constrain local feature selection, achieving synergistic optimization of efficiency and accuracy in multimodal understanding of engineering images. It receives linguistic guidance on the health diagnosis problem description and health status definition standards, and finally outputs a standardized bridge health status diagnosis report. This achieves a balance between detail preservation and computational efficiency in multimodal understanding of engineering images. Attached Figure Description

[0021] Figure 1 A flowchart of a bridge health diagnosis method based on a dynamic lightweight visual language model provided in an embodiment of the present invention; Figure 2 A schematic diagram of the core components for multi-scale feature sampling provided in an embodiment of the present invention; Figure 3 A dynamic selection architecture diagram is provided for embodiments of the present invention; Figure 4 This is a schematic diagram of the nonlinear projection layer for visual-text alignment implemented in this invention. Figure 5 This is an architectural diagram of a bridge health diagnosis device based on a dynamic lightweight visual language model, provided in an embodiment of the present invention. Detailed Implementation

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] Understandably, given the deficiencies in the background technology, this invention proposes a bridge health diagnosis method based on a dynamic lightweight visual language model, specifically as follows: Figure 1 As shown, it includes the following steps: S1: Obtain an ultra-high resolution original image of the bridge, and perform overlapping image partitioning processing on the original image; In this embodiment, a drone or fixed monitoring equipment is used to acquire original bridge images with a resolution of ≥4096×2160 pixels. A sliding window mechanism is used to segment the high-resolution bridge image into a global downsampled image and multiple overlapping local image blocks, and bilinear interpolation is employed to maintain geometric consistency. The sliding window mechanism is a technique that uses a fixed-size window to slide across the image and capture local regions, thus achieving image block processing.

[0024]

[0025] In the formula, Partition(I) represents the partitioning operation on the input image I; I is the original input image matrix. For a globally downsampled image, There are N local image patches.

[0026] Bilinear interpolation is used to maintain geometric consistency, and a sequence of spatially encoded image patches is output, calculated from the coordinate offset:

[0027] In the formula, The output is a scaled image matrix, where x and y are continuous coordinate positions in the target image, and i and j are neighborhood pixel offsets (taken as bilinear interpolation). ), These are the weighting coefficients.

[0028] S2: Encode the original image after partitioning, calculate cosine similarity, generate differential scores, and perform TopK filtering to obtain a dynamic multi-scale feature candidate set; The process of encoding the image after partitioning and calculating cosine similarity includes the following steps: First, local features of each image patch are extracted using a ViT-B / 16 encoder with shared weights; Subsequently, the cosine similarity matrix between representation units within the block is calculated, and expressed as:

[0029] In the formula, To represent the similarity between token j and k, For the j-th representation unit in the i-th block, It is represented as the k-th representation unit in the i-th image block.

[0030] The generation of differential scores and the implementation of TopK screening yield a multi-scale feature candidate set, including: Dynamically select the most critical representation units for bridge detection from high-density visual features:

[0031] In the formula, DS[s]( ( ) represents the feature matrix generated through dynamic sampling at resolution scale s. Let be the visual feature sequence of the i-th image patch, and D be the token importance score. The number of representation units reserved for the s-th scale ∈ .

[0032] Finally, the three-scale feature candidate set {n, n / 4, n / 16} is output. Figure 2 The diagram shows a multi-scale feature sampling component. The lower part of the diagram, x(n), represents the original representation unit. After cosine similarity scoring and TopK sampling, three candidate sets with different sampling rates, {n, n / 4, n / 16}, are obtained.

[0033] S3: Fuse multi-scale feature candidate sets, preserve the balance between details and global information, find the best single-scale feature, and output the optimized semantic feature vector; In this embodiment, the steps are divided into a training phase and an inference phase. In the training phase, features at various scales of the candidate set are used... With whole image encoding The semantic relevance is calculated by weighting:

[0034] in, For global image encoding, For the i-th block at scale s The following features Let s be the fusion weights for scale s.

[0035] Subsequently, the three-scale feature candidate sets are fused to preserve a balance between detail and global information:

[0036] DS[n]( ) is the feature matrix generated by dynamic sampling at resolution scale n.

[0037] Design a loss balance function Constrained multi-scale selection distribution; the core objective of this loss function is to prevent the dynamic scale selector from getting stuck in local optima during training by constraining the probability distribution, and always select a fixed scale ratio. The loss function is as follows:

[0038] in, For multi-scale selection of balance loss, α is the weight coefficient of the balance loss term, that is, the penalty strength for the scale selection frequency. The larger the value of α, the more the model forces the uniform use of different scales; n is the number of candidate sets, n=3, and N is the total number of samples in the current training batch. The frequency at which scale i is selected. When processing the j-th sample, this sample belongs to The sampling probability; this loss function ensures the stability of feature ratio selection at different scales.

[0039] like Figure 3 The diagram shows the training content during the training phase. The input on the left consists of three sampling features of the same region. After intermediate processing, the correlation calculation weights are obtained, and the output on the right is the filtered scale feature vector.

[0040] During the inference phase, the single-scale feature with the highest probability is selected, where arg max( The purpose of selecting an index for the optimal scale is to output the optimized semantic feature vector. : .

[0041] S4: Receive natural language text prompts, complete text encoding, input the optimized semantic feature vector into a nonlinear projection network, align the text prompts with visual features, and output a multi-dimensional feature vector; The natural language text prompts include: input guidance statements for health monitoring. The semantic text is used to combine the semantic guidance of the natural language prompts with visual focusing through dynamic multi-scale feature compression, i.e., text-image feature fusion / alignment.

[0042] The nonlinear projection network is constructed using the Mish activation function. The process of inputting the optimized semantic feature vector into the nonlinear projection network to align text prompts with visual features includes the following steps: S401: Accepts natural language text prompts and projects the bridge semantic feature vector Φ onto the language model space through a multi-layer Mish-activated MLP network, achieving cross-modal alignment between visual features and text prompt instructions.

[0043] like Figure 4 This diagram illustrates a nonlinear projection network. The nonlinear projection layer, constructed using the Mish activation function, aligns the bridge diagnostic text prompts with the guiding statements and visual features. It employs the Mish activation function to build the nonlinear projection network, achieving a two-stage transformation to map the visual feature space to the language embedding space. The goal is to achieve dimensional projection, completing the dimensional alignment between the image and text.

[0044] For input visual features, Let the projection weights be those of the i-th layer. For the i-th layer bias term, The output is activated for the i-th layer. The first layer of the nonlinear projection layer expands the 1024-dimensional visual features into a 1024-dimensional hidden layer, and the second projection layer compresses the 2048-dimensional features into 768 dimensions, where the 768-dimensional dimension is the standard input of the language model to achieve visual-text alignment; First layer projection:

[0045] Second layer projection:

[0046] The projection process employs a residual connection structure to ensure effective gradient backpropagation.

[0047] Finally, the alignment is completed, generating the following input sequence;

[0048] The output multidimensional feature vector includes: By fusion of multimodal features to align visual features with text instructions, knowledge graphs are used to retrieve and dynamically expand knowledge related to defects, and finally, a structured JSON report containing defect type, location coordinates and repair plan is automatically generated according to engineering specification templates, such as transverse crack + pier #3 + epoxy grouting.

[0049] In this implementation step, a structured defect report, component status description, and maintenance suggestion knowledge graph are generated in parallel based on cross-modal features to achieve multi-dimensional expression of the detection results.

[0050] S5: Encapsulate the generated content according to standard protocols to generate a diagnostic report on the health status of the bridge.

[0051] In this embodiment, the generated content is encapsulated according to standard protocols, supporting output in mainstream data exchange formats. The system implements an end-to-end processing pipeline to ensure the reliability of the entire link from feature input to final output. When deployed on edge devices, memory optimization strategies are used to ensure real-time requirements.

[0052] Furthermore, a structured knowledge graph containing defect reports, component status descriptions, and maintenance suggestions is generated in parallel based on cross-modal features. Finally, the output is encapsulated through a standard protocol, forming an end-to-end processing pipeline from feature input to report output. During the edge device deployment phase, a dynamic memory optimization strategy ensures that the system meets real-time requirements. The vision-language projector adopts a weight parameter sharing mechanism to reduce computational overhead. The entire process achieves a detection accuracy of over 92% in bridge health monitoring tasks.

[0053] In summary, the bridge health diagnosis method based on a dynamic lightweight visual language model described in this invention can improve the fine-grained question-answering accuracy of the Visual Question Answering (VQA) task to 49.6% (compared to 47.5% for MobileVLM V2). A lightweight nonlinear projection design based on multi-layer Mish activation replaces the traditional Q-Former structure, maintaining a comprehensive multimodal capability assessment (MMB) performance of 58.1% despite an 85% reduction in parameters. The introduction of a balanced multi-scale training strategy to regulate feature selection distribution ensures a stable anti-hallucination accuracy of 86.2% on the Phenomenon Spectrum Illusion (POPE) task. This achieves a comprehensive optimization of efficiency and accuracy in multimodal understanding of engineering images, receiving linguistic guidance on health diagnosis problem descriptions and health status definition standards, and ultimately outputting a standardized bridge health status diagnosis report. This achieves a balanced optimization of detail preservation and computational efficiency in multimodal understanding of engineering images.

[0054] According to a second aspect of the present invention, a bridge health diagnosis device based on a dynamic lightweight visual language model is provided, such as... Figure 5 The diagram shows the architecture of a bridge health diagnosis device based on a dynamic lightweight visual language model. The image represents a high-resolution bridge pattern, which is segmented, encoded, downsampled, scaled, and finally aligned with the text code vector through a nonlinear projection layer. The device includes: A visual encoder is used to acquire the original image of the bridge in ultra-high resolution, perform overlapping image partitioning on the original image, and encode the original image after partitioning, calculate cosine similarity, generate a differential score and implement TopK screening to obtain a dynamic multi-scale feature candidate set. The multi-scale downsampling layer is used to fuse multi-scale feature candidate sets, preserve the balance between details and global information, find the best single-scale feature, and output the optimized semantic feature vector. A text encoder is used to receive natural language text prompts and complete text encoding. The nonlinear projection layer is used to input the optimized semantic feature vector into the nonlinear projection network, align the text prompt words with the visual features, and output a multidimensional feature vector. A report generator is used to encapsulate the generated content according to standard protocols and generate diagnostic reports on the health status of bridges.

[0055] In summary, the bridge health diagnosis method and apparatus based on a dynamic lightweight visual language model provided in this invention, by inputting an ultra-high-resolution original image of the bridge, performs region segmentation and information density assessment on the high-resolution image through a dynamic spatial feature downsampling mechanism, and employs a global semantic-guided multi-scale fusion strategy to constrain local feature selection, achieving synergistic optimization of efficiency and accuracy in multimodal understanding of engineering images; it receives linguistic guidance on the description of the health diagnosis problem and the definition standards of the health status, and finally outputs a standardized bridge health status diagnosis report. This achieves a balanced optimization of detail preservation and computational efficiency in multimodal understanding of engineering images.

[0056] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0057] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0058] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0059] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

[0060] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A bridge health diagnosis method based on a dynamic lightweight visual language model, characterized in that, Includes the following steps: Obtain an ultra-high resolution original image of the bridge, and perform overlapping image partitioning processing on the original image; The original image after partitioning is encoded, cosine similarity is calculated, differential scores are generated, and TopK filtering is performed to obtain a dynamic multi-scale feature candidate set. By fusing multi-scale feature candidate sets, preserving the balance between details and global information, the optimal single-scale feature is obtained, and the optimized semantic feature vector is output. The system receives natural language text prompts, completes text encoding, inputs the optimized semantic feature vector into a nonlinear projection network, aligns the text prompts with visual features, and outputs a multidimensional feature vector. The generated content is packaged according to standard protocols to generate a diagnostic report on the health status of the bridge.

2. The bridge health diagnosis method based on a dynamic lightweight visual language model according to claim 1, characterized in that: The process of obtaining the original ultra-high resolution image of the bridge and performing overlapping image partitioning on the original image includes: The original superpixel bridge images are acquired by drones or fixed monitoring equipment. The superpixel bridge images are segmented into a global downsampled image and multiple overlapping local image patches using a sliding window mechanism. Bilinear interpolation is used to maintain geometric consistency, and the output is a sequence of image patches with spatial coding.

3. The bridge health diagnosis method based on a dynamic lightweight visual language model according to claim 1, characterized in that, The process of encoding the original image after partitioning, calculating cosine similarity, generating a differential score, and performing TopK filtering yields a dynamic multi-scale feature candidate set, including: Local features of each image patch are extracted using a ViT-B / 16 encoder with shared weights; Calculate the cosine similarity matrix between representation units within the block, and dynamically select the most critical representation units for the bridge detection task from high-density visual features; Finally, the multi-scale feature candidate set is output.

4. The bridge health diagnosis method based on a dynamic lightweight visual language model according to claim 3, characterized in that, The formula for dynamically selecting the most critical representation units for bridge detection from high-density visual features is as follows: In the formula, DS[s]( ( ) represents the feature matrix generated through dynamic sampling at resolution scale s. Let be the visual feature sequence of the i-th image patch, and D be the importance score of the representation unit. The number of representation units reserved for the s-th scale ∈ .

5. The bridge health diagnosis method based on a dynamic lightweight visual language model according to claim 1, characterized in that, The fusion of multi-scale feature candidate sets preserves a balance between detailed and global information, calculates the optimal single-scale feature, and outputs the optimized semantic feature vector, including: During the training phase, weights are calculated based on the features at each scale of the candidate set and the semantic relevance of the whole-image encoding; then the obtained multi-scale features are fused to preserve the balance between details and global information. Design a balanced loss function to constrain the multi-scale selection distribution; During the inference phase, the single-scale feature with the highest probability is selected, the index of the optimal scale is selected, and the optimized semantic feature vector is output.

6. The bridge health diagnosis method based on a dynamic lightweight visual language model according to claim 5, characterized in that, The balanced loss function prevents the dynamic scale selector from getting trapped in local optima during training by using probability distribution constraints, always selecting a fixed scale ratio; the loss function is expressed as follows: in, To select the balance loss for multi-scale, α is the weight coefficient of the balance loss term, n is the number of candidate sets, and N is the total number of samples in the current training batch. The frequency at which scale i is selected. When processing the j-th sample, this sample belongs to The sampling probability; this loss function ensures the stability of feature ratio selection at different scales.

7. The bridge health diagnosis method based on a dynamic lightweight visual language model according to claim 1, characterized in that, The natural language text prompts include: semantic text that guides the health monitoring process, which combines the semantic guidance of the natural language prompts with visual focusing through dynamic multi-scale feature compression, i.e., text-image feature fusion / alignment; the nonlinear projection network is constructed using the Mish activation function.

8. The bridge health diagnosis method based on a dynamic lightweight visual language model according to claim 1, characterized in that, The optimized semantic feature vector is input into a nonlinear projection network to align text prompts with visual features, outputting a multi-dimensional feature vector including: It accepts natural language text prompts and aligns health monitoring guidance text with visual features. Through a multi-layer Mish-activated MLP network, the bridge semantic feature vector is projected onto the language model space to complete the cross-modal alignment of visual features and text instructions. The system utilizes a knowledge graph to retrieve and dynamically expand knowledge related to defects, ultimately automatically generating a structured JSON report containing defect types, location coordinates, and repair plans according to engineering specification templates.

9. The bridge health diagnosis method based on a dynamic lightweight visual language model according to claim 1, characterized in that, The process of encapsulating the generated content according to standard protocols to generate a diagnostic report on the health status of the bridge includes: Based on cross-modal feature parallel generation, a structured knowledge graph containing defect reports, component status descriptions, and maintenance suggestions is generated. The output is encapsulated through standard protocols, forming an end-to-end processing pipeline from feature input to report output. When deployed on edge devices, memory optimization strategies ensure real-time requirements.

10. A bridge health diagnosis device based on a dynamic lightweight visual language model, characterized in that, include: A visual encoder is used to acquire ultra-high resolution raw images of the bridge and perform overlapping image partitioning processing on the raw images. The original image after partitioning is encoded, cosine similarity is calculated, differential scores are generated, and TopK filtering is performed to obtain a dynamic multi-scale feature candidate set. The multi-scale downsampling layer is used to fuse multi-scale feature candidate sets, preserve the balance between details and global information, find the best single-scale feature, and output the optimized semantic feature vector. A text encoder is used to receive natural language text prompts and complete text encoding. The nonlinear projection layer is used to input the optimized semantic feature vector into the nonlinear projection network, align the text prompt words with the visual features, and output a multidimensional feature vector. A report generator is used to encapsulate the generated content according to standard protocols and generate diagnostic reports on the health status of bridges.

Images