Bridge apparent image block construction method, system, device and storage medium

Abstract

The application discloses a bridge apparent image block construction method, system, device and storage medium. The bridge apparent image block construction method comprises the following steps: determining a second image based on a first image; inputting the second image into a visual semantic analysis model to obtain a first candidate data set; performing cropping on the first image to obtain a third image corresponding to each coarse candidate region; inputting the third image into the visual semantic analysis model to obtain a second candidate data set; screening all disease categories with a first preset flag value as a positioning starting flag value; determining a third candidate data set based on the disease categories, corresponding category positioning descriptions and the third image; and determining apparent image block data based on the first candidate data set, the second candidate data set and the third candidate data set, thereby improving the efficiency and accuracy of bridge apparent image block construction.

Description

Technical Field

[0001] This application relates to the technical field of bridge appearance image block construction, and in particular to a method, system, device and storage medium for bridge appearance image block construction. Background Technology

[0002] Bridge structures are prone to various surface defects during long-term service, such as concrete cracks, exposed rebar, and surface spalling. These defects generally exhibit significant scale differences, with small cracks and large areas of spalling potentially coexisting; their boundary morphology is highly irregular, making it difficult to accurately describe them using simple geometric models; and they are often accompanied by complex background interference factors, such as uneven lighting, shadow coverage, and surface stains. Furthermore, multiple types of defects often overlap and coexist in space, further increasing the difficulty of detection and analysis.

[0003] In existing technologies, the construction of bridge appearance image blocks typically relies on a single global model to process the entire image or adopts a fixed rule slicing strategy. However, when the high-resolution original image is directly scaled to a fixed input size to meet the model's processing requirements, the number of effective pixels for minor defects such as hairline cracks and exposed rebar edges is drastically reduced, resulting in a severe loss of key detail information. Subsequent analysis struggles to accurately capture such weak features. Furthermore, if a fixed-step sliding window method is used for image slicing, although it can achieve full area coverage, it repeatedly scans a large number of clean, defect-free surface areas, causing redundant consumption of computational resources and a significant increase in processing time. Ultimately, this makes the image block construction process inefficient and its practicality limited. Summary of the Invention

[0004] This application aims to at least address the technical problems existing in the prior art. To this end, this application proposes a method, system, device, and storage medium for constructing bridge appearance image blocks, which can improve the efficiency and accuracy of bridge appearance image block construction.

[0005] A first aspect of this application provides a method for constructing bridge appearance image blocks, the method comprising: In the case of acquiring a first image, a second image and coordinate mapping parameters between the first image and the second image are determined based on the first image, wherein the first image is the original appearance image of the bridge to be constructed, and the second image is a preview image of the first image; The second image is input into a visual semantic analysis model to obtain a first candidate dataset output by the visual semantic analysis model. The first candidate dataset includes an inspectable structural surface region and at least one coarse candidate region. The coarse candidate region is located within or intersects with the inspectable structural surface region and is determined to be a search region that may contain at least one disease category. Based on the coordinate mapping parameters, the coarse candidate regions are restored to the original coordinate system of the first image. The restored coarse candidate regions are expanded to obtain corresponding local kernel regions. The first image is then cropped according to the local kernel regions to obtain a third image corresponding to each coarse candidate region. The third image corresponding to each coarse candidate region and the corresponding disease category set are input into a visual semantic analysis model to obtain a second candidate dataset output by the visual semantic analysis model. The second candidate dataset includes local candidate regions, disease categories of the local candidate regions, and categorical localization descriptions and localization initiation flag values ​​of the disease categories in the disease category set. The target disease category with the location activation flag value as the first preset flag value is selected, and the categorized location description and the third image corresponding to the target disease category are determined. The target disease category, its corresponding categorized location description, and the third image are input into the text conditional target location model to obtain the third candidate dataset output by the text conditional target location model. The third candidate dataset includes multiple categorized location candidate regions. The candidate regions in the first candidate dataset, the second candidate dataset, and the third candidate dataset are unified into the original image coordinate system for multi-source candidate fusion to obtain fused candidate regions. Based on the fused candidate regions, the appearance image blocks of the bridge to be constructed are determined.

[0006] The bridge appearance image block construction method of this application has at least the following beneficial effects: This method employs a three-tiered cascaded analysis architecture: global coarse screening using preview images, local fine inspection using original images, and text-based conditional target localization. First, it rapidly locates inspectable structural surfaces and coarse candidates on low-resolution preview images, significantly reducing the computational cost of the global search and effectively avoiding detail loss caused by unnecessary global scaling of the entire high-resolution original image. Then, it returns to the original high-resolution image for local fine verification, improving the detection accuracy of small targets such as fine cracks. Finally, it triggers text-based conditional target localization only for categories with insufficient localization, specifically improving the recall rate of difficult-to-classify categories. This achieves a progressively decreasing computational cost and progressively increasing detection accuracy. This coarse-to-fine strategy significantly reduces repeated scanning and computation of disease-free areas, greatly improving the efficiency of image patch construction. Furthermore, it introduces a text-based conditional target localization model, using categorical localization descriptions of disease categories for auxiliary localization. This, combined with semantic information, enables more accurate identification and localization of diseases. Especially for diseases with indistinct visual features or those highly similar to the background, text descriptions can provide additional clues, enhancing the robustness of localization. Moreover, the method establishes coordinate mapping parameters while generating preview images, ensuring that low-resolution results accurately reflect the original image, providing a unified coordinate benchmark for multi-level processing, unifying the three-stage candidates to the original image coordinates for multi-source fusion, eliminating duplication by aggregating similar categories and preserving coexistence relationships across categories, and finally determining the apparent image blocks based on the fusion candidate regions, providing standardized input for downstream disease detection.

[0007] A second aspect of this application provides a bridge appearance image block construction system, the bridge appearance image block construction system comprising: An image conversion module is used to determine a second image and coordinate mapping parameters between the first image and the second image based on the first image, when a first image is acquired, wherein the first image is the original appearance image of the bridge to be constructed, and the second image is a preview image of the first image; The first candidate region extraction module is used to input the second image into the visual semantic analysis model to obtain the first candidate dataset output by the visual semantic analysis model. The first candidate dataset includes an inspectable structural surface region and at least one coarse candidate region. The coarse candidate region is located within or intersects with the inspectable structural surface region and is determined to be a search region that may contain at least one disease category. The second candidate region extraction module is used to restore the coarse candidate region to the original coordinate system of the first image based on the coordinate mapping parameters, expand the restored coarse candidate region to obtain the corresponding local kernel region, and crop the first image according to the local kernel region to obtain the third image corresponding to each coarse candidate region. The third image corresponding to each coarse candidate region and the corresponding disease category set are input into the visual semantic analysis model to obtain the second candidate dataset output by the visual semantic analysis model. The second candidate dataset includes local candidate regions, disease categories of local candidate regions, and categorical localization descriptions and localization initiation flag values ​​of disease categories in the disease category set. The third candidate region extraction module is used to filter out the target disease category whose positioning start flag value is a first preset flag value, and determine the categorized positioning description and the third image corresponding to the target disease category. The target disease category, its corresponding categorized positioning description, and the third image are input into the text conditional target positioning model to obtain the third candidate dataset output by the text conditional target positioning model. The third candidate dataset includes multiple categorized positioning candidate regions. The image patch extraction module is used to unify the candidate regions in the first candidate dataset, the second candidate dataset, and the third candidate dataset into the original image coordinate system for multi-source candidate fusion to obtain fused candidate regions, and to determine the appearance image patch of the bridge to be constructed based on the fused candidate regions.

[0008] A third aspect of this application provides an electronic device including at least one processor and a memory for communicatively connecting to the processor; the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to cause the at least one processor to perform a bridge appearance image block construction method as described in the first aspect of this application.

[0009] A fourth aspect of this application provides a computer-readable storage medium storing computer-executable instructions for causing a computer to perform a bridge appearance image block construction method as described in the first aspect of this application.

[0010] It should be noted that the beneficial effects of the second to fourth aspects of this application compared with the prior art are the same as the beneficial effects of the above-described method for constructing bridge appearance image blocks compared with the prior art, and will not be described in detail here.

[0011] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0012] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which: Figure 1 This is a flowchart illustrating the bridge appearance image block construction method provided in the embodiments of this application; Figure 2 This is a flowchart illustrating a method for constructing bridge appearance image blocks according to another embodiment of this application; Figure 3 This is a schematic diagram illustrating the three types of candidate coordinate restoration and the unification of coordinates with the original image provided in the embodiments of this application; Figure 4 This is a schematic diagram illustrating the coexistence of multi-source candidate fusion and multiple disease categories provided in the embodiments of this application; Figure 5 This is a schematic diagram of coverage determination and fallback scanning provided in the embodiments of this application; Figure 6 This is a schematic diagram showing the alignment of the candidate driving block with the boundary provided in an embodiment of this application; Figure 7 This is a schematic diagram of image block deduplication and record merging provided in an embodiment of this application; Figure 8 This is a schematic diagram of the bridge appearance image block construction system provided in the embodiments of this application; Figure 9 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0013] The embodiments of this application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application.

[0014] In the description of this application, the use of terms such as "first," "second," etc., is for the purpose of distinguishing technical features only and should not be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated or the order of the technical features indicated.

[0015] In the description of this application, it should be understood that the orientation descriptions, such as up, down, etc., are based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application.

[0016] In the description of this application, it should be noted that, unless otherwise explicitly defined, terms such as "setup," "installation," and "connection" should be interpreted broadly, and those skilled in the art can reasonably determine the specific meaning of the above terms in this application in conjunction with the specific content of the technical solution.

[0017] Bridge structures are prone to various surface defects during long-term service, such as concrete cracks, exposed rebar, and surface spalling. These defects generally exhibit significant scale differences, with millimeter-scale cracks and large areas of spalling potentially coexisting; their boundary morphology is highly irregular, making it difficult to accurately describe using simple geometric models; and they are often accompanied by complex background interference factors, such as uneven lighting, shadow coverage, and surface stains; furthermore, multiple types of defects often overlap and coexist in space, further increasing the difficulty of detection and analysis.

[0018] In existing technologies, the construction of bridge appearance image blocks typically relies on a single global model to process the entire image or adopts a fixed rule slicing strategy. However, when the high-resolution original image is directly scaled to a fixed input size to meet the model's processing requirements, the number of effective pixels for minor defects such as hairline cracks and exposed rebar edges is drastically reduced, resulting in a severe loss of key detail information. Subsequent analysis struggles to accurately capture such weak features. Furthermore, if a fixed-step sliding window method is used for image slicing, although it can achieve full area coverage, it repeatedly scans a large number of clean, defect-free surface areas, causing redundant consumption of computational resources and a significant increase in processing time. Ultimately, this makes the image block construction process inefficient and its practicality limited.

[0019] Some embodiments of this application are described with reference to... Figure 1 One embodiment of this application provides a method for constructing bridge appearance image blocks, including the following steps S110 to S150: Step S110: In the case of obtaining the first image, determine the second image and the coordinate mapping parameters between the first image and the second image based on the first image, wherein the first image is the original appearance image of the bridge to be constructed, and the second image is a preview image of the first image; Step S120: Input the second image into the visual semantic analysis model to obtain the first candidate dataset output by the visual semantic analysis model. The first candidate dataset includes an inspectable structural surface region and at least one coarse candidate region. The coarse candidate region is located within or intersects with the inspectable structural surface region and is judged to be a search region that may contain at least one disease category. Step S130: Based on the coordinate mapping parameters, the coarse candidate regions are restored to the original coordinate system of the first image. The restored coarse candidate regions are expanded to obtain the corresponding local kernel regions. The first image is then cropped according to the local kernel regions to obtain the third image corresponding to each coarse candidate region. The third image corresponding to each coarse candidate region and the corresponding disease category set are input into the visual semantic analysis model to obtain the second candidate dataset output by the visual semantic analysis model. The second candidate dataset includes local candidate regions, disease categories of local candidate regions, and categorical localization descriptions and localization initiation flag values ​​of disease categories in the disease category set. Step S140: Filter out the target disease category with the location start flag value as the first preset flag value, and determine the categorized location description and third image corresponding to the target disease category. Input the target disease category and its corresponding categorized location description and third image into the text conditional target location model to obtain the third candidate dataset output by the text conditional target location model. The third candidate dataset includes multiple categorized location candidate regions. Step S150: Unify the candidate regions in the first candidate dataset, the second candidate dataset, and the third candidate dataset into the original image coordinate system to perform multi-source candidate fusion to obtain fused candidate regions. Based on the fused candidate regions, determine the appearance image block of the bridge to be constructed.

[0020] For ease of understanding, the following explains some key terms in this embodiment: The first image refers to the high-resolution original image of the bridge structure to be inspected for defects.

[0021] The second image refers to a low-resolution preview image derived from the first image.

[0022] Coordinate mapping parameters refer to the mathematical relationships or transformation matrices used for coordinate transformation between the first and second images. These parameters allow for accurate location and recovery of regions between images of different resolutions, ensuring that regions identified in the preview image accurately correspond to the original high-resolution image.

[0023] Visual semantic analysis model is a computational model that can understand the content of an image and identify specific semantic information within it.

[0024] The inspectable structural surface area refers to the region in the bridge image that is identified as the actual structural surface. This area excludes sky, vegetation, vehicles, water surfaces, equipment obstructions, and obvious non-structural background.

[0025] The local verification region refers to the region obtained by appropriately expanding the coarse candidate region after restoring it from the coordinate system of the second image to the original coordinate system of the first image.

[0026] The third image refers to a high-resolution local image cropped from the first image based on the local verification region.

[0027] The defect category set refers to a predefined set of various defect types that may appear on bridge structures, such as cracks, spalling, and exposed rebar.

[0028] Local candidate regions refer to the more precise disease areas that are identified in a third image.

[0029] Categorized localization descriptions refer to textual descriptions of specific disease categories to help the target localization model understand the visual characteristics of the disease. For example, for crack disease, the description might include thin, irregular linear damage.

[0030] The text-conditional target localization model is a deep learning model capable of locating specific targets in an image based on textual descriptions. In this embodiment, it utilizes categorical localization descriptions of diseases to perform more accurate disease region localization in a third image.

[0031] Multi-source candidate fusion refers to the process of integrating and optimizing candidate regions from different detection stages (such as global coarse detection, local fine detection, and text conditional localization).

[0032] Apparent image patch refers to the bridge surface image patch that is finally determined based on the fusion candidate regions and used for subsequent detailed inspection and analysis.

[0033] This embodiment provides a method for constructing bridge appearance image blocks. Its main technical feature is designed to effectively address challenges such as scale differences, irregular boundaries, and background interference in bridge defect detection through a multi-stage, multi-scale analysis strategy.

[0034] First, in step S110, if the first image is acquired, the second image and the coordinate mapping parameters between them are determined based on the first image. The first image is the apparent original image (high-resolution original image) of the bridge to be constructed, and the second image is a preview of the first image. The high-resolution first image can be scaled down to a smaller second image by setting a scaling ratio, and the scaling ratio is recorded as the coordinate mapping parameters.

[0035] In step S120, the second image is input into the visual semantic analysis model to obtain a first candidate dataset output by the model. The first candidate dataset includes an inspectable structural surface region and at least one coarse candidate region. The coarse candidate region is located within or intersects with the inspectable structural surface region and is determined to be a search region that may contain at least one disease category. During the pre-training of the visual semantic analysis model, a set of disease categories can be input simultaneously. The training data (i.e., preview images corresponding to the training samples) enables the model to output regions that may contain one or more diseases; among which... A collection consisting of all disease categories, It is a pre-established set of categories, which may include category identifiers, category names, category aliases, and optional appearance descriptions.

[0036] This step describes the processing logic of the first stage, which achieves global coarse screening by filtering coarse candidate regions within the global image scope.

[0037] In step S130, the coarse candidate regions are restored to the original coordinate system of the first image based on the coordinate mapping parameters. The restored coarse candidate regions are expanded to obtain the corresponding local kernel regions. The first image is then cropped according to the local kernel regions to obtain the third image corresponding to each coarse candidate region.

[0038] The third image corresponding to each coarse candidate region and the corresponding disease category set are input into the visual semantic analysis model to obtain the second candidate dataset output by the visual semantic analysis model. The second candidate dataset includes local candidate regions, disease categories of local candidate regions, and categorical localization descriptions and localization initiation flag values ​​of disease categories in the disease category set. For example, after restoring the coarse candidate region to the original image coordinate system, its boundary can be fixedly extended outward by a certain number of pixels to form a local kernel region. Alternatively, its boundary can be extended outward by a fixed percentage area according to the size of the coarse candidate region to ensure that sufficient contextual information is included. Subsequently, the cropped third image is input into another visual semantic analysis model or into a visual semantic analysis model identical to that in step S120. In this step, the model can identify more accurate local candidate regions and disease categories.

[0039] Because the above steps perform global analysis on the preview image, which is a low-resolution image scaled down from the original high-resolution image, the purpose is to quickly search for potential disease locations within the entire image, significantly reducing the computational cost of the global search. However, precisely because the preview image has low resolution, the model struggles to accurately determine the disease category, and the output coarse candidate region is only a preliminary search box with rough boundaries and potential errors in category judgment, making it unsuitable as the final detection result. Therefore, this step describes the second stage, which involves cropping local high-resolution regions based on the coarse candidate region for fine-tuning. In the local high-resolution image, the texture, edges, color, and other details of the disease are fully presented, allowing the model to more accurately identify the disease category and refine the location boundaries of the candidate region. Simultaneously, it limits the check to only the set of disease categories associated with the coarse candidate, avoiding redundant calculations for irrelevant categories within local areas. The second stage solves the problems of inaccurate category judgment and imprecise location boundaries caused by resolution limitations in the first stage, representing a progression from coarse screening to fine-tuning.

[0040] In step S140, target disease categories with a location activation flag value of a first preset flag value are selected, and the categorical location description and third image corresponding to the target disease category are determined. The target disease category, its corresponding categorical location description, and the third image are input into the text conditional target localization model to obtain the third candidate dataset output by the text conditional target localization model. The third candidate dataset includes multiple categorical location candidate regions, and can also output the disease category identifier corresponding to each categorical location candidate region.

[0041] Because the second stage may result in low localization certainty and missing local candidate regions, the model struggles to reliably distinguish the category from similar backgrounds based solely on visual features. Directly incorporating these poorly localized results into multi-source fusion could lead to missed detections or misclassifications of difficult-to-classify categories. Therefore, this step describes the third stage. For these difficult-to-classify categories triggered by localization activation flags, textual conditions such as category names, appearance feature words, morphological words, and local location descriptions are input along with the local image into the text-conditional target localization model. These textual conditions provide clear semantic guidance, helping the model to specifically search for specific categories in the local image, improving recall and localization accuracy for categories with ambiguous boundaries and those easily confused. The third stage uses localization activation flags for on-demand triggering, only adding calculations for categories with insufficient localization in the second stage, avoiding repeated reasoning for reliably confirmed categories. This represents a progression from refined detection to targeted supplementation of difficult-to-classify categories.

[0042] In step S150, the candidate regions in the first candidate dataset, the second candidate dataset, and the third candidate dataset are unified to the original image coordinate system for multi-source candidate fusion to obtain fused candidate regions. Based on the fused candidate regions, the appearance image blocks of the bridge to be constructed are determined.

[0043] like Figure 2 For ease of understanding, examples of steps S110 to S150 above are as follows: (1) Let the first image be Its width is Height is ; Before generating the preview image, an image-level correlation check can be performed on the first image. The correlation check can be performed by rules, classification models, visual semantic models, or a combination thereof, which will not be described in detail here.

[0044] Then the image-level correlation quality value is obtained. (Normalized to [0,1]), set a high confidence threshold. and low confidence threshold ,exist At that time, it will proceed normally to the next stage of processing.

[0045] (2) Determine the second image (preview image) based on the first image, and establish coordinate mapping parameters between the first image and the second image; Here we assume the original image's coordinate system is... The coordinate system of the preview image is Specifically, the maximum allowed side length for global analysis is preset to be... According to the preview image scaling factor : ; Calculate the scaling ratio, according to , The first image is scaled while maintaining its aspect ratio to obtain the second image.

[0046] Simultaneously establish coordinate mapping parameters: original image coordinates ;in, , These are the coordinates in the second image.

[0047] The coordinate mapping parameters are used to accurately restore the spatial results obtained on the second image to the original coordinate system of the first image.

[0048] (3) Input the second image into the visual semantic analysis model to obtain the first candidate dataset output by the visual semantic analysis model. The first candidate dataset includes the surface regions of the inspectable structures. and at least one coarse candidate region .

[0049] Can inspect structural surface area This represents the visible concrete surface area in the first image suitable for inspecting bridge surface defects, used to exclude non-structural background elements such as sky, vegetation, vehicles, water surfaces, and equipment obstructions; coarse candidate region. Located in the inspectable structural surface area The search area is either inside or intersects with the inspectable structural surface area and is determined to potentially contain at least one type of defect.

[0050] In some embodiments, the model performs corresponding operations based on the first-stage global analysis instructions input by the user. The first-stage global analysis instructions are directed to the entire preview image and may require the model to identify inspectable structural surfaces, provide coarse candidates for multiple categories, and output first-stage category evidence for local verification. Here, the first-stage category evidence is a set of information used by the model at this stage to characterize the basis for disease category judgment. For example, when the model determines which disease category a candidate area belongs to, the multi-category probability distribution and confidence data it relies on are not limited to specific types of information.

[0051] (4) Using coordinate mapping parameters, the inspectable structural surface area is mapped. and coarse candidate region Restore the original coordinate system of the first image to the coordinate system of the second image. In the process, boundary clipping and validity checks are performed on the restored results, and the clipped and valid restored results are used for subsequent operations.

[0052] Specifically, based on coordinate mapping parameters, the coarse candidate region is... By restoring the original coordinate system of the first image, the restored coarse candidate region is expanded to obtain the corresponding local kernel region. .

[0053] Based on horizontal expansion factor and vertical expansion coefficient The recovered coarse candidate region is expanded outward and cropped to the effective range of the first image.

[0054] The first image is divided into local kernel regions. Crop the image to obtain the third image corresponding to each coarse candidate region. .

[0055] For example, the origin of the pixel coordinates in the first image is located at the top left corner. Axis to the right, Axial direction downwards. (First) Coarse candidate regions Restore to Then, the coarse candidate region is represented as: , , , The top left corner The bottom right corner is the local review area. The boundary is: ; ; ; ; in, , Input coordinates, , The horizontal and vertical scaling factors are greater than zero. , The coordinate offsets caused by scaling or filling, as described above , The formula is for restoring the coordinate relationship of points.

[0056] (5) Combine the third image corresponding to each coarse candidate region with the corresponding disease category set. Input the visual semantic analysis model to obtain the second candidate dataset output by the visual semantic analysis model.

[0057] in, For the first Coarse candidate regions A collection of disease categories, yes A subset of the visual semantic analysis model, which only applies to the third image. The task of identifying, refining, and locating the disease categories is scheduled.

[0058] The second candidate dataset may include local candidate regions, disease categories within the local candidate regions, and categorical localization descriptions for each disease category in the disease category set. And the location activation flag value. Among them, the categorized location description... This includes textual conditions such as the category name, aliases, appearance characteristics, morphological terms, and local location descriptions of the target disease category; the positioning activation flag value is used to control whether the third-stage textual condition target positioning is triggered.

[0059] It is worth noting that, similarly to the above, a second-stage local review instruction can also be set here. This instruction controls the model to perform corresponding operations. The second-stage local review instruction is responsible for local confirmation, refinement, and location scheduling. The second candidate dataset can also include second-stage category evidence, which is similar to the first-stage category evidence stage. Both are information sets used by the model in this stage to characterize the basis for disease category judgment (candidate valid status, judgment supporting information, etc.), but the specific types of information are not limited here.

[0060] (6) Select disease categories with a positioning start flag value of the first preset flag value as target disease categories (one or more), and determine the categorized positioning description corresponding to the target disease category. And the third image .

[0061] The location activation flag is a switch that controls whether the third-stage text conditional target location is triggered. It can be set to 1 or 0. During the model's operation, if the model believes that a disease category is not accurately located in the second stage (i.e., it is not accurately detected), the model will output a local location deterministic value (a normalized value) simultaneously. ,exist If the location is considered insufficient, set it to 1; if the location is considered sufficient, i.e. If the value is 0, then set it to 0, and set the first preset flag value to 1. Of course, the criteria for judging whether the localization is sufficient can also be whether the local candidate region is missing, whether the second type of evidence is complete, or a combination of the above; this is not limited here.

[0062] Target disease category and its corresponding categorized location description And the third image Input the data into the text conditional target localization model to obtain the third candidate dataset output by the text conditional target localization model.

[0063] The text-conditional target localization model searches for target disease categories in a third image (a locally high-resolution image) based on the textual conditions in the categorical localization description, outputting multiple categorical localization candidate regions. These candidate regions are first defined using the local input coordinate system of the text-conditional target localization model. The coordinates are saved in the local verification area. The top-left offset, scaling factor, and boundary parameters are restored to the original coordinate system of the first image. This is done in order to facilitate subsequent processing of the fusion region.

[0064] It is important to note that the categorized localization candidate regions and the local candidate regions in the second stage come from different sources.

[0065] (7) Unify the candidate regions in the first candidate dataset, the second candidate dataset, and the third candidate dataset to the original map coordinate system. Multi-source candidate fusion is performed to obtain fusion candidate regions, and the appearance image blocks of the bridge to be constructed are determined based on the fusion candidate regions.

[0066] Therefore, this method has the following beneficial effects: This method employs a three-tiered cascaded analysis architecture: global coarse screening using preview images, local fine inspection using original images, and text-based conditional target localization. First, it rapidly locates inspectable structural surfaces and coarse candidates on low-resolution preview images, significantly reducing the computational cost of the global search and effectively avoiding detail loss caused by unnecessary global scaling of the entire high-resolution original image. Then, it returns to the original high-resolution image for local fine verification, improving the detection accuracy of small targets such as fine cracks. Finally, it triggers text-based conditional target localization only for categories with insufficient localization, specifically improving the recall rate of difficult-to-classify categories. This achieves a progressively decreasing computational cost and progressively increasing detection accuracy. This coarse-to-fine strategy significantly reduces repeated scanning and computation of disease-free areas, greatly improving the efficiency of image patch construction. Furthermore, it introduces a text-based conditional target localization model, using categorical localization descriptions of disease categories for auxiliary localization. This, combined with semantic information, enables more accurate identification and localization of diseases. Especially for diseases with indistinct visual features or those highly similar to the background, text descriptions can provide additional clues, enhancing the robustness of localization. Moreover, the method establishes coordinate mapping parameters while generating preview images, ensuring that low-resolution results accurately reflect the original image, providing a unified coordinate benchmark for multi-level processing, unifying the three-stage candidates to the original image coordinates for multi-source fusion, eliminating duplication by aggregating similar categories and preserving coexistence relationships across categories, and finally determining the apparent image blocks based on the fusion candidate regions, providing standardized input for downstream disease detection.

[0067] In summary, this method effectively solves the technical problems of scale difference, loss of detail, computational redundancy and insufficient positioning accuracy in bridge defect detection by adopting a multi-stage, multi-scale and multi-source fusion strategy. It significantly improves the efficiency, accuracy and practicality of bridge appearance image block construction, and provides more advanced and reliable technical support for bridge structural health monitoring.

[0068] In some embodiments of this application, candidate regions from the first candidate dataset, the second candidate dataset, and the third candidate dataset are unified to the original image coordinate system for multi-source candidate fusion to obtain fused candidate regions, including: The coarse candidate regions in the first candidate dataset, the local candidate regions in the second candidate dataset, and the candidate regions that pass the validity test in the categorical localization candidate regions in the third candidate dataset are unified into the original image coordinate system to obtain a candidate region set; Candidate regions in the candidate region set are classified according to disease category, and the spatial overlap between every two candidate regions belonging to the same disease category is calculated. Candidate regions with spatial overlap greater than or equal to a preset overlap threshold are set in the same type of spatial candidate group to obtain multiple types of spatial candidate groups. Calculate the spatial union of all candidate regions within each similar spatial candidate group, and use the spatial union as the fusion candidate region.

[0069] Reference Figure 3 and Figure 4 In some embodiments, the coarse candidate regions in the first candidate dataset, the local candidate regions in the second candidate dataset, and the categorically located candidate regions in the third candidate dataset are each subjected to validity checks. For example, validity checks include ensuring that the candidate region coordinates meet certain conditions, such as: the candidate region area is greater than zero, the candidate region is in a valid source state, and the candidate region does not exceed the valid range of the original image. Candidate regions that pass the validity checks participate in subsequent fusion.

[0070] For each candidate region, the coordinates are unified to the original map coordinate system using their respective coordinate restoration methods. : The coarse candidate region is restored from the coordinate system of the preview image to the coordinate system of the original image using coordinate mapping parameters, and then the boundary is clipped. Local candidate regions utilize local verification regions The top-left offset and cropping parameters are used to restore the original image coordinate system from the local image coordinate system. Categorized localization of candidate regions utilizes local verification regions The top-left corner offset, the input scaling factor and boundary parameters of the text conditional target localization model are used to restore the original image coordinate system from the model input coordinate system.

[0071] After all candidate regions are unified to the original map coordinate system, a candidate region set is formed.

[0072] Based on the disease category, candidate regions in the candidate region set are classified. Candidate regions of the same disease category are grouped into the same category, while candidate regions of different disease categories are processed separately, without performing spatial aggregation across categories.

[0073] For any two candidate regions of the same disease category , (Assuming all areas are greater than zero and coordinates are valid), calculate the spatial overlap between the two. Spatial overlap includes the intersection-over-union ratio (IoU) and the coverage ratio (CR): Among them, the intersection-union ratio Coverage rate ; in, Represents the pixel area of ​​the region. Indicates intersection, It represents the union of sets.

[0074] Set a preset overlap threshold, including the intersection-overlap ratio threshold. and coverage threshold , , .

[0075] when or When two candidate regions are determined to have a similar spatial relationship, they are placed in the same similar spatial candidate group.

[0076] By using connected grouping, disjoint-set data structure, or equivalent methods, all candidate regions with similar spatial relationships are aggregated into multiple similar spatial candidate groups. The candidate regions within each similar spatial candidate group belong to the same disease category, and any two candidate regions within the group can be connected through direct or indirect spatial overlap.

[0077] For each spatial candidate group of the same type, the spatial union of all candidate regions within the group is calculated, and the spatial union is used as the fusion candidate region. The spatial union can be a non-rectangular region. For each non-empty spatial union, connected component analysis is performed and it is converted into an axis-aligned bounding rectangle for subsequent image patch construction, but the conversion does not change the true spatial union in the coverage calculation.

[0078] For example, let the first The same type of space candidate group is Its fusion candidate space reference region is the spatial union of the valid candidate boxes within the group: ,in It can be a non-rectangular region. For each non-empty region... Perform connected component analysis and convert each connected component into an axis-aligned bounding rectangle. The bounding rectangle of each connected component is used as an independent region to be constructed, or if the connected component itself is already a rectangle, it is directly used as the region to be constructed.

[0079] It is important to note that for candidate regions of different disease categories, even if there is spatial overlap, inclusion, or adjacency, their respective disease categories, sources, source validity status, source quality values, and category evidence are retained.

[0080] The multi-source candidate fusion method described in this paper merges multiple candidates for the same disease from different processing paths across three stages into a single fusion region by classifying them by disease type, calculating spatial overlap, aggregating overlapping candidates into similar spatial candidate groups, and taking the spatial union. This eliminates duplicate candidates and avoids repeated processing of the same disease in subsequent steps. Fusion is performed only within the same disease category; even if candidates from different categories spatially overlap, they are retained separately without mutual exclusion. This avoids the accidental deletion of overlapping diseases of different categories due to category-irrelevant non-maximum suppression, ensuring the detection integrity in scenarios with multiple coexisting diseases. Taking the spatial union of similar candidate groups, rather than a single optimal bounding box, ensures that the fusion region fully covers the entire spatial range of the same disease. Filtering invalid candidates before fusion improves accuracy, and source information is retained throughout the process to support result traceability.

[0081] In some embodiments of this application, determining the appearance image patch of the bridge to be constructed based on the fusion candidate region includes: Based on the fusion candidate region and the inspectable structural surface region, the bottom-up scanning region is determined; the bottom-up scanning region is different from the fusion candidate region. Based on the fusion candidate region and the bottom scan region, the apparent image block of the bridge to be constructed is determined.

[0082] The process of determining a catch-up scan region based on fusion candidate regions and the inspectable structural surface area aims to compensate for the potential coverage deficiencies when relying solely on fusion candidate regions, ensuring comprehensive coverage of the inspectable structural surface area. It serves as a supplementary mechanism to identify areas not covered by fusion candidate regions but which may still contain defects. This step generates the catch-up scan region by analyzing the portions of the inspectable structural surface area not covered by fusion candidate regions and performing connectivity analysis or region growing on these uncovered portions.

[0083] Unlike fusion candidate regions, the pocket scan region does not simply repeat the fusion candidate region; instead, it supplements the areas not covered or insufficiently covered by the fusion candidate region. This difference ensures the effectiveness and necessity of the pocket scan, avoiding redundant processing. The pocket scan region is generated based on the inspectable structural surface areas not covered by the fusion candidate region, which typically do not completely overlap or exhibit significant differences in image space.

[0084] Finally, based on the fusion candidate regions and the fallback scan regions, the apparent image patches for constructing the bridge are determined. This is the final step in generating image patches, which comprehensively considers the potential disease areas obtained through multi-source fusion (i.e., fusion candidate regions) and the supplementary areas discovered through the fallback mechanism (i.e., fallback scan regions), thereby ensuring that the generated set of image patches focuses on high-risk areas while also being comprehensive. Specifically, the fusion candidate regions and the fallback scan regions can be merged to form a total set of regions to be processed, and then image patches are constructed for each region in this set.

[0085] The above method introduces a fallback scanning region outside the fusion candidate region to supplement the scanning of areas on the inspectable structural surface not covered by defect candidates. This ensures complete coverage of the bridge's concrete structure surface and reduces subsequent defect detection omissions caused by missed front-end candidate detections. The fallback scanning region is clearly distinguished from the fusion candidate region. The fusion candidate region carries defect categories and evidence, while the fallback region only marks the scan source and does not generate category evidence. This ensures complete coverage while avoiding mislabeling defect-free areas as defects.

[0086] In some embodiments of this application, the bottom-up scanning region is determined based on the fusion candidate region and the inspectable structural surface region, including: Based on the fusion candidate regions, determine the covered areas, uncovered areas, and their coverage rates in the inspectable structural surface regions that are covered by the fusion candidate regions; A fallback scan is triggered if the coverage rate is less than a preset coverage threshold, if there is a connected region with an area larger than a preset area threshold in the uncovered area, or if the fusion candidate region is empty.

[0087] like Figure 5 In one embodiment, a specific example is as follows: Inspectable structural surface areas obtained from the first-stage global analysis Based on the baseline, the candidate regions are merged. Calculate the candidate coverage area. Uncovered areas and coverage .

[0088] When the fusion candidate is empty ( ), coverage Less than the preset coverage threshold Uncovered areas There exists an area larger than a preset area threshold. When the source validity status of a connected region or candidate region is invalid, a fallback scan is triggered.

[0089] After triggering the fallback scan, the uncovered areas will be... Performing connected component analysis yielded multiple uncovered connected regions. For areas larger than Uncovered connected regions are used to generate a fallback scan region. .

[0090] The fusion candidate region is derived from the detection results of the disease candidate regions in three stages, carrying the disease category and category evidence for each stage; the catch-all scan region is derived from the area on the surface of the inspectable structure that is not covered by the fusion candidate region, only recording the scan source, without adding new disease categories or generating category evidence.

[0091] Because the candidate detection in the above three stages may miss some areas, some areas of the inspectable structural surface may not be covered by the fusion candidates. If image blocks are constructed based solely on the fusion candidates, these uncovered areas will not enter the downstream disease detection, resulting in missed detections. The fallback scan supplements the uncovered areas by generating scan regions, ensuring complete coverage of the inspectable structural surface. Simultaneously, the fallback region only marks the scan source and does not generate disease category evidence, avoiding mislabeling disease-free areas as diseases. Therefore, this embodiment uses quantitative calculations of covered areas, uncovered areas, and coverage rate to drive the fallback trigger judgment with objective data, avoiding subjective judgment or redundant calculations from full-scale scanning. The three trigger conditions respectively cover three typical missed detection scenarios: insufficient overall coverage, large-area local uncovering, and complete candidate absence, ensuring timely activation of the fallback in different missed detection situations. Connected component analysis splits the uncovered areas into independent patches, generating fallback regions only for connected regions with an area greater than a threshold. This ensures that large-area uncovered areas are supplemented by scanning while avoiding redundant processing of small gaps, achieving a balance between coverage integrity and computational efficiency.

[0092] In some embodiments of this application, the apparent image blocks of the bridge to be constructed are determined based on the fusion candidate region and the bottom scan region, including: Connectivity component analysis is performed on the fusion candidate region and the bottom scan region respectively, and each connected component is converted into an axis-aligned bounding rectangle as the region to be constructed. Based on the preset effective size and preset overlap of the image block, an image block is constructed for each region to be constructed, resulting in multiple initial image blocks; The initial image blocks are deduplicated to obtain the appearance image blocks of the bridge to be constructed.

[0093] Specifically, connected component analysis is performed on the fusion candidate region and the bottom-up scan region separately, aiming to identify interconnected pixel regions in the image. For the fusion candidate region and the bottom-up scan region, connected component analysis can decompose them into independent, unconnected sub-regions, facilitating subsequent independent processing.

[0094] Subsequently, each connected component is converted into an axis-aligned bounding rectangle, which serves as the region to be patched. An axis-aligned bounding rectangle is the smallest rectangle that encloses a connected component, with its sides parallel to the coordinate axes. Converting irregular connected components into axis-aligned bounding rectangles simplifies the subsequent image patching process, providing a standardized and easy-to-manipulate bounding box as the basis for generating the final image patch.

[0095] Based on this, image blocks are constructed for each region to be constructed, using preset effective image block sizes and preset overlap amounts, resulting in multiple initial image blocks. This step aims to further subdivide larger or irregular regions to be constructed into standard image blocks that meet specific size and overlap requirements. Preset effective image block sizes ensure the standardization of output image blocks, facilitating subsequent unified processing and analysis; preset overlap amounts help ensure continuity between image blocks, avoiding information loss, especially in lesion edge areas. For example, a sliding window approach can be used, with the preset effective image block size as the window size and the preset overlap amount as the step size, to perform sliding cropping within each region to be constructed, generating initial image blocks.

[0096] Finally, the initial image patches are deduplicated to obtain the apparent image patches for constructing the bridge. During image patch construction, especially when overlaps exist, a large number of duplicate or highly similar image patches may be generated. Deduplication aims to eliminate this redundancy, reduce data volume, improve the efficiency of subsequent processing, and ensure that the final set of apparent image patches is concise and efficient.

[0097] like Figure 6 In some embodiments, specific examples are as follows: (1) Connectivity component analysis and determination of the region to be constructed; For fusion candidate regions and bottom scan area Perform connected component analysis (4-connected or 8-connected) separately, and convert each connected component into an axis-aligned circumscribed rectangle: ( , () as an independent region to be constructed.

[0098] If the width or height of a connected component is zero, it is marked as invalid and no image patch is generated.

[0099] (2) Construct image blocks of fixed size; Preset effective size of image blocks Horizontal overlap and vertical overlap All measurements are taken in pixels of the first image, with an effective step size. , .

[0100] ; ; in, The number of horizontal image patches. The number of vertical image blocks; that is, the width of the region to be constructed. And height At that time, a fixed effective size image block is generated, and its effective area is aligned with the starting boundary of the area to be constructed in the width and height directions to contain the area.

[0101] When the region to be constructed is greater than in the width direction At that time, the number of horizontal image patches The left boundary of the first image patch is aligned with the left boundary of the region to be constructed, the right boundary of the last image patch is aligned with the right boundary of the region to be constructed, and the left boundary of the middle image patch is aligned with the left boundary of the region to be constructed. The distribution is equidistant. The same applies to the height direction, which will not be elaborated here.

[0102] because , Using rounding up, the actual step size of the equidistant distribution is no greater than [missing value]. , The actual overlap is not less than , Furthermore, no image block exceeds the outer boundary of the region to be constructed.

[0103] when At that time, the left boundary of the first image patch and Align the left edge with the right edge of the last image patch. The right edge is aligned, and the middle image blocks are equidistant between the two ends; the left edge is horizontal. for: ; when At the same time, the same rule applies in the vertical direction: the upper boundary of the first image patch is... Align the upper boundary with the lower boundary of the last image block. The lower boundary is aligned, and the middle image blocks are evenly distributed; When the number of corresponding directions is 1, this direction is... The initial boundary alignment ensures that the position of individual image blocks within the region to be constructed, even those smaller than the effective size of the image block, is also determined. Because... , Using rounding up and equal-interval distribution, the actual step size obtained is no greater than [missing value]. , The actual overlap is not less than , And no image patch exceeds The outer boundary. If If the width or height of an area is zero, the area is marked as invalid and no image patch is generated.

[0104] (3) Physical deduplication; like Figure 7 Let the effective regions of the two image blocks be in The middle are respectively , The area of ​​each area is greater than zero.

[0105] like Then the center point Determined by the average value of the left, right, top, and bottom boundaries. Determined according to the same rules; For dimensionless deduplication and threshold comparison, For Center distance threshold in pixels.

[0106] Repeated checks are performed using conditions that the coordinates are completely identical, or by combining the intersection-union ratio and the center distance: , , In initial image blocks with the same original image identifier, the same effective image block size, and the same spatial recording version, the coordinates must be completely identical or the image blocks must be cross-jointly matched. And the distance from the center point The image block is determined to be a duplicate physical image block. After determining that it is a duplicate, only one image block is retained.

[0107] This embodiment uses connected component analysis to break down irregular fusion candidate regions and catch-up scan regions into independent patches and convert them into axis-aligned bounding rectangles, simplifying subsequent block calculations and ensuring clear boundaries. Image blocks are constructed based on preset fixed effective sizes and overlap amounts. Small regions are aligned with their starting boundaries, while large regions are aligned with their beginning and end boundaries and distributed at equal intervals in the middle, ensuring complete coverage of the region to be constructed without exceeding boundaries. The fixed size also adapts to the input requirements of downstream disease detection models. Physical deduplication is performed on the initial image blocks, merging image blocks with consistent coordinates or high overlap into a single physical image block and incorporating multi-source candidate association information. This avoids repeated inference calculations for the same physical region and ensures that each image block carries a complete candidate source and category evidence traceability chain, providing standardized and non-redundant input for backend cross-image block instance association and mask reconstruction.

[0108] In some embodiments of this application, determining the second image based on the first image includes: A correlation check is performed on the first image to obtain the image-level correlation quality value; If the image-level correlation quality value is greater than or equal to a preset low confidence threshold, obtain the width and height of the first image; Based on the preset global analysis of the maximum side length, width, and height, determine the scaling factor of the preview image; Based on the scaling factor of the preview image, the first image is scaled while maintaining the aspect ratio to obtain the second image, and coordinate mapping parameters are established based on the scaling factor of the preview image.

[0109] Specifically, a correlation check is performed on the first image to assess its overall quality and usability, ensuring the effectiveness of subsequent processing. Image-level correlation quality metrics can reflect key attributes such as image sharpness, brightness, contrast, and noise level. For example, image quality can be evaluated by calculating blur indices (such as Laplacian variance, gradient magnitude, etc.) or signal-to-noise ratio. Alternatively, deep learning models can be used to assess image quality; for instance, a classification model can be trained to determine whether an image meets preset quality standards, outputting a confidence score as a quality metric.

[0110] If the image-level correlation quality value is greater than or equal to a preset low-confidence threshold, the width and height of the first image are obtained. This step ensures that subsequent size acquisition and preview image generation operations are only performed when the quality of the first image meets a certain standard, avoiding invalid processing of low-quality images. The preset low-confidence threshold is used to define the acceptable lower limit of image quality. For example, the image width and height information can be obtained by directly reading the image file's metadata or pixel matrix by calling functions from an image processing library. Alternatively, the image width and height parameters can be extracted from the encoded information in the image file header during image loading by parsing the header information.

[0111] Based on the preset global analysis maximum side length, width, and height, the scaling factor of the preview image is determined. The purpose of this step is to calculate a suitable scaling ratio based on the original size of the first image and the preset maximum processing size, so as to generate a preview image that retains sufficient information while facilitating fast processing. The preset global analysis maximum side length is a processing upper limit set for the visual semantic analysis model to balance computational resources and analysis efficiency.

[0112] Based on the scaling factor of the preview image, the first image is scaled while maintaining its aspect ratio to obtain the second image. Coordinate mapping parameters are then established based on the scaling factor. This step uses the previously determined scaling factor to convert the first image into a smaller, more easily processed second image (preview image). Simultaneously, the scaling relationship is recorded so that the analysis results on the preview image can be mapped back to the original image later. Scaling while maintaining the aspect ratio avoids image content distortion.

[0113] This embodiment performs a correlation check on the original image before generating the preview image, filtering out images of non-bridge scenes and those without inspectable concrete surfaces. This avoids performing subsequent computationally intensive candidate detection and image patch construction on invalid images. The low confidence threshold allows edge cases to continue processing instead of being discarded directly, balancing filtering efficiency and recall. A scaling factor is calculated based on the preset maximum side length of the global analysis while maintaining aspect ratio scaling, avoiding distortion of the defect morphology caused by image deformation. Simultaneously, the preview image size is controlled within the range that the global analysis model can efficiently process, reducing the computational load of the first-stage global search. The scaling factor is also used to establish coordinate mapping parameters, ensuring a unique and accurate coordinate conversion relationship between the preview image and the original image. This guarantees that the low-resolution global analysis results can be accurately mapped back to the high-resolution original image, providing a unified and accurate coordinate reference for subsequent local cropping, multi-source fusion, and image patch construction.

[0114] In some embodiments of this application, the second candidate dataset further includes: local location deterministic values ​​and category evidence; the category evidence is used as the basis for judging the disease category; The process of setting the location startup flag value includes: If the local positioning certainty value is less than the preset positioning certainty threshold, the local candidate area is missing, or the category evidence is incomplete, the positioning activation flag value will be set to the first preset flag value.

[0115] This method optimizes the construction process of bridge appearance image patches by introducing local location certainty values ​​and category evidence, and dynamically setting a location initiation flag value based on this information. Specifically, when the visual semantic analysis model outputs the second candidate dataset, in addition to providing local candidate regions, their disease categories, and categorical location descriptions, it also provides the model's confidence in the local location results (local location certainty value) and the basis for supporting its category judgment (category evidence). When the model's confidence in the local location is low (i.e., the local location certainty value is less than the preset location certainty threshold), or the model fails to successfully detect local candidate regions, or the category evidence provided by the model is insufficient, the location initiation flag value is proactively set to the first preset flag value. This mechanism ensures that in cases where there is uncertainty or incomplete information in the initial analysis, the subsequent text-conditional target localization model can be triggered in a timely manner for more refined and comprehensive localization analysis. In this way, even if the visual semantic analysis model fails to provide highly certain results in some cases, the deficiency can be compensated for by activating a more professional text-conditional target localization model, avoiding the omission of potential disease areas due to early judgment errors, thereby improving the accuracy and recall rate of disease detection.

[0116] For ease of understanding, the following embodiments are also provided: This embodiment uses width Pixels, height Taking a first image of pixels as an example, a preset category set This includes cracks, peeling, exposed reinforcement, honeycomb-like surface, whitening, corrosion, and holes.

[0117] Configure global analysis maximum side length Pixels, effective size of image blocks =640×640 pixels, overlap amount Pixels. Correlation check on the image output. In a high-confidence state, generate a 1024×768 pixel preview image and save the coordinate mapping parameters.

[0118] Phase 1 Global Analysis Output A rough candidate region for cracks and a coarse peeling candidate region Cracks and spalling were written into the first candidate dataset. And associate them with their respective first-stage category evidence.

[0119] To expand the local verification area for coarse crack candidates, crop from the original image. , Including cracks, the second-stage review outputs a local crack candidate and a local location deterministic value. And the location activation flag value, and form a categorized location description. For coarse-grained candidates, Including peeling, the second-stage verification output of valid local candidates with a localization initiation flag value of 0, therefore, text conditional target localization is not invoked, and all are written into the second candidate dataset. .

[0120] Text conditional target localization model receiving Crack type and Output two categorical location candidates and restore them to... Forming a third candidate dataset .

[0121] , and Candidate regions in the original image coordinates are spatially aggregated of the same type; peeling candidates and crack candidates are retained even if they intersect spatially.

[0122] Assuming candidate regions are merged If there are uncovered connected regions with an area exceeding a threshold, a bottom-up scan region is generated in that region, and it is not marked as a crack, peeling or other category.

[0123] The regions to be constructed are determined for the crack fusion spatial reference region, the peeling fusion spatial reference region, and the bottom scan region, respectively.

[0124] Each small region generates a fixed effective size image block of 640×640; for large regions, the number of blocks in the horizontal and vertical directions is calculated, and the blocks are aligned end to end and equidistant in the middle.

[0125] If the candidate path and the bottom scan path generate physical image blocks with the same original image coordinates, then deduplication is performed, one image block file is retained, and the source records of cracks, peeling, and bottom scan paths are merged.

[0126] Additionally, refer to Figure 8 One embodiment of this application provides a bridge appearance image patch construction system, which includes: The image conversion module 1001 is used to determine a second image and coordinate mapping parameters between the first and second images based on the first image when the first image is acquired, wherein the first image is the original appearance image of the bridge to be constructed, and the second image is a preview image of the first image; The first candidate region extraction module 1002 is used to input the second image into the visual semantic analysis model to obtain the first candidate dataset output by the visual semantic analysis model. The first candidate dataset includes an inspectable structural surface region and at least one coarse candidate region. The coarse candidate region is located within or intersects with the inspectable structural surface region and is judged to be a search region that may contain at least one disease category. The second candidate region extraction module 1003 is used to restore the coarse candidate regions to the original coordinate system of the first image based on the coordinate mapping parameters, expand the restored coarse candidate regions to obtain the corresponding local kernel regions, and crop the first image according to the local kernel regions to obtain the third image corresponding to each coarse candidate region. The third image corresponding to each coarse candidate region and the corresponding disease category set are input into the visual semantic analysis model to obtain the second candidate dataset output by the visual semantic analysis model. The second candidate dataset includes local candidate regions, disease categories of local candidate regions, and categorical localization descriptions and localization initiation flag values ​​of disease categories in the disease category set. The third candidate region extraction module 1004 is used to filter out the target disease category whose positioning start flag value is the first preset flag value, and determine the categorized positioning description and third image corresponding to the target disease category. The target disease category and its corresponding categorized positioning description and third image are input into the text conditional target positioning model to obtain the third candidate dataset output by the text conditional target positioning model. The third candidate dataset includes multiple categorized positioning candidate regions. The image patch extraction module 1005 is used to unify the candidate regions in the first candidate dataset, the second candidate dataset, and the third candidate dataset into the original image coordinate system for multi-source candidate fusion to obtain fused candidate regions. Based on the fused candidate regions, the appearance image patch of the bridge to be constructed is determined.

[0127] It should be noted that the system embodiments described above are based on the same inventive concept as the method embodiments described above. Therefore, the relevant content of the method embodiments described above is also applicable to the system embodiments described above, and will not be repeated here.

[0128] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with relevant regulations. The acquisition, storage, use and processing of data in the technical solution of this application all comply with the relevant provisions of national laws and regulations.

[0129] like Figure 9 One embodiment of this application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described method for constructing bridge appearance image blocks. The electronic device includes: At least one memory; At least one processor; At least one program; The program is stored in memory, and the processor executes at least one program to implement a bridge appearance image block construction method according to the above embodiments of this application.

[0130] Electronic devices can be any smart terminal, including mobile phones, tablets, personal digital assistants (PDAs), and in-vehicle computers.

[0131] The electronic devices according to embodiments of this application will now be described in detail.

[0132] The processor 1600 can be implemented using a general-purpose central processing unit (CPU), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application. The memory 1700 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 1700 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program code is stored in the memory 1700 and is called and executed by the processor 1600 to implement a bridge appearance image block construction method according to the embodiments of this application.

[0133] The input / output interface 1800 is used to implement information input and output. The communication interface 1900 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.). Bus 2000 transmits information between various components of the device (e.g., processor 1600, memory 1700, input / output interface 1800, and communication interface 1900); The processor 1600, memory 1700, input / output interface 1800 and communication interface 1900 are connected to each other within the device via bus 2000.

[0134] This application also provides a storage medium, which is a computer-readable storage medium storing computer-executable instructions for causing a computer to execute the above-described bridge appearance image block construction method.

[0135] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0136] The above is a detailed description of the preferred embodiments of this application. However, the embodiments of this application are not limited to the above-described implementation methods. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the embodiments of this application. All such equivalent modifications or substitutions are included within the scope defined by the claims of the embodiments of this application.

Claims

1. A method for constructing bridge appearance image blocks, characterized in that, The method for constructing bridge appearance image blocks includes: In the case of acquiring a first image, a second image and coordinate mapping parameters between the first image and the second image are determined based on the first image, wherein the first image is the original appearance image of the bridge to be constructed, and the second image is a preview image of the first image; The second image is input into a visual semantic analysis model to obtain a first candidate dataset output by the visual semantic analysis model. The first candidate dataset includes an inspectable structural surface region and at least one coarse candidate region. The coarse candidate region is located within or intersects with the inspectable structural surface region and is determined to be a search region that may contain at least one disease category. Based on the coordinate mapping parameters, the coarse candidate regions are restored to the original coordinate system of the first image. The restored coarse candidate regions are expanded to obtain corresponding local kernel regions. The first image is then cropped according to the local kernel regions to obtain a third image corresponding to each coarse candidate region. The third image corresponding to each coarse candidate region and the corresponding disease category set are input into a visual semantic analysis model to obtain a second candidate dataset output by the visual semantic analysis model. The second candidate dataset includes local candidate regions, disease categories of the local candidate regions, and categorical localization descriptions and localization initiation flag values ​​of the disease categories in the disease category set. The target disease category with the location activation flag value as the first preset flag value is selected, and the categorized location description and the third image corresponding to the target disease category are determined. The target disease category, its corresponding categorized location description, and the third image are input into the text conditional target location model to obtain the third candidate dataset output by the text conditional target location model. The third candidate dataset includes multiple categorized location candidate regions. The candidate regions in the first candidate dataset, the second candidate dataset, and the third candidate dataset are unified into the original image coordinate system for multi-source candidate fusion to obtain fused candidate regions. Based on the fused candidate regions, the appearance image blocks of the bridge to be constructed are determined.

2. The method for constructing bridge appearance image blocks according to claim 1, characterized in that, Candidate regions from the first, second, and third candidate datasets are unified to the original image coordinate system for multi-source candidate fusion to obtain fused candidate regions, including: The coarse candidate regions in the first candidate dataset, the local candidate regions in the second candidate dataset, and the candidate regions that pass the validity test in the categorical localization candidate regions in the third candidate dataset are unified into the original image coordinate system to obtain a candidate region set; The candidate regions in the candidate region set are classified according to the disease category, and the spatial overlap between every two candidate regions belonging to the same disease category is calculated. Candidate regions with a spatial overlap greater than or equal to a preset overlap threshold are set in the same type of spatial candidate group to obtain multiple types of spatial candidate groups. Calculate the spatial union of all candidate regions within each of the aforementioned spatial candidate groups, and use the spatial union as the fusion candidate region.

3. The method for constructing bridge appearance image blocks according to claim 1, characterized in that, The step of determining the appearance image patch of the bridge to be constructed based on the fusion candidate region includes: Based on the fusion candidate region and the inspectable structural surface region, the bottom-up scanning region is determined; Based on the fusion candidate region and the bottom scan region, the apparent image block of the bridge to be constructed is determined.

4. The method for constructing bridge appearance image blocks according to claim 3, characterized in that, The step of determining the bottom-up scanning region based on the fusion candidate region and the inspectable structural surface region includes: Based on the fusion candidate regions, determine the covered areas, uncovered areas, and their coverage rates in the inspectable structural surface areas that are covered by the fusion candidate regions; A fallback scan is triggered when the coverage rate is less than a preset coverage threshold, when there is a connected region with an area larger than a preset area threshold in the uncovered region, or when the fusion candidate region is empty.

5. The method for constructing bridge appearance image blocks according to claim 3, characterized in that, The step of determining the apparent image block of the bridge to be constructed based on the fusion candidate region and the fallback scan region includes: Connectivity component analysis is performed on the fusion candidate region and the bottom scan region respectively, and each connected component is converted into an axis-aligned bounding rectangle as the region to be constructed. Based on the preset effective size and preset overlap of the image block, an image block is constructed for each region to be constructed, resulting in multiple initial image blocks; The initial image block is deduplicated to obtain the appearance image block of the bridge to be constructed.

6. The method for constructing bridge appearance image blocks according to claim 1, characterized in that, Determining the second image based on the first image includes: A correlation check is performed on the first image to obtain an image-level correlation quality value; If the image-level correlation quality value is greater than or equal to a preset low confidence threshold, the width and height of the first image are obtained. Based on the preset global analysis maximum side length, the width, and the height, the scaling factor of the preview image is determined; Based on the scaling factor of the preview image, the first image is scaled while maintaining the aspect ratio to obtain the second image, and the coordinate mapping parameters are established based on the scaling factor of the preview image.

7. The method for constructing bridge appearance image blocks according to claim 1, characterized in that, The second candidate dataset also includes: local location deterministic values ​​and category evidence; the category evidence is used to characterize the basis for judging the disease category; The process of setting the location activation flag value includes: If the local location certainty value is less than the preset location certainty threshold, the local candidate region is missing, or the category evidence is incomplete, the location activation flag value is set to the first preset flag value.

8. A bridge appearance image block construction system, characterized in that, The bridge appearance image patch construction system includes: An image conversion module is used to determine a second image and coordinate mapping parameters between the first image and the second image based on the first image, when a first image is acquired, wherein the first image is the original appearance image of the bridge to be constructed, and the second image is a preview image of the first image; The first candidate region extraction module is used to input the second image into the visual semantic analysis model to obtain the first candidate dataset output by the visual semantic analysis model. The first candidate dataset includes an inspectable structural surface region and at least one coarse candidate region. The coarse candidate region is located within or intersects with the inspectable structural surface region and is determined to be a search region that may contain at least one disease category. The second candidate region extraction module is used to restore the coarse candidate region to the original coordinate system of the first image based on the coordinate mapping parameters, expand the restored coarse candidate region to obtain the corresponding local kernel region, and crop the first image according to the local kernel region to obtain the third image corresponding to each coarse candidate region. The third image corresponding to each coarse candidate region and the corresponding disease category set are input into the visual semantic analysis model to obtain the second candidate dataset output by the visual semantic analysis model. The second candidate dataset includes local candidate regions, disease categories of local candidate regions, and categorical localization descriptions and localization initiation flag values ​​of disease categories in the disease category set. The third candidate region extraction module is used to filter out the target disease category whose positioning start flag value is a first preset flag value, and determine the categorized positioning description and the third image corresponding to the target disease category. The target disease category, its corresponding categorized positioning description, and the third image are input into the text conditional target positioning model to obtain the third candidate dataset output by the text conditional target positioning model. The third candidate dataset includes multiple categorized positioning candidate regions. The image patch extraction module is used to unify the candidate regions in the first candidate dataset, the second candidate dataset, and the third candidate dataset into the original image coordinate system for multi-source candidate fusion to obtain fused candidate regions, and to determine the appearance image patch of the bridge to be constructed based on the fused candidate regions.

9. An electronic device, characterized in that, It includes at least one processor and a memory for communicatively connecting with the processor; the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to cause the at least one processor to perform a bridge appearance image block construction method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions for causing a computer to perform a bridge appearance image block construction method as described in any one of claims 1 to 7.

Images