Method for measuring inclination of cultural relic protection building based on prompt segmentation model

Abstract

This invention proposes a method for measuring the tilt of cultural relic protected buildings based on a cue-based segmentation model. The method acquires images, building bounding boxes, internal and external cue points, and camera pose references. Low-resolution candidate mask logic values ​​are obtained through image and cue encoding and mask decoding. Left and right boundary displacements are extracted along the building height direction and decomposed into center offset and width variation components. The joint deviation amplitude, lateral boundary displacement occurrence, and outward or inward expansion direction are determined by combining four types of local boundary displacement references, forming directional boundary displacement coupling features and a signed boundary correction field. Position-by-position correction is performed between the logic value output and the native upsampling end. A building target mask is formed through native upsampling, Sigmoid, and threshold determination. The projected centerline is fitted, and the camera roll attitude is corrected to output the tilt direction and angle.

Description

Technical Field

[0001] This invention relates to the field of image measurement technology for cultural relic protection buildings, and in particular to a method for measuring the tilt of cultural relic protection buildings based on a cueable segmentation model. Background Technology

[0002] Cultural heritage buildings are typically characterized by their age, complex structure, and strict protection restrictions. Their tilt can be affected by uneven foundation settlement, component deformation, and environmental disturbances. Regular tilt monitoring of such buildings requires determining the orientation changes of the main structure from images or other on-site observation data, while minimizing contact with or attachment to measuring components. In actual shooting, building facades are often adjacent to or partially obscured by scaffolding, tree branches, eaves, and the sky background. Reliable extraction of the building's boundary and the correspondence between image coordinates and camera posture become fundamental issues in the image measurement process.

[0003] Among the publicly available building tilt monitoring schemes, some schemes use manually set identifiable targets and combine image geometric relationships to calculate building displacement and tilt angle. Taking CN107917695A "A Building Tilting Monitoring Method Based on Image Recognition Technology" as an example, this scheme sets up multiple concentric circular targets on each floor of the building, uses a drone to collect images of adjacent targets, establishes an image rectangular coordinate system, obtains the target center position based on the imaging law of convex lenses and circle fitting processing, calculates the relative displacement of adjacent targets, and obtains the building tilt angle based on the displacement relationship between the starting and ending targets. Through multiple measurements, the change in building tilt can also be obtained. The measurement object of this scheme is a pre-set target, and the image recognition result is transferred to the tilt angle calculation through the geometric center of the target and the inter-floor displacement.

[0004] The measurement object of this scheme is a pre-set target, which cannot directly process images of cultural relics buildings where it is inconvenient to set targets. When the facade of a cultural relic building is in contact with or obstructed by scaffolding, tree branches or other unilateral attachments, if the center line is directly fitted based on the overall outer contour of the candidate area of ​​the building or the final binary area, the unilateral outward convexity will be regarded as a change in the boundary of the main body of the building, resulting in the center position and tilt direction being pulled. The above-mentioned public scheme does not involve decomposing the displacement of the left and right boundaries and performing orientation correction according to the side of occurrence and the direction of expansion or contraction in the continuous logical value stage of the segmentation model output. Summary of the Invention

[0005] One objective of this invention is to propose a method for measuring the tilt of cultural relic protected buildings based on a cueable segmentation model. This application aims to solve the problem of identifying the convexity of candidate building boundaries caused by unilateral attachments such as scaffolding and tree branches in non-contact image measurement scenarios of cultural relic protected buildings. Before forming the building target mask and the building projection center line, the convexity of the boundary is oriented and corrected to match the internal logical value path of the segmentation model, so as to reduce its pull on the measurement of tilt direction and tilt angle. This problem also requires that the boundary analysis, correction position, correction polarity be consistent with the original upsampling order, and be connected with the measurement results after camera roll attitude correction.

[0006] The tilt measurement method for cultural relic protection buildings based on a promptable segmentation model according to an embodiment of the present invention includes:

[0007] S1. Acquire the target image containing the cultural relic under protection, architectural prompt information, and the camera attitude reference corresponding to the target image to form the tilt measurement input;

[0008] S2. Input the target image and building prompt information from the tilt measurement input into the prompt segmentation model. After image encoding, prompt encoding and mask decoding, obtain the low-resolution candidate mask logic value.

[0009] S3. The candidate regions represented by the logic values ​​of the low-resolution candidate mask are layered along the building height direction. The left and right boundary displacements of adjacent height layers are extracted. The center offset component is calculated based on the same-direction part of the left and right boundary displacements. The width change component is calculated based on the opposite-direction part of the left and right boundary displacements. Based on the joint deviation of the center offset component and the width change component relative to the local reference of the boundary displacement, the side of the boundary displacement, and the outward or inward direction of the boundary corresponding to the side of the boundary displacement, a directional boundary displacement coupling feature is formed. The correction amount, the position of the boundary to be corrected, and the correction polarity are determined based on the directional boundary displacement coupling feature. A signed boundary correction field of the same scale as the logic value of the low-resolution candidate mask is generated in the neighborhood of the boundary to be corrected. The signed boundary correction field is superimposed on the logic value of the low-resolution candidate mask between the output end of the mask decoder logic value and the native upsampling end of the prompting segmentation model to obtain the corrected mask logic value.

[0010] S4. Perform native upsampling and threshold determination on the correction mask logic value through the native upsampling end of the prompting segmentation model to form the architectural target mask of the cultural relic protection building to be tested.

[0011] S5. Extract multiple layered center points along the building height direction within the building target mask, and perform straight line fitting on the multiple layered center points to obtain the building projection center line. Determine the building projection tilt angle based on the building projection center line.

[0012] S6. Based on the camera attitude reference, perform roll attitude correction on the building projection tilt angle to determine the tilt direction and tilt angle of the cultural relic under test, and generate the tilt measurement results of the cultural relic under test.

[0013] Optionally, S1 specifically refers to:

[0014] Acquire a target image containing the cultural relic under protection;

[0015] In the target image, the building region bounding the cultural relic to be protected is determined, and the internal and external point prompts of the building are determined separately. The building prompt information is formed by the building region bounding, the internal and external point prompts of the building.

[0016] The camera roll attitude around the optical axis is acquired during target image acquisition, and the camera attitude reference corresponding to the target image is determined based on the roll attitude.

[0017] Establish a correspondence between the target image, building prompts, and camera attitude references to form the tilt measurement input.

[0018] Optionally, S2 specifically refers to:

[0019] Image patch projection and position encoding are performed on the target image. The resulting image patch features are then input into a multi-layer encoding structure containing a window attention encoding layer and a global attention encoding layer to form image encoding features that preserve the spatial positional relationships of buildings.

[0020] The coordinate position encoding and prompt type encoding of the building area bounding box, the internal point prompts and the external point prompts in the building prompt information are respectively performed to form prompt coding features corresponding to the image coding features;

[0021] Image encoding features and cue encoding features are input into the mask decoder. Through cue feature self-attention, cue feature to image feature cross-attention, and image feature to cue feature cross-attention, updated image features and mask label features are formed.

[0022] The updated image features are upsampled step by step, the mask label features are mapped to mask projection vectors, and the mask projection vectors are multiplied by the upsampled updated image features to obtain low-resolution candidate mask logical values.

[0023] Optionally, S3 specifically refers to:

[0024] Sigmoid operation and temporary foreground determination are performed on the logical values ​​of low-resolution candidate masks. Multiple adjacent height layers are divided along the building height direction, and the left and right boundary positions are determined based on the leftmost and rightmost foreground positions in each height layer.

[0025] The left boundary displacement is obtained by subtracting the left boundary positions of adjacent height layers, and the right boundary displacement is obtained by subtracting the right boundary positions of adjacent height layers.

[0026] Add the left boundary displacement and the right boundary displacement and take half to obtain the center offset component. Subtract the left boundary displacement from the right boundary displacement and take half to obtain the width variation component. Calculate the deviation of the center offset component and the width variation component from the local boundary displacement reference. Based on the deviation, calculate the joint deviation amplitude, determine the side where the boundary displacement occurs, and the outward or inward direction of the boundary corresponding to the side where the boundary displacement occurs, thus forming a directional boundary displacement coupling feature.

[0027] The correction amount, the position of the boundary to be corrected, and the correction polarity are determined based on the directional boundary displacement coupling characteristics. The correction value with the correction polarity is written into the neighborhood of the boundary to be corrected according to the position of the boundary to be corrected, generating a signed boundary correction field with the same scale as the low-resolution candidate mask logic value.

[0028] Between the output of the mask decoder logic value and the native upsampling end of the cue segmentation model, the signed boundary correction field is added position by position to the low-resolution candidate mask logic value to obtain the corrected mask logic value.

[0029] Optionally, S4 specifically refers to:

[0030] Input the correction mask logic value into the native upsampling end of the prompting segmentation model, and map the correction mask logic value to a high-resolution mask logic value that is consistent with the resolution of the target image according to the native upsampling path of the prompting segmentation model.

[0031] Perform a Sigmoid operation on the high-resolution mask logic value to obtain the building foreground probability corresponding to each image location;

[0032] The probability of building foreground is compared with a preset foreground determination threshold. The image positions where the probability of building foreground reaches the preset foreground determination threshold are determined as building foreground, and the remaining image positions are determined as building background, thus forming a building target mask.

[0033] Optionally, S5 specifically refers to:

[0034] The building target mask is divided into multiple measurement layers along the building height direction, and the left and right boundary x-coordinates of the building target mask are extracted in each measurement layer;

[0035] Add the left boundary x-coordinate and the right boundary x-coordinate of each measurement layer and take half of the result to obtain the layer center point of the corresponding measurement layer, and establish the correspondence between the layer center point and the height coordinate of the measurement layer.

[0036] The building projection centerline is obtained by fitting a straight line to multiple layer center points and their corresponding measured layer height coordinates.

[0037] The tilt angle of the building projection is determined based on the angle between the building projection centerline and the vertical direction of the image.

[0038] Optionally, S6 specifically refers to:

[0039] Extract the roll angle reference of the camera around the optical axis when the target image is acquired from the camera attitude reference, and determine the vertical direction reference of the target image;

[0040] The difference between the building projection tilt angle and the roll angle reference is calculated to obtain the building tilt angle after roll attitude correction.

[0041] The tilt direction of the cultural relic under test is determined by the sign of the tilt angle of the building after roll attitude correction, and the tilt angle of the cultural relic under test is determined by the absolute value of the tilt angle of the building after roll attitude correction.

[0042] By combining the tilt direction and tilt angle, the tilt measurement results of the cultural relic under protection are generated.

[0043] Optionally, for each height layer with adjacent height layers, the median value of the center offset component and the width variation component of the adjacent height layers is taken to obtain the local reference for the center offset component and the local reference for the width variation component. The median value of the left boundary displacement and the right boundary displacement of the adjacent height layers is taken to obtain the local reference for the left boundary displacement and the local reference for the right boundary displacement. The deviation amplitude between the center offset component and the local reference for the center offset component and the deviation amplitude between the width variation component and the local reference for the width variation component are combined to obtain the joint deviation amplitude. The deviation amplitude of the left boundary displacement relative to the local reference for the left boundary displacement and the deviation amplitude of the right boundary displacement relative to the local reference for the right boundary displacement are calculated to obtain the left boundary deviation amplitude and the right boundary deviation amplitude. The boundary displacement occurrence side when the left boundary deviation amplitude is greater than the right boundary deviation amplitude is determined as the left boundary. The boundary displacement occurrence side when the right boundary deviation amplitude is greater than the left boundary deviation amplitude is determined as the right boundary. The boundary displacement occurrence side when the left boundary deviation amplitude and the right boundary deviation amplitude are the same is determined as the common occurrence of the left and right boundaries.

[0044] Optionally, before superimposing the signed boundary correction field with the low-resolution candidate mask logic value, the Sigmoid operation and threshold determination for forming the building target mask are not performed on the low-resolution candidate mask logic value. Instead, the signed boundary correction field and the low-resolution candidate mask logic value are added position by position to form the correction mask logic value. The correction mask logic value is then directly input into the native upsampling end of the prompting segmentation model. The native upsampling end of the prompting segmentation model performs native upsampling, Sigmoid operation for forming the building target mask, and threshold determination on the correction mask logic value.

[0045] Optionally, the low-resolution candidate mask logic value output by the mask decoder logic value output terminal is transmitted along the mask logic value transmission path between the mask decoder logic value output terminal and the native upsampling terminal of the cueing segmentation model. Based on the candidate region boundary displacement represented by the low-resolution candidate mask logic value, a directional boundary displacement coupling feature is formed. Based on the directional boundary displacement coupling feature, a signed boundary correction field is generated. In the mask logic value transmission path, the signed boundary correction field is superimposed on the low-resolution candidate mask logic value to form a corrected mask logic value. The corrected mask logic value is then input into the native upsampling terminal of the cueing segmentation model along the native upsampling path of the cueing segmentation model.

[0046] The beneficial effects of this invention are:

[0047] (1) This invention proposes an improved candidate building boundary displacement analysis method. The temporary foreground region obtained by converting the logic value of the low-resolution candidate mask is layered along the building height direction. The left and right boundary displacements of adjacent height layers are extracted. The same-direction part of the left and right displacements forms the center offset component, and the opposite part forms the width change component. Local median benchmarks are established for the center offset, width change, left boundary displacement and right boundary displacement respectively. The deviation of each component relative to the corresponding benchmark is used to form the joint deviation amplitude. Combined with the side of the boundary displacement and the outward or inward direction of the corresponding boundary, a directional boundary displacement coupling feature is formed. This processing separates the center movement caused by the overall tilt of the building, the width change of the building itself and the local anomaly caused by the attachment on one side, so that the correction boundary and correction direction have scene basis. The generated signed boundary correction field can reduce the foreground logic value at the outward expansion position and increase the foreground logic value at the inward contraction position.

[0048] (2) This invention proposes a directional correction method that embeds the transmission path of the logic value of the model mask that can prompt segmentation. The signed boundary correction field is not externally clipped on the final binary mask, but is added position by position between the logic value output end of the mask decoder and the original upsampling end, and the low-resolution candidate mask logic value is added to form the correction mask logic value. The temporary sigmoid and temporary threshold are only used to extract the candidate region boundary. After the correction is completed, the continuous logic value is still processed along the original upsampling path of the model, and then the final sigmoid and threshold judgment is performed. Thus, the scene information obtained by the boundary displacement analysis directly changes the continuous input of the subsequent processing, retains the strength and spatial continuity of the boundary response, and fixes the correction position at the unique embedding point of the logic value path inside the model, avoiding the loss of boundary details caused by repair after binarization.

[0049] (3) This invention proposes a segmentation-geometry-attitude correction method for tilt measurement of cultural heritage buildings. The corrected building target mask is divided into measurement layers along the building height direction. The left and right boundary abscissas of the foreground of each layer are used to determine the layer center point. The height coordinate is used as the independent variable and the abscissa of the center point is used as the dependent variable to perform straight line fitting to obtain the building projection center line and the building projection tilt angle. Then, the projection tilt angle is compared with the camera roll angle reference corresponding to the acquired image in the same direction to output the tilt direction and tilt angle. This process connects prompt segmentation, single-sided boundary failure identification, continuous logic value correction, center line fitting and camera attitude compensation into a closed measurement chain, which can be used for non-contact image measurement of cultural heritage buildings adjacent to scaffolding, tree branches and other attachments. Attached Figure Description

[0050] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0051] Figure 1 This is a flowchart of a method for measuring the tilt of a cultural relic protection building based on a promptable segmentation model, as proposed in this invention.

[0052] Figure 2 A three-dimensional distribution diagram of overall performance;

[0053] Figure 3 For the joint deviation correction response surface plot;

[0054] Figure 4 The surface plot shows the coupling error of the boundary components;

[0055] Figure 5 Detailed diagram of the encoder and mask decoder;

[0056] Figure 6Comparison diagrams showing the correction of bulging building boundaries caused by unilateral attachment of tree branches;

[0057] Figure 7 A comparison diagram showing the correction of the layer center point and the building projection center line;

[0058] Figure 8 This is a schematic diagram of the tower body segmentation results;

[0059] Figure 9 This is a schematic diagram for tilt angle recognition. Detailed Implementation

[0060] In Example 1, reference Figures 1 to 9 A method for measuring the tilt of a protected historical building based on a suggestible segmentation model, comprising:

[0061] S1. Acquire the target image containing the cultural relic under protection, architectural prompt information, and the camera attitude reference corresponding to the target image to form the tilt measurement input;

[0062] S2. Input the target image and building prompt information from the tilt measurement input into the prompt segmentation model. After image encoding, prompt encoding and mask decoding, obtain the low-resolution candidate mask logic value.

[0063] S3. The candidate regions represented by the logic values ​​of the low-resolution candidate mask are layered along the building height direction. The left and right boundary displacements of adjacent height layers are extracted. The center offset component is calculated based on the same-direction part of the left and right boundary displacements. The width change component is calculated based on the opposite-direction part of the left and right boundary displacements. Based on the joint deviation of the center offset component and the width change component relative to the local reference of the boundary displacement, the side of the boundary displacement, and the outward or inward direction of the boundary corresponding to the side of the boundary displacement, a directional boundary displacement coupling feature is formed. The correction amount, the position of the boundary to be corrected, and the correction polarity are determined based on the directional boundary displacement coupling feature. A signed boundary correction field of the same scale as the logic value of the low-resolution candidate mask is generated in the neighborhood of the boundary to be corrected. The signed boundary correction field is superimposed on the logic value of the low-resolution candidate mask between the output end of the mask decoder logic value and the native upsampling end of the prompting segmentation model to obtain the corrected mask logic value.

[0064] S4. Perform native upsampling and threshold determination on the correction mask logic value through the native upsampling end of the prompting segmentation model to form the architectural target mask of the cultural relic protection building to be tested.

[0065] S5. Extract multiple layered center points along the building height direction within the building target mask, and perform straight line fitting on the multiple layered center points to obtain the building projection center line. Determine the building projection tilt angle based on the building projection center line.

[0066] S6. Based on the camera attitude reference, perform roll attitude correction on the building projection tilt angle to determine the tilt direction and tilt angle of the cultural relic under test, and generate the tilt measurement results of the cultural relic under test.

[0067] In this embodiment, step S1 specifically includes:

[0068] Each image acquisition record is treated as a processing unit, and the acquisition record index is used. The identification process involves capturing color images of the protected historical building using a camera, and then... The target image is denoted as ,in Indicates the image height. The image width is represented by 3, and the three color channels (red, green, and blue) are represented by 3. The top left corner of the image is used as the origin of the coordinate system. The horizontal coordinate increases along the width of the image, and the vertical coordinate increases along the height of the image. (Target image) Simultaneously record the data collection timestamp The building height direction is defined as from the bottom of the building to the top, which is opposite to the increasing direction of the vertical axis of the image;

[0069] In the target image The architectural area surrounding the building to be measured is defined in the diagram, and this architectural area is denoted as [reference needed]. ,in and These represent the x-coordinates of the left and right boundaries of the building area frame, respectively. and Let represent the ordinates of the upper and lower boundaries of the building area bounding box, respectively. The building area bounding box covers the visible main body of the cultural relic under test. When scaffolding or tree branches are in contact with the main body of the building, the scaffolding or tree branches can be located inside the building area bounding box. Based on the confirmed location of the main body of the building in the target image, the internal points of the building are determined and represented as . Each row contains the x and y coordinates of a point inside the building, located within the foreground area of ​​the tower, wall, or eaves. External building point cues are determined based on confirmed non-building locations in the target image and represented as... Each row contains the x and y coordinates of a point outside the building, located within scaffolding, tree branches, sky, ground, or other background areas, bounded by the building area. Building interior point prompts External points of the building Building prompts Building information The coordinates of the bounding box, interior points, and exterior points in the image are all taken from the target image. coordinate system;

[0070] According to the input size of the promptable segmentation model, the target image is segmented... Scale and pad to form the model input image. It also records the coordinate transformation from the original image coordinates to the model input coordinates. Transform coordinates Synchronous action on building area frame Building interior point prompts External points of the building Each of these forms a building area bounding box in the model input coordinate system. Building interior point prompts External points of the building The coordinate transformation results in the location of the prompts remaining within the valid coordinate range of the model input image. Prompt locations outside the valid range are not written into the model input prompt sequence. , and Formation and model input image Corresponding building information ;

[0071] The camera attitude record is read from the camera's built-in inertial measurement unit to obtain a timestamped roll attitude sequence. ,in Indicates the first The timestamp of the attitude record Indicates the first Camera roll angle in attitude recording, The total number of attitude records is represented by the target image acquisition timestamp. To match the baseline, select the appropriate value from the attitude recording sequence. The absolute value of the time difference is the smallest and does not exceed the preset attitude matching window. The attitude record is used to determine the corresponding roll angle as the target image. Roll angle reference Pose matching window The camera pose sampling period and image timestamp accuracy are preset, and in this embodiment, they are fixed at half of the camera pose sampling period. The increasing direction of the image's vertical coordinate is determined as the vertical reference of the image, and the angle units and positive and negative directions of the roll angle and image tilt angle are unified. The camera pose reference is formed by the vertical reference of the image. ;

[0072] Input the model into the image. Building prompts under the input coordinates of the model Input can prompt the segmentation model, and set the camera pose reference. Index of collected records Establish the correspondence and save it along the tilt measurement process, so that the actual input received by the segmentation model can be the model input image. and building information This makes the attitude data used for tilt angle correction the same as the model input image. Corresponding camera attitude reference .

[0073] In this embodiment, step S2 specifically includes:

[0074] The target image after coordinate transformation and building information The input can prompt the segmentation model, where, for Color image tensor, Including building region bounding boxes, interior building point cues, and exterior building point cues, the image encoder first processes... Image patch projection is performed using Conv2D with a kernel size of 16×16, a stride of 16, 3 input channels, and 768 output channels. This divides the input image into 64×64 image patches, forming 64×64×768 image patch features. Reshaping these image patch features flattens the 64×64 spatial locations into 4096 image tokens, resulting in an image patch feature sequence. Two-dimensional positional encoding is performed on the image block feature sequence. The two-dimensional positional encoding is generated according to the horizontal and vertical spatial positions of the image block, and then Add is performed position by position with the image block feature sequence to obtain image block features with attached building spatial positional relationships.

[0075] The image patch features after location encoding are input into a multi-layer encoding structure containing a window attention encoding layer and a global attention encoding layer. The window attention encoding layer divides the image token into local image windows and extracts local texture relationships between building edges, eaves, walls, and scaffolding or tree branches within the local windows. The global attention encoding layer establishes associations between all image tokens and extracts spatial relationships between different height regions of the building. The window attention encoding layer and the global attention encoding layer are different encoding layers of the same image encoder and do not constitute two independent model stages. Each encoding layer first performs LayerNorm, then generates queries, keys, and values ​​through Linear, performs Transpose on the keys, and performs MatMul with the queries to obtain the relevance. The correlation is scaled according to the feature dimension of a single attention head, and the attention weights are obtained through Softmax. The attention weights and values ​​are then processed by MatMul to obtain the attention output. The outputs of multiple attention heads are processed by Concat and Linear mapping, and then residuals are added with the input of the encoding layer. The attention update features obtained by residuals are then processed by LayerNorm again, and then passed through feedforward Linear, GELU and feedforward Linear in sequence. The feedforward output is then processed by residuals added with the attention update features to form the output features of the encoding layer. The image encoder has an image feature dimension of 768, and there are 12 image attention heads. Each attention head has 64-dimensional features. The feedforward network maps the 768-dimensional features to 3072-dimensional features, and then maps them back to 768-dimensional features.

[0076] The multi-layer coding structure outputs 64×64×768 image coding features. After adjusting the channel dimensions using Transpose, the features are sequentially passed through Conv2D with a 1×1 kernel, LayerNorm2D, Conv2D with a 3×3 kernel and a stride of 1, and LayerNorm2D, reducing the number of image feature channels from 768 to 256, thus forming the image coding features. LayerNorm normalizes along the feature dimension of a single token. LayerNorm2D normalizes 256 channel features at each spatial location. The normalization process does not change the image feature space size.

[0077] Building prompts The building bounding box, interior point cues, and exterior point cues are input into a cue encoder. The building bounding box is encoded using the coordinates of its top-left and bottom-right corners, while the interior and exterior point cues are encoded using point coordinates. Each cue coordinate is normalized and linearly mapped. The mapping results are then input into Sin and Cos functions, respectively. The sine and cosine position features are then concatenated to form a 256-dimensional coordinate position feature. Corner point type codes are superimposed on the top-left and bottom-right corner position features of the building bounding box, and interior point type codes are superimposed on the interior point cues. Similarly, exterior point cues are superimposed on the exterior point cues. Finally, the cue tokens corresponding to the corner points, interior points, and exterior points are concatenated along the sequence dimension to form sparse cue features. The sparse cue sequence is organized according to a preset length of 20. When the actual number of valid cue tokens is less than 20, it is padded with preset empty cue tokens. When no masked cue is input, a pre-set 256-dimensional maskless embedding is used, and the maskless embedding is expanded to 64×64 spatial positions to form dense cue features. ;

[0078] Encoding features of images With dense cue features The process involves position-by-position addition to generate a 64×64×256 fused image feature, which is then reshaped into a 4096×256 image feature sequence. The mask decoder constructs one intersection-over-union (IoU) token, three mask tokens, and 20 sparse cue tokens. These tokens are then concatenated along the sequence dimension to form a 24×256 decoded token. The mask decoder uses a bidirectional Transformer to process the decoded token and the fused image features. The decoded token first undergoes LayerNorm and token self-attention, then a Linear algorithm is used to generate queries, keys, and values, before performing a Transpose algorithm. e, MatMul, scaling, Softmax, attention weights and values, MatMul, Concat, Linear, and residual Add are used to form the updated decoding token. The updated decoding token is used as the query, and the 4096×256 fused image features are used as the key and value to perform cross-attention from the cue features to the image features, obtaining the cross-attention update amount. The updated decoding token is then updated by LayerNorm, Linear, GELU, Linear, and residual Add to complete the feedforward update. The fused image features are used as the query, and the updated decoding token is used as the key and value to perform cross-attention from the image features to the cue features, forming the updated image features. And form a 3×256 mask marking feature. ;

[0079] Update image features A progressive upsampling process is performed. First, ConvTranspose2D maps the 256 channels to 64 channels and adjusts the spatial size to 128×128. Then, LayerNorm2D and GELU are applied, followed by ConvTranspose2D mapping the 64 channels to 32 channels and adjusting the spatial size to 256×256. Finally, LayerNorm2D and GELU are applied again to form a 256×256×32 native mask feature. The three mask marker features are then processed by Linear, GELU, Linear, GELU, and Linear respectively, mapping the 256-dimensional feature to a 32-dimensional mask projection vector. The mask projection vector corresponding to the building cue is determined based on the candidate quality response formed by the intersection-union ratio (IU) token. The original mask features are reshaped to 65536×32, and the transpose of the mask projection vector and the original mask features is subjected to MatMul to obtain continuous logical values ​​corresponding to 65536 spatial locations. The MatMul result is then reshaped to 256×256×1 to form low-resolution candidate mask logical values. The low-resolution candidate mask logic value is a continuous logic value generated at the output of the mask decoder, which preserves the response strength and spatial continuity of the building boundary location. The final Sigmoid and threshold determination for forming the building target mask has not yet been performed.

[0080] In this embodiment, step S3 specifically includes:

[0081] The low-resolution candidate mask logic value output by the mask decoder is denoted as... ,right Perform a Sigmoid function on each spatial location to obtain temporary foreground probabilities, and then compare these temporary foreground probabilities with a pre-set temporary foreground determination threshold. Perform GreaterEqual to form candidate foreground regions. In the embodiments, Set the threshold to 0.5 and apply the same threshold across all height layers for the temporary foreground region. This is only used for boundary location extraction; the final Sigmoid and threshold determination used for building target mask formation are not performed at this location.

[0082] Along candidate foreground region The 256 image rows were divided according to the building height direction. A continuous height layer, Let be a preset integer not less than 4, and each height layer corresponds to a continuous image row, denoted as the th . The image row set of each height layer is ,in Extract the content that satisfies the requirements of each height layer. Given a set of foreground x-coordinates, use ReduceMin to obtain the left boundary position. Use ReduceMax to get the right boundary position. When a height layer does not contain foreground pixels, it is determined as an invalid height layer. Invalid height layers do not participate in boundary displacement, median, and local reference processing. Sub is performed on the left boundary position of the adjacent valid height layers to form a left boundary displacement sequence. Sub is applied to the right boundary position of adjacent effective height layers to form a right boundary displacement sequence. Each height layer is arranged according to the building height direction from the bottom to the top of the building. The left and right boundary displacements are obtained by subtracting the adjacent lower effective height layer boundary position from the higher effective height layer boundary position in the building height direction.

[0083] Add the left and right boundary displacements of adjacent height layer intervals, then multiply the Add result by 0.5 to form the center offset component. The center offset component represents the overall lateral movement of the candidate building area corresponding to the same direction of change in the left and right boundaries. The width change component is obtained by subtracting the left boundary displacement from the right boundary displacement of the same adjacent height level interval, and then multiplying the Sub result by 0.5. The width variation component represents the width variation of the candidate building area corresponding to the opposite changes of the left and right boundaries. The tilt of the actual building is manifested as the continuous change of the center offset component of multiple height layers. The tower body tapering and the eaves cantilever are manifested as the structural changes of the width variation component. The unilateral attachment interference caused by scaffolding or tree branches is manifested as the unilateral abnormal change of the left or right boundary displacement.

[0084] For the current height layer interval with effective adjacent height layers before and after, Median is performed on the center offset components on the front and rear sides respectively to form a local reference for the center offset components. Median is applied to the width variation components on the front and rear sides respectively to form local references for the width variation components. Median is applied to the left boundary displacements on both the front and rear sides to form a local reference for the left boundary displacement. Median is applied to the right boundary displacements on both the front and rear sides to form a local reference for the right boundary displacement. The center offset component and its local reference are subjected to Sub to form the center offset component deviation. The width variation component and its local reference are subjected to Sub to form the width variation component deviation. The two deviations are then subjected to Square, Add, and Sqrt respectively to form the joint deviation magnitude. The combined deviation magnitude represents the overall degree of deviation of the current height layer interval from the local boundary change pattern;

[0085] The left boundary displacement and the local reference for the left boundary displacement are subjected to Sub and Abs operations to form the left boundary deviation magnitude. The right boundary displacement and the local reference of the right boundary displacement are subjected to Sub and Abs operations to form the right boundary deviation magnitude. ,Compare and The boundary displacement occurrence sides are determined as follows: a larger deviation on the left boundary is identified as the left boundary, a larger deviation on the right boundary is identified as the right boundary, and the two deviations are equal, indicating a simultaneous occurrence on both the left and right boundaries. The expansion or contraction direction is determined based on the residual direction of the boundary displacement relative to the corresponding local datum: a decrease in the left boundary residual is identified as expansion, and an increase in the left boundary residual is identified as contraction; similarly, an increase in the right boundary residual is identified as expansion, and a decrease in the right boundary residual is identified as contraction. The combined deviation magnitude, boundary displacement occurrence sides, and expansion or contraction direction constitute a directional boundary displacement coupling characteristic. ,in This indicates that the left boundary, right boundary, or both boundaries occur simultaneously. Indicates the direction of expansion or contraction;

[0086] For the selected boundary to be corrected, at the height layer Grid position within Generate signed boundary correction values :

[0087] ;

[0088] In the formula, The correction value represents the location of the grid cells in the neighborhood of the boundary to be corrected. Indicates the side of the boundary to be corrected. Indicates the image acquisition record index. Indicates the index of adjacent height levels. Represents the ordinate in a low-resolution logical value grid. Represents the x-coordinate in a low-resolution logical value grid. Indicates the center offset component. Indicates the local reference for the center offset component. Indicates the width variation component. Indicates the local reference for the width variation component. This represents the proportionality coefficient between the pixel displacement deviation and the logic value correction range, pre-calibrated according to the logic value range and allowable correction range of the low-resolution candidate mask. This indicates the maximum allowable correction range for low-resolution logic values. This indicates that the correction range will be limited to 0 to... Within the range, This indicates the correction polarity; negative values ​​are used in the outward expansion direction, and positive values ​​are used in the inward contraction direction. This represents the x-coordinate of the boundary to be corrected, determined by the position of the left or right boundary. This represents the pre-defined positive integer boundary neighborhood half-width, in low-resolution grid pixels. This represents the lateral distance between the grid position and the boundary to be corrected. This means truncating negative weights outside the boundary neighborhood to zero. , , and All values ​​are expressed in pixel displacement units, and the results of the square root of the sum of squares are also expressed in pixel displacement units. The ratio of logical value units to pixel displacement units is used. and Use logical value units. Spatial decay weights are dimensionless;

[0089] The generated correction values ​​are written into a 256×256 spatial grid using Scatter to form a signed boundary correction field. When both left and right boundaries occur simultaneously, left and right boundary correction fields are generated separately, and an Add operation is performed at the same spatial location. Positive values ​​in the signed boundary correction field are used to increase the logical value of the building foreground, while negative values ​​are used to decrease the logical value of the building foreground at the location corresponding to a single-sided attachment. In the mask logic value transmission path between the mask decoder's logic value output and the native upsampling end of the cueable segmentation model, the... and Execute Add position by position to form the correction mask logic value. The correction mask logic value is then directly input into the native upsampling end of the segmentation model, which can provide hints.

[0090] In this embodiment, step S4 specifically includes:

[0091] The correction mask logic value formed by adding the signed boundary correction field and the low-resolution candidate mask logic value position by position is denoted as... Correct mask logic values Preserve the spatial coordinates and channel organization of low-resolution candidate mask logic values; store continuous logic values ​​after boundary correction at a single spatial location. Direct input can prompt the native upsampling end of the segmentation model; the input objects only include the correction mask logic values.

[0092] The example uses 4x bilinear upsampling for high-resolution image location. According to the pixel center coordinate rules, the high-resolution position is reverse-mapped to the low-resolution logical value grid. The vertical and horizontal coordinates are divided by 4 respectively, and then the pixel center is offset to obtain the low-resolution continuous coordinates. The low-resolution continuous coordinates are then subjected to a floor operation to obtain the integer coordinates. Then subtract the corresponding integer coordinate from the continuous coordinates to obtain the decimal coordinates. For high-resolution locations To form high-resolution mask logic values :

[0093] ;

[0094] In the formula, Represents high-resolution location The mask logic value, Indicates the image acquisition record index. Indicates the vertical position of the high-resolution mask logic value. Indicates the horizontal position of the high-resolution mask logic value. Represents low-resolution vertical integer coordinates. Represents low-resolution horizontal integer coordinates. Represents low-resolution vertical decimal coordinates. Represents low-resolution horizontal decimal coordinates. and The values ​​are 0 or 1, used to select the four low-resolution grid positions surrounding the continuous coordinates. This represents the correction mask logic value corresponding to the low-resolution grid position. The summation symbol indicates that a weighted summation is performed on four adjacent low-resolution grid positions. The low-resolution grid boundary positions are handled using a boundary-preserving method. Grid indices outside the range of 0 to 255 are fixed as the nearest boundary indices. All four interpolation weights are dimensionless values, and the sum of the four weights is 1. and All are mask logic values ​​with the same logic value dimensions. Bilinear interpolation does not change the positive or negative direction of the boundary correction value.

[0095] All high-resolution locations Composition of high-resolution mask logic values ,right Perform the final Sigmoid function on a spatial location-by-spatial basis, converting the high-resolution mask logic values ​​into building foreground probabilities. The foreground determination threshold used to form the building target mask is denoted as The threshold is fixed before the model is deployed. In the example, it will be... Setting it to 0.5 will increase the probability of the building being in the foreground. and Perform a GreaterEqual comparison; locations that reach the threshold are marked as building foreground, and locations below the threshold are marked as building background, thus forming a building target mask. The signed boundary correction field is first applied to the low-resolution continuous logic value. The correction mask logic value is then subjected to native upsampling, final Sigmoid, and final threshold determination. The temporary Sigmoid is only used for boundary localization of the low-resolution candidate region. The 1024×1024 building target mask formed here corresponds to the resolution of the model input image. When the image height and image width of the original target image are not 1024×1024, the filling area is removed according to the inverse correspondence of the coordinate transformation from the original image coordinates to the model input coordinates. The building target mask is then mapped to the image height and image width of the original target image. During mapping, the binary attribute of building foreground as 1 and building background as 0 is maintained.

[0096] In this embodiment, step S5 specifically includes:

[0097] Let the building target mask formed after native upsampling, final Sigmoid function, and final threshold determination be denoted as . In the building target mask, 1 represents the building foreground and 0 represents the building background. The image is divided along the vertical axis of the building target mask. A continuous measurement layer, The measurement layers are preset integers, arranged from the bottom to the top of the building, and denoted as the [number]th layer. The image row set of each measurement layer is ,in The measured floor height coordinates are obtained by subtracting the center ordinate of the corresponding image row from the ordinate of the bottom image of the building, so that the measured floor height coordinates increase along the building height direction from the bottom to the top of the building;

[0098] For the Each measurement layer extracts the required parameters. and The set of foreground x-coordinates of the building is used to extract the minimum x-coordinate in the set using ReduceMin, which forms the x-coordinate of the left boundary of the measurement layer. The maximum x-coordinate in the collection is extracted using ReduceMax to form the x-coordinate of the right boundary of the measurement layer. When a measurement layer does not contain building foreground pixels, it is marked as an invalid measurement layer. Invalid measurement layers do not generate boundary x-coordinates and do not participate in layer center point and line fitting. Valid measurement layers with left and right boundary x-coordinates are retained and rearranged according to the measurement layer height coordinates. The number of valid measurement layers is denoted as [missing information]. The number of effective measurement layers used for linear fitting should be no less than two;

[0099] Add the left and right boundary x-coordinates of each valid measurement layer, then multiply the result by 0.5 to obtain the x-coordinate of the layer center point. The x-coordinate of the layer center point and the corresponding measurement layer height coordinates are used to... Composition of layer center point , layer center point The vector is a two-dimensional vector. The first dimension represents the horizontal center position of the building on the current measurement floor, and the second dimension represents the vertical position of the building on the current measurement floor. The center points of the effective measurement floors are arranged into a center point sequence according to the building height. ;

[0100] The input for line fitting is established based on the center point sequence, and the height coordinates and constant terms of each layer center point are used to form the design matrix. The design matrix of the first Travel Composed of 1, the target vector is formed by combining the x-coordinates of the layer center points. The target vector is the first The elements are Let the line parameter vector be denoted as ,in Indicates the slope of the building's projected centerline. The intercept of the building projection centerline is represented by the line parameter determined using least squares matrix operations to minimize the sum of squared residuals between the x-coordinates of the layer center points and the predicted x-coordinates of the line. This is achieved when the design matrix... When the column rank is full, the normal equation is used to solve for the line parameters. When designing the matrix... When the column rank is not full, singular value decomposition is performed on the design matrix, and the inverse of the non-zero singular values ​​is taken to form the matrix pseudo-inverse. Then, the matrix pseudo-inverse is used to solve for the linear parameters.

[0101] The building projection centerline is formed based on the straight line parameters. The building projection centerline represents the relationship between the layer center position of the building target mask and the building height coordinates, and the slope of the building projection centerline. This represents the rate of lateral change of the building's center position as the measured floor height coordinates increase. When the horizontal coordinate of the image increases to the right and the measured floor height coordinates increase along the building's height from bottom to top, a positive slope indicates a change in the centerline towards increasing horizontal coordinates, while a negative slope indicates a change in the centerline towards decreasing horizontal coordinates. The slope of the projected building centerline is... Inputting Arctan yields the angle between the building's projection centerline and the vertical direction of the image, forming the building's projection tilt angle. Building projection tilt angle This indicates the tilt state of the building projection in the image coordinates, and has a different meaning from the building tilt angle after camera roll attitude correction.

[0102] In this embodiment, step S6 specifically includes:

[0103] The input data includes only the building projection tilt angle and the camera attitude reference bound to the target image. The camera attitude reference includes the roll angle reference of the camera around the optical axis at the time of target image acquisition and the vertical direction reference of the image. The roll angle reference is an angle value that can be read from the output of the camera inertial measurement unit, the camera attitude calculation record, or the target image acquisition metadata. A correspondence is established between the target image number or the acquisition time and the building projection tilt angle. When the attitude recording time is inconsistent with the target image acquisition time, time matching is performed between adjacent attitude records to obtain the roll angle reference corresponding to the target image acquisition time. The vertical direction reference of the image is determined by the target image coordinate system and is used to specify the zero-degree reference direction of the building projection tilt angle.

[0104] The target image coordinate system adopts a coordinate direction where the horizontal coordinate increases to the right and the vertical coordinate increases downward. The building projection tilt angle takes the vertical direction of the image as the zero-degree direction and observes the horizontal change of the building projection center line from the bottom of the building to the top of the building. When the building projection center line deflects in the direction of increasing horizontal coordinate of the image, it is defined as positive, and when it deflects in the direction of decreasing horizontal coordinate of the image, it is defined as negative. The camera roll angle reference adopts the same angle unit and positive and negative direction as the building projection tilt angle. The angle unit is uniformly adopted as degrees or radians. The units of the building projection tilt angle and the roll angle reference are kept consistent before performing the difference calculation.

[0105] Subtract the building projection tilt angle from the roll angle reference input to obtain the building tilt angle after roll attitude correction. The calculation relationship is expressed as follows:

[0106] ;

[0107] in, This indicates the building tilt angle after roll attitude correction, expressed in degrees or radians. The tilt angle of the building projection is derived from the angle between the building projection centerline and the vertical reference of the image, and is expressed in degrees or radians. This represents the roll angle reference of the camera around the optical axis during target image acquisition, in degrees or radians. The subscript "corr" indicates the corrected angle, the subscript "proj" indicates the projection tilt angle, and the subscript "roll" indicates the camera roll angle. Subtract performs an algebraic subtraction of two scalar angle values ​​with the same unit. The calculation result still maintains the same unit as the input angle. When the building projection tilt angle and roll angle reference use the same positive and negative directions, subtracting the roll angle reference can deduct the angular offset caused by the camera's rotation around the optical axis to the vertical reference direction of the image.

[0108] The Sign function is executed on the building tilt angle after roll attitude correction. Sign reads the positive and negative relationship of the building tilt angle after roll attitude correction: when the building tilt angle after roll attitude correction is greater than zero, Sign outputs a positive direction indicator; when the building tilt angle after roll attitude correction is less than zero, Sign outputs a negative direction indicator; when the building tilt angle after roll attitude correction is equal to zero, Sign outputs a reference direction indicator, indicating that the building projection center line coincides with the vertical direction of the image after roll attitude correction. A positive direction indicator corresponds to the building projection center line tilting in the direction of increasing image horizontal coordinate when it points from the bottom of the building to the top of the building; a negative direction indicator corresponds to the building projection center line tilting in the direction of decreasing image horizontal coordinate when it points from the bottom of the building to the top of the building; and a reference direction indicator corresponds to a building tilt angle of zero.

[0109] The absolute value function Abs is applied to the tilt angle of the building after roll attitude correction. Abs removes the positive and negative signs of the tilt angle after roll attitude correction, retaining only the angle magnitude to obtain the tilt angle. The tilt angle is a non-negative scalar, and the unit is consistent with the tilt angle after roll attitude correction. The tilt direction identifier output by Sign is combined with the tilt angle output by Abs to form the tilt measurement result of the cultural relic protection building to be measured. The tilt direction field in the tilt measurement result records the positive direction, negative direction or reference direction, and the tilt angle field records the absolute value of the angle after roll attitude correction.

[0110] Example 2

[0111] This embodiment describes the data organization method of the homologous ablation experiment based on a data scale of 120 target images. The target images are organized according to 30 building numbers, with 4 images from different perspectives for each building number. Each target image is bound to the building number, image number, acquisition time, building prompt information, camera roll angle reference, and the actual tilt angle of the building. Training, verification, and testing are isolated according to building numbers. Images from different perspectives with the same building number are not assigned to different data boundaries to avoid the same building appearance texture appearing in both training and testing data. The ablation experiment uses independent test image records, and the training and verification data are not included in the ablation statistics.

[0112] Based on the data records of four ablation conditions from 120 target images listed in Table 1, the three-dimensional performance distribution was plotted using the building boundary IoU, unilateral outward spread residual rate, and absolute error of the tilt angle as common evaluation coordinates. Figure 2 As shown, in this set of data records, the baseline condition samples are mainly distributed in the regions with lower building boundary IoU, higher unilateral expansion residual rate and higher tilt angle absolute error. The complete scheme samples are concentrated in the regions with higher building boundary IoU, lower unilateral expansion residual rate and lower tilt angle absolute error, indicating that after the unilateral attachment boundary expansion is suppressed, the traction on the building projection centerline is weakened.

[0113] The building body label uses a binary building target mask. The building body area, including the building walls, tower body, eaves, doors and windows, is marked as the building foreground. Scaffolding, branches, sky, ground, adjacent buildings and the background outside the building body are marked as the building background. When scaffolding and branches are in contact with the building body, the contact area is not written into the building foreground label, and the attachment side of the scaffolding or branches on the left, right, left and right sides is recorded separately. The building area bounding box is determined by the minimum x-coordinate, maximum x-coordinate, minimum y-coordinate and maximum y-coordinate of the building foreground label. The building interior points are selected from the building foreground at a position no less than 4 low-resolution grid pixels away from the foreground boundary. The building exterior points are preferentially selected from the scaffolding, branches and building background areas. The building interior points, building exterior points and building area bounding box adopt the same coordinate transformation rules as the target image. The coordinates are adjusted synchronously after scaling, filling or cropping.

[0114] The true tilt angle of the building uses the same vertical reference and positive and negative directions of the target image. The true tilt angle of the building is determined by the on-site measurement record of the building or the building projection centerline calibrated by the measurement reference. It is used to evaluate the tilt angle after the building projection centerline and roll attitude correction. Images without building body annotation, true tilt angle of the building or camera attitude reference are not included in the ablation statistics to ensure that the building boundary IoU, centerline deviation, absolute error of tilt angle and unilateral outward residual rate all have clear references.

[0115] The cue segmentation model uses the image encoder, cue encoder, and mask decoder already determined in Example 1. The candidate output selection process, which was not elaborated in Example 1, is as follows: The input to the candidate selection stage includes a 1×256 intersection-over-union (IoU) marker feature, a 3×256 mask marker feature, and a 256×256×32 native mask feature. After receiving the updated image features, sparse cue features, and dense cue features, the bidirectional Transformer outputs the IoU marker feature and three mask marker features. The IoU marker feature has 256 channels, and the mask marker features... The feature arrangement is 3×256. The three mask label features are processed by Linear, GELU, Linear, GELU and Linear respectively, and feature channel transformation, element-wise nonlinear transformation and mask projection are completed in sequence to form three 32-dimensional mask projection vectors. The size of the original mask feature is 256×256×32. The three mask projection vectors are respectively processed with the original mask feature by MatMul. Each time MatMul is performed along the 32-dimensional channel, weighted accumulation is performed to obtain three 256×256×1 low-resolution candidate mask logical values.

[0116] The intersection-union-ratio (IU) marker features are processed through Linear, GELU, Linear, and Linear again to output three candidate quality responses. The candidate quality responses are indexed and correspond to the three low-resolution candidate mask logic values ​​according to the order of the mask marker features. ArgMax is executed on the candidate quality responses, and the candidate mask logic value corresponding to the largest response is selected as the input for boundary analysis. The selected candidate mask logic value is then used for temporary Sigmoid, temporary foreground determination, height layering, boundary displacement extraction, and signed boundary correction field generation. Unselected candidate mask logic values ​​do not participate in boundary displacement calculation or tilt measurement output. The candidate quality response only undertakes the candidate mask selection under the same building prompt and does not undertake the calculation of building boundary correction amount.

[0117] During model training, the parameters of the image encoder and cue encoder are fixed. Only the parameters of the bidirectional Transformer, mask projection multilayer Linear, ConvTranspose2D, and Cross-Union Quality Output Layer in the mask decoder are updated. The low-resolution candidate mask logistic values ​​are natively upsampled and finally Sigmoided, and then compared pixel-by-pixel with the binary mask of the building body. The binary cross-entropy loss is obtained by performing Log, Multiply, and Add operations on the building foreground and background positions, respectively. The Dice loss uses the sum of the position-by-position Multiply results of the predicted foreground and building body labels as the overlap, and is used to predict foreground pixels. The sum of the foreground pixels and the label is used as a normalization factor, and a fixed small positive number is added before division. The cross-union ratio quality loss is obtained by subtracting, square, and mean operations on the cross-union ratio calculated from the candidate quality response, candidate mask, and building body label. The binary cross-entropy loss, Dice loss, and cross-union ratio quality loss are combined into batch training loss by multiplying and adding with fixed weights. The batch training loss is then backpropagated after mean operation. No independent image encoder, independent cue encoder, independent classifier, or independent external scoring branch is added during training. The signed boundary correction field is still generated using a deterministic operator based on boundary displacement.

[0118] In the complete scheme, the joint deviation magnitude is used to determine the correction amount, which is applied to the continuous logic values ​​in the boundary neighborhood through correction polarity. The relationship between the joint deviation magnitude, the correction amount, and the absolute error of the tilt angle is as follows: Figure 3 As shown, Figure 3 The sample points are distributed around the trend surface. The bottom contour lines represent the error variation area under different combinations of joint deviation amplitude and correction amount. In this set of data records, when the displacement of one side boundary increases abnormally, the joint deviation amplitude and correction amount increase synchronously. After logical value stage correction, the absolute error of the tilt angle is maintained in a low range.

[0119] The ablation experiment sets baseline conditions, center-offset-only conditions, post-threshold correction conditions, and complete scheme conditions. The baseline conditions do not generate a signed boundary correction field; low-resolution candidate mask logic values ​​are directly input to the native upsampling terminal. The center-offset-only condition retains the center-offset component and its local reference, generating correction values ​​only based on the degree of deviation of the center-offset component, without using the width variation component or its local reference. The post-threshold correction condition fully calculates the center-offset component, width variation component, four types of local references, joint deviation amplitude, boundary displacement laterality, and outward or inward expansion / contraction direction. However, it forms a binary representation in the final Sigmoid and final threshold determination. After the building target mask is applied, the binary building target mask is adjusted according to the position and correction polarity of the boundary to be corrected. The corresponding boundary neighborhood position is determined as the building background in the outward expansion direction and as the building foreground in the inward contraction direction. The post-threshold correction condition is only used to verify the embedding position of the continuous logic value path and is not an implementation of the method in the claims. The complete scheme condition uses the center offset component and the width change component simultaneously, and forms a joint deviation amplitude based on four types of local references. A signed boundary correction field is generated by combining the boundary displacement laterality and the outward or inward contraction direction, and a position-by-position Add is performed between the logic value output end of the mask decoder and the native upsampling end.

[0120] To distinguish the effects of the center offset component and the width variation component, a three-dimensional coupled comparison is performed between the center offset condition and the complete scheme, such as... Figure 4 As shown, the horizontal axis represents the deviation of the center offset, the vertical axis represents the deviation of the width change, and the vertical axis represents the absolute error of the tilt angle. In this set of data records, only the error trend surface of the center offset condition is higher than that of the complete scheme. Especially in the area where the deviation of the width change increases, the complete scheme still maintains a low error, indicating that the width change component can identify the boundary width changes caused by the tower body tapering, the eaves overhang, and the single-sided attachments, and avoid misjudging the symmetrical width change as the building center offset.

[0121] Four ablation conditions use the same set of trained model weights, without retraining. 120 target images, building labels, building cue information, camera pose reference, temporary foreground threshold, and final foreground threshold are fixed. Only the usage relationship of the center offset component and width variation component, or the embedding position of the signed boundary correction field, are changed. Building boundary IoU is compared by the intersection and union of the building target mask and the building label. Centerline deviation is compared by the angle between the fitted centerline of the building target mask and the reference centerline of the building. Tilt angle absolute error is determined by the difference between the tilt angle after roll attitude correction and the actual tilt angle of the building. Unilateral outward spread residual rate is determined by the proportion of foreground pixels retained in the scaffolding or tree branch attachment area within the building target mask. Mask area deviation is determined by the relative difference between the foreground area of ​​the building target mask and the foreground area of ​​the building label.

[0122] Table 1. Statistical results of key indicators for four ablation conditions.

[0123]

[0124] The statistical results of the data records listed in Table 1 show that the average IoU of the building boundary under the baseline condition is 0.9308, the average absolute error of the tilt angle is 0.6192 degrees, and the average residual rate of unilateral outward expansion is 53.9%. For the condition with only center offset, the three indicators are 0.9563, 0.4215 degrees, and 26.7%, respectively. For the condition with threshold correction, the three indicators are 0.9563, 0.3316 degrees, and 16.1%, respectively. For the complete scheme, the three indicators are 0.9825, 0.1392 degrees, and 4.9%, respectively. The complete scheme is relatively... The fact that the absolute error of the tilt angle and the residual rate of one-sided expansion are further reduced under the condition of only center offset indicates that the participation of the width change component in the joint deviation amplitude calculation can supplement the symmetrical width change information that the center offset component cannot express. The complete scheme further improves the building boundary IoU and reduces the residual rate of one-sided expansion compared to the post-threshold correction condition. This shows that the signed boundary correction field performs position-by-position Add between the low-resolution continuous logic value and the native upsampling end, which can change the boundary logic value before the final threshold determination, which is different from the external correction of the binary building target mask.

[0125] In summary, this embodiment discloses the process of candidate mask quality response selection, building body and single-sided attachment label construction, training parameter fixing, ablation condition setting and evaluation index formation. It also provides an ablation verification method for the technical route through three-dimensional performance distribution, joint deviation correction response surface and boundary component coupling error surface. The technical route is specifically manifested as follows: the left and right boundary displacements are decomposed into center offset components and width change components. Four types of local references are used to form a directional boundary displacement coupling feature composed of joint deviation amplitude, boundary displacement occurrence side and outward or inward direction. A signed boundary correction field is generated based on this feature. The signed boundary correction field is embedded into the continuous logic value transmission path between the mask decoder logic value output end and the native upsampling end. The corrected logic value continues to be natively upsampled, thresholded, fitted center line and rolled attitude corrected by the promptable segmentation model. In this way, the candidate boundary bulge caused by single-sided attachments such as scaffolding and tree branches can be internally corrected, reducing the traction of attachments on the layer center point and building projection center line, while retaining the ability to distinguish the actual building tilt, tower body taper and eave width change.

[0126] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for measuring the tilt of a cultural relic preservation building based on a suggestible segmentation model, characterized in that, include: S1. Acquire the target image containing the cultural relic under protection, architectural prompt information, and the camera attitude reference corresponding to the target image to form the tilt measurement input; S2. Input the target image and building prompt information from the tilt measurement input into the prompt segmentation model. After image encoding, prompt encoding and mask decoding, obtain the low-resolution candidate mask logic value. S3. The candidate regions represented by the logic values ​​of the low-resolution candidate mask are layered along the building height direction. The left and right boundary displacements of adjacent height layers are extracted. The center offset component is calculated based on the same-direction part of the left and right boundary displacements. The width change component is calculated based on the opposite-direction part of the left and right boundary displacements. Based on the joint deviation of the center offset component and the width change component relative to the local reference of the boundary displacement, the side of the boundary displacement, and the outward or inward direction of the boundary corresponding to the side of the boundary displacement, a directional boundary displacement coupling feature is formed. The correction amount, the position of the boundary to be corrected, and the correction polarity are determined based on the directional boundary displacement coupling feature. A signed boundary correction field of the same scale as the logic value of the low-resolution candidate mask is generated in the neighborhood of the boundary to be corrected. The signed boundary correction field is superimposed on the logic value of the low-resolution candidate mask between the output end of the mask decoder logic value and the native upsampling end of the prompting segmentation model to obtain the corrected mask logic value. S4. Perform native upsampling and threshold determination on the correction mask logic value through the native upsampling end of the prompting segmentation model to form the architectural target mask of the cultural relic protection building to be tested. S5. Extract multiple layered center points along the building height direction within the building target mask, and perform straight line fitting on the multiple layered center points to obtain the building projection center line. Determine the building projection tilt angle based on the building projection center line. S6. Based on the camera attitude reference, perform roll attitude correction on the building projection tilt angle to determine the tilt direction and tilt angle of the cultural relic under test, and generate the tilt measurement results of the cultural relic under test.

2. The method for measuring the tilt of cultural relic protected buildings based on a promptable segmentation model according to claim 1, characterized in that, S1 specifically refers to: Acquire a target image containing the cultural relic under protection; In the target image, the building region bounding the cultural relic to be protected is determined, and the internal and external point prompts of the building are determined separately. The building prompt information is formed by the building region bounding, the internal and external point prompts of the building. The camera roll attitude around the optical axis is acquired during target image acquisition, and the camera attitude reference corresponding to the target image is determined based on the roll attitude. Establish a correspondence between the target image, building prompts, and camera attitude references to form the tilt measurement input.

3. The method for measuring the tilt of cultural relic protected buildings based on a promptable segmentation model according to claim 1, characterized in that, S2 specifically refers to: Image patch projection and position encoding are performed on the target image. The resulting image patch features are then input into a multi-layer encoding structure containing a window attention encoding layer and a global attention encoding layer to form image encoding features that preserve the spatial positional relationships of buildings. The coordinate position encoding and prompt type encoding of the building area bounding box, the internal point prompts and the external point prompts in the building prompt information are respectively performed to form prompt coding features corresponding to the image coding features; Image encoding features and cue encoding features are input into the mask decoder. Through cue feature self-attention, cue feature to image feature cross-attention, and image feature to cue feature cross-attention, updated image features and mask label features are formed. The updated image features are upsampled step by step, the mask label features are mapped to mask projection vectors, and the mask projection vectors are multiplied by the upsampled updated image features to obtain low-resolution candidate mask logical values.

4. The method for measuring the tilt of cultural relic protected buildings based on a promptable segmentation model according to claim 1, characterized in that, S3 specifically refers to: Sigmoid operation and temporary foreground determination are performed on the logical values ​​of low-resolution candidate masks. Multiple adjacent height layers are divided along the building height direction, and the left and right boundary positions are determined based on the leftmost and rightmost foreground positions in each height layer. The left boundary displacement is obtained by subtracting the left boundary positions of adjacent height layers, and the right boundary displacement is obtained by subtracting the right boundary positions of adjacent height layers. Add the left boundary displacement and the right boundary displacement and take half to obtain the center offset component. Subtract the left boundary displacement from the right boundary displacement and take half to obtain the width variation component. Calculate the deviation of the center offset component and the width variation component from the local boundary displacement reference. Based on the deviation, calculate the joint deviation amplitude, determine the side where the boundary displacement occurs, and the outward or inward direction of the boundary corresponding to the side where the boundary displacement occurs, thus forming a directional boundary displacement coupling feature. The correction amount, the position of the boundary to be corrected, and the correction polarity are determined based on the directional boundary displacement coupling characteristics. The correction value with the correction polarity is written into the neighborhood of the boundary to be corrected according to the position of the boundary to be corrected, generating a signed boundary correction field with the same scale as the low-resolution candidate mask logic value. Between the output of the mask decoder logic value and the native upsampling end of the cue segmentation model, the signed boundary correction field is added position by position to the low-resolution candidate mask logic value to obtain the corrected mask logic value.

5. The method for measuring the tilt of cultural relic protected buildings based on a promptable segmentation model according to claim 1, characterized in that, S4 specifically refers to: Input the correction mask logic value into the native upsampling end of the prompting segmentation model, and map the correction mask logic value to a high-resolution mask logic value that is consistent with the resolution of the target image according to the native upsampling path of the prompting segmentation model. Perform a Sigmoid operation on the high-resolution mask logic value to obtain the building foreground probability corresponding to each image location; The probability of building foreground is compared with a preset foreground determination threshold. The image positions where the probability of building foreground reaches the preset foreground determination threshold are determined as building foreground, and the remaining image positions are determined as building background, thus forming a building target mask.

6. The method for measuring the tilt of cultural relic protected buildings based on a promptable segmentation model according to claim 1, characterized in that, S5 specifically refers to: The building target mask is divided into multiple measurement layers along the building height direction, and the left and right boundary x-coordinates of the building target mask are extracted in each measurement layer; Add the left boundary x-coordinate and the right boundary x-coordinate of each measurement layer and take half of the result to obtain the layer center point of the corresponding measurement layer, and establish the correspondence between the layer center point and the height coordinate of the measurement layer. The building projection centerline is obtained by fitting a straight line to multiple layer center points and their corresponding measured layer height coordinates. The tilt angle of the building projection is determined based on the angle between the building projection centerline and the vertical direction of the image.

7. The method for measuring the tilt of cultural relic protected buildings based on a promptable segmentation model according to claim 1, characterized in that, S6 specifically refers to: Extract the roll angle reference of the camera around the optical axis when the target image is acquired from the camera attitude reference, and determine the vertical direction reference of the target image; The difference between the building projection tilt angle and the roll angle reference is calculated to obtain the building tilt angle after roll attitude correction. The tilt direction of the cultural relic under test is determined by the sign of the tilt angle of the building after roll attitude correction, and the tilt angle of the cultural relic under test is determined by the absolute value of the tilt angle of the building after roll attitude correction. By combining the tilt direction and tilt angle, the tilt measurement results of the cultural relic under protection are generated.

8. The method for measuring the tilt of cultural relic protected buildings based on a promptable segmentation model according to claim 4, characterized in that, For each height layer with adjacent height layers, the median of the center offset component and the width variation component of the adjacent height layers is taken to obtain the local reference for the center offset component and the local reference for the width variation component. The median of the left boundary displacement and the right boundary displacement of the adjacent height layers is taken to obtain the local reference for the left boundary displacement and the local reference for the right boundary displacement. The deviation amplitude between the center offset component and the local reference for the center offset component and the deviation amplitude between the width variation component and the local reference for the width variation component are combined to obtain the joint deviation amplitude. The deviation amplitude of the left boundary displacement relative to the local reference for the left boundary displacement and the deviation amplitude of the right boundary displacement relative to the local reference for the right boundary displacement are calculated to obtain the left boundary deviation amplitude and the right boundary deviation amplitude. The boundary displacement occurrence side when the left boundary deviation amplitude is greater than the right boundary deviation amplitude is determined as the left boundary. The boundary displacement occurrence side when the right boundary deviation amplitude is greater than the left boundary deviation amplitude is determined as the right boundary. The boundary displacement occurrence side when the left boundary deviation amplitude and the right boundary deviation amplitude are the same is determined as the common occurrence of the left and right boundaries.

9. The method for measuring the tilt of cultural relic protected buildings based on a promptable segmentation model according to claim 4, characterized in that, Before superimposing the signed boundary correction field with the low-resolution candidate mask logic value, the Sigmoid operation and threshold determination for forming the building target mask are not performed on the low-resolution candidate mask logic value. The signed boundary correction field and the low-resolution candidate mask logic value are added position by position to form the correction mask logic value. The correction mask logic value is directly input into the native upsampling end of the prompting segmentation model. The native upsampling end of the prompting segmentation model performs native upsampling, Sigmoid operation for forming the building target mask, and threshold determination on the correction mask logic value.

10. The method for measuring the tilt of cultural relic protected buildings based on a promptable segmentation model according to claim 4, characterized in that, The low-resolution candidate mask logic value output from the mask decoder logic value output terminal is transmitted along the mask logic value transmission path between the mask decoder logic value output terminal and the native upsampling terminal of the cue segmentation model. Based on the candidate region boundary displacement represented by the low-resolution candidate mask logic value, a directional boundary displacement coupling feature is formed. Based on the directional boundary displacement coupling feature, a signed boundary correction field is generated. In the mask logic value transmission path, the signed boundary correction field is superimposed on the low-resolution candidate mask logic value to form a corrected mask logic value. The corrected mask logic value is then input into the native upsampling terminal of the cue segmentation model along the native upsampling path of the cue segmentation model.

Images