Building crack dimension factor transfer method

Abstract

The application provides a building crack scale factor transmission method and relates to the technical field of computer vision and image processing. According to multi-view images of a building surface, binocular disparity information, laser ranging information and inertial attitude information, a view overlap graph is established; crack detection is performed on each view image, and a plurality of crack sections are intercepted; cross-view crack sections are associated, matched and consistency checked, and a crack association graph is constructed; a local scale factor is generated at a local position of a part of the crack sections; the local scale factor is iteratively transmitted between the cross-view associated crack sections; a continuous scale field is constructed and optimized according to a building surface curvature field and the local scale factor of the local position of the crack section; the real width and real length of each crack section are calculated according to the final local scale factor at the local position of each crack section, and a plurality of crack sections belonging to the same physical crack are spliced to obtain the overall scale result and reliability of the whole crack.

Description

Technical Field

[0001] This invention relates to the field of computer vision and image processing technology, and more specifically, to a method for transferring the scale factor of building cracks. Background Technology

[0002] During their long service life, ancient buildings are susceptible to cracking, peeling, and localized hollowing in their walls, beams, columns, mortar joints, and decorative surfaces due to factors such as temperature and humidity changes, uneven foundation settlement, material aging, weathering and erosion, as well as earthquakes and vibrations. The actual width, length, bifurcation pattern, and evolution of cracks over time are crucial for assessing structural safety, developing repair plans, and evaluating the progression of damage. With the application of robotic inspection systems, binocular cameras, laser ranging units, and inertial measurement units in cultural heritage inspections, the acquisition of damage data for ancient buildings has gradually shifted from manual sampling to multi-view, continuous, and digital acquisition, providing new technological conditions for the quantitative assessment of cracks.

[0003] However, the relevant technologies lack methods for real-world crack measurement scenarios, making it difficult to achieve stable, real-scale recovery of long cracks, continuous cracks, and crack networks. Summary of the Invention

[0004] In view of this, the present invention provides a method for transferring building crack scale factors.

[0005] One aspect of the present invention provides a method for transferring building crack scale factors, comprising: establishing a view overlap graph with each view image as a node and overlapping view pairs as edges based on multi-view images of the building surface, binocular parallax information, laser ranging information, and inertial attitude information; performing crack detection on each view image, extracting multiple crack segments, and configuring structured description information for each crack segment; performing association matching and consistency verification on cross-view crack segments based on the overlapping view pair range defined by the view overlap graph and in combination with the structured description information, and constructing a crack association graph with crack segment objects as nodes and cross-view repeated observation relationships as edges; and determining the scale factor at local locations of some crack segments based on binocular parallax information, laser ranging information, and known component dimensions. Local scale factors are generated, and scale anchor points are determined and a scale anchor point table is constructed through confidence assessment. Based on the view overlap diagram and crack association diagram, and combined with the scale anchor point table, local scale factors are iteratively transferred between crack segments associated across views to obtain the local scale factors at local locations of each crack segment. Based on the optimization objective function, a continuous scale field is constructed and optimized according to the curvature field of the building surface and the local scale factors at local locations of crack segments to obtain the final local scale factors at local locations of each crack segment. The true width and true length of each crack segment are calculated based on the final local scale factors at local locations of each crack segment, and multiple crack segments belonging to the same physical crack are spliced ​​together to obtain the overall scale result and confidence level of the entire crack.

[0006] The present invention achieves the following beneficial effects through the above technical solution.

[0007] (1) By using binocular parallax information, laser ranging information, and known component dimensions to generate local scale factors, and then by using a transmission mechanism to extend a small number of scale anchor points to a larger crack area, the dependence on continuous high-precision reconstruction of the entire scene, dense laser scanning and manual point-by-point calibration is reduced, making it more suitable for application scenarios in ancient building inspection where the viewing angle is limited, the surface is complex and the on-site measurement conditions are limited.

[0008] (2) Further combining the wall curvature and continuous scale field as constraints, the global consistency optimization of the crack scale restoration results can suppress local scale drift under complex geometric conditions such as concavity and convexity, relief, brick joints, component corners, etc. on the ancient building wall, and improve the stability of the restoration of the true width and true length of the crack. It is especially suitable for scenarios such as daily inspection of ancient buildings, special inspection and multi-stage disease monitoring. Attached Figure Description

[0009] The above and other objects, features and advantages of the present invention will become more apparent from the following description of embodiments of the invention with reference to the accompanying drawings, in which:

[0010] Figure 1 An exemplary system architecture for applying the building crack scale factor transfer method according to an embodiment of the present invention is shown;

[0011] Figure 2 A flowchart of a method for transferring building crack scale factors according to an embodiment of the present invention is shown;

[0012] Figure 3 A flowchart of the method for transferring the scale factor of cracks in ancient buildings based on repeated observation of cracks in multiple views according to an embodiment of the present invention is shown.

[0013] Figure 4 A block diagram of a building crack scale factor transfer device according to an embodiment of the present invention is shown;

[0014] Figure 5 A block diagram of an electronic device suitable for implementing a method for transferring building crack scale factors according to an embodiment of the present invention is shown. Detailed Implementation

[0015] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the invention. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of the invention for ease of explanation. However, it will be apparent that one or more embodiments may be practiced without these specific details. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concept of the invention.

[0016] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0017] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0018] When using expressions such as "at least one of A, B and C", they should generally be interpreted in accordance with the meaning that is commonly understood by those skilled in the art (e.g., "a system having at least one of A, B and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B and C, etc.).

[0019] In the context of inspecting ancient buildings, cracks often span multiple bricks, mortar joints, or decorative components, making it difficult to completely cover the entire crack in a single photograph. Furthermore, due to the constantly changing relative position of the inspection platform to the wall, the same crack will exhibit differences in scale, viewing angle, clarity, and degree of occlusion in different views. For long cracks, bifurcated cracks, and cracks extending along curved surfaces, relying solely on single-view measurements or single close-range calibrations often only yields local pixel widths, making it difficult to reliably reconstruct the true physical scale.

[0020] Relevant image crack measurement schemes typically employ two technical approaches: one is based on manual rulers, single-target fixed plates, or fixed-distance measurements, converting the pixel width in a single image into the physical width; the other uses methods such as multi-view reconstruction or Structure from Motion (SfM) and Multi-View Stereo (MVS) to first restore the relative geometry of the scene, and then attempt to measure the crack size on the reconstructed model. The former relies on additional on-site calibration, has a limited measurement range, and is difficult to cover long cracks; while the latter can restore the scene geometry, its modeling objects are usually general feature points or surface fragments, rather than the "same crack segment" of the defect itself, which is prone to problems such as scale drift, local error accumulation, and unstable cross-view correspondence.

[0021] Furthermore, the walls of ancient buildings are not ideal planes; they often exhibit complex curvature variations such as reliefs, brick joints, bulges, weathering, and corners of components. If a uniform planar scale assumption is still adopted, the width of cracks may be magnified or reduced. If sparse laser ranging is used directly, it will be affected by the sparse distribution of laser points, occlusion, and the fact that the ranging does not completely coincide with the crack boundary, making it difficult to obtain a continuous and stable true scale throughout the entire crack area.

[0022] Therefore, there is an urgent need to propose a technical solution for the inspection of ancient buildings, which can establish a correlation between repeated observations of the same crack segment in multiple overlapping views, transfer local scale information obtained from a small number of reliable areas between overlapping views, and then combine wall curvature and continuous scale field for consistency optimization, so as to achieve the true scale restoration of long cracks, small cracks, bifurcated cracks and crack networks.

[0023] Existing multi-view reconstruction techniques primarily focus on camera pose determination, depth estimation, view selection, or image stitching, lacking methods tailored to real-world crack measurement scenarios. This makes it difficult to achieve stable, true-scale recovery for long cracks, continuous cracks, and crack networks. Therefore, this invention proposes a method for transferring building crack scale factors based on repeated observations of cracks from multiple views.

[0024] This invention does not merely use SfM or multi-view reconstruction as a scene geometry restoration tool. Instead, it establishes a cross-view correspondence of crack segments based on the repeated observation relationship of the same crack segment in different views. On this basis, it obtains reliable local scale information by combining binocular depth, laser ranging, known component dimensions, or local 3D reconstruction results. Through a scale factor transfer mechanism, a small number of local scales are extended to a larger crack area. Furthermore, by combining the curvature constraints and continuous scale field constraints of the wall surface where the crack is located, the scale restoration results are uniformly optimized, and finally the true width, true length, and dimensional change results along the crack direction of the crack are obtained.

[0025] Compared with related technologies, this invention no longer focuses solely on "how to reconstruct the scene," but rather on solving the problem of "how to stably restore the true scale of cracks under multi-view conditions." Therefore, it is applicable not only to long cracks, continuous cracks, and crack networks whose length exceeds the observation range of a single image, but also to the consistent measurement of a single small crack under multiple observation orientations. It can provide a foundation for the quantitative detection of cracks in ancient buildings and subsequent analysis of disease evolution.

[0026] This invention provides a method for transferring building crack scale factors, relating to the fields of computer vision and image processing technology. The method includes: establishing a view overlap graph with each view image as a node and overlapping view pairs as edges, based on multi-view images of the building surface, binocular parallax information, laser ranging information, and inertial attitude information; detecting cracks in each view image, extracting multiple crack segments, and configuring structured description information for each crack segment; performing association matching and consistency verification on cross-view crack segments based on the overlapping view pair range defined by the view overlap graph and the structured description information, constructing a crack association graph with crack segment objects as nodes and cross-view repeated observation relationships as edges; and generating local scale factors at local locations of some crack segments based on binocular parallax information, laser ranging information, and known component dimensions. Scale anchor points are determined and a scale anchor point table is constructed through confidence assessment. Based on the view overlap diagram and crack association diagram, and combined with the scale anchor point table, local scale factors are iteratively transferred between crack segments associated across views to obtain the local scale factors at local locations of each crack segment. Based on the optimization objective function, a continuous scale field is constructed and optimized according to the curvature field of the building surface and the local scale factors at local locations of crack segments to obtain the final local scale factors at local locations of each crack segment. The true width and true length of each crack segment are calculated based on the final local scale factors at local locations of each crack segment, and multiple crack segments belonging to the same physical crack are spliced ​​together to obtain the overall scale result and confidence level of the entire crack.

[0027] Figure 1 An exemplary system architecture for applying the building crack scale factor transfer method according to embodiments of the present invention is shown. It should be noted that... Figure 1 The examples shown are merely examples of system architectures that can be applied to embodiments of the present invention, in order to help those skilled in the art understand the technical content of the present invention, but do not mean that embodiments of the present invention cannot be used in other devices, systems, environments or scenarios.

[0028] like Figure 1As shown, the system architecture 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 serves as a medium for providing communication links between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, etc.

[0029] Users can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 via the network 104 to receive or send messages, etc. Various communication client applications can be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, and / or social media platform software, etc. (for example only).

[0030] The first terminal device 101, the second terminal device 102, and the third terminal device 103 can be various electronic devices with displays and support web browsing, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0031] Server 105 can be a server that provides various services, such as a backend management server that supports websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103 (this is just an example). The backend management server can analyze and process data such as received user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal devices.

[0032] It should be noted that the building crack scale factor transfer method provided in this embodiment of the invention can generally be executed by server 105. Correspondingly, the building crack scale factor transfer device provided in this embodiment of the invention can generally be located in server 105. The building crack scale factor transfer method provided in this embodiment of the invention can also be executed by a server or server cluster that is different from server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or server 105. Correspondingly, the building crack scale factor transfer device provided in this embodiment of the invention can also be located in a server or server cluster that is different from server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or server 105. Alternatively, the building crack scale factor transfer method provided in this embodiment of the invention can also be executed by the first terminal device 101, the second terminal device 102, or the third terminal device 103, or by other terminal devices different from the first terminal device 101, the second terminal device 102, or the third terminal device 103. Accordingly, the building crack scale factor transfer device provided in the embodiments of the present invention can also be installed in the first terminal device 101, the second terminal device 102, or the third terminal device 103, or in other terminal devices different from the first terminal device 101, the second terminal device 102, or the third terminal device 103.

[0033] For example, the multi-view images, binocular parallax information, laser ranging information, and inertial attitude information of the building surface can be originally stored in any one of the first terminal device 101, the second terminal device 102, or the third terminal device 103 (e.g., the first terminal device 101, but not limited thereto), or stored on an external storage device and can be imported into the first terminal device 101. Then, the first terminal device 101 can locally execute the building crack scale factor transfer method provided in the embodiments of the present invention, or send the multi-view images, binocular parallax information, laser ranging information, and inertial attitude information of the building surface to other terminal devices, servers, or server clusters, and the other terminal devices, servers, or server clusters that receive the multi-view images, binocular parallax information, laser ranging information, and inertial attitude information of the building surface can execute the building crack scale factor transfer method provided in the embodiments of the present invention.

[0034] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0035] Figure 2 A flowchart of a method for transferring building crack scale factors according to an embodiment of the present invention is shown.

[0036] like Figure 2 As shown, the method includes operations S201 to S207.

[0037] In operation S201, based on the multi-view images of the building surface, binocular parallax information, laser ranging information, and inertial attitude information, a view overlap graph is established with each view image as a node and overlapping view pairs as edges.

[0038] According to embodiments of the present invention, multi-view images are acquired by a binocular camera at different positions and angles. Binocular parallax information is calculated by stereo matching of the left and right images from the binocular camera. Laser ranging information is acquired by a laser ranging unit. Inertial attitude information is acquired by an inertial measurement unit. Overlapping view pairs represent local areas of the same building surface covered by the two view images in space.

[0039] For example, by acquiring multi-view images of the surface of ancient buildings, binocular parallax information, laser ranging information, and inertial attitude information, and obtaining a multi-view inspection dataset, a view overlap map can be established by completing basic geometric initialization. Basic geometric initialization may include, but is not limited to, multi-sensor time synchronization, extrinsic parameter calibration, and initial pose estimation.

[0040] In operation S202, crack detection is performed on each view image, multiple crack segments are extracted, and structured description information is configured for each crack segment.

[0041] For example, by performing crack detection, skeletonization, and boundary extraction on each view image, the detected cracks can be divided into multiple crack segments based on factors such as curvature, bifurcation, occlusion, and boundary truncation, and structured descriptive information can be established for each crack segment. This structured descriptive information characterizes the geometric shape, spatial location, and image quality attributes of the crack segment in each view image, and is not limited to these.

[0042] In operation S203, based on the range of overlapping view pairs defined by the view overlap graph and combined with the structured description information, the cross-view crack segments are associated and matched and their consistency is verified, and a crack association graph is constructed with crack segment objects as nodes and cross-view repeated observation relationships as edges.

[0043] According to an embodiment of the present invention, the cross-view repeated observation relationship characterizes the one-to-one correspondence between crack segments formed in different view images of the same physical crack.

[0044] For example, based on view overlap, geometric reprojection, and crack segment similarity features, it can be determined whether crack segments in different view images belong to the same physical crack segment. It can also be determined whether crack segments in different view images have cross-view repeated observation relationships, and crack association diagrams can be established based on these cross-view repeated observation relationships.

[0045] Based on the obtained crack correlation map, further multi-view methods can be used. Figure 1 Consistency constraints eliminate conflicting associations.

[0046] In operation S204, based on binocular parallax information, laser ranging information, and known component dimensions, local scale factors are generated at local locations in some crack segments, and scale anchor points are determined and a scale anchor point table is constructed through confidence assessment.

[0047] According to an embodiment of the present invention, the local scale factor refers to the mapping coefficient from the image pixel size to the actual physical size at a certain local location of the crack segment, which is used to convert the pixel width or pixel arc length at that location into the actual width or actual length.

[0048] According to embodiments of the present invention, local three-dimensional reconstruction can be performed first by combining at least one of binocular parallax information, laser ranging information, and known component dimensions. Then, local scale factors are generated at corresponding local locations in the partial crack segments based on the local three-dimensional reconstruction results.

[0049] The 3D reconstruction results can be flexibly selected based on the on-site acquisition conditions. For example, when laser ranging information is abundant, laser point clouds are used as the main input for 3D reconstruction; when laser data is sparse or missing, local 3D reconstruction can be performed based on multi-view images and binocular parallax information. The specific method for determining the 3D reconstruction results is not limited here.

[0050] According to embodiments of the present invention, each local scale factor may have corresponding confidence level information. Based on the confidence level, it can be determined whether the local scale factor is used as a scale anchor point or not. To ensure the accuracy of information transmission, only local scale factors used as scale anchor points are transmitted.

[0051] In operation S205, based on the view overlap diagram and crack association diagram, and combined with the scale anchor point table, the local scale factor is iteratively transferred between crack segments associated across views to obtain the local scale factor at the local location of each crack segment.

[0052] For example, based on crack correlation maps and view overlap maps, local scale information of scale anchor points can be transferred to other crack segment locations. Furthermore, the coverage of high-confidence scale results can be continuously expanded through multi-source fusion and iterative transfer.

[0053] In operation S206, based on the objective function, a continuous scale field is constructed and optimized according to the curvature field of the building surface and the local scale factor of the local location of the crack segment to obtain the final local scale factor at the local location of each crack segment.

[0054] According to an embodiment of the present invention, an optimization objective function can be constructed based on a scale anchor point consistency term, a transmission consistency term, a wall curvature constraint term, a scale field smoothing term, an anomaly scale suppression term, and the weighting coefficients of each term. Specifically, the scale anchor point consistency term ensures that the optimized continuous scale field values ​​at the scale anchor point positions are consistent with the scale factors recorded in the scale anchor point table; the transmission consistency term ensures that the optimized continuous scale field is consistent with the transmitted values ​​at the transmission positions; the wall curvature constraint term ensures that scale changes are consistent with the wall geometry; the scale field smoothing term suppresses abrupt scale changes between adjacent positions; and the anomaly scale suppression term suppresses unreliable scale values ​​that deviate from the normal range. Solving multiple common constraints ensures that the final scale field achieves global optimum while maintaining reliable anchor points, effective transmission, geometric rationality, spatial continuity, and the absence of anomalies. After obtaining the final local scale factors at the local locations of each crack segment, the final scale field can be determined accordingly.

[0055] For example, a building surface curvature field can be established based on binocular parallax information (such as binocular depth), laser ranging information (such as laser point cloud), and local surface fitting results. A continuous scale field can be constructed based on the building surface curvature field, and the final scale result can be obtained through an optimization objective function composed of anchor point consistency term, transitive consistency term, wall curvature constraint term, scale field smoothing term, and abnormal scale suppression term.

[0056] In operation S207, the true width and true length of each crack segment are calculated based on the final local scale factor at the local location of each crack segment. Multiple crack segments belonging to the same physical crack are spliced ​​together to obtain the overall scale result and reliability of the entire crack.

[0057] According to embodiments of the present invention, the final local scale factor at local locations of each crack segment can be used to calculate the true width, true length, width variation curve, and crack network information of each crack segment and the entire crack, while simultaneously outputting the measurement reliability of each crack segment and the entire crack. This can be extended to multi-phase monitoring.

[0058] Under multi-stage inspection conditions, further time-series registration and quantitative analysis of crack evolution can be performed.

[0059] Through the above embodiments of the present invention, local scale factors are generated using binocular parallax information, laser ranging information, and known component dimensions. A transfer mechanism then extends a small number of scale anchor points to a larger crack area, thereby reducing the reliance on continuous high-precision reconstruction of the entire scene, dense laser scanning, and manual point-by-point calibration. This makes it more suitable for application scenarios in ancient building inspections where viewing angles are limited, surfaces are complex, and on-site measurement conditions are limited. Furthermore, the present invention further incorporates wall curvature and a continuous scale field as constraints to optimize the global consistency of crack scale restoration results. This can suppress local scale drift under complex geometric conditions such as unevenness, reliefs, brick joints, and component corners on ancient building walls, improving the stability of crack width and length restoration. It is particularly suitable for scenarios such as daily inspections of ancient buildings, specialized testing, and multi-phase disease monitoring.

[0060] The following is combined with Figure 3 and specific embodiments, for Figure 2 The method shown will be further explained.

[0061] Figure 3 A flowchart illustrating the overall process of a method for transferring the scale factor of cracks in ancient buildings based on repeated observation of cracks from multiple views, according to an embodiment of the present invention, is shown.

[0062] like Figure 3 As shown, the method includes steps S1 to S7 (corresponding one-to-one with operations S201 to S207 described above). A detailed explanation follows.

[0063] S1. Establish a multi-view inspection data set and complete basic geometric initialization.

[0064] S1.1 Collect raw data from multi-view inspections.

[0065] According to embodiments of the present invention, multi-view images, binocular parallax information, laser ranging information, and inertial attitude information of building surfaces can be collected to form raw data for multi-view inspection.

[0066] For example, inspections can be conducted on the walls or components of ancient buildings to collect multi-view images, binocular parallax information, laser ranging information, and inertial attitude information. Multi-view images consist of multiple red-green-blue (RGB) images captured by the inspection terminal at different locations, times, or viewing angles of the same target area. When the inspection terminal is a binocular device, each acquisition moment includes at least a left-eye image and a right-eye image, from which binocular parallax information can be determined. Laser ranging information includes single-point ranging values, laser landing point coordinates, or a local sparse point set formed by multiple measurements. Inertial attitude information includes the pitch angle, roll angle, yaw angle, and their timestamps corresponding to the acquisition moment.

[0067] S1.2 Complete multi-sensor time synchronization and external parameter calibration.

[0068] According to embodiments of the present invention, time synchronization and extrinsic parameter calibration can be performed on the binocular camera, laser ranging unit, and inertial measurement unit to establish a unified sensor coordinate system.

[0069] Time synchronization ensures that images, depth, and laser information acquired at the same time are correlated one-to-one. Extrinsic parameter calibration determines the rigid transformation relationship between the binocular camera coordinate system, the laser coordinate system, and the inertial coordinate system. The rigid transformation relationship is used to transform the 3D points obtained by the binocular camera, the laser ranging points, and the attitude information obtained by the inertial measurement unit (IMU) into a unified spatial coordinate system.

[0070] For example, the coordinate system of the left eye camera at the first valid acquisition moment is preferably used as the local world coordinate system. When the inspection robot has already established its own map coordinate system, it can also directly use the robot's map coordinate system as the unified coordinate system. Specifically, the coordinate transformation between sensors is first completed based on the extrinsic parameters of the right eye camera, laser unit, and IMU relative to the left eye camera, and then the left eye camera pose is transformed to the unified world coordinate system based on the acquisition moment.

[0071] The above processing establishes a unified sensor coordinate system, enabling image information, depth information, and ranging information to be projected onto the same spatial reference frame.

[0072] S1.3. Determine the initial pose of each view image.

[0073] According to embodiments of the present invention, the initial pose of each view image in a unified sensor coordinate system can be calculated based on the calibrated inertial attitude information, image registration results, and multi-view geometric constraints.

[0074] The inertial attitude information is provided by the IMU and used to determine the initial rotation values ​​of the camera attitude, including pitch angle, roll angle, yaw angle, and corresponding timestamps. Image registration results are obtained by performing feature point matching, epipolar geometric constraints, and perspective-n-point (PnP) localization on adjacent view images, including relative rotation, relative translation, and the number of in-registration points between view images. Multi-view geometric constraints are constructed based on the feature matching relationships of multiple view images, combined with epipolar geometry, triangulation, or bundle adjustment, to ensure geometric consistency in the projection of the same spatial point in different views and to correct the pose parameters of the view images during joint optimization. The initial pose includes the camera center position, the camera optical axis direction (which can be determined by the camera attitude represented by a rotation matrix or Euler angles), and the relative displacement and relative rotation angle between the current view image and the preceding and following adjacent view images.

[0075] The initial pose does not need to achieve the final high accuracy, but it should be able to support subsequent candidate overlapping view filtering, local geometric projection, and cross-view correspondence search of crack segments.

[0076] S1.4. Establish a set of candidate overlapping view pairs.

[0077] According to an embodiment of the present invention, view image pairs that are adjacent in acquisition time, have an acquisition position distance less than a preset distance threshold, have a viewing angle difference less than a preset angle threshold, or have an image similarity greater than a preset similarity threshold can be selected based on the acquisition timestamp, camera center position, camera optical axis direction, and image registration results of each view image to form a candidate overlapping view pair set.

[0078] For example, the proximity of view acquisition times can be obtained first based on the difference in acquisition timestamps or frame numbers of each view image; the spatial proximity of the inspection trajectory can be obtained based on the Euclidean distance between the camera center positions of two view images obtained in S1.3; the viewing angle difference threshold can be obtained based on the angle between the camera optical axis directions of each view image; and coarse image similarity can be obtained by matching the number and proportion of local features such as Oriented Fast and Rotated BRIEF (ORB) and Scale-Invariant Feature Transform (SIFT) or global image feature similarity. Then, based on the proximity of view acquisition times, the spatial proximity of the inspection trajectory, the viewing angle difference threshold, or the coarse image similarity, all views are initially screened to form a set of candidate overlapping view pairs.

[0079] The initial screening follows these conditions: the difference in adjacent frame numbers does not exceed 5 frames, the distance between camera centers does not exceed 0.5 meters, the optical axis angle does not exceed 30 degrees, and the number of image feature matches is not less than 30. These are considered as candidate overlapping view pairs. The relevant values ​​in this embodiment can be adaptively adjusted according to the camera's field of view and inspection distance, and are not limited here.

[0080] This step is used to narrow down the scope of subsequent accurate overlap determination and reduce invalid comparisons made between all view images.

[0081] Subsequently, based on the initial pose and spatial location information of the building surface in each view image, and combined with the spatial overlap and image matching degree of each candidate overlapping view pair, a view overlap map can be constructed. The spatial location information of the building surface can be obtained by transforming binocular parallax information and laser ranging information into spatial coordinates, and is used to describe the three-dimensional coordinates of each pixel on the building surface in a unified sensor coordinate system. This information is used to project the pixel regions in the view images into three-dimensional space to support the construction of the view overlap map.

[0082] S1.5. Create an overlay view.

[0083] According to an embodiment of the present invention, based on the initial pose of each view image and the spatial position information of the building surface, the effective pixel region of the first view image in the candidate overlapping view pair set is projected onto the effective imaging range region of the second view image. The pixel ratio of pixels that fall into the effective imaging range region after projection and are not occluded to the total pixels in the effective imaging range region is counted. In response to detecting that the pixel ratio is greater than a preset geometric overlap threshold, it is determined that the first view image and the second view image satisfy the spatial overlap constraint. In response to detecting that the number of image matching feature points between the target first view image and the target second view image that satisfy the spatial overlap constraint is greater than a preset image quality threshold, it is determined that the target first view image and the target second view image constitute an overlapping view pair. A view overlap graph is constructed with each view image as a node and the overlapping view pair as an edge, wherein the edge weight in the view overlap graph represents the degree of overlap between the overlapping view pairs.

[0084] For example, by accurately determining candidate overlapping view pairs, overlapping view pairs can be obtained, and then a view overlap graph can be constructed.

[0085] The precise determination method is as follows: Based on the camera's initial pose, camera intrinsic and extrinsic parameters, and coarse depth or sparse surface support information, the effective region in view image i (i.e., the first view image) is mapped to view image j (i.e., the second view image). The proportion of pixels that still fall within the effective imaging range of view image j after projection and are not occluded is counted. If this pixel proportion is higher than a preset geometric overlap threshold, then view image i and view image j are constrained by spatial overlap. i and j are used to represent different view images in the multi-view image.

[0086] The coarse depth is primarily obtained by calculating the disparity from the binocular images. For areas where binocular matching fails, it is supplemented by laser ranging points or sparse 3D point sets formed by adjacent laser points. After transforming the binocular depth points and laser points to a unified sensor coordinate system, planar or curved surface fitting is performed in the local neighborhood to obtain surface support information that can be used for inter-image projection. The surface support information does not require the formation of a complete high-precision 3D model; it only needs to support local pixel back-projection and occlusion determination. The effective region is the area in the source view image that can be projected. It represents the pixel area in the view image that, after distortion correction, is not in the image black border, severely overexposed, severely underexposed, or significantly motion-blurred, and has effective depth or surface support information. The effective imaging range is the range of positions in the target view image that can accept projection points. It refers to the actual usable image boundary range after distortion correction, such as pixel coordinates being within the image width and height range, further excluding distortion correction black borders and pre-marked invalid image areas.

[0087] For view image pairs constrained by spatial overlap, the number of matching feature points, the number of registered intrapoints, or the proportion of clear and usable pixels in the common area can all be used as the number of matching feature points.

[0088] The process involves extracting feature points from two view images using methods such as ORB and SIFT, performing descriptor matching, and then filtering by distance ratio to obtain the number of valid matching feature points. Next, the Random Sample Consensus (RANSAC) algorithm, combined with the fundamental matrix, essential matrix, or PnP model, is used to eliminate incorrect matches, and the remaining number of matching points is taken as the number of in-registration points. For the common projection area of ​​the two view images, overexposed, underexposed, and blurred pixels are marked based on indicators such as grayscale saturation, gradient strength, and Laplacian variance. The proportion of remaining clear pixels to the total number of pixels in the common area is then calculated to obtain the proportion of clear, usable pixels within the common area.

[0089] If at least a predetermined number of indicators, including the number of matching feature points, the number of registered intrapoints, and the proportion of clear and usable pixels in the common area, are higher than the corresponding preset image quality threshold, then the view image pair is determined to constitute an overlapping view pair.

[0090] A view overlap graph is constructed using each view image as a node and overlapping view pairs as edges. The edge weights can represent the proportion of the common area, the registration quality, or the overall overlap confidence.

[0091] The common region ratio is the area of ​​the effective region of the source view image that remains within the effective imaging range of the target after being projected onto the target view image and passes the occlusion check, which is the proportion of the effective region area of ​​the source view image. The registration quality can be characterized by the RANSAC in-point rate and the average reprojection error. The overall overlap confidence is obtained by weighting the common region ratio, the registration in-point rate, the reprojection error, and the common region sharpness after normalization.

[0092] View overlap diagrams are used to determine the view range within which cross-view searches of crack segments are permitted.

[0093] S1.6 Establish surface support information.

[0094] According to embodiments of the present invention, a geometric description of the building surface (i.e., surface support information) can be established based on the initial pose of each view image and the spatial position information of the building surface. The geometric description of the building surface includes information on local depth, normal, local flatness, and surface continuity.

[0095] By utilizing binocular depth maps, laser impact points, and local sparse point sets, surface support information is established for the walls or components of ancient buildings. This surface support information includes at least local depth, coarse normal, local flatness, and surface continuity information, which are used for subsequent geometric projection of crack segments, scale transfer constraints, and geometric prior construction in the optimization of the continuous scale field of the wall surface.

[0096] The binocular depth map is obtained by disparity matching of the left and right eye images after stereo correction, and calculated based on the camera focal length and binocular baseline. The laser impact point is obtained by combining the distance value output by the laser ranging unit with its extrinsic parameter conversion with the camera. The local sparse point set is constructed in a unified sensor coordinate system from multiple laser impact points, effective binocular depth points, or a fusion of both.

[0097] For example, effective binocular depth points and laser points can be uniformly transformed to the world coordinate system, and local neighborhoods can be divided on the wall surface to be processed. For each neighborhood, local plane fitting or quadratic surface fitting is performed using neighboring 3D points. The center depth of the fitted surface is used as the local depth, and the normal of the fitted plane or the local normal of the surface is used as the coarse normal. The mean square residual from the point to the fitted surface is used to characterize the local flatness. The depth difference and normal angle between adjacent local regions are used to characterize surface continuity information. Based on this, surface support information can be established.

[0098] S2. Extract the crack region from each view image and construct the crack segment object.

[0099] S2.1 Perform crack candidate region detection.

[0100] According to an embodiment of the present invention, crack detection can be performed on each view image after crack enhancement processing to obtain crack candidate regions.

[0101] For example, crack candidate region detection is performed on the RGB image of each view to obtain a crack probability map or a crack binary region map. In some implementations, crack edge enhancement or linear structure enhancement processing can be performed on each view image first to highlight the boundary features and elongated structural features of the crack relative to the background texture, and then candidate regions are extracted based on the enhancement results.

[0102] The RGB image used for crack candidate region detection can be a camera-calibrated RGB image, preferably after at least lens distortion correction and invalid boundary cropping. Brightness normalization or noise reduction may also be performed if necessary. Crack edge enhancement refers to using edge detection, directional filtering, linear structure enhancement, or a combination thereof to enhance the responses of the crack's lateral boundaries and thin, dark lines, making the crack region more prominent in the image. Crack detection methods may include: semantic segmentation methods based on deep learning; traditional image processing methods based on linear structure enhancement; threshold segmentation methods based on grayscale, texture, and directional information; and combinations of the above methods. The detection output should be able to characterize the location, extent, and preliminary morphology of the crack in the image.

[0103] In a preferred embodiment, depth information can be used for geometric consistency screening of candidate crack regions or for the removal of false cracks, but it is not the primary input for crack appearance detection. Specifically, cracks are first detected based on RGB appearance, with depth only used as a post-hoc check. For example, if a candidate crack actually corresponds to a suspended cable outside the brickwork, an object edge, a significant shadow boundary, or foreground occlusion, its candidate region usually has a significant depth abrupt change or does not belong to the same continuous surface as the surrounding wall, and its confidence level can be reduced or it can be removed. Conversely, candidate regions that are depth-continuous with adjacent walls and show compatible crack responses in multiple views are retained.

[0104] S2.2, Generate the crack centerline and boundary.

[0105] According to an embodiment of the present invention, the crack centerline and the left and right boundary information of the crack can be extracted based on the crack candidate region.

[0106] Specifically, the crack probability map output by S2.1 can be binarized according to a preset probability threshold, and the final binary crack region can be obtained through connected component filtering, morphological denoising, etc., which serves as the crack candidate region input by S2.2. When S2.1 directly outputs the binary crack region, the processed binary crack region is directly used as the crack candidate region input by S2.2.

[0107] Skeletonization is performed on the crack candidate region to obtain the crack centerline while preserving the left and right boundary information of the crack. Based on this, the pixel width is calculated at each sampling position of the crack centerline. The pixel width can be obtained from the boundary spacing in the normal direction of the centerline.

[0108] S2.3, Divide the crack into segments.

[0109] According to an embodiment of the present invention, the cracks represented by the crack candidate region can be divided into multiple crack segments based on preset key locations.

[0110] For example, a complete crack is divided into multiple crack segments based on the following locations: locations with significant changes in centerline curvature, bifurcation points, locations with abrupt changes in width, occlusion locations, and image boundary truncation locations. Each segment is then used as the smallest processing unit for subsequent scale generation, transfer, and optimization.

[0111] S2.4: Establish crack segment attribute descriptions.

[0112] According to embodiments of the present invention, structured description information can be configured for each crack segment. This structured description information includes at least the crack segment number, its associated view number, centerline coordinate sequence, crack boundary coordinate sequence, pixel width sequence, start-point coordinates, end-point coordinates, local direction vector, local curvature description, texture descriptor, and image sharpness confidence. If necessary, information such as component type, surface normal, and local reflectivity in the crack segment's neighborhood can also be added.

[0113] The local orientation vector can be obtained by linear fitting or principal component analysis (PCA) based on adjacent sampling points along the centerline. Local curvature can be calculated based on the rate of change of tangent direction and discrete curvature at adjacent sampling points along the centerline. Texture descriptors can extract descriptive features such as grayscale gradients, local binary patterns (LBP), histograms of oriented gradients (HOG), and SIFT from a predetermined width neighborhood on both sides of the crack centerline. Image sharpness confidence can be obtained based on the Laplacian variance, edge gradient intensity, and the stability of the crack's left and right boundary responses.

[0114] S3. Establish cross-view repeated observation associations for the same physical crack segment.

[0115] S3.1 Generate a set of candidate crack segment pairs.

[0116] According to an embodiment of the present invention, within the scope of the overlapping view pair defined by the view overlap diagram, the crack segment in the target first view image can be projected onto the target second view image, and crack segments in the target second view image whose distance from the projected crack segment is less than a preset distance threshold and which satisfy the basic geometric constraints can be selected to form a candidate crack segment pair set.

[0117] For example, within the scope of the view overlay map established in step S1.5, for any pair of view images i,j that are allowed to be associated, using the crack segment in view image i as a reference object, the crack segment is mapped to a candidate region in view image j based on the camera's initial pose, camera intrinsic and extrinsic parameters, local depth information, and geometric reprojection relationship. Within this candidate region of view image j, crack segments that are close in position, morphologically compatible, and satisfy basic geometric constraints are selected to form a set of candidate corresponding crack segments.

[0118] Local depth information represents the local surface depth of the sampling location of the crack centerline and its adjacent wall area. Since reliable binocular parallax may not be available inside the crack, the depth of the surface containing the crack is estimated using the surface support information established by S1.6 for the adjacent walls on both sides of the crack. Based on camera intrinsic and extrinsic parameters, the initial pose of each view image, and local depth information, the crack centerline or its local sampling points in view i are back-projected into three-dimensional spatial points according to depth, transformed to the camera coordinate system of view j, and then the target image coordinates are obtained by projecting using the camera intrinsic parameters of view j, thus forming a geometric reprojection relationship.

[0119] After length normalization, candidate crack segments are compared in terms of centerline shape, direction, and width changes. For example, if the average distance between two centerlines after reprojection is less than a preset average centerline distance threshold, the angle between the principal directions is less than a preset angle threshold, and the normalized centerline shape similarity is higher than a preset centerline similarity threshold, the cracks are considered morphologically compatible. For cracks with bifurcations, the number of adjacent bifurcations and their directions are further compared. In specific embodiments, the angle between the principal directions does not exceed 20°, and the average distance between reprojected centerlines does not exceed 10 pixels, etc., but the specific values ​​can be adjusted according to the imaging resolution and are not limited here.

[0120] The basic geometric constraints include at least reprojection deviation, local depth difference, and centerline direction difference not exceeding a preset threshold. The output of this step is not the final matching result, but rather a set of candidate segments that may belong to the same physical crack segment.

[0121] S3.2 Construct the associated features of the crack segment.

[0122] According to embodiments of the present invention, the similarity of centerline shape, local direction consistency, width distribution similarity, texture profile consistency, bifurcation topology consistency, reprojection error, and local depth consistency of each candidate crack segment pair in the candidate crack segment pair set can be extracted to obtain the correlation features between candidate crack segment pairs.

[0123] Among these, centerline shape similarity characterizes whether the overall extension morphology of two crack segments is consistent. Local direction consistency characterizes whether the changes in the crack segment's direction along the line are compatible. Width distribution similarity characterizes whether the width variation trends of the two crack segments along the line are similar. Texture profile consistency characterizes whether the grayscale, texture, or edge distribution around the crack is similar. Bifurcation topology consistency characterizes whether the structures of adjacent bifurcations, intersections, or endpoints of crack segments are consistent. Reprojection error characterizes the positional deviation of a crack segment after it is mapped to another view. Local depth consistency characterizes whether the depth and normal relationship of the surface where the crack is located are consistent.

[0124] S3.3 Calculate the comprehensive correlation score.

[0125] According to an embodiment of the present invention, a comprehensive correlation score between candidate crack segment pairs can be calculated based on correlation features. In response to detecting that the comprehensive correlation score is greater than a preset correlation threshold, a cross-view repeated observation relationship is determined to exist between the candidate crack segment pairs.

[0126] For example, let candidate crack segment pairs be a, b, and the m-th associated feature be denoted as . Its weight is Then the overall correlation score It can be expressed as formula (1).

[0127] (1).

[0128] Where M is the total number of associated features. γ is a preset association threshold.

[0129] when When candidate crack segments a and b are determined to be repeated observations of the same physical crack segment in different view images, it indicates the existence of a cross-view repeated observation relationship.

[0130] S3.4. Establish a crack correlation diagram.

[0131] According to an embodiment of the present invention, a crack association graph can be constructed using crack segment objects as nodes and cross-view repeated observation relationships as edges. The edge weights in the crack association graph represent the association confidence level.

[0132] In one implementation, the association confidence can be directly obtained by using the comprehensive association score calculated in step S3.3. In this case, the higher the comprehensive association score, the higher the confidence that the two crack segments belong to the same physical crack.

[0133] In another implementation, the association confidence score can also be the result of converting the comprehensive association score. For example, the comprehensive association score can be normalized to map its value range to the interval of 0 to 1, which can then be used as the association confidence score; or the continuous score can be discretized into high, medium, and low confidence levels according to the interval in which the comprehensive association score is located; or the comprehensive association score can be weighted and corrected by combining the degree of overlap of corresponding edges in the view overlap diagram to obtain the corrected association confidence score.

[0134] The above conversion methods are all within the protection scope of this invention.

[0135] S3.5, Execute Multi-View Figure 1 Induced elimination.

[0136] According to an embodiment of the present invention, for crack segments that have cross-view repeated observation relationships in a preset number of view images, the overall consistency of multiple views can be detected; in response to determining that multiple associated edges in the crack association graph constitute a conflict relationship based on the overall consistency of multiple views, associated edges with an association confidence of less than a preset association confidence threshold, inconsistent transmission, or corresponding geometric deviation greater than a preset geometric deviation threshold are deleted.

[0137] For example, for crack segments with candidate correlations in three or more view images, further checks should be performed on their overall consistency across multiple views. The consistency check should include at least one or more of the following: Unique correspondence constraint: In principle, the same source crack segment should not simultaneously correspond to multiple spatially separated and topologically discontinuous independent crack segments in the target view image. However, when the same physical crack is divided into two or more continuous sub-segments in the target view due to occlusion, image boundaries, segmentation, etc., one-to-many correspondences should be allowed, as long as these sub-segments, after reprojection, are located in different continuous intervals of the source crack segment and satisfy topological continuity. Transitive consistency constraint: If crack segment A corresponds to crack segment B, and crack segment B corresponds to crack segment C, then crack segment A and crack segment C should have compatible geometric relationships. Closed-loop geometric consistency constraint: The centerline position, direction, and reprojection relationship of the same crack segment in multiple views should be overall self-consistent.

[0138] The centerline position, direction, and reprojection relationship refer to the correspondence of the same physical crack segment in three or more views. They can be mutually verified in terms of spatial position, local extension direction, and geometric projection relationship between different views, and there are no obvious contradictions.

[0139] Specifically, for crack segments A, B, and C that have established candidate association relationships, multiple sampling points are first selected at preset intervals along the centerline of crack segment A. Then, using the camera intrinsic parameters, camera extrinsic parameters, initial poses of each view image, and local depth information of the corresponding positions obtained in step S1, each sampling point is back-projected onto a unified sensor coordinate system and then projected onto the view images of crack segments B and C, respectively.

[0140] For each projected sampling point, check the following relationships.

[0141] (1) Consistency of centerline position. After projecting the centerline sampling point of crack segment A onto view B or view C, calculate the shortest distance between the projected point and the centerline of the corresponding target crack segment. When the distance of most sampling points is less than the preset position deviation threshold, the corresponding crack segment is considered to be consistent in the centerline position; if the projection result deviates from the target crack segment as a whole, or some sampling points correspond to another independent crack, the candidate association is considered to have a position conflict.

[0142] (2) Consistency of centerline direction. Based on the local direction vector of the crack segment obtained in step S2.4, compare the tangential directions of the centerlines at corresponding positions. Since the imaging directions are different in different views, the directions can be converted to a unified spatial coordinate system based on the camera pose and local surface geometry, and then the direction angles at the corresponding positions can be calculated. When the direction angle is less than the preset direction difference threshold and the trend of change along the crack direction is basically consistent, the direction relationship is considered to be compatible; when the corresponding crack segments show obvious reverse deflection, intersection, or significantly inconsistent direction change trends, the association is considered to have a direction conflict.

[0143] (3) Reprojection Consistency. For the same centerline sampling position, reprojection is performed using the geometric relationships from A to B, B to C, and A to C, respectively. If the position obtained by directly projecting from A to C basically coincides with the corresponding position in B determined by A and then in C determined by B, then the association is considered to satisfy transitive consistency. Furthermore, the sampling position can be closed-loop correspondented in the order of A→B→C→A. When the deviation between the centerline position after returning to view A and the original position is less than the preset closed-loop error threshold, the association is considered to satisfy closed-loop geometric consistency.

[0144] For example, a sampling position on the centerline of crack segment A, after geometric projection, corresponds to the middle position of the centerline of crack segment B, and further corresponds to the corresponding position in crack segment C. When crack segment C is mapped back to crack segment A, the returned position is still near the original sampling position, and the centerline extension direction of the corresponding position in the three views is consistent. Therefore, A, B, and C are considered to be repeated observations of the same physical crack segment. If a candidate association causes the closed-loop return position to deviate significantly from the original position, or the centerline direction of the corresponding position is significantly inconsistent with other views, the confidence of the candidate association is reduced or the candidate association edge is deleted.

[0145] Therefore, the aforementioned "overall self-consistency" does not require that the pixel coordinates of the crack centerline in different view images be the same. Rather, it requires that after transformation by camera pose, local depth, and geometric reprojection relationship, the spatial position, line order, and extension direction of the same physical crack local location in multiple views can be mutually verified.

[0146] When a candidate associated edge constitutes a conflict relationship with other associated edges, candidate associated edges with low association confidence, inconsistent transmission, or large geometric deviations are deleted, and only crack segment correspondences that satisfy the overall consistency across multiple views are retained. Here, associated edges refer to the candidate cross-view correspondences obtained in S3.4 based on S3.1-S3.3. Each edge connects two crack segment nodes located in different views that are judged to be repeatedly observed as the same physical crack segment, and the comprehensive association score is recorded as the edge weight. Conflicting associated edges refer to associated edges among multiple candidate cross-view correspondences that cannot simultaneously satisfy uniqueness, topological relationship, or closed-loop geometric constraints.

[0147] S4. Generate local scale factors and establish scale anchor points.

[0148] S4.1 Define the local scale factor.

[0149] For any crack segment, the pixel size in the image at different locations is usually not exactly the same as its corresponding real physical size because camera viewpoint, target distance, surface curvature, and local imaging conditions can all change. Therefore, this invention does not use a single global scale parameter, but instead defines a local scale factor for the local location of the crack segment.

[0150] For any sampling position p on the crack segment, if its pixel width is The local scale factor is Then the actual width of that position It can be represented as shown in formula (2).

[0151] (2).

[0152] Similarly, if the pixel length of a certain local arc segment of the crack centerline is The corresponding true length can then be calculated using the local scale factor. In this invention, the local scale factor is not pre-assumed as a constant, but is derived from local geometric measurements and is allowed to vary along the crack direction.

[0153] According to an embodiment of the present invention, a local scale factor is used to describe "the physical size in actual space corresponding to a pixel in the image" at a certain local location of the crack. This scale factor is not uniformly set based on the entire image, but is calculated at a local location of the crack using the crack pixel position, pixel width, camera intrinsic and extrinsic parameters, camera initial pose, local depth information, and local surface geometry information obtained in the aforementioned steps.

[0154] Taking the determination of local scale factors using binocular depth information as an example. First, the crack centerline and left and right boundaries are obtained according to step S2.2. For a certain sampling position on the centerline, the left and right crack boundary points are determined along the normal direction of the centerline at that position, and the pixel distance between the two boundary points in the current image is calculated. For example, the distance between the left and right boundaries at a certain position is 8 pixels. Second, based on the camera intrinsic parameters, the initial camera pose, and the binocular depth near the crack position or the local surface support information established in step S1.6, the spatial positions of the left and right boundary points on the actual wall are determined. Specifically, when both left and right boundary points have reliable depth values, the two boundary pixels can be directly back-projected into three-dimensional space; when the crack interior or boundary position lacks stable depth, a local plane or local surface can be established using the effective depth points of the adjacent walls on both sides of the crack, and the imaging rays corresponding to the left and right boundary pixels are intersected with the local surface, thereby obtaining the spatial positions of the two boundary points on the actual wall. Then, the actual distance between the two spatial boundary points along the local wall is calculated. Assuming the actual crack width corresponding to the aforementioned 8 pixels is 2.0 mm, the local scale factor at this location is "2.0 mm corresponds to 8 pixels," meaning each pixel corresponds to 0.25 mm. Subsequent pixel widths detected at this sampling location can then be converted to the actual crack width using this ratio.

[0155] A similar approach can be used to determine the scale of the true crack length. First, select two adjacent sampling points along the centerline to obtain the pixel arc length between the two points. Then, based on camera parameters, local depth, and local surface support information, backproject the two centerline sampling points onto the actual wall surface to determine their corresponding true surface distance. Comparing the true surface distance with the corresponding pixel arc length yields the scale relationship for crack length conversion at that local location. For example, if the length of a local arc segment along the centerline in the image is 10 pixels, and the true length of this local arc segment is 3.0 mm after 3D backprojection and local wall geometry calculation, then the local scale along the crack extension direction at that location is 0.30 mm per pixel.

[0156] When the wall surface is approximately planar and the imaging tilt is small, the same local scale factor can be used to convert the pixel size within that local area. When the wall surface is tilted, the curvature or the viewing angle changes significantly, it is preferable to determine the local scale relationship based on the normal direction and the tangential direction of the crack centerline, respectively. The scale relationship in the normal direction is used to calculate the true width of the crack, and the scale relationship in the tangential direction is used to calculate the true length of the crack, so as to reduce the scale conversion error caused by perspective distortion and local surface tilt.

[0157] S4.2 Determine the calculation location of the local scale factor.

[0158] According to embodiments of the present invention, the local location of a crack segment can be determined based on image clarity, crack boundary integrity, binocular parallax effectiveness, laser landing point reliability, degree of occlusion, degree of reflection, and degree of blurring at various locations within the crack segment.

[0159] Since not every crack segment location has reliable depth or ranging information, it is preferable to first select a set of candidate local locations on the crack segment to generate local scale factors. These candidate local locations can be selected based on the following principles: high image clarity in the vicinity; intact crack boundary with distinguishable left and right boundaries; effective binocular parallax with minimal depth fluctuation; presence of a laser impact point or reliable local distance information nearby; not located in areas of significant occlusion, strong reflection, or severe blurring; and not located in the very nearest region to a crack bifurcation or in an area with highly unstable boundaries.

[0160] Reliable local distance information can be determined based on the following areas: continuous binocular depth and neighborhood depth variance below a threshold; or the presence of a laser impact point near the crack with completed extrinsic parameter conversion, and the difference between the laser distance and the estimated binocular distance being below the allowable error; or the presence of an identifiable component / scale of known size. For example, if the local binocular depth standard deviation is less than 5 mm and the relative deviation between the binocular and laser distances is less than 2%-5%, it can be determined that the location has reliable local distance information.

[0161] S4.3, Generate the initial local scale factor.

[0162] According to an embodiment of the present invention, a local scale factor at a local location of a crack segment can be generated based on at least one of binocular depth, laser ranging information, known component dimensions, manual scale information, or local three-dimensional reconstruction results at a local location of the crack segment.

[0163] For example, a local scale factor can be generated at the corresponding location on the crack segment based on at least one of the following information sources: binocular depth; laser ranging value; spatial geometric relationship between the laser landing point and the crack boundary; known dimensions of ancient building components; artificial ruler information; and local three-dimensional reconstruction results.

[0164] Among these methods, the local scale factor can be obtained by back-projecting the crack boundary points or adjacent points of the centerline into 3D space according to depth and calculating the ratio of the true distance to the 2D pixel distance. Based on laser ranging information, the surface distance and local surface model are determined using laser points near the crack, and the crack pixel rays are intersected with this surface; the local scale factor is calculated from the distance between the spatial intersection points. Based on the laser impact point and crack boundary, a local wall surface is fitted with multiple laser points near the crack, and the true distance is calculated after back-projecting the left and right boundary pixels of the crack onto this wall surface; the local scale factor can be obtained by back-projecting the pixels of the left and right boundaries of the crack onto this wall surface and calculating the true distance. Based on known component dimensions, the local scale factor can be obtained by identifying two known feature points of the component and establishing a local scale using the known true dimensions and the image / surface distance between the two points. Based on artificial ruler information, the local scale is obtained by using the known scale length and corresponding pixel length. Based on the local 3D reconstruction results, the corresponding points of the crack are found on a local 3D model that already has absolute scale, and the local scale factor is calculated from the 3D distance and the image pixel distance.

[0165] When multiple sources exist simultaneously, they should be merged according to their confidence level, rather than arbitrarily selected.

[0166] S4.4, Assess the confidence level of the scale factor.

[0167] According to embodiments of the present invention, the initial confidence value of the local scale factor can be calculated based on at least one of the following: depth variance at a local location of the crack segment, binocular and laser consistency, image sharpness, crack boundary integrity, degree of occlusion, degree of surface reflectivity, and local geometric continuity.

[0168] Because the quality of scale estimation varies at different local locations of cracks, a reliability assessment of the local scale factor is necessary. This reliability assessment can be based on one or more of the aforementioned indicators. Specifically, depth variance describes the stability of the binocular depth results within the local neighborhood. Binocular-laser consistency describes the proximity of the two scale sources. Image sharpness reflects the stability of crack boundary detection. Crack boundary integrity reflects the reliability of crack width measurements. Occlusion level determines whether the location is at risk of being obscured by components, shadows, or other objects. Surface reflectivity suppresses false scales caused by specular reflection or bright areas. Local geometric continuity determines whether the location is in a region of significant geometric abrupt change.

[0169] Based on the above indicators, calculate the initial confidence value for each local scale factor.

[0170] S4.5 Establish scale anchor points.

[0171] According to an embodiment of the present invention, in response to detecting that the initial confidence value is greater than a preset confidence threshold, the local location of the corresponding crack segment can be marked as a scale anchor point.

[0172] For example, when the initial confidence value of a local scale factor is higher than a preset confidence threshold, the corresponding local location of the crack segment is marked as a scale anchor point. A scale anchor point refers to a location or local region on the crack segment where a reliable mapping relationship between pixel size and actual size has been established.

[0173] In this invention, it is not required that every crack segment contain a scale anchor point. Establishing a small number of scale anchor points at reliable locations is sufficient to provide initial scale support for subsequent cross-view scale transfer.

[0174] S4.6 Establish a scale anchor point table.

[0175] According to embodiments of the present invention, a scale anchor table can be constructed based on the scale anchor point number, the corresponding crack segment number, the corresponding view number, the local (pixel point and / or arc length, etc.) location index, the local scale factor value, the scale source type, and the initial confidence value.

[0176] To facilitate subsequent scale transfer and global optimization, a scale anchor table can be established for all scale anchor points based on the above information. Specifically, for local location indexes, the centerline sampling point (u, v) can be recorded in the image pixel coordinate system after distortion correction within the relevant view, where u is the column coordinate and v is the row coordinate. The sampling sequence number or normalized arc length position of this point on the crack centerline is also recorded. For scale anchor points with reliable 3D positions, their 3D coordinates in a unified sensor coordinate system can also be recorded. Scale source types can include at least one of the following: binocular depth scale, laser ranging scale, binocular-laser fusion scale, known component size scale, artificial ruler scale, and local 3D reconstruction scale.

[0177] The scale anchor point table can also record local normals, local depths, and neighborhood geometric states when necessary. The neighborhood geometric states can include at least one of the following: approximately planar regions, continuously gently curving surface regions, high curvature regions, component corner regions, regions with abrupt depth changes, and regions with insufficient surface support. For each state, the local normal, curvature value, and fitting residual of the corresponding region can also be recorded, instead of just storing a text label.

[0178] S5. Transfer local scale factor between overlapping views.

[0179] S5.1 Determine the source and target crack segments for executable scale transfer.

[0180] According to embodiments of the present invention, based on the cross-view repeated observation relationship in the crack association diagram, the overlapping view pairs in the view overlap diagram, and the distribution of scale anchor points in each crack segment, crack segments that already have scale anchor points can be identified as source crack segments, and crack segments that have cross-view association relationships with source crack segments but have not yet established scale anchor points can be identified as target crack segments. The cross-view association relationship includes cross-view repeated observation relationships and association relationships corresponding to overlapping view pairs.

[0181] For example, after completing the cross-view repeated observation association of crack segments, different crack segments may still differ in the completeness of scale information. Crack segments that already have reliable scale anchors or high-confidence local scale factors are defined as source crack segments in the current transfer; crack segments that have completed cross-view matching but have not yet established reliable scales or have low scale confidence are defined as target crack segments in the current transfer. The local scale factor on one crack segment is allowed to be transferred to another crack segment only when the following conditions are met: a cross-view repeated observation relationship is established between the two crack segments in step S3; at least one reliable scale anchor or an updated high-confidence scale factor exists in the source crack segment; there is an effective overlap relationship between the views where the source crack segment and the target crack segment are located; and there is no severe occlusion, extreme blur, or significant distortion in the vicinity of the target crack segment.

[0182] In this context, a valid overlap exists between the source crack segment and the target crack segment in the view. Specifically, this is manifested in two ways: first, the two view images already form a valid edge in the view overlap map in S1.5; second, the local positions of the two corresponding crack segments for which scale transfer is to be performed are also located within a common visible area. For example, a crack segment is considered to have a valid overlap only when at least 30% of its local length is visible and unobstructed in both view images, and a stable local position correspondence can be established. The corresponding position of the target crack segment refers to the mapping from a centerline sampling position of the source crack segment to the corresponding centerline sampling position of the target crack segment and its adjacent width section, based on the repeated observation relationship of the same physical crack segment in step S5.2. For example, the scale anchor point at the 35% arc length of the source crack segment corresponds to approximately the 35% arc length position of the target crack segment after direction unification, and is further finely adjusted by local curvature or texture.

[0183] S5.2 Establish local location correspondences.

[0184] According to embodiments of the present invention, a local positional mapping between the source location in the source crack segment and the target location in the target crack segment can be established based on the crack geometry and cross-view correlation. The source location represents the location with a scale anchor point.

[0185] Specifically, one or more of the following methods can be used to determine the local position correspondence: centerline normalized arc length correspondence, endpoint correspondence constraint, curvature segmentation correspondence, and local texture correspondence.

[0186] For example, suppose the source crack segment A and the target crack segment B are determined to belong to the same physical crack. First, the start and end directions of the two crack segments are determined based on the reprojection results; if the directions of the two crack segments are opposite, the sampling order of B is reversed. Then, the total length of the two centerlines is normalized to 0 to 1, and the anchor point with an arc length ratio of 0.35 on A is first mapped to approximately 0.35 on B; then, the curvature peak near this position and the texture on both sides of the crack are compared, and the position with the highest texture similarity and the lowest reprojection error is selected within a predetermined search window before and after the target position (e.g., the 0.35 position) as the final corresponding point. For example, the 42nd sampling point of A is finally determined to correspond to the 51st sampling point of B.

[0187] After the above processing, each scale anchor point or high confidence scale position on the source crack segment can be found in the corresponding local position in the target crack segment.

[0188] S5.3 Calculate the scale transfer weight.

[0189] According to an embodiment of the present invention, the scale transfer weight at the target location corresponding to the source location can be calculated based on at least one of the following: the association confidence between the source crack segment and the target crack segment, the reprojection error, the image sharpness, the local depth confidence, the degree of occlusion, the orientation consistency, and the viewing angle.

[0190] Scale transfer weights are used to characterize the reliability of scale information transfer from a source location to a target location. The scale transfer weight at the target location corresponding to the source fracture segment s is denoted as... .

[0191] S5.4. Generate candidate scale factors for the target crack segment.

[0192] According to an embodiment of the present invention, the local scale factor at the source location can be transferred to the target location to obtain the candidate local scale factor at the target location.

[0193] For example, for a certain local location on the source crack segment Its local scale factor is denoted as Step S5.2 determines its corresponding position in the target crack segment as follows: This can form a candidate local scale factor for the target location, denoted as . .

[0194] Specifically, once the local scale factor of the source location is known, the true physical size of the corresponding local segment can be obtained using a pixel width or pixel arc length at the source location. Since the source and target locations have already been confirmed in S5.2 to be observing the same physical local segment, the candidate local scale factor of the target location can be obtained by dividing the true size by the pixel width or pixel arc length of the corresponding local segment in the target view. For example, if the pixel length corresponding to the source location region is 10 pixels and the local scale factor at the source location is 0.20 mm / pixel, then the corresponding true length is 2 mm; if the pixel length corresponding to the same local segment in the target location region is 8 pixels, then the candidate local scale factor at the target location should be 0.25 mm / pixel.

[0195] It should be noted that the candidate local scale factor is not the final scale result of the target crack segment, but rather a scale estimate provided by the source crack segment for the target crack segment.

[0196] S5.5, Integrate multiple candidate local scale factors.

[0197] According to an embodiment of the present invention, the candidate local scale factors can be weighted and fused according to the scale transfer weight to obtain the transferred local scale factor at the target location.

[0198] For example, the same crack segment is often observed repeatedly in multiple view images, so a certain local location may simultaneously receive multiple candidate local scale factors. To avoid the direct impact of a single transmission path error on the final result, this invention performs a fusion process on multiple candidate local scale factors. The fusion methods may include: weighted fusion, robust median fusion, and fusion after outlier removal.

[0199] Taking weighted fusion as an example, the local scale factor after the target location is transferred. It can be calculated using formula (3).

[0200] (3).

[0201] S5.6 Update the scale status of the target crack segment.

[0202] According to embodiments of the present invention, the overall scale state of a target crack segment can be updated based on the post-transmission local scale factor of the target crack segment at at least one target location and the post-transmission confidence value of the post-transmission local scale factor. The overall scale state characterizes whether the target crack segment is used as the source crack segment in the next transmission process or not.

[0203] For example, after a target crack segment obtains its scale factors at several local locations, the overall scale state of the target crack segment is updated. The update includes: marking which locations of the target crack segment already possess valid scale information; updating the confidence level of each scale factor of the target crack segment after transmission; and determining whether the target crack segment has transformed from a scale-less state to a high-confidence scale state that can serve as a transmission source. If the target crack segment meets the preset confidence conditions after the update, it can be included in the source crack segment set for the next round of transmission.

[0204] Valid scale information must simultaneously meet the following conditions for transmission results: the transmission weight is higher than a preset transmission weight threshold; when from multiple transmission sources, the dispersion of each transmitted local scale factor is lower than a preset dispersion threshold; the transmitted local scale sub-values ​​are within a reasonable range determined based on camera distance and local surface geometry; and there is no severe occlusion or abrupt change in surface geometry at that location. Only after these conditions are met is it marked as valid scale information.

[0205] Unlike the initial confidence value, which reflects the quality of direct measurement and is primarily derived from depth stability, binocular-laser consistency, image sharpness, and boundary integrity, updating the confidence of local scale factors after each transmission for the target crack segment requires additional consideration of source scale confidence, current transmission weight, transmission path length, and consistency among multiple transmission sources. For example, higher source scale confidence, smaller source target reprojection error, shorter transmission path, and closer scales from multiple independent sources result in higher target scale confidence. A certain attenuation should be set after each round of indirect transmission to prevent the infinite spread of low-quality results. This design also enhances the technical effect of "few anchor points but suppressed error accumulation."

[0206] S5.7, Perform iterative propagation.

[0207] According to an embodiment of the present invention, the local scale factor in the source crack segment can be iteratively passed until the iteration stopping condition is met.

[0208] For example, in the joint graph structure composed of the crack association graph and the view overlap graph, the transfer process of steps S5.1-S5.6 is iteratively executed. The view overlap graph is used to define the range of transferable view images; the crack association graph is used to define the correspondence of transferable crack segments; the scale anchor table provides the transfer starting point; and the updated overall scale state determines which crack segments can continue to serve as new transfer sources. The above process is repeated until any of the following stopping conditions are met: the number of newly added high-confidence scale results is less than a preset new threshold; the change in transfer results for two consecutive rounds is less than a preset change threshold; or the number of transfer rounds reaches a preset upper limit.

[0209] S5.8 Output the local scale results after transmission.

[0210] After completing the iterative transfer, the local scale factor and its corresponding post-transfer confidence value for each crack segment at different local locations can be obtained. The results include: the set of locations where the scale has been restored in each crack segment; the local scale factor value after transfer at each location; the number of transfer sources at each location; and the post-transfer confidence value at each location.

[0211] S6. Construct the wall curvature field and continuous scale field, and perform consistency optimization.

[0212] S6.1 Establish the wall curvature field.

[0213] According to an embodiment of the present invention, the curvature field of a building surface can be calculated based on the geometric description of the building surface.

[0214] For example, based on binocular depth and laser point clouds, a geometric description of the building surface is calculated through local surface fitting. This allows for the calculation of local curvature, surface normal, surface smoothness, and rate of curvature change in various regions of the building surface, thus constructing a curvature field. This curvature field describes the geometric changes at different locations on the building surface. For any point on the building surface, the curvature field provides information such as the curvature value, normal direction, intensity of curvature change, and degree of surface smoothness at that location.

[0215] Local surface fitting includes: selecting a certain spatial radius or a fixed number of binocular 3D points and laser points centered on the location of the wall to be processed in a unified coordinate system; firstly, outliers are removed using RANSAC. Least squares plane fitting is used when the neighborhood is close to a plane; quadratic surface fitting is used when the plane fitting residual exceeds a threshold. The surface normal and local curvature are obtained based on the differential geometric parameters of the fitted surface. For example, at least 20 effective 3D support points can be selected within a 50mm radius for fitting. The specific neighborhood size can be adjusted according to the inspection distance and point cloud density, and is not limited here.

[0216] Local curvature characterizes the degree of local bending of the wall surface. Surface normal characterizes the spatial orientation of a local wall surface, used to determine the relationship between the camera's viewing direction and the wall's tilt. Surface flatness characterizes the degree of deviation of a local 3D point from the fitted plane or fitted surface, used to distinguish between flat walls and irregular areas. The rate of change of curvature characterizes the drastic change in curvature as the wall moves along its position, used to identify geometrical abrupt changes such as corners and bulges.

[0217] The wall curvature field records at least the three-dimensional position of the surface, local curvature, surface normal, surface flatness, and rate of curvature change at each spatial location; it can also record surface fitting residuals, surface continuity, local depth, support point density / geometric confidence, and corner or depth abrupt change indicators.

[0218] S6.2 Establish the correspondence between the crack segment and the curvature field.

[0219] According to an embodiment of the present invention, the local location of each crack segment can be projected onto the curvature field of the building surface to establish a correspondence between the local location of the crack segment and the curvature field of the building surface.

[0220] For example, by projecting each crack segment and its local location onto the wall curvature field, a correspondence can be established between the local scale factor of the crack and the geometric state of the wall. Through this mapping, it can be clarified whether a certain scale result is located in a flat wall area or a region of abrupt curvature change; whether the scale results of different crack segments are on the same continuous surface; whether adjacent scale results should maintain approximate continuity; and which locations should be allowed to change scale more rapidly and which locations should be subject to stronger smoothing constraints.

[0221] S6.3 Construct a continuous scale field.

[0222] According to an embodiment of the present invention, an initial continuous scale field of the building surface can be constructed based on the above correspondence and the local scale factor at the local location of the crack segment.

[0223] The initial continuous scale field is a spatial distribution function defined on the surface of the target wall or component, used to represent the mapping relationship from pixel scale to the actual physical scale at different locations. The initial continuous scale field should satisfy the following: the scale value at the scale anchor point is consistent with the anchor point scale factor; the scale value at the transferred result is as consistent as possible with the transferred value; the scale factor change within the same continuous wall area remains piecewise smooth; and the smoothing constraint strength is reduced at component corners, local abrupt changes, or occlusion boundaries.

[0224] S6.4 Construct the optimization objective function.

[0225] According to embodiments of the present invention, an optimization objective function can be constructed based on a scale anchor point consistency term, a transitive consistency term, a curvature constraint term, a scale field smoothing term, and an anomalous scale suppression term.

[0226] The objective function is optimized as shown in formula (4).

[0227] (4).

[0228] in, For the scale anchor point consistency term; To transmit consistent items; This is a wall curvature constraint term; For scale field smoothing terms; This is an outlier scale suppression term; represents the weight coefficients for each item; E is the objective function value.

[0229] S6.5 Solve for the final scale result.

[0230] According to an embodiment of the present invention, the initial continuous scale field is optimized by minimizing the optimization objective function to obtain the optimized continuous scale field; the optimized scale value at the spatial position corresponding to the local position of the crack segment is read from the optimized continuous scale field as the final local scale factor at the local position of the crack segment, and as the final scale result.

[0231] S7. Output the true scale results and reliability of the cracks.

[0232] S7.1 Calculate the true width and true length of the crack.

[0233] According to an embodiment of the present invention, the true width at each sampling position can be calculated based on the pixel width and the final local scale factor of each sampling position of the crack segment; and the true length of the corresponding crack segment can be calculated based on the pixel arc length between each adjacent sampling point of the crack centerline and the final local scale factor.

[0234] For example, for each sampling location on the crack segment, the pixel width and corresponding final local scale factor of that sampling location are read, and the true width of that location is calculated. Here, the pixel width of the sampling location refers to the position of the current width sampling point on the crack centerline, and the corresponding final local scale factor should be the final local scale factor at the width corresponding to that sampling location along the crack normal.

[0235] For example, the pixel arc length between adjacent sampling points is calculated based on the coordinate sequence of the crack centerline, and the pixel length is converted into the true length by combining the final local scale factor at the corresponding position. Specifically, for the local arc segment formed by two adjacent centerline sampling points, the final local scale factor at the corresponding position can be converted using the length scale factor of the midpoint of the arc segment, or the average of the length scale factors at both ends.

[0236] Through the above calculations, the true width distribution along the line of each crack segment and the true length of the crack segment can be obtained. The true length of the crack segment can be expressed as shown in formula (5).

[0237] (5).

[0238] Where L represents the actual length of the crack segment; Γ represents the set of discrete local arc segments that constitute the crack centerline; and P represents one of the local arc segments or the representative position of that arc segment. The vertex vector represents the pixel arc length of local arc segment P in the image; k(P) represents the final length scale factor of the corresponding position of local arc segment P along the tangential direction of the crack, with units such as mm / pixel. The true length of the crack segment can be obtained by summing the true arc lengths of each local arc segment.

[0239] S7.2, splice continuous cracks and output the overall geometric result.

[0240] According to an embodiment of the present invention, in response to the detection that multiple crack segments are spatially adjacent, topologically continuous and belong to the same physical crack, the crack segments can be spliced ​​together to obtain the overall scale result of the entire crack.

[0241] During the splicing process, the connection order between crack segments is determined, duplicate measurements caused by repeated observation areas are removed, and the connection relationship between the main crack and the branch cracks is preserved. After splicing, the total length, maximum width, minimum width, average width, width variation curve, branch length, and crack network topology information of the entire crack are output.

[0242] Specifically, firstly, the endpoints of the centerlines of each crack segment are transformed to a unified sensor coordinate system. For two crack segments, a continuous connection is established when the spatial distance between their endpoints is lower than a preset spatial distance threshold (e.g., 10mm, but not limited to this), the tangential angle at the endpoints is lower than a preset tangential angle threshold (e.g., 30°, but not limited to this), and they are adjacent in the crack association diagram or spatial topology. A crack network topology is constructed using crack segments as edges and connecting endpoints and bifurcation points as nodes. The order of crack segments is determined based on the node connection relationship. For overlapping portions caused by multiple view images, the centerlines of two crack segments are projected onto a unified space, and the overlapping intervals are compared. When the spatial distance is consistently lower than a preset overlap spatial distance threshold, it is determined as a repeated observation, and only one length measurement is retained. The width result of the view image with higher reliability is retained first, or multiple width results are fused. When a node connects three or more crack segments, the node is marked as a bifurcation node, and each branch is retained separately, thus forming a trunk-branch crack network.

[0243] S7.3, Output the reliability of crack scale results.

[0244] According to embodiments of the present invention, the local confidence level of each crack segment and the overall confidence level of the entire crack can be determined based on at least one of the following: the number of repeated observations of each crack segment, the reliability of the scale anchor point, the length of the scale transfer path, the consistency of multi-view results, the sharpness of the local image, the stability of the local depth, and the change of the local curvature field.

[0245] For example, after obtaining the true-scale results for crack segments and the entire crack, the reliability of their measurements can be assessed. The reliability assessment can comprehensively consider the following factors: the number of repeated observations of the crack segment; the reliability of the scale anchor source; the length of the scale transfer path; the consistency of multi-view results; local image sharpness; local depth stability; and local curvature field variations. Based on these factors, the reliability score or reliability level of each crack segment and the entire crack is output to reflect the stability and acceptability of the current scale results.

[0246] The credibility score or credibility level can be determined as follows: First, normalize all the aforementioned factors to 0-1: the more repeated observations, the higher the score; the more reliable the scale anchor, the higher the score; the longer the transmission path, the lower the score; the smaller the difference between the scale results of multiple views, the higher the score; the clearer the image and the smaller the depth fluctuation, the higher the score; and the higher the score when it is on a flat and continuous surface and the curvature field is reliable. Then, the scores are weighted and summed to obtain the comprehensive credibility of 0-1.

[0247] For example, anchor point reliability and multiple views Figure 1 Consistency is given a higher weight, while propagation path, number of repeated observations, image quality, depth stability, and geometric state are given lower weights. For example, a final score ≥0.85 is a high confidence level, 0.70-0.85 is a high confidence level, 0.50-0.70 is a medium confidence level, and below 0.50 is a low confidence level. The specific weights and level thresholds can be adaptively adjusted based on calibration experiments.

[0248] Through the above embodiments of the present invention, by establishing high-confidence local scale anchor points at a small number of reliably measurable locations, and combining the cross-view repeated observation relationship of the same crack segment with the geometric prior of the building surface, local scale information is transferred between different view images and different crack locations. This makes the restoration of the true scale of cracks no longer dependent on high-precision measurement directly at each location. It can realize the true scale restoration of long cracks, continuous cracks and crack networks, and improve the consistency of measurement results of small cracks under multi-view conditions.

[0249] Figure 4 A block diagram of a building crack scale factor transfer device according to an embodiment of the present invention is shown.

[0250] like Figure 4 As shown, the building crack scale factor transfer device 400 includes a view overlay diagram establishment module 410, a crack segment extraction module 420, a crack association diagram construction module 430, a scale anchor point determination module 440, a local scale factor acquisition module 450, a local scale factor optimization module 460, and a crack overall scale result acquisition module 470.

[0251] The view overlap graph creation module 410 is used to create a view overlap graph with each view image as a node and overlapping view pairs as edges, based on the multi-view images of the building surface, binocular parallax information, laser ranging information, and inertial attitude information.

[0252] The crack segment extraction module 420 is used to detect cracks in each view image, extract multiple crack segments, and configure structured description information for each crack segment.

[0253] The crack association graph construction module 430 is used to perform association matching and consistency verification on cross-view crack segments based on the overlapping view pair range defined by the view overlap graph and combined with structured description information, and construct a crack association graph with crack segment objects as nodes and cross-view repeated observation relationships as edges.

[0254] The scale anchor point determination module 440 is used to generate local scale factors at local locations in some crack segments based on binocular parallax information, laser ranging information, and known component dimensions, and to determine scale anchor points and construct a scale anchor point table through confidence evaluation.

[0255] The local scale factor acquisition module 450 is used to iteratively transfer the local scale factor between crack segments associated across views based on the view overlap diagram and crack association diagram, combined with the scale anchor point table, to obtain the local scale factor at the local location of each crack segment.

[0256] The local scale factor optimization module 460 is used to construct and optimize a continuous scale field based on the building surface curvature field and the local scale factor of the local location of the crack segment, according to the optimization objective function, so as to obtain the final local scale factor at the local location of each crack segment.

[0257] The module 470 for obtaining the overall scale result of the crack is used to calculate the true width and true length of each crack segment based on the final local scale factor at the local location of each crack segment, and to splice together multiple crack segments belonging to the same physical crack to obtain the overall scale result and reliability of the entire crack.

[0258] According to an embodiment of the present invention, the multi-view images are acquired based on a binocular camera, the binocular parallax information is determined by stereo matching of the left and right images of the binocular camera, the laser ranging information is acquired based on a laser ranging unit, and the inertial attitude information is acquired based on an inertial measurement unit. The view overlap graph establishment module includes a coordinate system unit, an initial pose calculation unit, a candidate overlapping view pair formation unit, and a view overlap graph construction unit.

[0259] The coordinate system unit is used for time synchronization and extrinsic parameter calibration of the binocular camera, laser ranging unit, and inertial measurement unit, and to establish a unified sensor coordinate system.

[0260] The initial pose calculation unit is used to calculate the initial pose of each view image in a unified sensor coordinate system based on the calibrated inertial attitude information, image registration results and multi-view geometric constraints. The initial pose includes the camera center position and the camera optical axis direction.

[0261] The candidate overlapping view pair set forming unit is used to filter view image pairs that are adjacent in acquisition time, have an acquisition position distance less than a preset distance threshold, have a viewing angle difference less than a preset angle threshold, or have an image similarity greater than a preset similarity threshold, based on the acquisition timestamp, camera center position, camera optical axis direction, and image registration results of each view image, in order to form a candidate overlapping view pair set.

[0262] The view overlap graph construction unit is used to construct a view overlap graph based on the initial pose of each view image and the spatial position information of the building surface, combined with the spatial overlap and image matching degree of each candidate overlapping view pair in the candidate overlapping view pair set.

[0263] According to an embodiment of the present invention, the view overlay graph construction unit includes an effective pixel ratio statistics subunit, a spatial overlap constraint subunit, an overlapping view pair determination subunit, and a view overlay graph construction subunit.

[0264] The effective pixel ratio statistics subunit is used to project the effective pixel area of ​​the first view image in the candidate overlapping view pair set onto the effective imaging range area of ​​the second view image based on the initial pose and spatial position information of each view image, and to count the pixel ratio of the pixels that fall into the effective imaging range area after projection and are not occluded to the total pixels in the effective imaging range area.

[0265] The spatial overlap constraint subunit is used to determine that the first view image and the second view image satisfy the spatial overlap constraint in response to the detection that the pixel ratio is greater than a preset geometric overlap threshold.

[0266] The overlapping view pair determination subunit is used to determine that the target first view image and the target second view image constitute an overlapping view pair in response to detecting that the number of image matching feature points between the target first view image and the target second view image that satisfy the spatial overlap constraint is greater than a preset image quality threshold.

[0267] The view overlap graph construction sub-unit is used to construct a view overlap graph with each view image as a node and overlapping view pairs as edges. The edge weights in the view overlap graph represent the degree of overlap between overlapping view pairs.

[0268] According to an embodiment of the present invention, the crack segment extraction module includes a crack detection unit, a crack information extraction unit, a crack division unit, and a crack segment configuration unit.

[0269] The crack detection unit is used to detect cracks in the enhanced images of each view and obtain crack candidate regions.

[0270] The crack information extraction unit is used to extract the crack centerline and left and right boundary information of the crack based on the crack candidate region.

[0271] The crack segmentation unit is used to divide the cracks represented by the crack candidate region into multiple crack segments according to preset key locations.

[0272] The crack segment configuration unit is used to configure structured description information for each crack segment. The structured description information includes crack segment number, view number, centerline coordinate sequence, crack boundary coordinate sequence, pixel width sequence, start coordinates, end coordinates, local direction vector, local curvature description, texture descriptor, and image sharpness confidence.

[0273] According to an embodiment of the present invention, the crack association graph construction module includes a candidate crack segment pair set composition unit, an association feature extraction unit, a comprehensive association score calculation unit, a cross-view repeated observation relationship determination unit, and a crack association graph construction unit.

[0274] The candidate crack segment pair set is a unit used to project crack segments in the target first view image onto the target second view image within the scope of the overlapping view pair defined by the view overlap image, and select crack segments from the target second view image whose distance from the projected crack segment is less than a preset distance threshold and which meet the basic geometric constraints to form a candidate crack segment pair set.

[0275] The association feature extraction unit is used to extract the centerline shape similarity, local direction consistency, width distribution similarity, texture profile consistency, bifurcation topology consistency, reprojection error, and local depth consistency of each candidate crack segment pair in the candidate crack segment pair set, so as to obtain the association features between the candidate crack segment pairs.

[0276] The comprehensive correlation score calculation unit is used to calculate the comprehensive correlation score between candidate crack segment pairs based on correlation characteristics.

[0277] The cross-view repeated observation relationship determination unit is used to determine the existence of a cross-view repeated observation relationship between candidate crack segment pairs in response to the detection that the comprehensive association score is greater than a preset association threshold.

[0278] The crack association graph construction unit is used to construct a crack association graph with crack segment objects as nodes and cross-view repeated observation relationships as edges. The edge weights in the crack association graph represent the association confidence.

[0279] According to an embodiment of the present invention, the crack association graph construction module further includes a multi-view overall consistency detection unit and an association edge deletion unit.

[0280] The multi-view overall consistency detection unit is used to detect the overall consistency of crack segments that have cross-view repeated observation relationships in a preset number of view images.

[0281] The associated edge deletion unit is used to delete associated edges that have a lower association confidence level than a preset association confidence level threshold, are inconsistent in transmission, or have a corresponding geometric deviation greater than a preset geometric deviation threshold, in response to the determination that multiple associated edges in the crack association graph constitute a conflict relationship based on the overall consistency of the multi-view.

[0282] According to an embodiment of the present invention, the scale anchor point determination module includes a crack segment local location determination unit, a local scale factor generation unit, an initial confidence value calculation unit, a scale anchor point marking unit, and a scale anchor point table construction unit.

[0283] The crack segment local location determination unit is used to determine the local location of the crack segment based on the image clarity, crack boundary integrity, binocular parallax effectiveness, laser landing point reliability, degree of occlusion, degree of reflection, and degree of blurring at each location of the crack segment.

[0284] The local scale factor generation unit is used to generate the local scale factor at the local location of the crack segment based on at least one of the following: binocular depth, laser ranging information, known component size, manual scale information, or local three-dimensional reconstruction results.

[0285] The initial confidence value calculation unit is used to calculate the initial confidence value of the local scale factor based on at least one of the following: depth variance at the local location of the crack segment, binocular and laser consistency, image sharpness, crack boundary integrity, degree of occlusion, degree of surface reflectivity, and local geometric continuity.

[0286] The scale anchor point marking unit is used to mark the local location of the crack segment as a scale anchor point in response to detecting that the initial confidence value is greater than the preset confidence threshold.

[0287] The scale anchor table construction unit is used to construct a scale anchor table based on the scale anchor number, the crack segment number to which it belongs, the view number to which it belongs, the local location index, the local scale factor value, the scale source type, and the initial confidence value.

[0288] According to an embodiment of the present invention, the local scale factor acquisition module includes a source / target crack segment determination unit, a local scale factor transfer unit, a scale transfer weight calculation unit, a local scale factor calculation unit after transfer, and an overall scale state update unit.

[0289] The source / target crack segment determination unit is used to determine crack segments with scale anchors as source crack segments based on cross-view repeated observation relationships in the crack association diagram, overlapping view pairs in the view overlap diagram, and the distribution of scale anchor points in each crack segment. Crack segments that have cross-view association relationships with source crack segments but have not yet established scale anchor points are determined as target crack segments. The cross-view association relationships include cross-view repeated observation relationships and association relationships corresponding to overlapping view pairs.

[0290] The local scale factor transfer unit is used to establish a local position mapping between the source position in the source crack segment and the target position in the target crack segment based on the crack geometry and cross-view correlation, and transfer the local scale factor at the source position to the target position to obtain the candidate local scale factor at the target position. Here, the source position represents the position with a scale anchor point.

[0291] The scale transfer weight calculation unit is used to calculate the scale transfer weight at the target location corresponding to the source location based on at least one of the following: the association confidence between the source crack segment and the target crack segment, the reprojection error, the image sharpness, the local depth confidence, the degree of occlusion, the orientation consistency, and the angle of view.

[0292] The local scale factor calculation unit after transfer is used to perform weighted fusion calculation on the candidate local scale factors according to the scale transfer weight, so as to obtain the local scale factor after transfer at the target location.

[0293] The overall scale state update unit is used to update the overall scale state of the target crack segment based on the local scale factor after transmission at at least one target location and the confidence value of the local scale factor after transmission. The overall scale state represents whether the target crack segment is used as the source crack segment in the next transmission process or not, so as to iteratively transmit the local scale factor in the source crack segment until the iteration stop condition is met, and obtain the local scale factor after transmission and the confidence value after transmission of each crack segment.

[0294] According to an embodiment of the present invention, the local scale factor optimization module includes a building surface geometric description establishment unit, a building surface curvature field calculation unit, a correspondence establishment unit, an initial continuous scale field construction unit, an optimization objective function construction unit, an initial continuous scale field optimization unit, and a final local scale factor determination unit.

[0295] The building surface geometry description establishment unit is used to establish the building surface geometry description based on the initial pose of each view image and the spatial position information of the building surface. The building surface geometry description includes local depth, normal, local flatness and surface continuity information.

[0296] The building surface curvature field calculation unit is used to calculate the building surface curvature field based on the geometric description of the building surface.

[0297] The correspondence establishment unit is used to project the local position of each crack segment onto the curvature field of the building surface, and establish the correspondence between the local position of the crack segment and the curvature field of the building surface.

[0298] The initial continuous scale field construction unit is used to construct the initial continuous scale field of the building surface based on the correspondence and the local scale factor at the local location of the crack segment.

[0299] An optimization objective function construction unit is used to construct an optimization objective function based on scale anchor consistency terms, transitive consistency terms, curvature constraint terms, scale field smoothing terms, and anomalous scale suppression terms.

[0300] The initial continuous scale field optimization unit is used to optimize the initial continuous scale field by minimizing the optimization objective function, so as to obtain the optimized continuous scale field.

[0301] The final local scale factor determination unit is used to read the optimized scale value at the spatial location corresponding to the local location of the crack segment from the optimized continuous scale field, and use it as the final local scale factor at the local location of the crack segment.

[0302] According to an embodiment of the present invention, the crack overall scale result acquisition module includes a true width calculation unit, a true length calculation unit, a crack segment splicing unit, and a reliability determination unit.

[0303] The true width calculation unit is used to calculate the true width at each sampling location based on the pixel width and the final local scale factor of each sampling location in the crack segment.

[0304] The true length calculation unit is used to calculate the true length of the corresponding crack segment based on the pixel arc length between each adjacent sampling point of the crack centerline and the final local scale factor.

[0305] The crack segment splicing unit is used to splice the crack segments in response to the detection that multiple crack segments are spatially adjacent, topologically continuous and belong to the same physical crack, so as to obtain the overall scale result of the entire crack.

[0306] The credibility determination unit is used to determine the local credibility of each crack segment and the overall credibility of the entire crack based on at least one of the following: the number of repeated observations of each crack segment, the reliability of the scale anchor point, the length of the scale transfer path, the consistency of the multi-view results, the sharpness of the local image, the stability of the local depth, and the change of the local curvature field.

[0307] Any one or more of the modules, units, and subunits, or at least part of the functions of any one or more of them, according to embodiments of the present invention, can be implemented in one module. Any one or more of the modules, units, and subunits according to embodiments of the present invention can be implemented by dividing them into multiple modules. Any one or more of the modules, units, and subunits according to embodiments of the present invention can be at least partially implemented as hardware circuits, such as field-programmable gate arrays (FPGAs), programmable logic arrays (PLAs), systems-on-a-chip, systems-on-a-substrate, systems-on-package, application-specific integrated circuits (ASICs), or implemented by hardware or firmware through any other reasonable means of integrating or packaging circuits, or implemented in any one of software, hardware, and firmware, or in a suitable combination of any of these. Alternatively, one or more of the modules, units, and subunits according to embodiments of the present invention can be at least partially implemented as computer program modules, which, when run, can perform corresponding functions.

[0308] For example, any multiple modules among the following can be combined into one module / unit / subunit: view overlay map creation module 410, crack segment extraction module 420, crack association map construction module 430, scale anchor point determination module 440, local scale factor acquisition module 450, local scale factor optimization module 460, and crack overall scale result acquisition module 470. Alternatively, any one of these modules / units / subunits can be split into multiple modules / units / subunits. Or, at least some of the functionality of one or more of these modules / units / subunits can be combined with at least some of the functionality of other modules / units / subunits and implemented within one module / unit / subunit. According to embodiments of the present invention, at least one of the view overlay map creation module 410, crack segment extraction module 420, crack correlation map construction module 430, scale anchor point determination module 440, local scale factor acquisition module 450, local scale factor optimization module 460, and crack overall scale result acquisition module 470 can be at least partially implemented as hardware circuits, such as field-programmable gate arrays (FPGAs), programmable logic arrays (PLAs), systems-on-a-chip, systems-on-a-substrate, systems-on-package, application-specific integrated circuits (ASICs), or any other reasonable means of integrating or packaging circuits, or implemented in hardware or firmware, or in any one of software, hardware, and firmware implementations, or in a suitable combination of any of these. Alternatively, at least one of the view overlay map creation module 410, crack segment extraction module 420, crack correlation map construction module 430, scale anchor point determination module 440, local scale factor acquisition module 450, local scale factor optimization module 460, and crack overall scale result acquisition module 470 can be at least partially implemented as computer program modules, which can perform corresponding functions when the computer program module is run.

[0309] Figure 5 A block diagram of an electronic device suitable for implementing a method for transferring building crack scale factors according to an embodiment of the present invention is shown. Figure 5 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments of the present invention.

[0310] like Figure 5As shown, an electronic device 500 according to an embodiment of the present invention includes a processor 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage portion 508 into a random access memory (RAM) 503. The processor 501 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 501 may also include onboard memory for caching purposes. The processor 501 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0311] RAM 503 stores various programs and data required for the operation of electronic device 500. Processor 501, ROM 502, and RAM 503 are interconnected via bus 504. Processor 501 executes various operations of the method flow according to embodiments of the present invention by executing programs in ROM 502 and / or RAM 503. It should be noted that the programs may also be stored in one or more memories other than ROM 502 and RAM 503. Processor 501 may also execute various operations of the method flow according to embodiments of the present invention by executing programs stored in said one or more memories.

[0312] According to an embodiment of the present invention, the electronic device 500 may further include an input / output (I / O) interface 505, which is also connected to a bus 504. The electronic device 500 may also include one or more of the following components connected to the input / output (I / O) interface 505: an input section 506 including a keyboard, mouse, etc.; an output section 507 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a LAN card, modem, etc. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the input / output (I / O) interface 505 as needed. A removable medium 511, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 510 as needed so that computer programs read from it can be installed into the storage section 508 as needed.

[0313] According to embodiments of the present invention, the method flow according to embodiments of the present invention can be implemented as a computer software program. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a computer-readable storage medium, the computer program containing program code for performing the method shown in the flowchart. In such embodiments, the computer program can be downloaded and installed from a network via communication section 509, and / or installed from removable medium 511. When the computer program is executed by processor 501, it performs the functions defined in the system of the embodiments of the present invention. According to embodiments of the present invention, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0314] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs, which, when executed, implement the building crack scale factor transfer method according to embodiments of the present invention.

[0315] According to embodiments of the present invention, the computer-readable storage medium may be a non-volatile computer-readable storage medium. Examples include, but are not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In the present invention, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0316] For example, according to embodiments of the present invention, a computer-readable storage medium may include the ROM 502 and / or RAM 503 described above and / or one or more memories other than ROM 502 and RAM 503.

[0317] Embodiments of the present invention also include a computer program product comprising a computer program containing program code for performing the methods provided in the embodiments of the present invention. When the computer program product is run on an electronic device, the program code is used to enable the electronic device to implement the building crack scale factor transfer method provided in the embodiments of the present invention.

[0318] When the computer program is executed by the processor 501, it performs the functions defined in the system / apparatus of this embodiment of the invention. According to embodiments of the invention, the systems, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0319] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and may be downloaded and installed via the communication section 509, and / or installed from a removable medium 511. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0320] According to embodiments of the present invention, program code for executing the computer programs provided in the embodiments of the present invention can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C", or similar programming languages. The program code can be executed entirely on the user's computing device, partially on the user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0321] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions. Those skilled in the art will understand that the features recited in the various embodiments and / or claims of the present invention can be combined and / or combined in various ways, even if such combinations or combinations are not expressly stated in the present invention. In particular, the features described in the various embodiments and / or claims of this invention can be combined and / or combined in various ways without departing from the spirit and teachings of this invention. All such combinations and / or combinations fall within the scope of this invention.

[0322] The embodiments of the present invention have been described above. However, these embodiments are merely illustrative and not intended to limit the scope of the invention. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. The scope of the invention is defined by the appended claims and their equivalents. Various substitutions and modifications can be made by those skilled in the art without departing from the scope of the invention, and all such substitutions and modifications should fall within the scope of the invention.

Claims

1. A method for transferring building crack scale factors, characterized in that, The method includes: Based on the multi-view images of the building surface, binocular parallax information, laser ranging information, and inertial attitude information, a view overlap graph is established with each view image as a node and overlapping view pairs as edges. Crack detection is performed on each view image, multiple crack segments are extracted, and structured description information is configured for each crack segment; Based on the scope of overlapping view pairs defined by the view overlap graph, and combined with the structured description information, cross-view crack segments are associated and matched and their consistency is verified to construct a crack association graph with crack segment objects as nodes and cross-view repeated observation relationships as edges. Based on the binocular parallax information, the laser ranging information, and the known component dimensions, local scale factors are generated at local locations in some crack segments, and scale anchor points are determined and a scale anchor point table is constructed through confidence evaluation. Based on the view overlap diagram and the crack association diagram, and in conjunction with the scale anchor point table, the local scale factor is iteratively transferred between crack segments associated across views to obtain the local scale factor at the local location of each crack segment. Based on the objective function, a continuous scale field is constructed and optimized according to the curvature field of the building surface and the local scale factor of the local location of the crack segment, so as to obtain the final local scale factor at the local location of each crack segment. The true width and true length of each crack segment are calculated based on the final local scale factor at the local location of each crack segment. Multiple crack segments belonging to the same physical crack are spliced ​​together to obtain the overall scale result and reliability of the entire crack.

2. The method according to claim 1, characterized in that, The multi-view images are acquired based on a binocular camera, the binocular parallax information is determined by stereo matching of the left and right images of the binocular camera, the laser ranging information is acquired based on a laser ranging unit, and the inertial attitude information is acquired based on an inertial measurement unit. The step of establishing a view overlap graph based on multi-view images of the building surface, binocular parallax information, laser ranging information, and inertial attitude information, with each view image as a node and overlapping view pairs as edges, includes: Time synchronization and extrinsic parameter calibration are performed on the binocular camera, the laser ranging unit, and the inertial measurement unit to establish a unified sensor coordinate system; Based on the calibrated inertial attitude information, image registration results, and multi-view geometric constraints, the initial pose of each view image in the unified sensor coordinate system is calculated, wherein the initial pose includes the camera center position and the camera optical axis direction; Based on the acquisition timestamps, camera center positions, camera optical axis directions, and image registration results of each view image, view image pairs that are adjacent in acquisition time, have an acquisition position spacing of less than a preset distance threshold, have a viewing angle difference of less than a preset angle threshold, or have an image similarity greater than a preset similarity threshold are selected to form a candidate overlapping view pair set. Based on the initial pose of each view image and the spatial location information of the building surface, and combined with the spatial overlap and image matching degree of each candidate overlapping view pair in the candidate overlapping view pair set, a view overlap map is constructed.

3. The method according to claim 2, characterized in that, The step of constructing a view overlap map based on the initial pose and spatial position information of each view image, combined with the spatial overlap and image matching degree of each candidate overlapping view pair in the candidate overlapping view pair set, includes: Based on the initial pose and spatial position information of the building surface of each view image, the effective pixel area of ​​the first view image in the candidate overlapping view pair set is projected onto the effective imaging range area of ​​the second view image, and the pixel ratio of the pixels that fall into the effective imaging range area after projection and are not occluded to the total pixels in the effective imaging range area is counted. In response to detecting that the pixel ratio is greater than a preset geometric overlap threshold, it is determined that the first view image and the second view image satisfy spatial overlap constraints; In response to detecting that the number of image matching feature points between a target first view image and a target second view image that satisfy the spatial overlap constraint is greater than a preset image quality threshold, it is determined that the target first view image and the target second view image constitute an overlapping view pair; The view overlap graph is constructed using each view image as a node and the overlapping view pairs as edges, wherein the edge weights in the view overlap graph represent the degree of overlap between the overlapping view pairs.

4. The method according to claim 1, characterized in that, The process of detecting cracks in each view image, extracting multiple crack segments, and configuring structured description information for each crack segment includes: Crack detection is performed on each view image after crack enhancement to obtain crack candidate regions; Based on the candidate crack region, extract the crack centerline and the left and right boundary information of the crack; Based on preset key locations, the cracks represented by the crack candidate region are divided into multiple crack segments; Each crack segment is configured with structured description information, which includes crack segment number, view number, centerline coordinate sequence, crack boundary coordinate sequence, pixel width sequence, start coordinate, end coordinate, local direction vector, local curvature description, texture descriptor, and image sharpness confidence.

5. The method according to claim 3, characterized in that, The process of constructing a crack association graph, which uses the overlapping view pair range defined by the view overlap graph and the structured description information, to perform association matching and consistency verification on cross-view crack segments and construct a crack association graph with crack segment objects as nodes and cross-view repeated observation relationships as edges, includes: Within the overlapping view pair defined by the view overlap diagram, the crack segment in the target first view image is projected onto the target second view image. Crack segments that are less than a preset distance threshold and satisfy basic geometric constraints are selected from the target second view image to form a candidate crack segment pair set. For each candidate crack segment pair in the candidate crack segment pair set, extract the centerline shape similarity, local direction consistency, width distribution similarity, texture profile consistency, bifurcation topology consistency, reprojection error, and local depth consistency to obtain the correlation features between the candidate crack segment pairs; Calculate the comprehensive correlation score between the candidate crack segment pairs based on the correlation features; In response to the detection that the comprehensive correlation score is greater than a preset correlation threshold, it is determined that there is a cross-view repeated observation relationship between the candidate crack segment pairs; Using crack segment objects as nodes and the cross-view repeated observation relationship as edges, the crack association graph is constructed, wherein the edge weights in the crack association graph represent the association confidence.

6. The method according to claim 5, characterized in that, The step of constructing a crack association graph with crack segment objects as nodes and cross-view repeated observation relationships as edges, based on the overlapping view pair range defined by the view overlap graph and combined with the structured description information, further includes: For crack segments that exhibit cross-view repeated observation relationships in a preset number of view images, detect the overall consistency of the multi-view images; In response to the determination based on the overall consistency of the multi-view that multiple associated edges in the crack association graph constitute a conflict relationship, associated edges with an association confidence level less than a preset association confidence level threshold, inconsistent transmission, or corresponding geometric deviation greater than a preset geometric deviation threshold are deleted.

7. The method according to claim 1, characterized in that, The step of generating local scale factors at local locations in some crack segments based on the binocular parallax information, the laser ranging information, and the known component dimensions, and determining scale anchor points and constructing a scale anchor point table through confidence evaluation includes: The local location of the crack segment is determined based on the image clarity, crack boundary integrity, binocular parallax effectiveness, laser landing point reliability, degree of occlusion, degree of reflection, and degree of blurring at various locations within the crack segment. Based on at least one of the following: binocular depth measurement information, laser ranging information, known component dimensions, manual scale information, or local 3D reconstruction results at a local location of the crack segment, a local scale factor at the local location of the crack segment is generated. The initial confidence value of the local scale factor is calculated based on at least one of the following: depth variance at a local location of the crack segment, binocular and laser consistency, image sharpness, crack boundary integrity, degree of occlusion, degree of surface reflectivity, and local geometric continuity. In response to detecting that the initial confidence value is greater than a preset confidence threshold, the local location of the crack segment is marked as a scale anchor point; A scale anchor table is constructed based on the scale anchor point number, the crack segment number to which it belongs, the view number to which it belongs, the local location index, the local scale factor value, the scale source type, and the initial confidence value.

8. The method according to claim 1, characterized in that, The step of iteratively transferring local scale factors between cross-view associated crack segments based on the view overlap diagram and the crack association diagram, combined with the scale anchor point table, to obtain the local scale factors at local locations of each crack segment includes: Based on the cross-view repeated observation relationship in the crack association diagram, the overlapping view pairs in the view overlap diagram, and the distribution of scale anchor points in each crack segment, the crack segments that already have scale anchor points are identified as source crack segments, and the crack segments that have cross-view association relationships with the source crack segments but have not yet established scale anchor points are identified as target crack segments. The cross-view association relationship includes the cross-view repeated observation relationship and the association relationship corresponding to the overlapping view pairs. Based on the crack geometry and cross-view correlation, a local position mapping is established between the source position in the source crack segment and the target position in the target crack segment. The local scale factor at the source position is then transferred to the target position to obtain the candidate local scale factor at the target position. The source position represents the position with a scale anchor point. Calculate the scale transfer weight at the target location corresponding to the source location based on at least one of the following: the correlation confidence between the source and target crack segments, reprojection error, image sharpness, local depth confidence, occlusion degree, orientation consistency, and view angle. Based on the scale transfer weight, the candidate local scale factors are weighted and fused to obtain the transferred local scale factor at the target location. Based on the local scale factor of the target crack segment at at least one target location after transmission and the confidence value of the local scale factor after transmission, the overall scale state of the target crack segment is updated. The overall scale state indicates whether the target crack segment is used as the source crack segment in the next transmission process or not, so as to iteratively transmit the local scale factor in the source crack segment until the iteration stopping condition is met, and the local scale factor and confidence value after transmission of each crack segment are obtained.

9. The method according to claim 1, characterized in that, The process involves constructing and optimizing a continuous scale field based on the objective function, the curvature field of the building surface, and the local scale factors at the local locations of the crack segments. The final local scale factors at each crack segment's local location include: Based on the initial pose of each view image and the spatial position information of the building surface, a geometric description of the building surface is established, wherein the geometric description of the building surface includes local depth, normal, local flatness and surface continuity information; Calculate the curvature field of the building surface based on the geometric description of the building surface; The local locations of each crack segment are projected onto the curvature field of the building surface to establish the correspondence between the local locations of the crack segments and the curvature field of the building surface. Based on the correspondence and the local scale factor at the local location of the crack segment, an initial continuous scale field is constructed for the building surface; An optimization objective function is constructed based on the scale anchor consistency term, the transitive consistency term, the curvature constraint term, the scale field smoothing term, and the anomalous scale suppression term. The initial continuous-scale field is optimized by minimizing the optimization objective function to obtain the optimized continuous-scale field; The optimized scale value at the spatial location corresponding to the local location of the crack segment is read from the optimized continuous scale field and used as the final local scale factor at the local location of the crack segment.

10. The method according to claim 1, characterized in that, The process involves calculating the true width and length of each crack segment based on the final local scale factor at each local location, and then splicing together multiple crack segments belonging to the same physical crack to obtain the overall scale result and reliability of the entire crack, including: The true width at each sampling location is calculated based on the pixel width and the final local scale factor of the crack segment. The true length of the corresponding crack segment is calculated based on the pixel arc length between each adjacent sampling point of the crack centerline and the final local scale factor. In response to the detection of multiple crack segments that are spatially adjacent, topologically continuous and belong to the same physical crack, the crack segments are spliced ​​together to obtain the overall scale result of the entire crack. The local confidence level of each crack segment and the overall confidence level of the entire crack are determined based on at least one of the following: the number of repeated observations of each crack segment, the reliability of the scale anchor point, the length of the scale transfer path, the consistency of the multi-view results, the sharpness of the local image, the stability of the local depth, and the change of the local curvature field.

Images