An automatic road and bridge defect diagnosis system based on image recognition
By establishing a mechanical mapping between defect pixels and structural mesh models and multi-view image recognition, the problem of the inability to quantify the impact of damage on structural safety in existing technologies has been solved. This enables three-dimensional identification and automatic diagnosis of defects in roads and bridges, providing a scientific basis for damage assessment and reinforcement decisions.
Patent Information
- Application Number
- CN202611628125.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-10-21
- Publication Date
- 2026-11-17
AI Technical Summary
Existing technologies can only identify the surface geometric features of roads and bridges, but cannot quantify the substantial impact of damage on structural safety, and are difficult to automatically infer the causes of defects and assess the risk of damage evolution.
By establishing a mechanical mapping between defect pixels and structural mesh models, introducing physical evolution trajectory attributes and local energy density, and employing multi-view image recognition and 3D reconstruction techniques, crack features are extracted and crack driving force indices and stability reduction factors are calculated to generate automatic diagnostic results.
It achieves a precise leap from two-dimensional pixels to three-dimensional structures, enabling the quantification of crack physical dimensions, identification of crack evolution networks, scientific determination of crack trends, calculation of stability reduction factors, and provision of diagnostic conclusions with mechanical support.
Smart Images

Figure CN123118650A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the technical field of defect diagnosis, and in particular to an automatic defect diagnosis system for roads and bridges based on image recognition. Background Technology
[0002] In recent years, the detection of defects in roads and bridges has mainly relied on manual inspections, which suffers from limitations such as low efficiency and strong subjectivity. Although computer vision technology has been introduced, existing technologies mostly focus on pixel-level recognition of two-dimensional images, only extracting geometric features such as length and width. This leads to a serious disconnect between visual damage representation and the intrinsic stress nature of the structure, making it impossible to perceive the distribution of defects in three-dimensional space and their impact on structural stability. Furthermore, due to the lack of precise mapping with the structural design coordinate system and mechanical reasoning models, the system struggles to identify the varying degrees of danger of defects at different component locations, and cannot automatically determine the specific causes of defects, making it difficult for the detection results to directly guide scientific maintenance. Therefore, achieving a leap from surface image recognition to in-depth mechanical evaluation is a pressing technical challenge for the industry.
[0003] Currently, Chinese invention patent application CN202410667740.4 discloses a method and apparatus for predicting the development and attitude of structural fractures. This includes: constructing a three-dimensional surface data volume of structural and fault planes; obtaining the fracture attitude and principal curvature distribution of the three-dimensional surfaces based on the data volume; obtaining the fault curvature, tensional zone curvature, and shear zone curvature of the target block based on a three-dimensional geomechanical model; and predicting the structural fracture attitude of the target block based on the fracture attitude, fault curvature, tensional zone curvature, and shear zone curvature of the three-dimensional surfaces of the structural and fault planes. This invention introduces fracture attitude prediction into the overall fracture prediction process, enabling structural fractures under strong stress backgrounds to not only have the concept of the number of fractures but also attitude information such as dip angle and strike, making the effectiveness evaluation of structural fractures under strong stress backgrounds more targeted and beneficial to the application of fracture prediction results in oil and gas exploration and development. However, related technologies can only identify surface geometric features and cannot quantify the substantial impact of damage on structural safety. This invention addresses the pain points of difficulty in automatically inferring the cause of defects and the inability to quantitatively assess the risk of damage evolution by establishing a mechanical mapping between defect pixels and structural mesh models and introducing physical evolution trend attributes and local energy density. Summary of the Invention
[0004] The technical problem addressed by this invention is that related technologies can only identify surface geometric features and cannot quantify the substantial impact of damage on structural safety. This invention solves the pain points of difficulty in automatically inferring defect causes and quantitatively assessing damage evolution risks by establishing a mechanical mapping between defect pixels and structural mesh models, and introducing physical evolution trajectory attributes and local energy density.
[0005] To solve the above-mentioned technical problems, the present invention provides the following technical solution:
[0006] An automatic diagnosis system for road and bridge defects based on image recognition includes an acquisition module, an image construction module, a feature extraction module, and a result generation module.
[0007] The acquisition module is used to acquire multi-view defect image sequences, extract defect segmentation mask images based on the multi-view defect image sequences, and construct a structural mesh model.
[0008] The graph construction module is used to map the defect pixels in the defect segmentation mask image to a set of three-dimensional defect points located in the structural mesh model, and to construct the three-dimensional defect point set of crack type as a surface embedded crack image, and to construct the three-dimensional defect point set of peeling type as a peeling region on the structural mesh model.
[0009] The feature extraction module is used to extract crack features based on the surface-embedded crack map and to calculate the cross-sectional weakening index of each component of the road and bridge structure based on the spalling area.
[0010] The result generation module is used to calculate the crack driving force index, comprehensive crack network index and crack stability reduction factor based on crack characteristics and cross-sectional weakening index, and calculate the comprehensive cause score of each component location range based on the fault cause assumption, and combine them to generate automatic defect diagnosis results.
[0011] Preferably, the acquisition module includes an image acquisition unit and a 3D mesh construction unit;
[0012] The image acquisition unit is used to acquire defect images covering the surface of the road and bridge structure to be detected, obtain a multi-view defect image sequence, and use a convolutional neural network to segment the defect images to obtain a defect segmentation mask.
[0013] The three-dimensional mesh building unit is used to perform multi-view geometric reconstruction based on multi-view defect image sequences to obtain the camera pose and sparse three-dimensional feature point coordinates corresponding to the defect images.
[0014] Multi-view stereo matching is performed on multi-view defect image sequences based on sparse 3D feature point coordinates to generate 3D point cloud data.
[0015] The 3D point cloud data is processed into a mesh to obtain a structural mesh model.
[0016] Preferably, the graph construction module includes a point set generation unit, a graph generation unit, and a peeling region construction unit;
[0017] The point set generation unit is used to back-project the defect pixels in the defect segmentation mask into spatial straight line rays according to the camera pose corresponding to the defect image.
[0018] By performing spatial geometric intersection operations between the spatial straight line ray and the structural mesh model, three-dimensional intersection points are obtained. The three-dimensional intersection points corresponding to all defect pixels are then obtained to obtain a three-dimensional defect point set.
[0019] The graph generation unit is used to construct a surface-embedded crack graph and calculate the local crack width based on the structural mesh model;
[0020] The peeling region construction unit is used to extract the region boundary of the three-dimensional defect point set with the defect type of peeling on the structural mesh model, and the region boundary is represented as the boundary curve of the peeling region.
[0021] Preferably, in the graph generation unit, constructing a surface-embedded crack graph and calculating the local crack width based on the structural mesh model specifically includes:
[0022] In the local neighborhood of the structural mesh model, a three-dimensional skeleton is extracted and curve fitting is performed on the three-dimensional defect point set of the crack type to obtain the three-dimensional crack skeleton curve.
[0023] The three-dimensional crack skeleton curve is discretized into crack nodes according to a preset discretization strategy. The curve segments between adjacent crack nodes are used as crack edges to obtain a surface-embedded crack map.
[0024] Record the spatial coordinates of each crack node in the surface-embedded crack diagram within the structural mesh model and the spatial path of each crack edge within the structural mesh model;
[0025] Based on the set of defect pixels corresponding to each crack edge in the defect segmentation mask, the defect boundary position is searched along the normal direction of one of the crack edges, and the local crack width of the crack edge is calculated based on the defect boundary position.
[0026] Preferably, the feature extraction module includes a feature extraction unit, a labeling unit, and a calculation unit;
[0027] The feature extraction unit is used to extract crack features from a set of three-dimensional defect points of type crack. The crack features include a set of geometric features, a set of topological features, and a set of location features.
[0028] The annotation unit is used to annotate the region type of crack nodes and crack edges according to a preset spatial region division rule in the road and bridge structure design coordinate system.
[0029] The types of the regions include mid-span region, end-span region, support-near region, and component edge region;
[0030] The calculation unit is used to select a typical section that intersects with the boundary curve of the spalled area in the road and bridge structure design coordinate system, calculate the area of the geometric difference region between the original section profile and the projected profile of the spalled area in the typical section, use the ratio of the geometric difference region area to the original section area as the section weakening ratio, and select the maximum section weakening ratio as the section weakening index.
[0031] Preferably, in the feature extraction unit, extracting the crack features of the three-dimensional defect point set of crack type specifically includes:
[0032] The path length of the crack edge is calculated according to the spatial path of the crack edge in the structural mesh model. The path length and local crack width of all crack edges are counted, and a set of geometric features is constructed. The set of geometric features includes the crack length distribution and crack width distribution.
[0033] The number of crack edges connected to crack nodes is counted and used as the degree attribute of crack nodes. Based on graph theory, closed paths formed by different crack edges in the surface embedded crack graph are identified to obtain the topological role attribute of crack edges. The topological role attribute includes the number of branches and the number of simple loop structures.
[0034] The set of topological features is obtained by analyzing statistical properties, the number of branches, and the number of simple loop structures.
[0035] Based on the coordinate transformation relationship between the structural mesh model coordinate system and the road and bridge structural design coordinate system, the spatial coordinates of the crack nodes and the spatial coordinates of the representative points on the crack edge spatial path are converted into structural coordinates under the road and bridge structural design coordinate system to obtain the structural coordinate system. In the structural coordinate system, the crack edge and crack node are associated with the corresponding component positions to obtain the set of position features.
[0036] The component positions include span position, height position, correspondence with the tension region, and correspondence with the compression region.
[0037] Preferably, the result generation module includes a mechanical calculation unit, a risk quantification unit, and a structural health index calculation unit;
[0038] The mechanical calculation unit is used to obtain the nominal tensile stress value corresponding to the position of each component based on the design information and location feature set of the road and bridge structure, and calculate the crack driving force index according to the fracture mechanics rules by using the nominal tensile stress value and the path length of the crack edge.
[0039] The risk quantification unit is used to calculate the local strain energy of the structural mesh model using mechanical analytical formulas, extract the local strain energy at the crack node and label it as local energy density, extract the direction of the maximum principal stress at the crack node based on the maximum principal stress criterion, and label the physical evolution direction attribute of the crack edge.
[0040] Calculate the comprehensive crack network index based on the surface-embedded crack map and crack characteristics;
[0041] The structural health index calculation unit is used to calculate the crack stability reduction factor based on local energy density and physical evolution trajectory properties, specifically including:
[0042] The local energy density at the crack node is obtained. The local energy density of the crack nodes located at both ends of the same crack edge is arithmetically averaged to obtain the energy index of the crack edge. The energy index of the crack edge is normalized with the preset reference energy level to obtain the energy coefficient.
[0043] Based on the included angle deviation, the directional coefficient is obtained according to the preset monotonically decreasing mapping relationship;
[0044] The instability index of the crack edge is obtained by multiplying the energy coefficient, direction coefficient and path length of the crack edge. Within the component location range, the instability index of each crack edge is weighted and averaged according to the path length of the crack edge to obtain the crack stability reduction factor.
[0045] The structural health index is calculated based on the cross-sectional weakening index of the component location, the maximum value of the crack driving force index, the comprehensive crack network index, and the crack stability reduction factor, according to a preset mapping relationship.
[0046] Preferably, in the risk quantification unit, the calculation of the comprehensive crack network index based on the surface-embedded crack map specifically includes:
[0047] The geometric weight is calculated by combining the local energy density of the crack node, the path length of the crack edge, and the local crack width. The calculation expression is as follows:
[0048] ;
[0049] in, Geometric weights The preset kinetic energy sensitivity coefficient, This refers to the width of the local crack. This represents the path length of the crack edge. Local energy density;
[0050] Normalize the nominal tensile stress value at the component location corresponding to the crack edge, and retrieve the score corresponding to the stress interval to which the normalized nominal tensile stress value belongs in the mechanical grade mapping table, which is used as the mechanical weight.
[0051] Assign values based on the topological role attributes and physical evolution direction attributes of the crack edge, and calculate the topological weight;
[0052] The geometric weights, mechanical weights, and topological weights are multiplied to obtain the crack network contribution value. The crack network contribution values of all crack edges are then summed to obtain the comprehensive crack network index.
[0053] Preferably, topological weights are obtained by assigning values based on the topological role attributes and physical evolution direction attributes of the crack edges, specifically including:
[0054] The angle deviation between the actual geometric orientation and the physical evolution orientation of the crack edge is calculated by vector dot product;
[0055] If the crack edge belongs to a simple loop structure and the included angle deviation is within a preset deviation threshold, then a first topological weight is assigned.
[0056] If the crack edge is identified as the end branch connected to a crack node with a degree attribute of 1, and the included angle deviation exceeds a preset deviation threshold, then a second topological weight is assigned.
[0057] Among them, the first topological weight > 1.0 > the second topological weight.
[0058] Preferably, the result generation module further includes a diagnostic generation unit;
[0059] The diagnostic generation unit is used to perform weighted calculations based on the logical conditions of the disease cause hypothesis to obtain the cause scores of crack edges and associated crack nodes, summarize the cause scores of each crack edge within the component location range, and obtain the comprehensive cause score corresponding to each disease cause hypothesis.
[0060] The structural health index, the cross-sectional weakening index of each component location, the crack driving force index, the comprehensive crack network index, and the comprehensive cause score corresponding to each defect cause hypothesis are compiled into diagnostic output data, and the automatic defect diagnosis results are output.
[0061] The beneficial effects of this invention are as follows: By utilizing multi-view geometric reconstruction and 3D mesh mapping, a precise transition from 2D pixels to 3D structural semantics of defects is achieved. The physical dimensions of cracks can be accurately quantified, and complex crack evolution networks can be identified by extracting 3D skeletons and topological features. Furthermore, the physical evolution direction attribute and local energy density are introduced. By comparing the angular deviation between the crack's geometric direction and the direction of maximum principal stress, the evolution trend of cracks can be scientifically determined, and stability reduction factors can be calculated. This enables a deeper understanding of the underlying forces in structural safety assessment, moving beyond visual representation. The system automatically calculates the section weakening ratio and crack driving force index at the component scale. Combined with a comprehensive causal scoring logic, it can automatically generate diagnostic conclusions with mechanical support. Attached Figure Description
[0062] Figure 1This is a basic flowchart of an automatic diagnosis system for road and bridge defects based on image recognition, provided as an embodiment of the present invention. Detailed Implementation
[0063] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.
[0064] Example 1, referring to Figure 1 As one embodiment of the present invention, an automatic diagnosis system for road and bridge defects based on image recognition is provided, including an acquisition module, an image construction module, a feature extraction module, and a result generation module:
[0065] The acquisition module is used to acquire multi-view defect image sequences, extract defect segmentation mask images based on the multi-view defect image sequences, and construct a structural mesh model;
[0066] The graph construction module is used to map defect pixels in the defect segmentation mask map to a set of 3D defect points located in the structural mesh model, and to construct a surface-embedded crack map for the 3D defect point set of crack type, and to construct a peeling region on the structural mesh model for the 3D defect point set of peeling type.
[0067] The feature extraction module is used to extract crack features based on the surface-embedded crack map and to calculate the cross-sectional weakening index of each component of the road and bridge structure based on the spalling area.
[0068] The results generation module is used to calculate the crack driving force index, comprehensive crack network index, and crack stability reduction factor based on crack characteristics and cross-sectional weakening index. It also calculates the comprehensive cause score of each component location range based on the fault cause assumption and combines them to generate automatic defect diagnosis results.
[0069] This invention utilizes a three-dimensional mesh model and surface-embedded crack maps to achieve a dimensional upgrade from two-dimensional image recognition to three-dimensional structural semantics, solving the pain points of inaccurate defect localization and lack of spatial description. It not only extracts the geometric features of cracks and spalling but also deeply explores the substantial impact of damage on structural bearing capacity by calculating crack driving force indices, comprehensive crack network indices, and stability reduction factors, achieving a quantitative assessment of the risk of disease evolution. Combined with hypotheses about the causes of the disease, a comprehensive scoring of component dimensions is performed, providing a mechanically supported decision-making basis for the scientific maintenance and precise reinforcement of bridges.
[0070] In a specific embodiment, the acquisition module includes an image acquisition unit and a three-dimensional mesh construction unit;
[0071] The image acquisition unit is used to acquire defect images covering the surface of the road and bridge structure to be detected, obtain a multi-view defect image sequence, and use a convolutional neural network to segment the defect images to obtain a defect segmentation mask.
[0072] Three-dimensional mesh building units are used for multi-view geometric reconstruction based on multi-view defect image sequences to obtain the camera pose and sparse three-dimensional feature point coordinates corresponding to the defect images;
[0073] Multi-view stereo matching is performed on multi-view defect image sequences based on sparse 3D feature point coordinates to generate 3D point cloud data.
[0074] The 3D point cloud data is processed into a mesh to obtain a structural mesh model.
[0075] Specifically, multi-view geometric reconstruction based on multi-view defect image sequences includes:
[0076] For a multi-view defect image sequence covering the surface of a road or bridge structure to be inspected, local feature points are extracted from the defect images, and feature point matching is performed between adjacent views to obtain cross-view feature correspondences. The camera extrinsic parameters corresponding to each defect image are estimated, and the positions of the feature points in three-dimensional space are calculated through triangulation to form a sparse three-dimensional feature point cloud.
[0077] Camera pose includes the camera's spatial position and orientation information in the coordinate system of the structured mesh model;
[0078] Multi-view stereo matching of multi-view defect image sequences based on sparse 3D feature point coordinates specifically includes:
[0079] Using sparse 3D feature points as initial constraints, the consistency of their projection positions on defect images from various viewpoints is checked to limit the search range and viewpoint combination for pixel matching. For the image pixels to be reconstructed, pixel block similarity matching is performed among multiple images that satisfy multi-view geometric constraints, and the cross-viewpoint pixel correspondence is determined based on color or texture similarity. The matched multi-view pixels are converted into 3D coordinates through triangulation to form a dense 3D point set.
[0080] The process of meshing 3D point cloud data to obtain a structured mesh model specifically includes:
[0081] First, the point cloud is denoised, downsampled, and normal vector estimated to remove isolated points and obvious erroneous points, ensuring the uniformity and reliability of the point cloud. Then, based on the implicit surface reconstruction method, the discrete point cloud is fitted into a closed or semi-closed surface composed of a large number of small triangular facets to obtain a structural mesh model that approximates the surface of the actual road and bridge structure.
[0082] In a specific embodiment, the graph construction module includes a point set generation unit, a graph generation unit, and a peeling region construction unit;
[0083] The point set generation unit is used to back-project the defect pixels in the defect segmentation mask into spatial straight line rays according to the camera pose corresponding to the defect image;
[0084] By performing spatial geometric intersection operations between the spatial straight line ray and the structural mesh model, three-dimensional intersection points are obtained. The three-dimensional intersection points corresponding to all defect pixels are then obtained to obtain a three-dimensional defect point set.
[0085] The graph generation unit is used to construct a surface-embedded crack map and calculate the local crack width based on the structural mesh model;
[0086] The peeling region building element is used to extract the region boundary of the three-dimensional defect point set with the peeling type on the structural mesh model, and the region boundary is represented as the boundary curve of the peeling region.
[0087] Specifically, during the defect image acquisition process, the camera used to acquire the defect image is calibrated to obtain the camera's internal parameters and external parameters obtained through multi-view geometric reconstruction.
[0088] External parameters are used to describe the orientation of the camera coordinate system relative to the structured mesh model coordinate system, including rotation and translation, and are used to realize the coordinate transformation relationship from the camera coordinate system to the structured mesh model coordinate system.
[0089] For any pixel coordinate in the defect image, based on the camera's internal parameters, the pixel coordinate is transformed from image plane coordinates to a normalized imaging direction in the camera coordinate system, resulting in a line of sight originating from the camera's optical center and passing through that pixel. Using the rotation and translation parameters given in the camera pose, the line of sight is transformed from the camera coordinate system to the structured mesh model coordinate system, resulting in a spatial straight ray represented in the structured mesh model coordinate system. This ray takes the position of the camera's optical center in the structured mesh model coordinate system as its starting point and the transformed direction vector as its direction.
[0090] The spatial straight ray is intersected with each triangular facet in the structured mesh model to determine which triangular facets the ray intersects with, and the effective intersection point closest to the camera's optical center is selected as the corresponding 3D point of the pixel in the coordinate system of the structured mesh model.
[0091] By repeating the above steps for all target pixels in the defect image, the coordinate transformation relationship between the camera pose and the structural mesh model can be used to map the pixel coordinates of the defect image to the three-dimensional coordinates in the coordinate system of the structural mesh model, thus obtaining the three-dimensional defect point set corresponding to the defect region.
[0092] In a specific embodiment, the process of constructing a surface-embedded crack map and calculating the local crack width in the graph generation unit, based on the structural mesh model, specifically includes:
[0093] In the local neighborhood of the structural mesh model, a three-dimensional skeleton is extracted and curve fitting is performed on the three-dimensional defect point set of the crack type to obtain the three-dimensional crack skeleton curve.
[0094] The three-dimensional crack skeleton curve is discretized into crack nodes according to a preset discretization strategy. The curve segments between adjacent crack nodes are used as crack edges to obtain a surface-embedded crack map.
[0095] It should be noted that this invention back-projects pixels in the defect segmentation mask and intersects them with the mesh to obtain a three-dimensional defect point set of cracks and spalling on the structural surface. Furthermore, the crack type point set is constructed as a surface-embedded crack map, and the spalling type point set is constructed as the spalling region boundary curve on the mesh. Unlike existing methods that simply represent cracks and spalling with line segments or regions on a plane, this invention forms a crack network with a node-edge structure and precise spalling geometric boundaries on a real curved surface. This tightly couples the spatial distribution of defects with the morphology of structural components, significantly improving the accuracy and robustness of subsequent topology analysis and section weakening calculations.
[0096] Specifically, the extraction of the 3D skeleton and curve fitting of the 3D defect point set of the crack type within the local neighborhood of the structured mesh model includes:
[0097] After obtaining the 3D defect point cluster of a crack and its corresponding local mesh neighborhood, the point set generation unit and graph generation unit can perform 3D skeleton extraction within this local neighborhood. The steps include:
[0098] Within the crack cluster, an adjacency graph is constructed with 3D defect points as nodes, based on the spatial distance between points or the adjacency relationship on the local mesh. The weights of the edges in the adjacency graph can be Euclidean distance between points or geodesic distance along the local surface. Main direction analysis and connectivity analysis are performed on the adjacency graph to identify the main extension direction of the crack and possible branch locations. Using a centerline extraction method based on the shortest path, one or more center paths with widths reduced to the single-point level are extracted from the original crack cluster, resulting in a 3D skeleton point sequence representing the approximate orientation of the crack.
[0099] Based on the connectivity of the skeleton points in the adjacency graph and their arrangement along the main direction, the skeleton point sequence is sorted and organized into one or more ordered skeleton polylines. For cracks with branches, multiple sub-skeleton curves are divided at the branch nodes. Spline curve fitting is used to smoothly fit the ordered skeleton points in the coordinate system of the structural mesh model, resulting in a smooth and continuous three-dimensional crack skeleton curve. During the fitting process, the fitted curve can be restricted to fall near the local mesh neighborhood to ensure that the fitting result is consistent with the actual surface shape of the structure.
[0100] The local neighborhood of the structural mesh model is a local mesh sub-region truncated on the structural mesh model according to a preset spatial range, centered on the set of three-dimensional defect points corresponding to a certain crack, and is used to approximate the curved surface part that bears the crack.
[0101] The preset discretization strategy sets the maximum node spacing along the arc length of the crack skeleton as the scale parameter, based on structural dimensions, crack length, and diagnostic accuracy requirements. Along the arc length of each 3D crack skeleton curve, sampling is performed from one end at a spacing not exceeding the preset maximum, and the spatial points corresponding to the sampling locations are designated as crack nodes. For regions with abrupt curvature changes or inflection points, the sampling spacing can be adaptively reduced according to the curvature change, and crack nodes are added at the turning points to more finely characterize the crack geometry. For multi-branch cracks, crack nodes are forcibly set as connection nodes at the bifurcation and intersection points of the skeleton curves to ensure that the subsequently constructed surface-embedded crack map can correctly represent the topology of the crack network.
[0102] Record the spatial coordinates of each crack node in the surface-embedded crack diagram within the structural mesh model and the spatial path of each crack edge within the structural mesh model;
[0103] Based on the set of defect pixels corresponding to each crack edge in the defect segmentation mask, the defect boundary position is searched along the normal direction of one of the crack edges, and the local crack width of the crack edge is calculated based on the defect boundary position.
[0104] In a specific embodiment, the feature extraction module includes a feature extraction unit, a labeling unit, and a calculation unit;
[0105] The feature extraction unit is used to extract crack features from a set of three-dimensional defect points of type crack. The crack features include a set of geometric features, a set of topological features, and a set of location features.
[0106] The annotation unit is used to annotate the region type of crack nodes and crack edges according to the preset spatial region division rules in the coordinate system of road and bridge structure design;
[0107] The types of areas include mid-span area, end-span area, support-adjacent area, and component edge area;
[0108] The calculation unit is used to select a typical section that intersects with the boundary curve of the spalled area in the coordinate system of road and bridge structure design, calculate the area of the geometric difference region between the original section profile and the projected profile of the spalled area in the typical section, take the ratio of the area of the geometric difference region to the area of the original section as the section reduction ratio, and select the maximum section reduction ratio as the section reduction index.
[0109] Specifically, the preset spatial area division rules are as follows:
[0110] Division of mid-span and end-span areas: Within each span, the span coordinates are segmented according to the design length L of the span. A certain proportion of the area near the center line of the two end supports is divided into the end-span area, and the remaining span section in the middle is divided into the mid-span area.
[0111] Delineation of the area near the support: Based on the position of the center line or axis of each support in the design coordinate system, a distance threshold range is set on both sides along the span and vertical directions. The spatial location within this distance threshold range is marked as the area near the support.
[0112] Component edge area division: Based on the original cross-sectional outline of the bridge component in the design drawings, a side band of a certain width is reserved along the normal direction inward or outward from the cross-sectional outline. The spatial position falling within this side band is marked as the component edge area.
[0113] The calculation of the area of the geometric difference region between the original cross-sectional profile and the projected profile of the spalled area in a typical cross-section specifically includes:
[0114] In the coordinate system of road and bridge structure design, a typical section that intersects with the boundary curve of the spalling area is selected according to the design information. The equation of the plane of the typical section is determined, and the original section profile of the typical section is extracted from the design model. The original section profile is represented as a closed profile line in the plane of the section.
[0115] By utilizing the coordinate transformation relationship between the structural mesh model and the road and bridge structural design coordinate system, the three-dimensional boundary curve of the spalling area is transformed into the road and bridge structural design coordinate system. Then, taking the typical cross-sectional plane as the projection plane, the boundary curve of the spalling area is orthogonally projected onto the cross-sectional plane to obtain the projection contour line of the spalling area in the typical cross-section, and the projection contour line is represented as a closed contour line in the cross-sectional plane.
[0116] In a typical cross-sectional plane, the region corresponding to the original cross-sectional profile is regarded as the original cross-sectional region, and the region corresponding to the projected profile of the spalled region is regarded as the spalled projection region. Planar Boolean operations are used to perform a geometric difference operation on the two to obtain the difference region between the original cross-sectional region and the spalled projection region. This difference region represents the geometric region that is weakened from the original cross-section due to the spalling effect.
[0117] The aforementioned geometric difference region is discretized into polygonal segments within the cross-sectional plane. The areas of these polygonal segments are summed using a planar polygon area calculation method to obtain the area of the geometric difference region between the original cross-sectional profile and the projected profile of the peeled area in this typical cross-section.
[0118] It should be noted that this invention accurately transforms crack nodes and crack edges from the structural mesh model coordinate system to the road and bridge structural design coordinate system, and marks their respective mid-span, end-span, support vicinity, and component edge regions according to preset spatial region division rules. Simultaneously, it projects the spalling area onto a typical cross-section and calculates the geometric difference area to form the cross-sectional weakening ratio and the maximum cross-sectional weakening index. Compared with traditional diagnostic methods that only provide the location, length, width, and other appearance features of cracks in images, this invention not only clarifies which span, component, and stress area the crack and spalling are located in, but also directly quantifies spalling as a weakening ratio of the cross-sectional bearing area. This directly links defect evaluation with structural bearing capacity, significantly enhancing the structural semantics and engineering guidance value of the diagnostic results.
[0119] In a specific embodiment, the feature extraction unit extracts crack features from the three-dimensional defect point set of crack types, specifically including:
[0120] The path length of the crack edge is calculated according to the spatial path of the crack edge in the structural mesh model. The path length and local crack width of all crack edges are counted, and a geometric feature set is constructed, which includes the crack length distribution and crack width distribution.
[0121] The number of crack edges connected to crack nodes is counted and used as the degree attribute of crack nodes. Based on graph theory, closed paths formed by different crack edges in the surface embedded crack graph are identified to obtain the topological role attributes of crack edges, including the number of branches and the number of simple loop structures.
[0122] The set of topological features is obtained by analyzing statistical properties, the number of branches, and the number of simple loop structures.
[0123] Based on the coordinate transformation relationship between the structural mesh model coordinate system and the road and bridge structural design coordinate system, the spatial coordinates of the crack nodes and the spatial coordinates of the representative points on the crack edge spatial path are converted into structural coordinates under the road and bridge structural design coordinate system to obtain the structural coordinate system. In the structural coordinate system, the crack edge and crack node are associated with the corresponding component positions to obtain the set of position features.
[0124] The component location includes its span position, height position, correspondence with the tension zone, and correspondence with the compression zone.
[0125] It should be noted that in the feature extraction unit of this invention, the crack edge path length and local width are statistically analyzed into a set of geometric features such as the total crack length and width distribution. Simultaneously, based on the surface-embedded crack graph, the node degree, branch number, and number of simple loop structures are statistically analyzed to form a set of topological features reflecting the complexity and connectivity of the crack network. Then, through coordinate transformation, the cracks are precisely associated with their span position, height position, and tension or compression zone, constituting a set of location features. Compared to the traditional approach that only uses single geometric indicators such as maximum crack width or crack number, this invention systematically characterizes crack features from three aspects: geometry, topology, and stress location. It can identify both locally dangerous cracks and reveal the potential impact of large-scale crack networks on the overall structural performance, providing a rich feature foundation for subsequent crack driving force calculation, network indicator quantification, and causal analysis.
[0126] Specifically, the identification of closed paths formed by different crack edges in a surface-embedded crack graph based on graph theory methods includes:
[0127] Using crack nodes as nodes in the graph and crack edges connecting adjacent crack nodes as edges, an adjacency list or adjacency matrix is established based on the surface-embedded crack graph to record the relationship between each crack node and its connected crack edges.
[0128] For each crack node, count the number of crack edges connected to it and use this number as the degree attribute of the crack node; when the degree of a crack node is greater than 2, mark the node as a branch node, and the number of branch nodes in the corresponding crack connected subgraph is used as the number of branches of the crack network.
[0129] Based on the above graph structure, the method for extracting the set of cyclic edges after eliminating tree branches is used to enumerate the sequence of crack edges that may form a loop in the graph to obtain a candidate set of closed paths. The candidate set of closed paths is then topologically verified one by one, and closed paths that meet the following conditions are retained: each crack node in the closed path is connected to at least two crack edges in the path, the closed path does not contain self-intersecting or repeated edges, and the closed path cannot be further divided into smaller closed sub-paths.
[0130] Closed paths that satisfy the above conditions are marked as simple loop structures. The number of crack edges contained in the simple loop structures is counted to obtain the number of simple loop structures in each crack connected subgraph, and this number is used as one of the topological role attributes of the crack network.
[0131] Based on the coordinate transformation relationship between the structural mesh model coordinate system and the road and bridge structure design coordinate system, during the on-site measurement or design stage, several baselines that can be uniquely corresponding in both the structural mesh model and the design model are selected. Based on the spatial coordinates of these corresponding points in the structural mesh model coordinate system and the road and bridge structure design coordinate system, the rigid body transformation relationship between the two coordinate systems is solved to obtain the coordinate transformation parameters used to describe rotation and translation.
[0132] By using coordinate transformation parameters, the spatial coordinates of crack nodes in the structural mesh model coordinate system, as well as the spatial coordinates of pre-selected representative points on the crack edge spatial path, are uniformly transformed to the road and bridge structural design coordinate system, thus obtaining the structural coordinates of each crack node and crack edge representative point in the structural design coordinate system.
[0133] Based on the structural design information of the road and bridge, the geometric boundaries, span segment ranges, and height distribution ranges of each component are obtained in the design coordinate system. The structural coordinates of the crack nodes and crack edge representative points are matched with the spatial range of each component to determine the component number and corresponding span position segment into which they fall, thereby marking the span position and the component to which the crack nodes and crack edges belong.
[0134] In the typical section design information, the geometry and neutral axis position of each component section are obtained, and the spatial division of the tension and compression regions of the section is determined according to the design conditions or structural analysis results. The structural coordinates of the crack nodes and crack edge representative points are projected onto the typical section of the corresponding component, and their position relative to the neutral axis is used to determine whether they are located in the tension or compression region. This correspondence is marked as part of the position feature.
[0135] In a specific embodiment, the result generation module includes a mechanical calculation unit, a risk quantification unit, and a structural health index calculation unit;
[0136] The mechanical calculation unit is used to obtain the nominal tensile stress value corresponding to the location of each component based on the design information and location feature set of the road and bridge structure. By using the nominal tensile stress value and the path length of the crack edge, the crack driving force index is calculated according to the fracture mechanics rules.
[0137] The risk quantification unit is used to calculate the local strain energy of the structural mesh model using mechanical analytical formulas, extract the local strain energy at the crack node and label it as local energy density, extract the direction of the maximum principal stress at the crack node based on the maximum principal stress criterion, and label the physical evolution direction attribute of the crack edge.
[0138] Calculate the comprehensive crack network index based on the surface-embedded crack map and crack characteristics;
[0139] Specifically, the design information for road and bridge structures includes the geometric position and cross-sectional distribution of each component in the road and bridge structure design coordinate system, used to determine the component where the crack is located and the corresponding mid-span region, end region, and support region; the original design geometric profile and cross-sectional dimension parameters of typical sections of each component, used to obtain the original cross-sectional profile and original cross-sectional area; the position of the neutral axis of each typical section determined by the design analysis and the tension and compression regions divided therefrom; and the design internal forces at typical sections of each component or the nominal tensile stress distribution converted from the design internal forces, used to determine the nominal tensile stress value corresponding to the position of each component.
[0140] The specific parameters for calculating the crack driving force according to fracture mechanics rules include:
[0141] Based on the set of location features, determine the tension and compression regions corresponding to each crack edge; combine the design information of the road and bridge structure, and obtain the nominal tensile stress value at the location of the crack edge under the corresponding component cross section and working conditions.
[0142] By using the surface-embedded crack map and geometric feature set, the path length of each crack edge is obtained. Adjacent crack edges that are continuous with the crack edge and have the same direction can be merged as needed to obtain the equivalent crack length characterizing the crack development along the principal tensile stress direction.
[0143] Based on the relationship between crack stress intensity factor, nominal tensile stress, and crack length in linear elastic fracture mechanics, a crack driving force index is calculated using the nominal tensile stress value and the path length of the crack edge to characterize the magnitude of the fracture driving force at the crack edge. The crack driving force index increases monotonically with the increase of nominal tensile stress and crack length, and is used to reflect the potential danger of the crack continuing to expand or penetrate under the current stress state.
[0144] The structural health index calculation unit is used to calculate the crack stability reduction factor based on local energy density and physical evolution trajectory properties. Specifically, it includes:
[0145] The local energy density at the crack node is obtained. The local energy density of the crack nodes located at both ends of the same crack edge is arithmetically averaged to obtain the energy index of the crack edge. The energy index of the crack edge is normalized with the preset reference energy level to obtain the energy coefficient.
[0146] Based on the included angle deviation, the directional coefficient is obtained according to the preset monotonically decreasing mapping relationship;
[0147] The instability index of the crack edge is obtained by multiplying the energy coefficient, direction coefficient and path length of the crack edge. Within the component location range, the instability index of each crack edge is weighted and averaged according to the path length of the crack edge to obtain the crack stability reduction factor.
[0148] The structural health index is calculated based on the cross-sectional weakening index of the component location, the maximum value of the crack driving force index, the comprehensive crack network index, and the crack stability reduction factor, according to a preset mapping relationship.
[0149] Specifically, the preset reference energy level is used to distinguish between local energy near cracks that is within the normal fluctuation range and that is significantly higher. The local energy density of a large number of nodes in the structural mesh model is statistically analyzed, and the maximum value is selected as the reference energy level for subsequent energy index normalization.
[0150] The instability indices for each crack edge are weighted and summarized according to the path length of the crack edge, specifically including:
[0151] Within a certain component location range, the instability index of all crack edges within that range and the corresponding crack edge path length are collected. Using the crack edge path length as the weight, the instability index of each crack edge is weighted and averaged to obtain a comprehensive instability index that characterizes the overall crack instability within the component range. This results in crack edges with longer path lengths and higher instability indexes contributing more to the comprehensive instability index.
[0152] Based on the monotonically decreasing relationship, the above-mentioned comprehensive instability index is converted into a crack stability reduction factor, so that when the comprehensive instability index is small, the crack stability reduction factor is close to 1, and when the comprehensive instability index increases, the crack stability reduction factor gradually decreases, which is used to reduce the load-bearing capacity of the component in the calculation of the structural health index.
[0153] The preset mapping settings and calculation of the structural health index specifically include:
[0154] The cross-sectional weakening index, maximum crack driving force index, and comprehensive crack network index of the component location are normalized to the interval [0,1] according to their dimensions, and it is agreed that the larger the value, the higher the risk; the crack stability reduction factor is used as the reduction coefficient in the interval [0,1], and the smaller the value, the more unstable the crack.
[0155] Based on historical engineering data, weighting coefficients are assigned to the cross-section weakening index, the maximum crack driving force index, and the comprehensive crack network index. In this embodiment, these coefficients are preferably 0.5, 0.3, and 0.2. The three normalized indices are then weighted and combined to obtain the comprehensive risk index of the component. Based on the comprehensive risk index, a crack stability reduction factor is introduced to amplify or reduce the risk index, so that when the crack stability reduction factor is small, the comprehensive risk index is correspondingly higher. According to a monotonically decreasing function, the comprehensive risk index is mapped to a structural health index. The structural health index ranges from [0, 100], and the larger the comprehensive risk index, the smaller the structural health index. The structural health index is used as a quantitative evaluation result of the health status of the component or the overall structure.
[0156] In a specific embodiment, the calculation of the comprehensive crack network index based on the surface-embedded crack map in the risk quantification unit specifically includes:
[0157] The geometric weight is calculated by combining the local energy density of the crack node, the path length of the crack edge, and the local crack width. The calculation expression is as follows:
[0158] ;
[0159] in, Geometric weights The preset kinetic energy sensitivity coefficient, This refers to the width of the local crack. This represents the path length of the crack edge. Local energy density;
[0160] It should be noted that the function To comprehensively reflect the influence of crack edge path length L and local crack width W on geometric weights, in this embodiment, the preferred value is... ;
[0161] Normalize the nominal tensile stress value at the component location corresponding to the crack edge, and retrieve the score corresponding to the stress interval to which the normalized nominal tensile stress value belongs in the mechanical grade mapping table, which is used as the mechanical weight.
[0162] Assign values based on the topological role attributes and physical evolution direction attributes of the crack edge, and calculate the topological weight;
[0163] The geometric weights, mechanical weights, and topological weights are multiplied to obtain the crack network contribution value. The crack network contribution values of all crack edges are then summed to obtain the comprehensive crack network index.
[0164] Specifically, the mechanical grade mapping table is as follows:
[0165] Stress range Mechanical weight score 0.8~1.0 1.5 0.6~0.8 1.2 0.4~0.6 1.0 0.2~0.4 0.8 0~0.2 0.5
[0166] Table 1
[0167] In a specific embodiment, topological weights are obtained by assigning values based on the topological role attributes and physical evolution direction attributes of the crack edge, specifically including:
[0168] The angle deviation between the actual geometric orientation and the physical evolution orientation of the crack edge is calculated by vector dot product;
[0169] If the crack edge belongs to a simple loop structure and the included angle deviation is within a preset deviation threshold, then a first topological weight is assigned.
[0170] If the crack edge is identified as the end branch connected to a crack node with a degree attribute of 1, and the included angle deviation exceeds a preset deviation threshold, then a second topological weight is assigned.
[0171] Among them, the first topological weight > 1.0 > the second topological weight.
[0172] Specifically, the angle deviation between the actual geometric orientation and the physical evolution orientation of the crack edge, calculated using vector dot product, includes:
[0173] Perform a dot product operation between the direction vector of the crack edge and the direction vector indicated by the physical evolution direction attribute. Based on the dot product result, deduce the size of the angle between the two vectors. If the angle is less than or equal to 15°, the angle deviation is considered to be within the preset deviation threshold. If the angle is greater than 15°, the angle deviation is considered to exceed the preset deviation threshold. The preset deviation threshold is 15°.
[0174] The first and second topological weights are coefficients used to quantify the topological importance of crack edges in the crack network. The first topological weight is assigned when the crack edge belongs to a simple loop structure and the angle deviation between its actual geometric orientation and physical evolution orientation is within a preset deviation threshold. The preferred value of the first topological weight is 1.2. Its function is to amplify the contribution value of the crack edge in the comprehensive crack network index calculation, reflecting the system's higher attention to closed loop damage that conforms to mechanical laws. The second topological weight is assigned when the crack edge is identified as the terminal branch connected to a crack node with a degree attribute of one and the angle deviation between its actual geometric orientation and physical evolution orientation exceeds a preset deviation threshold. The preferred value of the second topological weight is 0.8. Its function is to reduce the contribution value of the crack edge in the comprehensive crack network index calculation, reflecting the system's assignment of a relatively low risk weight to terminal cracks that deviate from the principal stress field.
[0175] In a specific embodiment, the result generation module further includes a diagnosis generation unit;
[0176] The diagnosis generation unit is used to perform weighted calculations based on the logical conditions of the disease cause hypothesis to obtain the cause score of crack edge and associated crack node, summarize the cause scores of each crack edge within the component location range, and obtain the comprehensive cause score corresponding to each disease cause hypothesis.
[0177] The structural health index, the cross-sectional weakening index of each component location, the crack driving force index, the comprehensive crack network index, and the comprehensive cause score corresponding to each defect cause hypothesis are compiled into diagnostic output data, and the automatic defect diagnosis results are output.
[0178] Specifically, the logical conditions for the hypothesis of disease causes include:
[0179] The logical conditions for the load overload assumption include: the crack direction is basically consistent with the direction of the principal tensile stress and the included angle deviation is less than 15 degrees; the crack is located in the mid-span region or the tension zone; the crack width changes significantly with the load; the local energy density exceeds the threshold; and the crack edge path length is in a moderate ratio to the component size.
[0180] The logical conditions for the support settlement type hypothesis include: cracks developing at a 45-degree angle, cracks located in the vicinity of the support, cracks radiating from the support to the mid-span, multiple components near the same support exhibiting similar crack patterns, and crack width being maximum at the support and decreasing towards the distance.
[0181] The logical conditions for the temperature stress type assumption include cracks penetrating the component cross section, relatively uniform crack spacing, cracks located in the edge region of the component, crack width changing significantly with temperature, and crack orientation without obvious principal stress direction preference.
[0182] The logical conditions for the material aging hypothesis include rough crack surface with peeling edges, multiple crack branches forming a network structure, low local energy density, uneven crack width, and a large number of simple loop structures in the crack edge topology.
[0183] The logical conditions for the fatigue damage hypothesis include that the crack originates from the stress concentration point, the crack propagation path is straight, the crack tip is obvious, there are many terminal branches with a crack edge attribute of 1, and the physical evolution direction deviates little from the principal stress direction.
[0184] The weighted calculation based on the logical conditions of the disease cause hypothesis specifically includes: defining a basic scoring function to calculate the basic matching degree of each crack edge to each disease cause hypothesis, where each logical condition has a corresponding weight coefficient, and the function value of the degree to which each crack edge satisfies the hypothesis condition is between zero and one. Then, the degree of satisfaction of each logical condition is calculated, including: the calculation of the consistency between crack direction and principal stress direction based on the deviation between the actual direction and the expected direction; the calculation of regional matching degree based on whether the area where the crack edge is located belongs to the assumed expected region set; the calculation of energy density matching degree based on the closeness of the local energy density of the crack edge to the assumed expected energy density range; the calculation of topological role matching degree based on the matching degree of the topological role attribute of the crack edge to the assumed expected topological feature; and the calculation of crack width feature matching degree based on the closeness of the crack width to the assumed expected width. Next, weight coefficients are set according to different cause hypotheses, and the cause score of the crack edge is calculated based on the weight coefficients, which is the sum of the product of the degree of satisfaction of each logical condition and the corresponding weight. Finally, the score is corrected by associating crack nodes, considering the influence of crack nodes connected to the crack edge on the score of the hypothesis, with a node influence coefficient of 0.2. Next, within the component location range, the scores of all crack edges are weighted and summarized according to the crack edge path length to obtain the score of that component location for that hypothesis. Then, the scores of all component locations are weighted a second time according to the component's importance weight to obtain the comprehensive causal score of the entire structure for that hypothesis. Finally, the scores of all causal hypotheses are normalized to obtain the relative probability. The normalized value represents the relative probability of each disease causal hypothesis, which can be directly used for diagnostic output.
[0185] Causal hypothesis Towards Weight Regional weight Energy weight Spatial weight Width weight Overload 0.3 0.25 0.2 0.1 0.15 Support settlement 0.25 0.35 0.1 0.15 0.15 Temperature stress 0.15 0.2 0.15 0.2 0.3 Material aging 0.1 0.15 0.1 0.35 0.3 fatigue injury 0.25 0.2 0.2 0.25 0.1
[0186] Table 2
[0187] It should be noted that the above numerical parameters, such as energy reference level, stress range division, angle threshold, and various weighting coefficients, can be adjusted based on actual engineering experience and historical data under different bridge types and working conditions. The values given in the specification are only preferred values in this embodiment.
[0188] The mechanical calculation unit no longer relies solely on empirical judgments based on crack length and width. Instead, it combines nominal tensile stress at the component location and crack edge path length to construct a crack driving force index according to fracture mechanics rules. This quantifies the potential danger of crack propagation from the perspectives of stress and crack scale, reflecting the failure trend under actual stress more accurately than traditional visual or empirical grading. The risk quantification unit obtains local strain energy density and the direction of maximum principal stress through mechanical analysis. This assigns a physical evolution direction to the crack edge and, together with crack length and width, forms a geometric weight. Combined with the mechanical weight obtained from nominal tensile stress mapping and the topological weight determined by topological roles and orientation deviations such as branches and simple loops, a crack network contribution value and a comprehensive crack network index are constructed. This is an innovative point that couples energy field information with crack network structure for evaluation, which is more advanced than the traditional approach of only considering... The single-index method for the longest or widest crack more comprehensively characterizes the overall deterioration effect of the crack system on structural performance. This invention utilizes the consistency between local energy density and geometric orientation and physical evolution orientation to define crack edge instability and summarize it into a crack stability reduction factor within the component. This significantly amplifies the risk of unstable, high-energy cracks that conform to the principal tensile stress direction in the health index, which is superior to the existing method of not distinguishing between static cracks and developing cracks. Finally, the diagnostic generation unit scores the crack edges and their nodes according to the logical conditions of the fault cause hypothesis, and outputs the structural health index and cause diagnosis results together with the section weakening index, crack driving force, comprehensive crack network index, and stability reduction factor. This has a greater diagnostic depth and engineering decision-making value than traditional systems that only provide a defect list or a rough health level.
[0189] This invention significantly improves the intelligence and accuracy of road and bridge defect diagnosis by deeply integrating geometric damage identified by computer vision with structural mechanics assessment logic. Compared with traditional techniques that rely solely on image processing, this invention utilizes multi-view geometric reconstruction and 3D mesh mapping to achieve a precise leap from two-dimensional pixels to three-dimensional structural semantics of defects. It can not only accurately quantify the physical size of cracks but also identify complex crack evolution networks by extracting 3D skeletons and topological features. Furthermore, it introduces physical evolution direction attributes and local energy density. By comparing the angular deviation between the crack's geometric direction and the direction of maximum principal stress, it can scientifically determine the crack's evolution trend and calculate stability reduction factors, thus achieving a deep dive from visual representation to the essence of stress in structural safety assessment. The system automatically calculates the section weakening ratio and crack driving force index at the component scale. Combined with comprehensive causal scoring logic, it can automatically generate diagnostic conclusions with mechanical support, effectively solving the pain points of strong subjectivity and inability to quantify inherent structural risks in manual inspections. This provides highly reliable decision support for the precise reinforcement and preventive maintenance of bridges.
[0190] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0191] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the protection scope of the present invention.
Claims
1. An automatic diagnosis system for road and bridge defects based on image recognition, characterized in that, It includes a data acquisition module, a graph construction module, a feature extraction module, and a result generation module: The acquisition module is used to acquire multi-view defect image sequences, extract defect segmentation mask images based on the multi-view defect image sequences, and construct a structural mesh model. The graph construction module is used to map the defect pixels in the defect segmentation mask image to a set of three-dimensional defect points located in the structural mesh model, and to construct the three-dimensional defect point set of crack type as a surface embedded crack image, and to construct the three-dimensional defect point set of peeling type as a peeling region on the structural mesh model. The feature extraction module is used to extract crack features based on the surface-embedded crack map and to calculate the cross-sectional weakening index of each component of the road and bridge structure based on the spalling area. The result generation module is used to calculate the crack driving force index, comprehensive crack network index and crack stability reduction factor based on crack characteristics and cross-sectional weakening index, and calculate the comprehensive cause score of each component location range based on the fault cause assumption, and combine them to generate automatic defect diagnosis results.
2. The automatic road and bridge defect diagnosis system based on image recognition as described in claim 1, characterized in that, The acquisition module includes an image acquisition unit and a 3D mesh construction unit; The image acquisition unit is used to acquire defect images covering the surface of the road and bridge structure to be detected, obtain a multi-view defect image sequence, and use a convolutional neural network to segment the defect images to obtain a defect segmentation mask. The three-dimensional mesh building unit is used to perform multi-view geometric reconstruction based on multi-view defect image sequences to obtain the camera pose and sparse three-dimensional feature point coordinates corresponding to the defect images. Multi-view stereo matching is performed on multi-view defect image sequences based on sparse 3D feature point coordinates to generate 3D point cloud data. The 3D point cloud data is processed into a mesh to obtain a structural mesh model.
3. The automatic road and bridge defect diagnosis system based on image recognition as described in claim 1, characterized in that, The graph construction module includes a point set generation unit, a graph generation unit, and a peeling region construction unit; The point set generation unit is used to back-project the defect pixels in the defect segmentation mask into spatial straight line rays according to the camera pose corresponding to the defect image. By performing spatial geometric intersection operations between the spatial straight line ray and the structural mesh model, three-dimensional intersection points are obtained. The three-dimensional intersection points corresponding to all defect pixels are then obtained to obtain a three-dimensional defect point set. The graph generation unit is used to construct a surface-embedded crack graph and calculate the local crack width based on the structural mesh model; The peeling region construction unit is used to extract the region boundary of the three-dimensional defect point set with the defect type of peeling on the structural mesh model, and the region boundary is represented as the boundary curve of the peeling region.
4. The automatic road and bridge defect diagnosis system based on image recognition as described in claim 1, characterized in that, In the graph generation unit, based on the structural mesh model, the specific steps of constructing a surface-embedded crack graph and calculating the local crack width include: In the local neighborhood of the structural mesh model, a three-dimensional skeleton is extracted and curve fitting is performed on the three-dimensional defect point set of the crack type to obtain the three-dimensional crack skeleton curve. The three-dimensional crack skeleton curve is discretized into crack nodes according to a preset discretization strategy. The curve segments between adjacent crack nodes are used as crack edges to obtain a surface-embedded crack map. Record the spatial coordinates of each crack node in the surface-embedded crack diagram within the structural mesh model and the spatial path of each crack edge within the structural mesh model; Based on the set of defect pixels corresponding to each crack edge in the defect segmentation mask, the defect boundary position is searched along the normal direction of one of the crack edges, and the local crack width of the crack edge is calculated based on the defect boundary position.
5. The automatic diagnosis system for road and bridge defects based on image recognition as described in claim 1, characterized in that, The feature extraction module includes a feature extraction unit, a labeling unit, and a calculation unit; The feature extraction unit is used to extract crack features from a set of three-dimensional defect points of type crack. The crack features include a set of geometric features, a set of topological features, and a set of location features. The annotation unit is used to annotate the region type of crack nodes and crack edges according to a preset spatial region division rule in the road and bridge structure design coordinate system. The types of the regions include mid-span region, end-span region, support-near region, and component edge region; The calculation unit is used to select a typical section that intersects with the boundary curve of the spalled area in the road and bridge structure design coordinate system, calculate the area of the geometric difference region between the original section profile and the projected profile of the spalled area in the typical section, use the ratio of the geometric difference region area to the original section area as the section weakening ratio, and select the maximum section weakening ratio as the section weakening index.
6. The automatic road and bridge defect diagnosis system based on image recognition as described in claim 1, characterized in that, In the feature extraction unit, the extraction of crack features from the three-dimensional defect point set of crack types specifically includes: The path length of the crack edge is calculated according to the spatial path of the crack edge in the structural mesh model. The path length and local crack width of all crack edges are counted, and a set of geometric features is constructed. The set of geometric features includes the crack length distribution and crack width distribution. The number of crack edges connected to crack nodes is counted and used as the degree attribute of crack nodes. Based on graph theory, closed paths formed by different crack edges in the surface embedded crack graph are identified to obtain the topological role attribute of crack edges. The topological role attribute includes the number of branches and the number of simple loop structures. The set of topological features is obtained by analyzing statistical properties, the number of branches, and the number of simple loop structures. Based on the coordinate transformation relationship between the structural mesh model coordinate system and the road and bridge structural design coordinate system, the spatial coordinates of the crack nodes and the spatial coordinates of the representative points on the crack edge spatial path are converted into structural coordinates under the road and bridge structural design coordinate system to obtain the structural coordinate system. In the structural coordinate system, the crack edge and crack node are associated with the corresponding component positions to obtain the set of position features. The component positions include span position, height position, correspondence with the tension region, and correspondence with the compression region.
7. The automatic road and bridge defect diagnosis system based on image recognition as described in claim 1, characterized in that, The result generation module includes a mechanical calculation unit, a risk quantification unit, and a structural health index calculation unit. The mechanical calculation unit is used to obtain the nominal tensile stress value corresponding to the position of each component based on the design information and location feature set of the road and bridge structure, and calculate the crack driving force index according to the fracture mechanics rules by using the nominal tensile stress value and the path length of the crack edge. The risk quantification unit is used to calculate the local strain energy of the structural mesh model using mechanical analytical formulas, extract the local strain energy at the crack node and label it as local energy density, extract the direction of the maximum principal stress at the crack node based on the maximum principal stress criterion, and label the physical evolution direction attribute of the crack edge. Calculate the comprehensive crack network index based on the surface-embedded crack map and crack characteristics; The structural health index calculation unit is used to calculate the crack stability reduction factor based on local energy density and physical evolution trajectory properties, specifically including: The local energy density at the crack node is obtained. The local energy density of the crack nodes located at both ends of the same crack edge is arithmetically averaged to obtain the energy index of the crack edge. The energy index of the crack edge is normalized with the preset reference energy level to obtain the energy coefficient. Based on the included angle deviation, the directional coefficient is obtained according to the preset monotonically decreasing mapping relationship; The instability index of the crack edge is obtained by multiplying the energy coefficient, direction coefficient and path length of the crack edge. Within the component location range, the instability index of each crack edge is weighted and averaged according to the path length of the crack edge to obtain the crack stability reduction factor. The structural health index is calculated based on the cross-sectional weakening index of the component location, the maximum value of the crack driving force index, the comprehensive crack network index, and the crack stability reduction factor, according to a preset mapping relationship.
8. The automatic road and bridge defect diagnosis system based on image recognition as described in claim 1, characterized in that, In the risk quantification unit, the calculation of the comprehensive crack network index based on the surface-embedded crack map specifically includes: The geometric weight is calculated by combining the local energy density of the crack node, the path length of the crack edge, and the local crack width. The calculation expression is as follows: ; in, Geometric weights The preset kinetic energy sensitivity coefficient, This refers to the width of the local crack. This represents the path length of the crack edge. Local energy density; Normalize the nominal tensile stress value at the component location corresponding to the crack edge, and retrieve the score corresponding to the stress interval to which the normalized nominal tensile stress value belongs in the mechanical grade mapping table, which is used as the mechanical weight. Assign values based on the topological role attributes and physical evolution direction attributes of the crack edge, and calculate the topological weight; The geometric weights, mechanical weights, and topological weights are multiplied to obtain the crack network contribution value. The crack network contribution values of all crack edges are then summed to obtain the comprehensive crack network index.
9. The automatic road and bridge defect diagnosis system based on image recognition as described in claim 1, characterized in that, The topological weights are obtained by assigning values based on the topological role attributes and physical evolution direction attributes of the crack edges, specifically including: The angle deviation between the actual geometric orientation and the physical evolution orientation of the crack edge is calculated by vector dot product; If the crack edge belongs to a simple loop structure and the included angle deviation is within a preset deviation threshold, then a first topological weight is assigned. If the crack edge is identified as the end branch connected to a crack node with a degree attribute of 1, and the included angle deviation exceeds a preset deviation threshold, then a second topological weight is assigned. Among them, the first topological weight > 1.0 > the second topological weight.
10. The automatic road and bridge defect diagnosis system based on image recognition as described in claim 1, characterized in that, The result generation module also includes a diagnostic generation unit; The diagnostic generation unit is used to perform weighted calculations based on the logical conditions of the disease cause hypothesis to obtain the cause scores of crack edges and associated crack nodes, summarize the cause scores of each crack edge within the component location range, and obtain the comprehensive cause score corresponding to each disease cause hypothesis. The structural health index, the cross-sectional weakening index of each component location, the crack driving force index, the comprehensive crack network index, and the comprehensive cause score corresponding to each defect cause hypothesis are compiled into diagnostic output data, and the automatic defect diagnosis results are output.
Citation Information
Patent Citations
Method and device for predicting development and occurrence of tectonic fracture
CN121028200A
