A municipal road construction defect inspection system and method based on visual detection
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- JIANGSU HUATAI ROAD & BRIDGE ENG CO LTD
- Filing Date
- 2025-05-27
- Publication Date
- 2026-06-16
Smart Images
Figure CN120685644B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of visual inspection technology, specifically to a visual inspection system and method for inspecting defects in municipal road construction. Background Technology
[0002] The quality of municipal road construction directly affects the safety and service life of urban infrastructure. With the increasing complexity of road construction processes (such as the increase of hidden works such as anchor grouting), the demand for refined control of construction quality is becoming increasingly prominent. Existing road construction inspection mainly relies on sensors (such as laser rangefinders and visible light cameras), resulting in a high rate of missed detection of hidden defects: problems such as internal voids and uneven grout penetration cannot be detected by surface observation, leading to a high rate of false defects and untimely early warning.
[0003] In summary, existing technologies suffer from hidden defects such as incomplete grouting and material delamination, which are easily overlooked and prevent timely and comprehensive inspection of large construction areas, thus affecting the progress of road construction. Summary of the Invention
[0004] This application provides a visual inspection-based system and method for inspecting defects in municipal road construction, aiming to solve the technical problems in the prior art where hidden defects such as incomplete grouting and material delamination are easily overlooked, making it impossible to conduct timely and comprehensive inspections of large construction areas and affecting the progress of road construction.
[0005] In view of the above problems, the technical solution to achieve the present application is as follows:
[0006] This application provides a visual inspection system for municipal road construction defects, comprising: a scanning module for setting a monitoring device at a first location coordinate associated with an anchor bolt maintenance point; scanning a target construction area of the municipal road using the monitoring device; a similarity comparison module for performing multi-scale similarity comparison between the 3D point cloud data obtained by the monitoring device and standard 3D point cloud data stored in historical qualified samples; an association mapping module for setting a multispectral imaging device at a second location coordinate associated with the anchor bolt maintenance point; simultaneously acquiring a first texture feature corresponding to the visible light band and a second texture feature corresponding to the near-infrared band of the target construction area using the multispectral imaging device; a parameter search module for searching the grouting pump operating parameters of the anchor bolt maintenance point during a primary grouting process; searching the grouting pump operating parameters of the anchor bolt maintenance point during a secondary grouting process; and a defect alert module for alerting construction defects based on multi-scale similarity comparison information, the first texture feature corresponding to the visible light band, the second texture feature corresponding to the near-infrared band, and the grouting pump operating parameters.
[0007] Preferably, the three-dimensional point cloud data and standard three-dimensional point cloud data are subjected to voxel meshing to generate multiple resolution voxel meshes; point cloud feature descriptors are determined for each resolution voxel mesh in the multiple resolution voxel meshes, and the point cloud feature descriptors include at least one of normal direction, curvature, and local point density; multi-scale similarity comparison is performed using the point cloud feature descriptors at different scales.
[0008] Preferably, the multi-scale similarity comparison index includes any one or more of the following: point-to-point distance, normal vector consistency, and feature point distribution density.
[0009] Preferably, an RGB image of the target construction area is captured, and color features, edge features, and texture frequency features in the RGB image are extracted as first texture features; a near-infrared image of the target construction area is captured, and moisture content features, material density features, and hidden defect features in the near-infrared image are extracted as second texture features.
[0010] Preferably, a construction parameter state space is constructed using the operating parameters of the grouting pump and the pressure grouting pump; fuzzy probability derivation of defects is performed in the construction parameter state space to obtain the construction defect type and the fuzzy probability value of the defect; based on the construction defect type and the fuzzy probability value of the defect, it is determined whether to trigger the construction defect reminder.
[0011] Preferably, the operating parameters of the grouting pump corresponding to the primary grouting construction at the anchor bolt maintenance point are determined, including grouting time, grouting pressure, grouting flow rate, and grout mix ratio; and the operating parameters of the grouting pump corresponding to the secondary grouting construction at the anchor bolt maintenance point are determined, including grouting time, peak grouting pressure, pressure stabilization time, and grouting volume.
[0012] Preferably, if the construction defect alert is triggered, the defect range is initially defined by multi-scale similarity comparison information; the boundary region of the initially defined defect range is clipped according to the first texture feature corresponding to the visible light band; and the boundary region of the initially defined defect range is clipped according to the second texture feature corresponding to the near-infrared band.
[0013] Preferably, the RGB image within the initially defined defect range is segmented into multiple superpixel blocks with similar color features; the color histogram features, local binary mode features, and gray-level co-occurrence matrix features of the multiple superpixel blocks are extracted to construct the feature vector of the superpixel blocks; and the boundary region of the initially defined defect range is cropped using the feature vector of the superpixel blocks and the anisotropic features of the defect region edge.
[0014] Preferably, a similarity threshold τ is defined based on the cosine similarity between the feature vector of the superpixel block and the feature vector of the defect region; multiple superpixel blocks are traversed, and superpixel blocks with a cosine similarity less than the similarity threshold τ are removed from the initially defined defect range; the edges of the cropped defect region are extracted; corner points and abrupt change points on the edges of the cropped defect region are identified, a normal vector field of the boundary pixels is generated, the angle distribution between the normal vector and the local texture direction is determined, and the anisotropic characteristics of the defect region edges are evaluated.
[0015] In another aspect, this application provides a visual inspection method for detecting defects in municipal road construction. The method includes: setting an inspection and monitoring device at a first location coordinate associated with an anchor bolt maintenance point; scanning a target construction area of the municipal road using the inspection and monitoring device; performing a multi-scale similarity comparison between the three-dimensional point cloud data obtained from the scan and the standard three-dimensional point cloud data stored as historical qualified samples; setting a multispectral imaging device at a second location coordinate associated with the anchor bolt maintenance point; simultaneously acquiring a first texture feature corresponding to the visible light band and a second texture feature corresponding to the near-infrared band of the target construction area using the multispectral imaging device; searching for the grouting pump operating parameters of the anchor bolt maintenance point during a primary grouting process; searching for the grouting pump operating parameters of the anchor bolt maintenance point during a secondary grouting process; and issuing a construction defect alert based on the multi-scale similarity comparison information, the first texture feature corresponding to the visible light band, the second texture feature corresponding to the near-infrared band, and the grouting pump operating parameters.
[0016] In summary, one or more technical solutions provided in this application achieve multi-scale similarity comparison analysis of the differences between the target construction area and the standard model, more comprehensively detect structural defects, and integrate visual inspection data with construction process parameters for comprehensive evaluation of construction quality from multiple perspectives, thereby determining the specific spatial location of defects and providing defect location alerts. Attached Figure Description
[0017] Figure 1 This application provides a structural schematic diagram of a vision-based inspection system for municipal road construction defects.
[0018] Figure 2 This application provides a flowchart illustrating a vision-based inspection method for defects in municipal road construction.
[0019] Explanation of reference numerals in the attached diagram: Scanning module M100, Similarity comparison module M200, Association mapping module M300, Parameter search module M400, Defect alert module M500. Detailed Implementation
[0020] Example 1
[0021] The present application will now be described in detail with reference to the accompanying drawings, such as... Figure 1 As shown, this application provides a vision-based inspection system for municipal road construction defects, wherein the system includes:
[0022] The scanning module M100 is used to inspect the first position coordinates of the monitoring equipment set in the anchor maintenance point association mapping; and to use the monitoring equipment to scan the target construction area of the municipal road; the similarity comparison module M200 is used to perform multi-scale similarity comparison between the three-dimensional point cloud data obtained by the monitoring equipment and the standard three-dimensional point cloud data stored in the historical qualified sample storage.
[0023] Specifically, anchor bolt maintenance points are pre-defined key monitoring locations, and the first location coordinates associated with them are the specific installation locations of the inspection and monitoring equipment. The inspection and monitoring equipment acquires 3D point cloud data of the target construction area through 3D scanning technology. This data can accurately reflect the geometric shape, surface features, and other information of the construction area. The standard 3D point cloud data stored in the historical qualified sample storage refers to the 3D point cloud data corresponding to the construction areas that have been verified as qualified in previous construction projects. It contains characteristic information such as the construction shape and structure that meet the standards. Multi-scale similarity comparison refers to comparing the 3D point cloud data of the target construction area with the standard 3D point cloud data at different resolution scales to identify differences and potential defects.
[0024] Execution steps: First, based on the first location coordinates associated with the anchor bolt maintenance point, the inspection and monitoring equipment is precisely installed near the target construction area of the municipal road; then, the equipment begins to scan the target construction area to obtain detailed three-dimensional point cloud data; then, the three-dimensional point cloud data obtained from these scans is compared with the standard three-dimensional point cloud data of the stored historical qualified samples for multi-scale similarity comparison.
[0025] By performing voxel meshing and hierarchical processing, the data is divided into voxel meshes of different resolutions, and point cloud feature descriptors such as normal direction and curvature are determined at each resolution. Similarity comparisons are performed at multiple scales. By performing multi-scale similarity comparisons on the 3D point cloud data of the target construction area, it was found that compared with the standard data, the target area has small-scale depressions in some areas. These depressions are difficult to detect in low-resolution comparisons, but are accurately identified in high-resolution comparisons, thereby effectively improving the recognition rate of minor defects and providing an accurate basis for subsequent construction defect alerts.
[0026] The association mapping module M300 is used to set the second position coordinates of the multispectral imaging device on the anchor bolt maintenance point association mapping; using the multispectral imaging device, it simultaneously acquires the first texture feature corresponding to the visible light band and the second texture feature corresponding to the near-infrared band of the target construction area; the parameter search module M400 is used to search for the grouting pump operation parameters of the anchor bolt maintenance point during the first grouting construction process; and to search for the grouting pump operation parameters of the anchor bolt maintenance point during the second pressure grouting construction process; the defect reminder module M500 is used to provide construction defect reminders based on multi-scale similarity comparison information, the first texture feature corresponding to the visible light band, the second texture feature corresponding to the near-infrared band, and the grouting pump operation parameters.
[0027] Specifically, anchor bolt maintenance points are key locations for monitoring and evaluation during municipal road construction. Their associated mapping of secondary location coordinates is used to determine the installation location of multispectral imaging equipment. Multispectral imaging equipment can simultaneously capture images in the visible and near-infrared bands and extract texture features of the target construction area. The visible light band is used to capture visible defects such as surface cracks, while the near-infrared band is used to detect hidden defects such as water penetration and internal cavities. The operating parameters of grouting pumps and pressure grouting pumps include grouting time, grouting pressure, and grouting flow rate, which reflect the quality and status of grouting and pressure grouting during construction.
[0028] Execution steps: First, set the multispectral imaging device to the second position coordinates associated with the anchor bolt maintenance point to ensure that the device can effectively cover the target construction area; then, the device simultaneously acquires the first texture features corresponding to the visible light band and the second texture features corresponding to the near-infrared band of the target construction area. Through the visible light image and near-infrared image acquired by the multispectral imaging device, extract color features, edge features, and moisture content features, etc. These features can comprehensively reflect the surface and internal conditions of the construction area and improve the detection rate of hidden defects.
[0029] Simultaneously, the system searches for the operating parameters of the grouting pump and pressure grouting pump at the anchor maintenance point during the primary grouting and secondary pressure grouting processes. These parameters are used to derive the fuzzy probability of defects through the construction parameter state space. Combined with multi-scale similarity comparison information and texture features, the system comprehensively assesses and alerts on construction defects. Furthermore, by analyzing the fluctuations in grouting pressure and flow rate, and combining multispectral imaging results, the system can more accurately identify hidden defects such as incomplete grouting and material delamination, thereby effectively improving the quality control level of municipal road construction.
[0030] Furthermore, the similarity comparison module M200 is used to perform the following method:
[0031] The three-dimensional point cloud data and standard three-dimensional point cloud data are processed into voxel meshes to generate multiple resolution voxel meshes; point cloud feature descriptors are determined for each resolution voxel mesh in the multiple resolution voxel meshes, and the point cloud feature descriptors include at least one of normal direction, curvature, and local point density; multi-scale similarity comparison is performed using the point cloud feature descriptors at different scales.
[0032] Specifically, voxel meshing is a method that discretizes a continuous three-dimensional space into regular cubes (voxels) for layering and simplifying three-dimensional point cloud data. Different resolution voxel meshes refer to voxelizing three-dimensional point cloud data at different scales. High-resolution meshes provide detailed local features, while low-resolution meshes provide overall structural information. Point cloud feature descriptors are parameters used to describe the local geometric properties of point clouds, including normal direction (representing surface orientation), curvature (representing the degree of surface bending), and local point density (representing the distribution density of point clouds in local regions). Multi-scale similarity comparison quantifies the degree of similarity between target point clouds and standard point clouds by comparing the feature descriptors of target point clouds at different resolutions.
[0033] Execution steps: First, the 3D point cloud data obtained by the inspection and monitoring equipment is processed into voxel meshes with the standard 3D point cloud data to generate multiple resolution voxel meshes. Specifically, at a voxel side length of 1cm, the fine geometric features of the target construction area can be captured, while at a side length of 10cm, the focus is on the overall structural features. Next, the point cloud feature descriptors under each resolution voxel mesh are determined, such as normal direction, curvature, and local point density. By calculating the normal direction of the points in each voxel, the directional distribution of the surface of the construction area can be obtained, and the local point density can reflect the density of the point cloud.
[0034] By utilizing these feature descriptors, multi-scale similarity comparisons are performed at different scales. By comparing the similarity of the normal directions of the target and the standard point cloud at different scale resolutions, deviations in local surface directions and differences in the overall structure can be identified, effectively improving the recognition rate of minute defects.
[0035] Furthermore, the similarity comparison module M200 is also used to perform the following method:
[0036] Multi-scale similarity comparison metrics include any one or more of the following: point-to-point distance, normal vector consistency, and feature point distribution density.
[0037] Specifically, multi-scale similarity comparison quantifies the similarity between the target construction area's 3D point cloud data and standard 3D point cloud data by performing feature analysis on voxel grids at different resolutions; point-to-point distance refers to the Euclidean distance between corresponding points in the target point cloud and the standard point cloud at the same or different scales, used to measure the deviation between point positions; normal vector consistency determines whether the surface orientation is consistent by comparing the normal vector directions of corresponding points in the target and standard point clouds; feature point distribution density refers to the distribution of the number of key feature points in a specific area, used to evaluate the richness of detail in the point cloud data.
[0038] Execution steps: When performing multi-scale similarity comparison, firstly, the point-to-point distance between the target and the standard point cloud under different resolution voxel grids is calculated. By analyzing the point-to-point distance between the target and the standard point cloud under different resolution voxel grids, it is found that the average distance exceeds a preset threshold, indicating that there is a significant geometric deviation in the target construction area locally. Next, the consistency of normal vectors is analyzed. By calculating the angle between the normal vectors of corresponding points of the target and the standard point cloud, if the angle exceeds a set threshold, it indicates that there is a difference in surface orientation. Finally, the distribution density of feature points is evaluated. By comparing the number of feature points of the target and the standard point cloud in a specific area, if the feature point density of the target point cloud is significantly lower than that of the standard point cloud, it suggests that there is a certain probability that there are problems such as missing materials or loose structure in that area.
[0039] By combining indicators such as point-to-point distance, normal vector consistency, and feature point distribution density, hidden defects such as incomplete grouting and material delamination can be identified more accurately, effectively improving the quality control level of municipal road construction. Multi-scale similarity comparison in municipal road construction quality inspection can comprehensively evaluate construction quality from multiple perspectives.
[0040] Furthermore, the association mapping module M300 is used to perform the following method:
[0041] RGB images of the target construction area are captured, and color features, edge features, and texture frequency features in the RGB images are extracted as first texture features; near-infrared images of the target construction area are captured, and moisture content features, material density features, and hidden defect features in the near-infrared images are extracted as second texture features.
[0042] Specifically, an RGB image refers to an image composed of three color channels: red, green, and blue. It can capture the surface color and texture information of the target construction area. Color features identify the characteristics of different materials or defects by analyzing the color distribution of each region in the image. Edge features identify structural boundaries by detecting areas with significant changes in grayscale values in the image. Texture frequency features identify surface roughness and other characteristics by analyzing the repetition frequency of texture patterns in the image.
[0043] Near-infrared imaging refers to imaging using light in the near-infrared band, which can penetrate surfaces and detect internal information that cannot be detected by traditional visible light. Moisture content characteristics identify the moisture content in materials by the interaction between near-infrared light and water molecules. Material density characteristics identify the compactness of materials by the difference in reflection of near-infrared light in materials of different densities. Hidden defect characteristics identify hidden defects such as internal voids by the sensitivity of near-infrared light to internal structures.
[0044] Execution steps: RGB and near-infrared images of the target construction area are captured using a multispectral imaging device. For the RGB image, color features, edge features, and texture frequency features are extracted as the first texture features using image processing algorithms. Color features can be obtained by calculating the color histogram of each region in the image. Edge features can be detected using the Sobel or Canny operator, and texture frequency features can be analyzed using a Gabor filter. These features help identify visible defects such as surface cracks and material color differences.
[0045] For near-infrared images, moisture content, material density, and hidden defect features are extracted as secondary texture features. Moisture content can be calculated from the reflectance of a specific wavelength band, material density can be inferred from the grayscale distribution of the image, and hidden defect features can be identified through abnormal reflection patterns in the image. By fusing features from RGB and near-infrared images, hidden defects such as internal voids and moisture infiltration can be detected, enabling a more comprehensive assessment of construction quality and ensuring the accuracy and reliability of defect identification.
[0046] Furthermore, the defect alert module M500 is used to perform the following method:
[0047] A construction parameter state space is constructed using the operating parameters of the grouting pump and the pressure grouting pump. Defect fuzzy probability derivation is performed within this state space to obtain the construction defect type and its fuzzy probability value. Based on the construction defect type and its fuzzy probability value, it is determined whether to trigger the construction defect alert.
[0048] Specifically, the operating parameters of grouting pumps and pressure grouting pumps refer to multiple indicators during equipment operation, such as grouting time, grouting pressure, grouting flow rate, and grout mix ratio. Constructing the construction parameter state space involves integrating these parameters to form a multi-dimensional data space, which is used to comprehensively describe the state of the construction process. Defect fuzzy probability derivation uses fuzzy mathematics theory to calculate the probability and type of construction defects based on the data in the construction parameter state space. Construction defect alert is an early warning mechanism that is triggered when the calculated defect fuzzy probability value exceeds a set threshold, reminding construction personnel to pay attention to and deal with potential defects.
[0049] Execution steps: First, collect operational parameters during the grouting and pressure grouting processes at the anchor bolt maintenance points, such as grouting time, grouting pressure, and grouting flow rate. Using the operating parameters of the grouting pump and pressure grouting pump, construct a construction parameter state space to provide a data foundation for defect analysis. Preferably, the grouting pressure parameter reflects the density of the grout filling; abnormal grouting pressure indicates insufficient grouting. The pressure grouting time parameter reflects the adequacy of the pressure grouting process; excessively short pressure grouting time suggests insufficient pressure grouting.
[0050] In the state space of construction parameters, fuzzy probability derivation of defects is performed to obtain the types of construction defects and their fuzzy probability values. Specifically, using a fuzzy mathematical model, based on parameters such as grouting pressure and grouting time, the fuzzy probability of void defects appearing in the target area is derived. Based on the types of construction defects and their fuzzy probability values, it is determined whether a construction defect alert is triggered, and construction personnel are promptly notified so that corresponding measures can be taken, such as re-grouting or strengthening grouting, to effectively improve the reliability and safety of construction quality.
[0051] Furthermore, the defect alert module M500 is also used to perform the following method:
[0052] Determine the grouting pump operating parameters corresponding to the primary grouting construction at the anchor bolt maintenance point, including grouting time, grouting pressure, grouting flow rate, and grout mix ratio; determine the grouting pump operating parameters corresponding to the secondary grouting construction at the anchor bolt maintenance point, including grouting time, peak grouting pressure, pressure stabilization time, and grouting volume.
[0053] Specifically, anchor maintenance points are key locations for monitoring and evaluation during municipal road construction. The operating parameters of grouting pumps and pressure grouting pumps correspond to the technical indicators in primary grouting and secondary pressure grouting construction, respectively. The operating parameters of grouting pumps for primary grouting construction include grouting time, grouting pressure, grouting flow rate, and grout mix ratio. These parameters represent the duration of the grouting process, the applied pressure, the flow rate of the grout, and the composition ratio of the grout, respectively. The operating parameters of pressure grouting pumps for secondary pressure grouting construction include pressure grouting time, peak pressure, pressure stabilization time, and pressure grouting volume. These parameters represent the duration of the pressure grouting process, the maximum pressure value reached, the period of pressure stabilization, and the total amount of grout injected, respectively, collectively reflecting the quality and status of grouting and pressure grouting construction.
[0054] Execution steps: First, determine the operating parameters of the grouting pump corresponding to the primary grouting construction at the anchor bolt maintenance point, including grouting time, grouting pressure, grouting flow rate, and grout mix ratio. Simultaneously, determine the operating parameters of the grouting pump corresponding to the secondary grouting construction, including grouting time, peak grouting pressure, pressure stabilization time, and grout volume. Generally, insufficient grouting time may result in incomplete grout filling; insufficient grouting pressure may lead to incomplete grouting; too small a grouting flow rate may result in low grouting efficiency; and improper grout mix ratio may result in grout performance not meeting requirements. Insufficient grouting time may result in insufficient grouting; too low a peak grouting pressure may result in poor grouting effect; insufficient pressure stabilization time may result in poor stability after grouting; and insufficient grout volume may result in insufficient compaction after grouting.
[0055] These parameters are used to derive the fuzzy probability of defects through the state space of construction parameters, which affects the triggering of construction defect alerts. By analyzing the operating parameters of the grouting pump and the pressure grouting pump, it was found that the grouting pressure of a single grouting was lower than the normal range. Combined with other parameters, the fuzzy probability of void defects was derived, triggering construction defect alerts and prompting construction personnel to handle them, effectively improving the reliability and safety of construction quality.
[0056] Furthermore, the defect alert module M500 is also used to perform the following method:
[0057] If the construction defect alert is triggered, the defect range is initially defined by multi-scale similarity comparison information; the boundary region of the initially defined defect range is clipped according to the first texture feature corresponding to the visible light band; the boundary region of the initially defined defect range is clipped according to the second texture feature corresponding to the near-infrared band.
[0058] Specifically, multi-scale similarity comparison information is obtained by comparing the 3D point cloud data of the target construction area with standard 3D point cloud data at different resolutions, and is used to initially determine the possible range of defects. The first texture features corresponding to the visible light band include color features, edge features, and texture frequency features. These features can clearly outline the details of the surface of the construction area and provide explicit basis for boundary trimming. The second texture features corresponding to the near-infrared band cover moisture content features, material density features, and hidden defect features, which can reveal potential problems inside the construction area and help determine the defect boundary from a hidden perspective.
[0059] Execution steps: First, use multi-scale similarity comparison information to quickly locate the area where the defect may exist and obtain a preliminary definition of the defect range. By comparison, it is found that there is a significant difference between the target construction area and the standard model in a certain local area, and it is preliminarily judged that there may be a defect in this area. Next, use the first texture feature corresponding to the visible light band to trim the boundary area of the preliminarily defined defect range.
[0060] By analyzing color features, edge features, and texture frequency features in RGB images, the boundaries of defects are accurately identified. Specifically, color features help distinguish different materials or identify color differences, while edge features clearly outline the contours of defects. Furthermore, the second texture feature corresponding to the near-infrared band is used to further optimize the cropping of defect boundaries.
[0061] By analyzing the moisture content, material density, and hidden defect features in near-infrared images, potential problems inside defects can be revealed, ensuring the accuracy of boundary trimming. Specifically, moisture content features can help identify internal voids caused by moisture infiltration, while material density features can reflect the density of the material. Through multi-dimensional feature fusion analysis, the location and extent of defects can be determined more accurately.
[0062] Furthermore, the defect alert module M500 is also used to perform the following method:
[0063] The RGB image within the initially defined defect area is segmented into multiple superpixel blocks with similar color features. The color histogram features, local binary mode features, and gray-level co-occurrence matrix features of the multiple superpixel blocks are extracted to construct the feature vector of the superpixel blocks. The boundary region of the initially defined defect area is cropped using the feature vector of the superpixel blocks and the anisotropic features of the defect area edge.
[0064] Specifically, superpixel segmentation refers to dividing an image into multiple superpixel blocks with similar color features. These superpixel blocks can better reflect the local features and structural information of the image than traditional pixels. Color histogram features describe the distribution of colors in the image, local binary mode features are used to capture the local texture patterns of the image, gray-level co-occurrence matrix features are used to quantify the spatial relationship and texture features of gray values in the image, and anisotropy features reflect the directionality and non-uniformity of the edges of defective regions and are used to evaluate the complexity and irregularity of the edges.
[0065] Execution steps: Within the initially defined defect range, firstly, superpixel segmentation is performed on the RGB image to divide the image into multiple superpixel blocks with similar color features; then, the color histogram features, local binary mode features, and gray-level co-occurrence matrix features of each superpixel block are extracted to construct the feature vector of each superpixel block.
[0066] Color histogram features help identify the distribution of different color regions, local binary mode features can capture edge and texture details, and gray-level co-occurrence matrix features can quantify the roughness and directionality of the texture. Finally, by using the feature vectors of superpixel blocks and the anisotropic features of the defect region edges, the boundary region of the initially defined defect range is cropped. Specifically, by using the cosine similarity between the feature vectors of superpixel blocks and the feature vectors of the defect region, as well as the anisotropic features of the defect region edges, the defect boundary region is accurately identified and cropped, effectively improving the accuracy of defect boundary identification, ensuring the accuracy of defect localization, and providing reliable data support for subsequent defect repair.
[0067] Furthermore, the defect alert module M500 is also used to perform the following method:
[0068] Based on the cosine similarity between the feature vector of the superpixel block and the feature vector of the defect region, a similarity threshold τ is defined; multiple superpixel blocks are traversed, and superpixel blocks with a cosine similarity less than the similarity threshold τ are removed from the initially defined defect range; the edges of the cropped defect region are extracted; corner points and abrupt change points on the edges of the cropped defect region are identified, the normal vector field of the boundary pixels is generated, the angle distribution between the normal vector and the local texture direction is determined, and the anisotropic characteristics of the defect region edges are evaluated.
[0069] Specifically, the superpixel block feature vector is constructed by extracting the color histogram features, local binary mode features, and gray-level co-occurrence matrix features of the superpixel block, and is used to describe the visual characteristics of the superpixel block. The defect region feature vector is obtained by statistically analyzing the features of known defect regions and is used to compare with the superpixel block feature vector.
[0070] Cosine similarity is an index that measures the degree of similarity between two vectors. Its value ranges from 0 to 1. The closer the value is to 1, the higher the similarity. The similarity threshold τ is a pre-set judgment standard used to distinguish whether a superpixel block belongs to a defect region. Anisotropic features reflect the directionality and non-uniformity of the edge of the defect region and are used to evaluate the complexity and irregularity of the edge. The normal vector field is a vector field that describes the direction of the boundary pixel, while the local texture direction is the main direction of the texture near the boundary pixel.
[0071] Execution steps: First, a similarity threshold τ is defined based on the cosine similarity between the superpixel block feature vector and the defect region feature vector. Specifically, through training with a large amount of sample data, it is determined that when the cosine similarity is less than 0.6, the superpixel block is likely to belong to a non-defect region. Then, multiple superpixel blocks within the initially defined defect range are traversed, and the cosine similarity between each superpixel block and the defect region feature vector is calculated. Superpixel blocks with a cosine similarity less than the similarity threshold τ are removed from the initially defined defect range, thereby more accurately narrowing the defect range and making the defect region more prominent.
[0072] Next, the edges of the cropped defect region are extracted, and corner points and abrupt change points on the edges are identified. These key points usually indicate the start or end position of the defect and are crucial for accurate defect localization. Then, the normal vector field of the boundary pixels is generated. By calculating the angle distribution between the normal vector and the local texture direction, the anisotropic characteristics of the defect region edge are evaluated. Specifically, if the angle distribution between the normal vector and the local texture direction shows a Gaussian distribution, it indicates that the defect edge is relatively regular; if the angle distribution is relatively discrete, it indicates that the defect edge is relatively complex. The analysis method based on feature vectors and anisotropic characteristics can effectively improve the accuracy of defect boundary recognition and ensure the accuracy of defect localization.
[0073] In summary, the beneficial effects of the embodiments of this application are:
[0074] The system employs a scanning module for setting the first position coordinates of the inspection and monitoring equipment at the anchor bolt maintenance point in an associated mapping; scanning the target construction area on the municipal road using the inspection and monitoring equipment; a similarity comparison module for performing multi-scale similarity comparison between the 3D point cloud data obtained by the inspection and monitoring equipment and the standard 3D point cloud data stored as historical qualified samples; an association mapping module for setting the second position coordinates of the multispectral imaging equipment at the anchor bolt maintenance point in an associated mapping; and multispectral imaging equipment to simultaneously acquire the first texture features corresponding to the visible light band and the second texture features corresponding to the near-infrared band of the target construction area; a parameter search module for searching the grouting pump operating parameters of the anchor bolt maintenance point during the first grouting construction process; and searching the grouting pump operating parameters of the anchor bolt maintenance point during the second grouting construction process; and a defect alert module for alerting construction defects based on multi-scale similarity comparison information, the first texture features corresponding to the visible light band, the second texture features corresponding to the near-infrared band, and the grouting pump operating parameters. This application provides a visual inspection-based system and method for inspecting defects in municipal road construction. It achieves multi-scale similarity comparison analysis of the differences between the target construction area and the standard model, more comprehensively detects structural defects, and integrates visual inspection data with construction process parameters for comprehensive evaluation of construction quality from multiple perspectives. It also determines the specific spatial location of defects and provides defect location alerts.
[0075] Example 2
[0076] Based on the same inventive concept as the vision-based municipal road construction defect inspection system described in the foregoing embodiments, such as... Figure 2 As shown in the figure, this application provides a method for inspecting defects in municipal road construction based on visual detection, wherein the method includes:
[0077] S1: The inspection and monitoring equipment is set at the first location coordinate associated with the anchor bolt maintenance point; the inspection and monitoring equipment is used to scan the target construction area of the municipal road.
[0078] S2: Perform a multi-scale similarity comparison between the three-dimensional point cloud data obtained by scanning the inspection and monitoring equipment and the standard three-dimensional point cloud data stored in the historical qualified sample storage.
[0079] S3: The multispectral imaging device is set at the second position coordinates associated with the anchor bolt maintenance point; using the multispectral imaging device, the first texture feature corresponding to the visible light band and the second texture feature corresponding to the near-infrared band of the target construction area are acquired simultaneously;
[0080] S4: Search for the grouting pump operating parameters of the anchor bolt maintenance point during the first grouting construction process; search for the grouting pump operating parameters of the anchor bolt maintenance point during the second grouting construction process.
[0081] S5: Based on multi-scale similarity comparison information, the first texture feature corresponding to the visible light band, and the second texture feature corresponding to the near-infrared band, combined with the grouting pump operating parameters and the pressure grouting pump operating parameters, construction defect reminders are provided.
[0082] Furthermore, the method of this application involves performing multi-scale similarity comparison between the 3D point cloud data obtained by the inspection and monitoring equipment and the standard 3D point cloud data stored as historical qualified samples.
[0083] The three-dimensional point cloud data and standard three-dimensional point cloud data are processed into voxel meshes to generate multiple resolution voxel meshes; point cloud feature descriptors are determined for each resolution voxel mesh in the multiple resolution voxel meshes, and the point cloud feature descriptors include at least one of normal direction, curvature, and local point density; multi-scale similarity comparison is performed using the point cloud feature descriptors at different scales.
[0084] Furthermore, the method of this application includes performing multi-scale similarity comparison using the point cloud feature descriptors at different scales:
[0085] Multi-scale similarity comparison metrics include any one or more of the following: point-to-point distance, normal vector consistency, and feature point distribution density.
[0086] Furthermore, the method of this application simultaneously acquires the first texture feature corresponding to the visible light band and the second texture feature corresponding to the near-infrared band of the target construction area.
[0087] RGB images of the target construction area are captured, and color features, edge features, and texture frequency features in the RGB images are extracted as first texture features; near-infrared images of the target construction area are captured, and moisture content features, material density features, and hidden defect features in the near-infrared images are extracted as second texture features.
[0088] Furthermore, this application's method for identifying construction defects by combining the operating parameters of the grouting pump and the pressure grouting pump includes:
[0089] A construction parameter state space is constructed using the operating parameters of the grouting pump and the pressure grouting pump. Defect fuzzy probability derivation is performed within this state space to obtain the construction defect type and its fuzzy probability value. Based on the construction defect type and its fuzzy probability value, it is determined whether to trigger the construction defect alert.
[0090] Furthermore, the method of this application includes:
[0091] Determine the grouting pump operating parameters corresponding to the primary grouting construction at the anchor bolt maintenance point, including grouting time, grouting pressure, grouting flow rate, and grout mix ratio; determine the grouting pump operating parameters corresponding to the secondary grouting construction at the anchor bolt maintenance point, including grouting time, peak grouting pressure, pressure stabilization time, and grouting volume.
[0092] Furthermore, after determining whether the construction defect alert has been triggered, the method of this application further includes:
[0093] If the construction defect alert is triggered, the defect range is initially defined by multi-scale similarity comparison information; the boundary region of the initially defined defect range is clipped according to the first texture feature corresponding to the visible light band; the boundary region of the initially defined defect range is clipped according to the second texture feature corresponding to the near-infrared band.
[0094] Furthermore, based on the first texture features corresponding to the visible light band, the boundary region of the initially defined defect range is clipped. The method of this application includes:
[0095] The RGB image within the initially defined defect area is segmented into multiple superpixel blocks with similar color features. The color histogram features, local binary mode features, and gray-level co-occurrence matrix features of the multiple superpixel blocks are extracted to construct the feature vector of the superpixel blocks. The boundary region of the initially defined defect area is cropped using the feature vector of the superpixel blocks and the anisotropic features of the defect area edge.
[0096] Furthermore, the method of this application includes:
[0097] Based on the cosine similarity between the feature vector of the superpixel block and the feature vector of the defect region, a similarity threshold τ is defined; multiple superpixel blocks are traversed, and superpixel blocks with a cosine similarity less than the similarity threshold τ are removed from the initially defined defect range; the edges of the cropped defect region are extracted; corner points and abrupt change points on the edges of the cropped defect region are identified, the normal vector field of the boundary pixels is generated, the angle distribution between the normal vector and the local texture direction is determined, and the anisotropic characteristics of the defect region edges are evaluated.
[0098] In summary, any step can be stored as a computer instruction or program in an unrestricted computer memory and can be called and recognized by an unrestricted computer processor; no further restrictions are imposed here.
[0099] Furthermore, the above technical solutions only embody the preferred technical solutions of the embodiments of this application. Any changes that those skilled in the art may make to certain parts of these solutions embody the novel principles of the embodiments of this application. Obviously, those skilled in the art can make various modifications and variations to this application without departing from the scope of this application.
Claims
1. A visual inspection system for inspecting defects in municipal road construction, characterized in that, The system includes: The scanning module is used to inspect the first location coordinates of the monitoring equipment set in the anchor bolt maintenance point association mapping; and to use the monitoring equipment to scan the target construction area of the municipal road. The similarity comparison module is used to perform multi-scale similarity comparison between the three-dimensional point cloud data obtained by the inspection and monitoring equipment and the standard three-dimensional point cloud data stored in the historical qualified sample storage. The association mapping module is used to set the second position coordinates of the multispectral imaging device in the anchor bolt maintenance point association mapping; using the multispectral imaging device, the first texture feature corresponding to the visible light band and the second texture feature corresponding to the near-infrared band of the target construction area are acquired simultaneously. The parameter search module is used to search for the grouting pump operating parameters of the anchor bolt maintenance point during the first grouting construction process; and to search for the grouting pump operating parameters of the anchor bolt maintenance point during the second grouting construction process. The defect alert module is used to alert users to construction defects based on multi-scale similarity comparison information, the first texture feature corresponding to the visible light band, the second texture feature corresponding to the near-infrared band, and the operating parameters of the grouting pump and the pressure grouting pump. Among them, construction defect alerts are provided by combining the operating parameters of the grouting pump and the pressure grouting pump, including: A construction parameter state space is constructed using the operating parameters of the grouting pump and the pressure grouting pump. Defect fuzzy probability derivation is performed in the state space of the construction parameters to obtain the construction defect type and defect fuzzy probability value; Based on the construction defect type and the defect fuzzy probability value, determine whether to trigger the construction defect alert; Determine the grouting pump operating parameters corresponding to a single grouting operation at the anchor bolt maintenance point, including grouting time, grouting pressure, grouting flow rate, and grout mix ratio; Determine the grouting pump operating parameters corresponding to the secondary grouting construction at the anchor bolt maintenance point, including grouting time, peak grouting pressure, pressure stabilization time, and grouting volume; After determining whether the construction defect alert has been triggered, the process further includes: If the construction defect alert is triggered, the scope of the defect is initially defined through multi-scale similarity comparison information; Based on the first texture feature corresponding to the visible light band, the boundary region of the initially defined defect range is clipped; based on the second texture feature corresponding to the near-infrared band, the boundary region of the initially defined defect range is clipped. Among them, the first texture feature corresponding to the visible light band and the second texture feature corresponding to the near-infrared band of the target construction area are acquired simultaneously, including: Capture an RGB image of the target construction area, and extract color features, edge features, and texture frequency features from the RGB image as the first texture feature; Near-infrared images of the target construction area are captured, and the moisture content features, material density features, and hidden defect features in the near-infrared images are extracted as second texture features.
2. The municipal road construction defect inspection system based on vision detection as described in claim 1, characterized in that, The 3D point cloud data obtained by the inspection and monitoring equipment is compared with the standard 3D point cloud data stored in the historical qualified sample database using a multi-scale similarity comparison, including: The three-dimensional point cloud data and standard three-dimensional point cloud data are subjected to voxel meshing to generate multiple resolution voxel meshes; Determine the point cloud feature descriptor for each resolution voxel grid in the plurality of resolution voxel grids, wherein the point cloud feature descriptor includes at least one of normal direction, curvature, and local point density; Multi-scale similarity comparison is performed using the point cloud feature descriptor at different scales.
3. The municipal road construction defect inspection system based on vision detection as described in claim 2, characterized in that, Multi-scale similarity comparison is performed using the point cloud feature descriptors at different scales, including: Multi-scale similarity comparison metrics include any one or more of the following: point-to-point distance, normal vector consistency, and feature point distribution density.
4. The municipal road construction defect inspection system based on vision detection as described in claim 1, characterized in that, Based on the first texture feature corresponding to the visible light band, the boundary region of the initially defined defect range is trimmed, including: The RGB image within the initially defined defect range is segmented into multiple superpixel blocks with similar color features by performing superpixel segmentation. Extract color histogram features, local binary mode features, and gray-level co-occurrence matrix features from multiple superpixel blocks to construct the feature vector of the superpixel blocks; The boundary region of the initially defined defect range is clipped using the feature vectors of superpixel blocks and the anisotropic features of the defect region edges.
5. A visual inspection system for municipal road construction defects as described in claim 4, characterized in that, include: A similarity threshold τ is defined based on the cosine similarity between the feature vector of the superpixel block and the feature vector of the defect region. Traverse multiple superpixel blocks and remove superpixel blocks with a cosine similarity less than the similarity threshold τ from the initially defined defect range; extract the edges of the cropped defect region; Identify corner points and abrupt change points on the edge of the cropped defect region, generate the normal vector field of the boundary pixels, determine the angular distribution between the normal vector and the local texture direction, and evaluate the anisotropic characteristics of the defect region edge.
6. A method for inspecting defects in municipal road construction based on visual detection, characterized in that, For implementing the vision-based inspection system for municipal road construction defects according to any one of claims 1-5, the method includes: The inspection and monitoring equipment is set at the first location coordinates associated with the anchor bolt maintenance point; the inspection and monitoring equipment is used to scan the target construction area of the municipal road. A multi-scale similarity comparison is performed between the 3D point cloud data obtained by the inspection and monitoring equipment and the standard 3D point cloud data stored in the historical qualified sample storage. The multispectral imaging device is set at the second position coordinates associated with the anchor bolt maintenance point; using the multispectral imaging device, the first texture feature corresponding to the visible light band and the second texture feature corresponding to the near-infrared band of the target construction area are acquired simultaneously; Search for the grouting pump operating parameters of the anchor bolt maintenance point during the first grouting construction process; search for the grouting pump operating parameters of the anchor bolt maintenance point during the second grouting construction process; Based on multi-scale similarity comparison information, the first texture feature corresponding to the visible light band, and the second texture feature corresponding to the near-infrared band, construction defect alerts are provided by combining the grouting pump operating parameters and the pressure grouting pump operating parameters.