Municipal road and bridge defect detection method and system based on infrared imaging

Abstract

The present application relates to the technical field of road and bridge defect detection, and provides a municipal road and bridge defect detection method and system based on infrared imaging. The method includes: obtaining spatial structure information; introducing detection constraints, scanning and detecting by an infrared thermal imager, and generating an infrared image set; traversing the infrared image set to perform temperature analysis and obtain temperature analysis results; extracting features based on spatial structure information, and then performing defect detection and identification to obtain a defect data set; evaluating based on the defect data set, constructing a defect detection report, and sending it to a remote data processing unit. The present application solves the technical problem that traditional detection technology requires a lot of time and manpower to conduct detailed inspections of every part of the road and bridge, and it is difficult to find hidden defects or early damage. It realizes the use of infrared imaging technology to quickly scan the surface of the road and bridge, improves the detection speed and efficiency, and can detect small temperature changes, thereby finding small defects or damage.

Description

Technical Field

[0001] The present application relates to the technical field of road and bridge construction, specifically to the technical field of road and bridge defect detection, and in particular to a municipal road and bridge defect detection method and system based on infrared imaging. Background Art

[0002] With the acceleration of urbanization and the continuous improvement of infrastructure construction, municipal roads and bridges, as an important part of urban transportation, are increasingly receiving widespread attention from the society for their quality and safety. However, with the continuous increase in traffic volume and the extension of the service life of roads and bridges, the defect detection and maintenance of municipal roads and bridges are facing increasing challenges. Traditional detection technology may require detailed inspection of every part of the road and bridge, which requires a lot of time and manpower, and it is difficult for traditional detection technology to detect hidden defects or early damage. Summary of the invention

[0003] This application provides a municipal road and bridge defect detection method and system based on infrared imaging, aiming to solve the technical problem that traditional detection technology requires a lot of time and manpower to conduct detailed inspections of each part of the road and bridge, and it is difficult to detect hidden defects or early damage.

[0004] In view of the above problems, the present application provides a municipal road and bridge defect detection method and system based on infrared imaging.

[0005] The first aspect disclosed in the present application provides a municipal road and bridge defect detection method based on infrared imaging, the method comprising: performing three-dimensional modeling on a target road and bridge to obtain spatial structural information of the target road and bridge; introducing detection constraints based on the spatial structural information, scanning and detecting the target road and bridge according to the detection constraints by an infrared thermal imager, and generating an infrared image set of the target road and bridge; traversing the infrared image set to perform temperature analysis, construct a temperature distribution map sequence, and obtain a temperature analysis result based on the temperature distribution map sequence; performing feature extraction based on the spatial structural information to determine a plurality of structural features, and performing defect detection and identification based on the temperature analysis results and in combination with the plurality of structural features to obtain a road and bridge defect data set; performing an evaluation based on the road and bridge defect data set, constructing a defect detection report based on the road and bridge defect evaluation result, and sending the defect detection report to a remote data processing unit through an early warning signal.

[0006] Another aspect disclosed in the present application provides a municipal road and bridge defect detection system based on infrared imaging, the system comprising: a three-dimensional modeling module, the three-dimensional modeling module is used to perform three-dimensional modeling of a target road and bridge to obtain spatial structural information of the target road and bridge; a scanning detection module, the scanning detection module is used to introduce detection constraints based on the spatial structural information, scan and detect the target road and bridge according to the detection constraints through an infrared thermal imager, and generate an infrared image set of the target road and bridge; a temperature analysis module, the temperature analysis module is used to traverse the infrared image set to perform temperature analysis, construct a temperature distribution map sequence, and obtain a temperature analysis result according to the temperature distribution map sequence; a defect detection and identification module, the defect detection and identification module is used to perform feature extraction based on the spatial structural information, determine multiple structural features, perform defect detection and identification based on the temperature analysis result and in combination with the multiple structural features, and obtain a road and bridge defect data set; a defect assessment module, the defect assessment module is used to perform assessment based on the road and bridge defect data set, construct a defect detection report based on the road and bridge defect assessment result, and send the defect detection report to a remote data processing unit through an early warning signal.

[0007] One or more technical solutions provided in this application have at least the following technical effects or advantages:

[0008] The above-mentioned municipal road and bridge defect detection method based on infrared imaging uses three-dimensional modeling technology to accurately capture the spatial structure information of the target road and bridge, providing basic data for subsequent detection work. Subsequently, based on these spatial structure information, a series of detection constraints are set. These conditions ensure that the infrared thermal imager can perform comprehensive and accurate scanning and detection of the target road and bridge according to the predetermined path and angle. In this way, a set of data sets containing infrared images of different parts of the target road and bridge is obtained. After that, temperature analysis is performed on these infrared image sets. By comparing and analyzing the temperature distribution in the image, a temperature distribution map sequence is constructed, and the temperature analysis results are obtained from it. After obtaining the temperature analysis results, feature extraction is performed in combination with the three-dimensional spatial structure information. The purpose of this step is to identify multiple key features from the structure of the road and bridge, which will serve as an important basis for defect detection. Then, the temperature analysis results are combined with the extracted structural features to detect and identify defects on the target road and bridge. Through this process, a detailed data set about road and bridge defects is obtained. Finally, an evaluation is performed based on this defect data set, and a defect detection report is constructed accordingly. This report not only contains the specific location and type of the defect, but also gives corresponding treatment suggestions. In order to ensure that the relevant departments can quickly understand and deal with these problems, the defect detection report is sent to the remote data processing unit through an early warning signal so that appropriate measures can be taken in time.

[0009] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0011] Figure 1 The figure is a flow chart of a municipal road and bridge defect detection method based on infrared imaging in one embodiment.

[0012] Figure 2 The present invention is an architecture diagram of a municipal road and bridge defect detection system based on infrared imaging in one embodiment.

[0013] Explanation of the accompanying drawings: three-dimensional modeling module 1, scanning detection module 2, temperature analysis module 3, defect detection and identification module 4, defect assessment module 5. DETAILED DESCRIPTION

[0014] The embodiments of the present application provide a municipal road and bridge defect detection method and system based on infrared imaging to solve the technical problem that traditional detection technology requires a lot of time and manpower to conduct detailed inspections of each part of the road and bridge, and it is difficult to detect hidden defects or early damage.

[0015] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0016] It should be noted that the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or server that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or modules that are not explicitly listed or inherent to these processes, methods, products or devices.

[0017] Embodiment 1

[0018] like Figure 1As shown, the present application provides a municipal road and bridge defect detection method based on infrared imaging, the method comprising:

[0019] The target road and bridge are three-dimensionally modeled to obtain the spatial structure information of the target road and bridge.

[0020] Infrared imaging technology can achieve rapid scanning of large areas of road and bridge surfaces, improving the speed and efficiency of detection. Infrared imaging technology is highly sensitive to temperature changes and can detect tiny temperature changes, thereby discovering tiny defects or damage.

[0021] In the embodiment of the present application, the system terminal performs three-dimensional modeling of the target road and bridge by using multiple two-dimensional image data from multiple angles to perform structural recovery, and digitally replicate and reconstruct the actual form and structure of the road and bridge. The purpose of this step is to obtain the precise spatial structural information of the road and bridge, including its geometric shape, size, positional relationship, etc. Through three-dimensional modeling, a detailed and accurate three-dimensional digital model can be obtained, which contains the spatial structural information of the target road and bridge, can fully reflect the actual situation of the road and bridge, and provide basic data support for subsequent detection and analysis.

[0022] Furthermore, the present application provides a method for performing three-dimensional modeling on a target road or bridge to obtain spatial structure information of the target road or bridge, including:

[0023] Retrieve a historical image database of the target road and bridge, and extract multiple two-dimensional image data from multiple angles based on the historical image database; perform image matching on the multiple two-dimensional image data from multiple angles, perform structural restoration based on the matching results, and extract multiple three-dimensional point cloud data; perform three-dimensional modeling based on the multiple three-dimensional point cloud data to construct the spatial structure information of the target road and bridge.

[0024] Preferably, in order to construct the three-dimensional spatial structure information of the target road and bridge, the system terminal retrieves the historical image database of the target road and bridge. This database contains two-dimensional image data of the road and bridge taken from multiple different angles and time periods. Subsequently, by performing image matching on these multi-angle two-dimensional image data, the corresponding feature points and regions between different images are identified. Afterwards, the initial position of the three-dimensional point is calculated using the triangulation method based on the matched feature points and the intrinsic parameters of the camera, such as focal length, principal point, etc., to generate an initial three-dimensional point cloud. Afterwards, based on the matched feature points, the system terminal estimates the relative position and posture of the camera at different viewing angles, i.e., the external parameters of the camera. The reprojection error of the feature points is then minimized by the bundle adjustment method to further accurately estimate the external parameters of the camera. The bundle adjustment method is an iterative optimization process that optimizes the camera parameters and the three-dimensional point coordinates by minimizing the difference between the actual observed feature point positions and the theoretical positions calculated based on the camera parameters and the three-dimensional points. In this process, the external parameters of the camera and the position of the three-dimensional points are continuously adjusted until the preset maximum number of iterations is reached. After optimization by the bundle adjustment method, the initial three-dimensional point cloud becomes more accurate and reliable, and three-dimensional point cloud data is obtained. These point cloud data represent a set of discrete points of the road bridge in three-dimensional space, and their position and density reflect the real shape and structure of the road bridge. Then, based on these three-dimensional point cloud data, the system terminal performs three-dimensional modeling. This step is to convert the discrete point cloud data into a continuous three-dimensional surface model, thereby constructing the complete spatial structure information of the target road bridge. This three-dimensional model is used for subsequent applications such as defect detection.

[0025] Based on the spatial structure information, detection constraints are introduced, and the target road and bridge are scanned and detected by an infrared thermal imager according to the detection constraints to generate an infrared image set of the target road and bridge.

[0026] In one embodiment, the three-dimensional spatial structural information of the target bridge is obtained, which provides the system terminal with a detailed view of the geometry and structure of the bridge. This view can help understand the structural layout, material distribution and potential thermal properties of the bridge. Subsequently, the system terminal determines the detection focal length constraint information, detection direction constraint information and detection position constraint information based on these structural information, and organizes these constraint information to generate detection constraint conditions. These conditions are intended to ensure that the infrared thermal imager can capture the thermal radiation information of each important part and key node of the bridge. Afterwards, the system terminal uses the infrared thermal imager to scan and detect the target bridge according to these detection constraints. The infrared thermal imager can receive and display the thermal radiation image of the surface temperature distribution of the target bridge. Since different materials and structures have different heat conduction and radiation characteristics, the temperature difference of different areas on the bridge can be observed through the thermal imager, thereby discovering potential defects, abnormalities or faults. During the scanning process, the thermal imager generates a series of infrared images, which constitute the infrared image set of the target bridge. These image sets not only contain temperature distribution information on the road and bridge surface, but also metadata such as time and location, providing a rich data source for subsequent data analysis and processing.

[0027] Furthermore, the present application provides a method for introducing detection constraints based on the spatial structure information, and the method further includes:

[0028] Based on the spatial structure information, a target to be detected is determined, a detection distance is calculated according to the target to be detected, and detection focal length constraint information is set; based on the spatial structure information, a range to be detected is determined, a detection angle is calculated according to the range to be detected, and detection direction constraint information is set; based on the target to be detected, a preset detection position is evaluated in combination with the spatial structure information, and detection position constraint information is determined; the detection focal length constraint information, the detection direction constraint information, and the detection position constraint information are integrated to generate the detection constraint condition.

[0029] Preferably, the system terminal identifies key targets to be detected based on the spatial structural information of the target road and bridge. These targets include key structural points of the bridge, potential stress concentration areas or other parts that require special attention. For the detection focal length, the system terminal calculates the detection distance for each target to be detected. The detection distance refers to the distance between the target to be detected and the detection device. The calculation of this distance is based on the spatial position information of the target to be detected, and this distance can be calculated. The determination of the detection distance is crucial for selecting appropriate detection equipment and adjusting detection parameters. After determining the detection distance, the system terminal further sets the detection focal length constraint information. The focal length constraint information is determined based on the detection distance and the performance parameters of the detection equipment, which ensures that the detection equipment can obtain clear and accurate detection images at the correct focal length. For the detection direction, the system terminal defines one or more ranges to be detected based on the spatial structural information. These ranges include key parts such as the bridge deck, piers, and bridge joints of the road and bridge. Then, according to the spatial position and shape of the range to be detected, a suitable detection angle is calculated. The detection angle is the line of sight direction or scanning angle of the detection device relative to the target to be detected or the range to be detected. Based on the calculated detection angle, the system terminal sets the detection direction constraint information. These constraint information ensures that the detection equipment can detect in the predetermined direction and cover all important areas within the range to be detected. For the detection position, the system terminal combines the target to be detected and the spatial structure information to evaluate whether the preset detection position is appropriate. The evaluation process involves analyzing the impact of the detection position on the detection effect, the accessibility of the detection position, the safety and other factors. Among them, the preset detection position is the location where the detection equipment is deployed. Based on the evaluation results of the preset detection position, the system terminal determines the detection position constraint information. These constraint information guides the final deployment position of the detection equipment to ensure that the detection task can be carried out efficiently and safely. Finally, the system terminal integrates the detection focal length constraint information, the detection direction constraint information and the detection position constraint information to generate complete detection constraint conditions. These constraints provide clear guidance and basis for subsequent detection work, ensuring that the detection can be carried out according to the predetermined goals and requirements.

[0030] Furthermore, the present application provides a method of scanning and detecting a target road or bridge according to the detection constraint conditions by an infrared thermal imager to generate an infrared image set of the target road or bridge, and the method further includes:

[0031] Based on the detection position constraint information, the infrared thermal imager is deployed to determine the detection position information set; based on the detection position information set, the detection direction information set of the infrared thermal imager is set according to the detection direction constraint information; based on the detection focal length constraint information, the detection focal length information set of the infrared thermal imager is obtained by adjusting the detection position information set and the detection direction information set.

[0032] Optionally, for the layout of the detection position, the system terminal accurately arranges the infrared thermal imager according to the detection position constraint information to determine its appropriate detection position information set. These information sets contain parameters such as the specific location and height where the thermal imager should be placed to ensure that it can effectively cover the target detection area. For the layout of the rotation direction, the system terminal sets the detection direction information set of the infrared thermal imager according to the detection direction constraint information. This direction information set allows the thermal imager to rotate in multiple directions during the detection process so as to monitor the detection area in all directions. The angle allowed for rotation in the direction information set must be within the detection direction constraint information. For the layout of the detection focal length, the system terminal adjusts and obtains the detection focal length information set according to the detection focal length constraint information. This process is completed based on the detection position information set and the detection direction information set. The core purpose of the detection focal length information set is to determine the appropriate focal length setting for the long and short distance of the target to be detected according to the detection direction and the detection position within the detection focal length constraint range. Since in actual detection, the target objects may be distributed at different distances, which may be far or near, there may be multiple detection focal lengths in the geometric structure. These focal lengths must be set strictly within the detection focal length constraint information. This setting method ensures that the infrared thermal imager can efficiently and safely perform all-round detection of the target area.

[0033] An infrared scanning plan is formulated based on the detection position information set, the detection direction information set, and the detection focal length information set, and the infrared thermal imager is activated to execute the infrared scanning plan to obtain multiple infrared images; the multiple infrared images are combined according to the spatial structure information to generate the infrared image set of the target road and bridge.

[0034] Optionally, the system terminal determines the specific location where the infrared thermal imager needs to be placed or moved based on the detection position information set. This information includes the starting position of the thermal imager, the moving path, and the specific area to be covered. Subsequently, the detection direction information set is used to determine the direction in which the infrared thermal imager should point at each position. This information ensures that the thermal imager can capture all aspects of the road and bridge, both horizontally and vertically, so as to obtain complete scanning data. Thereafter, the focus of the infrared thermal imager is adjusted according to the detection focal length information set to ensure that the infrared image of the target road and bridge can be clearly captured. After determining the detection position, direction, and focal length, the system terminal formulates an infrared scanning plan. This plan describes in detail the operation of the thermal imager at each position, including the speed, range, and time of the scan. Once the scanning plan is formulated, the system terminal activates the infrared thermal imager to execute the plan. The thermal imager moves according to the set position and direction, and adjusts the focal length at each position to capture the infrared image of the road and bridge. Further, the system terminal calculates the relative position relationship between different images by identifying common features between infrared images, such as landmarks, textures, etc. These relative position relationships are then used to transform all images into a unified coordinate system. Subsequently, in this unified coordinate system, the system terminal determines the stitching order and position of the images based on the spatial structure information and performs the image stitching operation. After that, the stitched images are cropped according to the actual range and spatial structure information of the target bridge to remove unnecessary background and interference information. At the same time, the image is precisely positioned according to the spatial structure information to ensure that each part can accurately correspond to the actual position of the target bridge. After the above steps, the infrared images finally obtained by the system terminal are organized in a certain format and order to form a complete infrared image set. This image set includes multiple infrared images at different angles, different focal lengths and different time points to fully reflect the temperature distribution and structural characteristics of the target bridge for subsequent analysis and research.

[0035] The infrared image set is traversed to perform temperature analysis, a temperature distribution map sequence is constructed, and a temperature analysis result is obtained according to the temperature distribution map sequence.

[0036] In one embodiment, the system terminal traverses the acquired infrared image set, that is, processes each image one by one. During the processing, the image processing unit is used to extract the temperature information in the image. Subsequently, based on the extracted temperature information, the system terminal constructs a sequence of temperature distribution graphs. The temperature distribution graph is an image that intuitively displays temperature information, which uses color or brightness to indicate the temperature of different areas or points. This sequence can reflect the change of temperature over time or space. Afterwards, temperature analysis is performed based on the obtained temperature distribution graph sequence to understand the temperature change trend and obtain the temperature analysis results. By deeply analyzing these results, a comprehensive understanding of the temperature distribution of the object can be obtained, and strong support can be provided for subsequent defect detection and identification.

[0037] Furthermore, the present application provides traversing the infrared image set to perform temperature analysis, constructing a temperature distribution map sequence, and obtaining a temperature analysis result according to the temperature distribution map sequence, and the method also includes:

[0038] The infrared image set is traversed, and each image is loaded into an image processing unit for information extraction to obtain a temperature information set, wherein the temperature information set corresponds to the infrared image set; each pixel in the infrared image set is calculated based on the temperature information set to generate multiple temperature extremes and multiple temperature averages.

[0039] Preferably, in the infrared image processing process, the system terminal traverses the infrared image set and loads it into the image processing unit one by one. The image processing unit is a pre-set key component, which is specifically used to extract information from infrared images and has a built-in image processing algorithm pre-written by professionals. When the infrared image set is traversed and loaded into the image processing unit one by one, the unit will use the built-in algorithm to extract information from each image and construct a temperature information set. These temperature information sets include temperature data for each pixel in the image, and this temperature information set maintains a one-to-one correspondence with the infrared image set to ensure that subsequent temperature analysis can be accurate. After obtaining the temperature information set, the system terminal extracts the corresponding multiple temperature values ​​from the temperature information set for each pixel block in the infrared image set. Subsequently, each pixel block in the infrared image set is calculated. This process includes identifying and extracting the highest and lowest temperatures in each pixel block, and calculating the average temperature in each pixel block, generating multiple temperature extremes and multiple temperature means. These calculation results provide important data support for subsequent temperature analysis and processing, helping to better understand and analyze the temperature distribution in infrared images.

[0040] Based on the multiple temperature extreme values ​​and the multiple temperature average values, the temperature values ​​of the infrared image set are marked in turn to generate multiple temperature distribution maps, and the multiple temperature distribution maps have a corresponding relationship with the infrared image set; the multiple temperature distribution maps are spliced ​​according to the acquisition time sequence of the infrared image set to generate a temperature distribution map sequence.

[0041] Preferably, after obtaining multiple temperature extremes and temperature means, the system terminal uses these temperature data to mark the temperature value of each image in the infrared image set and generate a series of temperature distribution maps. These temperature distribution maps intuitively show the temperature distribution of the target bridge in different areas or pixels, and maintain a one-to-one correspondence with the original infrared image set. In order to show the temperature changes of the target bridge at different times or locations, the system terminal splices these temperature distribution maps according to the acquisition time series of the infrared image set. This splicing process ensures the timing of the temperature distribution map, so that the generated temperature distribution map sequence can continuously reflect the temperature changes of the target bridge. Through this temperature distribution map sequence, the system terminal can more clearly understand the temperature differences and changing trends of the target bridge at different times or locations, providing strong support for subsequent defect detection and identification.

[0042] The target road and bridge are analyzed according to the temperature distribution diagram sequence to obtain a plurality of temperature change trends; and the plurality of temperature change trends are added to the temperature analysis result.

[0043] Preferably, when performing temperature analysis on the target bridge, the generated temperature distribution graph sequence is used to capture the temperature changes of the target bridge at different times or locations. These temperature distribution graph sequences record in detail the temperature data of the target bridge over time or location. By analyzing these sequences, the system terminal can identify multiple temperature change trends. These temperature change trends include the continuous rise or fall of the temperature in a specific area, the expansion or reduction of the temperature difference between different areas, etc. These trends reflect the temperature behavior of the target bridge under different conditions, which helps to understand the changing laws and possible reasons of its temperature distribution. Finally, the temperature change trends obtained from these analyses are added to the temperature analysis results. This temperature analysis result has important reference value for defect detection of the target bridge.

[0044] Feature extraction is performed based on the spatial structure information to determine multiple structural features, and defect detection and identification is performed based on the temperature analysis result in combination with the multiple structural features to obtain a road and bridge defect data set.

[0045] In one embodiment, the system terminal divides the spatial structure information into multiple pixel regions, each of which contains multiple spatial structure feature parameters. Subsequently, the system terminal assigns weights to these spatial structure feature parameters and determines multiple structural features based on the assignment results. These structural features can help the system terminal understand the physical properties and design features of the road and bridge. Afterwards, the system terminal compares the temperature data and structural features through the defect detection model, identifies the association between temperature anomalies and structural features, and generates a road and bridge defect data set. This data set contains detailed information such as the location and type of the defect, which can be used for subsequent evaluation and maintenance work.

[0046] Furthermore, the present application provides a method for performing defect detection and identification based on the temperature analysis result and in combination with the multiple structural features to obtain a road and bridge defect data set, and the method further includes:

[0047] Retrieve the historical temperature record log of the target road and bridge, extract the temperature record data set under normal operating conditions according to the historical temperature record log; and determine the first abnormal temperature area set based on comparing the temperature record data set with the temperature analysis result.

[0048] Optionally, the system terminal retrieves the historical temperature record log of the target road and bridge from the data platform. These records include temperature data at different time points, covering the normal operating status of the road and bridge. After obtaining the complete historical temperature records, the system terminal selects the temperature data representing the normal operating status of the road and bridge. These data are collected without faults or abnormalities and serve as the benchmark for subsequent comparisons. Subsequently, the extracted temperature record data set under normal operating conditions is compared with the current temperature analysis results. The purpose of this step is to find out the current temperature data that is inconsistent with the historical data or exceeds the normal range. Through comparison, those current temperature data that are significantly different from the historical data can be identified, thereby forming a preliminary determined abnormal temperature area set, namely the first abnormal temperature area set. This set represents areas where there may be problems, but these abnormal areas are based on the results of the preliminary comparison and may not be completely accurate.

[0049] Performing correlation analysis on the plurality of structural features and the first abnormal temperature region set to generate correlation analysis factors; and using the correlation analysis factors to train a defect detection model to construct the defect detection model.

[0050] Optionally, after determining the first set of abnormal temperature regions, the system terminal performs association analysis on these abnormal temperature regions and the extracted multiple structural features. This association analysis aims to find the potential connection between abnormal temperature and specific structural features. Specifically, the system terminal matches each abnormal temperature region with the structural features of the road bridge. This involves the correspondence of spatial positions, that is, determining which structural features are spatially associated with which abnormal temperature regions. Subsequently, association indicators are constructed for each associated structural feature and abnormal temperature region. These indicators are obtained by counting the frequency of occurrence of specific structural features related to the abnormal temperature region. After that, all the association indicators are combined to construct association analysis factors. These factors are a comprehensive expression of the association relationship between abnormal temperature and structural features, and contain comprehensive information of multiple structural features and abnormal temperature regions. Further, the system terminal designs a convolutional neural network model according to the complexity of the problem and the characteristics of the data, and determines the number of convolutional layers, the size of the convolution kernel, the step size, the padding and other parameters. Subsequently, the data in the association analysis factor is divided into a training set and a test set. The training set is used for model training, and the test set is used to evaluate the performance of the model. After that, the convolutional neural network model is trained using the training set. During the training process, the training data will pass through multiple convolutional layers for feature extraction, and will be reduced in dimension through the pooling layer, and then classified through the fully connected layer. Then, the back propagation algorithm and Adam are used to update the weights and bias terms of the model to minimize the cross entropy loss function. When the training reaches the maximum number of iterations, the system terminal uses the test set to evaluate the trained convolutional neural network model and calculate the accuracy of the model. If the accuracy meets the expected expectations, the trained convolutional neural network model is output as a defect detection model. Otherwise, the number of convolutional layers, convolution kernel size, step size, etc. are adjusted, and retraining is performed.

[0051] The multiple temperature change trends are synchronized to the defect detection model to obtain a second abnormal temperature region set, and defect detection is performed on the second abnormal temperature region set through the defect detection model to output a defect detection result.

[0052] Optionally, during the defect detection process, the system terminal synchronizes the obtained multiple temperature change trends to the defect detection model, and the defect detection model analyzes these trends to predict future temperature change trends. This step is to more accurately identify potential temperature anomaly areas. Subsequently, the defect detection model combines the predicted future temperature change trends with the temperature record data set to obtain a second abnormal temperature area set. Compared with the previous first abnormal temperature area set, this set is more accurate and reliable because it combines the predicted data. Afterwards, the defect detection model automatically identifies and classifies these abnormal temperature areas according to the determined second abnormal temperature area set, and outputs the corresponding defect detection results. These results will help operators quickly locate and solve potential problems, thereby improving the safety and reliability of roads and bridges.

[0053] A defect information set is identified according to the defect detection result, wherein the defect information set includes defect location information, defect type information, and defect weight information; and the defect location information, the defect type information, and the defect weight information are added to the road and bridge defect data set.

[0054] Optionally, after obtaining the defect detection results, the system terminal extracts a defect information set from the defect detection results. This information set contains three parts, namely, defect location information, defect type information, and defect weight information. Defect location information refers to the specific location of the defect in the road and bridge structure, which is an area range. Defect type information describes the type or nature of the defect, such as cracks, corrosion, deformation, etc. This is very important for subsequent maintenance and reinforcement work, because different types of defects may require different treatment methods. Defect weight information is a quantitative assessment of the severity of the defect. It is a numerical value that represents the potential impact of the defect on the safety and stability of the road and bridge structure. After obtaining these detailed defect information, the system terminal adds this information to the road and bridge defect data set. This data set is used to generate subsequent defect detection reports, and this data set is also an important reference for road and bridge maintenance and management.

[0055] An assessment is performed based on the road and bridge defect data set, a defect detection report is constructed according to the road and bridge defect assessment result, and the defect detection report is sent to a remote data processing unit through an early warning signal.

[0056] In one embodiment, the system terminal conducts a comprehensive and in-depth evaluation of the defect status of the road and bridge based on the acquired road and bridge defect data set, that is, the road and bridge defect data set is parsed, the defect weight information in the road and bridge defect data set is compared with the defect risk level table, the risk level of the defect corresponding to the defect weight information is determined, and an evaluation result is generated. This defect risk level table is constructed based on historical experience and professional knowledge. Subsequently, based on this evaluation result, the system terminal constructs a detailed defect detection report, which will clearly list all detected defects, including their location, type and risk level. Once the defect detection report is completed, the system terminal sends it to the remote data processing unit through an early warning signal. This unit is a center for centralized management and analysis of data, which can further process and analyze the received defect detection reports and provide timely decision support for operators.

[0057] In summary, the embodiments of the present application have at least the following technical effects:

[0058] The embodiment of the present application performs three-dimensional modeling of the road and bridge by retrieving the historical image database of the target road and bridge to obtain spatial structure information. Subsequently, based on the spatial structure information, detection constraints are set to ensure that the infrared thermal imager can scan the target road and bridge according to the predetermined detection distance, focal length, direction and position to generate a complete infrared image set. After that, the infrared image set is traversed and temperature analyzed to construct a temperature distribution map sequence and extract multiple temperature change trends. At the same time, feature extraction is performed based on the spatial structure information to determine multiple key structural features, and correlation analysis is performed in combination with the temperature analysis results to generate correlation analysis factors. These factors are used to train the defect detection model to improve its detection accuracy. Once the defect detection model training is completed, multiple temperature change trends are synchronized to the model, defect detection is performed, and detailed defect detection results are output. These results include the location, type and weight information of the defect, which are integrated into a road and bridge defect data set. Then, based on this data set, a defect detection report is constructed and sent to a remote data processing unit through an early warning signal for further analysis and processing. These technical effects jointly solve the technical problems that traditional detection technology requires a lot of time and manpower to conduct detailed inspections of every part of the road and bridge, and it is difficult to detect hidden defects or early damage. It realizes the use of infrared imaging technology to quickly scan the surface of the road and bridge, improves the detection speed and efficiency, and can detect tiny temperature changes at the same time, thereby discovering tiny defects or damage.

[0059] Embodiment 2

[0060] Based on the same inventive concept as the municipal road and bridge defect detection method based on infrared imaging in the aforementioned embodiment, Figure 2As shown, the present application provides a municipal road and bridge defect detection system based on infrared imaging, the system comprising:

[0061] Three-dimensional modeling module 1: The three-dimensional modeling module 1 is used to perform three-dimensional modeling on the target road or bridge to obtain the spatial structure information of the target road or bridge.

[0062] Scanning and detection module 2: The scanning and detection module 2 is used to introduce detection constraints based on the spatial structure information, scan and detect the target road and bridge according to the detection constraints through an infrared thermal imager, and generate an infrared image set of the target road and bridge.

[0063] Temperature analysis module 3: The temperature analysis module 3 is used to traverse the infrared image set to perform temperature analysis, construct a temperature distribution map sequence, and obtain a temperature analysis result according to the temperature distribution map sequence.

[0064] Defect detection and identification module 4: The defect detection and identification module 4 is used to perform feature extraction based on the spatial structure information, determine multiple structural features, perform defect detection and identification based on the temperature analysis results and in combination with the multiple structural features, and obtain a road and bridge defect data set.

[0065] Defect assessment module 5: The defect assessment module 5 is used to perform assessment based on the road and bridge defect data set, construct a defect detection report according to the road and bridge defect assessment result, and send the defect detection report to the remote data processing unit through an early warning signal.

[0066] Furthermore, the three-dimensional modeling module 1 is also used to perform the following method:

[0067] Retrieve a historical image database of the target road and bridge, and extract multiple two-dimensional image data from multiple angles based on the historical image database; perform image matching on the multiple two-dimensional image data from multiple angles, perform structural restoration based on the matching results, and extract multiple three-dimensional point cloud data; perform three-dimensional modeling based on the multiple three-dimensional point cloud data to construct the spatial structure information of the target road and bridge.

[0068] Furthermore, the scanning detection module 2 is also used to perform the following method:

[0069] Based on the spatial structure information, a target to be detected is determined, a detection distance is calculated according to the target to be detected, and detection focal length constraint information is set; based on the spatial structure information, a range to be detected is determined, a detection angle is calculated according to the range to be detected, and detection direction constraint information is set; based on the target to be detected, a preset detection position is evaluated in combination with the spatial structure information, and detection position constraint information is determined; the detection focal length constraint information, the detection direction constraint information, and the detection position constraint information are integrated to generate the detection constraint condition.

[0070] Furthermore, the scanning detection module 2 is also used to perform the following method:

[0071] Based on the detection position constraint information, infrared thermal imagers are deployed to determine a detection position information set; based on the detection position information set, a detection direction information set of the infrared thermal imager is set according to the detection direction constraint information; based on the detection focal length constraint information, a detection focal length information set of the infrared thermal imager is obtained by adjusting the detection position information set and the detection direction information set; based on the detection position information set, the detection direction information set and the detection focal length information set, an infrared scanning plan is formulated, the infrared thermal imager is activated to execute the infrared scanning plan, and a plurality of infrared images are obtained; and the plurality of infrared images are combined according to the spatial structure information to generate the infrared image set of the target road and bridge.

[0072] Furthermore, the temperature analysis module 3 is also used to perform the following method:

[0073] The infrared image set is traversed, and each image is loaded into an image processing unit for information extraction to obtain a temperature information set, and the temperature information set corresponds to the infrared image set; each pixel in the infrared image set is calculated based on the temperature information set to generate multiple temperature extremes and multiple temperature averages; based on the multiple temperature extremes and the multiple temperature averages, the infrared image set is labeled with temperature values ​​in turn to generate multiple temperature distribution maps, and the multiple temperature distribution maps correspond to the infrared image set; the multiple temperature distribution maps are spliced ​​according to the acquisition time sequence of the infrared image set to generate a temperature distribution map sequence; the target road and bridge are analyzed according to the temperature distribution map sequence to obtain multiple temperature change trends; and the multiple temperature change trends are added to the temperature analysis result.

[0074] Furthermore, the temperature analysis module 3 is also used to perform the following method:

[0075] Retrieve the historical temperature record log of the target road and bridge, and extract the temperature record data set under normal operating conditions according to the historical temperature record log; determine the first abnormal temperature area set based on the comparison of the temperature record data set with the temperature analysis result; perform correlation analysis with the first abnormal temperature area set in combination with the multiple structural features to generate correlation analysis factors; use the correlation analysis factors to train the defect detection model and construct the defect detection model; synchronize the multiple temperature change trends to the defect detection model to obtain the second abnormal temperature area set, and perform defect detection on the second abnormal temperature area set through the defect detection model, and output the defect detection result; identify the defect information set according to the defect detection result, and the defect information set includes defect location information, defect type information, and defect weight information; add the defect location information, the defect type information, and the defect weight information to the road and bridge defect data set.

[0076] Furthermore, the defect detection and identification module 4 is also used to perform the following method:

[0077] The spatial structure information is divided using a regular grid to generate a plurality of pixel areas; spatial structure feature parameters of the plurality of pixel areas are extracted, weights are assigned based on the spatial structure feature parameters, and spatial structure feature weight data is generated; the spatial structure feature weight data is sorted in descending order to generate a weight sequence, the spatial structure feature parameters are extracted according to the weight sequence, and the plurality of structural features are determined.

[0078] It should be noted that the above-mentioned sequence of the embodiments of the present application is only for description and does not represent the advantages and disadvantages of the embodiments. And the above-mentioned specific embodiments of this specification are described. The processes depicted in the accompanying drawings do not necessarily require the specific order and continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0079] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

[0080] This specification and drawings are merely exemplary illustrations of the present application and are deemed to cover any and all modifications, variations, combinations or equivalents within the scope of the present application. Obviously, a person skilled in the art may make various modifications and variations to the present application without departing from the scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the present application and its equivalents, the present application intends to include these modifications and variations.

Claims

1. A municipal road and bridge defect detection method based on infrared imaging, characterized in that: The method comprises: Carry out three-dimensional modeling of the target road and bridge to obtain the spatial structure information of the target road and bridge; Introducing detection constraints based on the spatial structure information, scanning and detecting the target road and bridge according to the detection constraints by an infrared thermal imager, and generating an infrared image set of the target road and bridge; Traversing the infrared image set to perform temperature analysis, constructing a temperature distribution map sequence, and obtaining a temperature analysis result according to the temperature distribution map sequence; Perform feature extraction based on the spatial structure information to determine multiple structural features, perform defect detection and identification based on the temperature analysis result and in combination with the multiple structural features to obtain a road and bridge defect data set; Performing an evaluation based on the road and bridge defect data set, constructing a defect detection report according to the road and bridge defect evaluation result, and sending the defect detection report to a remote data processing unit through an early warning signal; Traversing the infrared image set to perform temperature analysis, constructing a temperature distribution map sequence, and obtaining a temperature analysis result according to the temperature distribution map sequence, the method comprising: Traversing the infrared image set, loading each image into an image processing unit for information extraction, and obtaining a temperature information set, wherein the temperature information set corresponds to the infrared image set; Calculating each pixel in the infrared image set based on the temperature information set to generate multiple temperature extreme values ​​and multiple temperature average values; Based on the multiple temperature extreme values ​​and the multiple temperature average values, the infrared image set is sequentially marked with temperature values ​​to generate multiple temperature distribution graphs, wherein the multiple temperature distribution graphs correspond to the infrared image set; splicing the multiple temperature distribution maps according to the acquisition time sequence of the infrared image set to generate a temperature distribution map sequence; Analyze the target road and bridge according to the temperature distribution diagram sequence to obtain multiple temperature change trends; adding the plurality of temperature change trends to the temperature analysis result; According to the temperature analysis result, defect detection and identification are performed in combination with the multiple structural features to obtain a road and bridge defect data set, the method comprising: Retrieving the historical temperature record log of the target road and bridge, and extracting the temperature record data set under normal operating conditions according to the historical temperature record log; Determine a first abnormal temperature region set based on comparing the temperature record data set with the temperature analysis result; performing correlation analysis on the plurality of structural features and the first abnormal temperature region set to generate a correlation analysis factor; Using the association analysis factors to train a defect detection model to construct the defect detection model; Synchronizing the multiple temperature change trends to the defect detection model to obtain a second abnormal temperature region set, and performing defect detection on the second abnormal temperature region set through the defect detection model to output a defect detection result; Identify a defect information set according to the defect detection result, wherein the defect information set includes defect location information, defect type information, and defect weight information; The defect location information, the defect type information, and the defect weight information are added to the road and bridge defect dataset.

2. The method according to claim 1, characterized in that The target road and bridge are subjected to three-dimensional modeling to obtain spatial structural information of the target road and bridge, and the method includes: Retrieving a historical image database of the target road and bridge, and extracting a plurality of two-dimensional image data at multiple angles based on the historical image database; Performing image matching on the multiple two-dimensional image data at multiple angles, performing structural restoration according to the matching results, and extracting multiple three-dimensional point cloud data; Three-dimensional modeling is performed based on the multiple three-dimensional point cloud data to construct the spatial structure information of the target road and bridge.

3. The method according to claim 1, characterized in that Introducing detection constraints based on the spatial structure information, the method includes: Determine a target to be detected based on the spatial structure information, calculate a detection distance according to the target to be detected, and set detection focal length constraint information; Determine a range to be detected based on the spatial structure information, calculate a detection angle according to the range to be detected, and set detection direction constraint information; Based on the target to be detected, the preset detection position is evaluated in combination with the spatial structure information to determine the detection position constraint information; The detection focal length constraint information, the detection direction constraint information, and the detection position constraint information are integrated to generate the detection constraint condition.

4. The method according to claim 3, characterized in that Scanning and detecting the target road and bridge according to the detection constraint conditions by an infrared thermal imager to generate an infrared image set of the target road and bridge, the method comprising: Deploy infrared thermal imagers based on the detection position constraint information to determine a detection position information set; Based on the detection position information set, setting the detection direction information set of the infrared thermal imager according to the detection direction constraint information; Based on the detection focal length constraint information, a detection focal length information set of the infrared thermal imager is obtained by adjusting the detection position information set and the detection direction information set; Formulate an infrared scanning plan based on the detection position information set, the detection direction information set, and the detection focal length information set, activate the infrared thermal imager to execute the infrared scanning plan, and obtain multiple infrared images; The multiple infrared images are combined according to the spatial structure information to generate the infrared image set of the target road and bridge.

5. The method according to claim 1, characterized in that Based on the spatial structure information, feature extraction is performed to determine multiple structural features, and the method includes: Using a regular grid to divide the spatial structure information to generate a plurality of pixel regions; Extracting spatial structure characteristic parameters of the plurality of pixel regions, performing weight assignment based on the spatial structure characteristic parameters, and generating spatial structure characteristic weight data; The spatial structure feature weight data is sorted in descending order to generate a weight sequence, and the spatial structure feature parameters are extracted according to the weight sequence to determine the multiple structural features.

6. The municipal road and bridge defect detection system based on infrared imaging is characterized by: The system is used to implement the steps of the municipal road and bridge defect detection method based on infrared imaging as described in any one of claims 1 to 5, and comprises: 3D modeling module: perform 3D modeling on the target road and bridge to obtain the spatial structure information of the target road and bridge; Scanning and detection module: introducing detection constraints based on the spatial structure information, scanning and detecting the target road and bridge according to the detection constraints through an infrared thermal imager, and generating an infrared image set of the target road and bridge; Temperature analysis module: traverses the infrared image set to perform temperature analysis, constructs a temperature distribution map sequence, and obtains a temperature analysis result according to the temperature distribution map sequence; Defect detection and identification module: extracting features based on the spatial structure information, determining multiple structural features, performing defect detection and identification based on the temperature analysis results and combining the multiple structural features to obtain a road and bridge defect data set; Defect assessment module: performs assessment based on the road and bridge defect data set, constructs a defect detection report according to the road and bridge defect assessment result, and sends the defect detection report to the remote data processing unit through an early warning signal.

Images