Bridge structure health monitoring method and system assisted by unmanned aerial vehicle

Abstract

The invention provides a bridge structure health monitoring method and system assisted by an unmanned aerial vehicle, and relates to the technical field of bridge health monitoring, and the method comprises the steps: obtaining the image information of a to-be-monitored bridge according to a multispectral imager carried by the unmanned aerial vehicle; performing three-dimensional reconstruction according to the image information to obtain a beam body three-dimensional model of the to-be-monitored bridge; and performing health monitoring on the bridge to be monitored according to the three-dimensional model of the bridge body. According to the bridge structure health monitoring method assisted by the unmanned aerial vehicle provided by the invention, the structure safety of the bridge is evaluated more scientifically. The accuracy and reliability of structural health assessment are improved, and a more targeted basis is provided for bridge maintenance.

Description

Background Art

[0002] As a key component of transportation infrastructure, the structural safety of bridges is crucial to ensuring smooth and safe transportation. With the increase in the service life of bridges and the continuous increase in traffic volume, bridge structures are easily damaged and deteriorated by various factors, such as environmental erosion, fatigue loads, natural disasters, etc. Therefore, effective health monitoring of bridges is a necessary means to timely discover potential safety hazards and formulate reasonable maintenance strategies.

[0003] Traditional bridge health monitoring methods mainly include manual regular inspections and monitoring technologies based on single sensors. Manual inspections rely on the experience and expertise of inspectors, and use visual inspections, simple measuring tools and other means to check for obvious damage on the bridge surface. This method is inefficient and difficult to detect some hidden damage, such as internal steel corrosion and internal cracks in concrete. Monitoring technologies based on single sensors, such as strain gauges and accelerometers, can monitor changes in certain mechanical parameters of bridges to a certain extent, but can only obtain local information and cannot fully reflect the overall structural state of the bridge. For example, strain gauges can only measure the strain of the adhesive part, and it is difficult to detect damage or structural changes far away from the sensor position; accelerometers are mainly used to monitor the vibration characteristics of bridges, and have limited monitoring capabilities for static deformation of structures and changes in internal material characteristics.

[0004] In addition, in terms of bridge appearance inspection, most of the existing image monitoring technologies are based on visible light images. Although visible light images can intuitively display the appearance of the bridge surface, they are not able to obtain information related to some internal structural defects and changes in material properties. For example, visible light images are difficult to provide effective diagnostic basis for changes in moisture content inside the concrete beam, early micro cracks, and microstructural changes caused by stress concentration inside the steel. Summary of the invention

[0005] In response to the above technical problems, the present application provides a drone-assisted bridge structure health monitoring method, which at least partially solves the problems existing in the prior art.

[0006] In a first aspect of the present application, a method for monitoring the health of a bridge structure assisted by a drone is provided, the method comprising:

[0007] Obtain image information of the bridge to be monitored using the multispectral imager carried by the UAV;

[0008] Perform three-dimensional reconstruction based on the above image information to obtain a three-dimensional model of the beam of the bridge to be monitored;

[0009] According to the above three-dimensional model of the beam body, health monitoring is performed on the above-mentioned bridge to be monitored.

[0010] In a second aspect of the present application, a drone-assisted bridge structure health monitoring system is provided, the system comprising:

[0011] An information acquisition unit, used for acquiring image information of the bridge to be monitored according to a multispectral imager carried by the UAV;

[0012] A model acquisition unit, used for performing three-dimensional reconstruction according to the above image information to obtain a three-dimensional model of the beam of the bridge to be monitored;

[0013] The monitoring unit is used to perform health monitoring on the bridge to be monitored based on the three-dimensional model of the beam.

[0014] This application has at least the following beneficial effects:

[0015] The drone-assisted bridge structure health monitoring method provided in this application uses a drone equipped with a multi-spectral imager to obtain richer beam information, not only limited to the surface appearance, but also to assist in judging the health status of the beam from the perspective of changes in the internal characteristics of the material. Then, a corresponding three-dimensional model of the beam is constructed, and finally, combined with finite element analysis software, the quantitative deformation data is input to simulate the stress and strain distribution of the beam under actual load, so as to more scientifically evaluate the structural safety of the bridge. The accuracy and reliability of structural health assessment are improved, providing a more targeted basis for bridge maintenance. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0017] Figure 1 A flow chart of a method for monitoring the health of a bridge structure assisted by a drone provided in an embodiment of the present application;

[0018] Figure 2 A structural block diagram of the drone-assisted bridge structure health monitoring system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0019] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0020] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0021] It should be noted that various aspects of the embodiments within the scope of the appended claims are described below. It should be apparent that the aspects described herein may be embodied in a wide variety of forms, and any specific structure and / or function described herein is merely illustrative. Based on the present application, it should be understood by those skilled in the art that an aspect described herein may be implemented independently of any other aspect, and two or more of these aspects may be combined in various ways. For example, any number of aspects described herein may be used to implement the device and / or practice the method. In addition, other structures and / or functionalities other than one or more of the aspects described herein may be used to implement this device and / or practice this method.

[0022] Please refer to Figure 1 As shown, an embodiment of the present application provides a method for monitoring the health of a bridge structure assisted by a drone, the method comprising:

[0023] Step S100, obtaining image information of the bridge to be monitored using the multispectral imager carried by the UAV.

[0024] Specifically, a multispectral imager is mounted on the drone, where a multispectral imager is an instrument that can obtain image information of a target object within a spectral range of multiple different bands. It can decompose the electromagnetic radiation reflected or emitted by the target into multiple narrow bands and image them separately, thereby obtaining the image characteristics of the target in different spectral bands. These bands usually include visible light bands, near infrared bands, short-wave infrared bands, etc., and each band can reflect the specific physical or chemical properties of the target.

[0025] In this embodiment, since the imager can simultaneously collect image information of multiple spectral bands such as visible light, near infrared and thermal infrared, the material properties of the bridge beam will show different reflection and radiation characteristics in different spectral bands. For example, in the near infrared band, the difference in moisture content inside the concrete and the tiny cracks in the structure will show different image contrast from the visible light band, and the thermal infrared band can reflect the temperature distribution difference of the beam due to internal stress changes or structural damage.

[0026] Therefore, compared with cameras and other equipment in related technologies, the image information of the bridge to be monitored obtained by the spectral imager in this embodiment can obtain richer beam information, which is not only limited to the surface appearance, but also can assist in judging the health status of the beam from the perspective of changes in the internal characteristics of the material.

[0027] Step S200: Perform three-dimensional reconstruction based on the above image information to obtain a three-dimensional model of the beam of the bridge to be monitored.

[0028] Specifically, multispectral images are processed to achieve high-precision 3D reconstruction. First, images of different spectral bands are registered and fused, and the feature differences of each pixel in the image under different spectra are used to improve the accuracy of feature extraction. For example, when the visible light and near-infrared images are fused, the edge detection of cracks on the beam surface will be more accurate.

[0029] Then, a point cloud-based 3D reconstruction method is used to convert the fused image information into 3D point cloud data. Different from the traditional 3D reconstruction based only on visible light images, the integration of multispectral information enables the point cloud data to more accurately reflect the real structural morphology of the beam, especially when dealing with areas with similar surface textures but different internal structures.

[0030] Step S300: performing health monitoring on the bridge to be monitored based on the three-dimensional model of the beam.

[0031] Specifically, by comparing the three-dimensional models of the beam reconstructed at different times, the deformation of each part of the beam can be calculated. The method based on spatial geometric analysis is not only a simple comparison of the overall displacement of the model, but also the analysis of complex deformation characteristics such as the local curvature change and distortion of the beam. For example, for the sinking of one end and the bending deformation of the middle part of the beam caused by uneven settlement, the deformation parameters of different parts can be accurately quantified.

[0032] Combined with finite element analysis software, the quantitative deformation data is input to simulate the stress and strain distribution of the beam under actual load, so as to more scientifically evaluate the structural safety of the bridge. This embodiment deeply combines the deformation data obtained by multi-spectral three-dimensional reconstruction with finite element analysis, improves the accuracy and reliability of structural health assessment, and provides a more targeted basis for bridge maintenance.

[0033] In summary, this embodiment uses a multi-spectral imager equipped with a drone to obtain richer information about the beam, not only limited to the surface appearance, but also to assist in judging the health of the beam from the perspective of changes in the internal characteristics of the material. Then, a corresponding three-dimensional model of the beam is constructed, and finally, combined with finite element analysis software, the quantitative deformation data is input to simulate the stress and strain distribution of the beam under actual load, so as to more scientifically evaluate the structural safety of the bridge. The accuracy and reliability of structural health assessment are improved, providing a more targeted basis for bridge maintenance.

[0034] In an exemplary embodiment of the present application, the above step S100 further includes the following steps:

[0035] Step S110, according to the target bridge structure type of the bridge to be monitored, determine the target critical loss type list corresponding to the bridge to be monitored; wherein the target bridge structure type is one of the preset bridge structure types; each preset bridge structure type has a corresponding critical loss type list; each critical loss type list includes several critical loss types; each critical loss type has a corresponding critical area.

[0036] Specifically, the target bridge structure type may be a beam structure, an arch structure, a cable structure, etc. Each bridge structure has a corresponding key loss type list, and the key loss type lists corresponding to different bridge structures may include the same key loss types, or may include completely different key loss types.

[0037] Each key loss type list includes several key loss types. As an example: the key loss type list corresponding to the beam structure may include flexural deformation, fatigue cracking, web oblique cracks, concrete spalling and exposed reinforcement, support voiding, etc.; wherein each key loss type has a corresponding key area, as an example: the key area corresponding to flexural deformation is the mid-span of the beam, the key area corresponding to fatigue cracking is the bottom of the mid-span of the beam, the key area corresponding to web oblique cracks is the web area of ​​the beam close to the support, the key area corresponding to concrete spalling and exposed reinforcement is the end of the beam, and the key area corresponding to support voiding is the support.

[0038] Step S120, determining a target monitoring area list of the bridge to be monitored according to the target critical loss type list; wherein the target monitoring area list includes a corresponding critical area for each critical loss type in the target critical loss type list.

[0039] Specifically, each key loss type in each key loss type list has a corresponding key area, which is mapped to the bridge to be monitored. The bridge to be monitored can obtain several target monitoring area lists according to the corresponding target key loss type list, and the target monitoring area list contains the key area corresponding to each key loss type in the target key loss type list. Here, each key area of ​​the bridge to be monitored included in the target monitoring area list is a part that is prone to problems or accidents.

[0040] Step S130, obtaining the material properties and damage development stage corresponding to each target monitoring area.

[0041] Specifically, different parts of the same bridge may be made of different materials. Different materials will exhibit different reflection and radiation characteristics for different spectral bands. For example, in the near-infrared band, differences in moisture content inside the concrete and tiny cracks in the structure will show different image contrast from the visible light band. The thermal infrared band can reflect the temperature distribution differences in the beam due to internal stress changes or structural damage.

[0042] Furthermore, the above-mentioned material properties may include: concrete structure, steel structure, composite structure, etc.

[0043] In addition, the above damage development stages are used to describe the damage degree of a certain damage of the bridge. As an example, the damage development stages may include: early damage stage, middle damage stage and late damage stage.

[0044] Step S140, setting a corresponding target spectral band for the corresponding target monitoring area according to the material characteristics and damage development stage corresponding to each target monitoring area.

[0045] Specifically, step S140 also includes:

[0046] Step S141, determining a basic spectral band corresponding to each target monitoring area according to the material characteristics corresponding to each target monitoring area.

[0047] Here, the basic spectral band range is determined according to the main components of the main materials of the bridge. For example, the cement, aggregate, water and other components of concrete correspond to the near-infrared and mid-infrared bands; in the near-infrared band (780-2500nm), the moisture in concrete has key spectral characteristics. The main components of steel are iron and a small amount of carbon and other alloying elements. In the near-ultraviolet band (200-400nm), the initial oxidation and crystal structure changes on the surface of steel will cause changes in the absorption and reflection characteristics of light, which is very helpful for detecting microcracks in the early stage of fatigue damage. At the same time, the thermal infrared characteristics of steel (3-5μm and 8-14μm) are very important for detecting heat release during corrosion, because steel corrosion will generate heat, and the rusted area can be found by temperature anomalies in the thermal infrared band. The carbon fiber and resin matrix of the composite material correspond to specific infrared bands and are assisted by visible light and near-infrared bands. Carbon fiber reinforced polymer (CFRP) is mainly composed of carbon fiber and resin matrix. In certain infrared bands, the chemical groups of the resin (such as epoxy groups) will have characteristic absorption peaks, and changes in these peaks can reflect the aging or damage of the composite material. The orientation of carbon fibers makes the material exhibit optical anisotropy in different directions, and its reflectivity in different bands will vary depending on the fiber direction.

[0048] Step S142, determining a corresponding target spectral band within the corresponding basic spectral band according to the damage development stage corresponding to each target monitoring area.

[0049] Here, step S142 also includes:

[0050] Step S142a: If the damage development stage is the initial stage of damage, the change sensitive band corresponding to the corresponding material characteristic is determined as the corresponding target spectral band.

[0051] Among them, the damage development stage can be obtained based on information such as bridge maintenance records. The health monitoring in this application is performed once every preset time interval, and the bridge maintenance record can be used to record the data of each health monitoring. The early stage of damage is slight damage. As an example: for concrete materials, the corresponding basic spectral band is the near-infrared band (780-2500nm), which is the maximum spectral band required for testing concrete materials. The newly generated microcracks will cause water to accumulate locally. Water has a specific absorption peak in the near-infrared band, such as in the range of 1400-1900nm. The accumulation of water causes the reflectivity to change, which can effectively detect microcracks. That is, if a target monitoring area is the early stage of damage to concrete materials (that is, the bridge maintenance record is not recorded as moderate or severe damage), the spectral band of 1400-1900nm can be used.

[0052] Step S142b, if the damage development stage is the mid-stage of damage, the corresponding target spectral band is determined based on the change sensitive band and the specific damage band corresponding to the corresponding material property; the mid-stage of damage is the damage characteristic change period determined based on historical damage data.

[0053] Among them, if the above damage development stage is the middle stage of damage, the middle stage of damage is the damage characteristic change period determined according to historical damage data. That is, in the middle stage of damage, the damage characteristics have changed significantly. At this time, compared with the early stage of damage, the corresponding spectral band needs to be adjusted. There are specific spectral bands for damage characteristics that may appear in the middle stage of damage. As an example: concrete cracks may expose the surface steel bars in the middle stage of damage. Then, it is considered that the exposure of the surface steel bars is a damage characteristic that has changed significantly. At this time, based on the detection in the range of 1400-1900nm, it may be necessary to set the corresponding spectral band for whether the surface steel bars are exposed, that is, the specific damage band. The above specific damage band can be determined based on historical data.

[0054] Step S142c: if the damage development stage is the late damage stage, the basic spectral band is determined as the corresponding target spectral band.

[0055] When the damage development stage is in the late stage of damage, multiple problems may arise. At this time, the bridge is comprehensively monitored based on the basic spectral band. Here, the basic spectral band includes several detection sub-bands. In this embodiment, the bridge is comprehensively monitored based on the several detection sub-bands.

[0056] Step S150, acquiring image information corresponding to each target monitoring area according to the target spectral band corresponding to each target monitoring area.

[0057] In this embodiment, since the material characteristics and damage development stages corresponding to each target monitoring area may be different, a corresponding target spectral band is determined for each target monitoring area, so as to obtain image information corresponding to each target monitoring area.

[0058] In an exemplary embodiment of the present application, after step S110, the method further includes:

[0059] Step S160, determining a non-target monitoring area list of the bridge to be monitored according to the above-mentioned target critical loss type list; wherein the non-target monitoring area list includes all areas of the bridge to be monitored except for the corresponding critical areas for each critical loss type in the target critical loss type list.

[0060] Step S170, obtaining material properties corresponding to each non-target monitoring area.

[0061] Step S180, according to the material characteristics corresponding to each non-target monitoring area, a corresponding target spectral band is set for the corresponding non-target monitoring area.

[0062] Step S190, acquiring image information corresponding to each non-target monitoring area according to the target spectral band corresponding to each non-target monitoring area.

[0063] In this embodiment, for each non-target monitoring area in the non-target monitoring area list, it is an area where problems are relatively less likely to occur. Therefore, the target spectral band is determined for each non-target monitoring area only based on the material characteristics of each non-target monitoring area, thereby saving resources and improving monitoring efficiency.

[0064] In an exemplary embodiment of the present application, the above step S200 includes:

[0065] Step S210: pre-processing the above image information.

[0066] Specifically, the above image information is firstly subjected to radiation correction, as follows: Due to the influence of factors such as lighting conditions in different spectral bands, there are radiation differences in multispectral images. For each spectral band image, radiation calibration is first performed. By collecting image data of a standard radiation source (such as a whiteboard and a blackboard with known reflectivity) before or after imaging, a radiation response model is established, and the original grayscale value of the image is converted into a radiation brightness value to eliminate the radiation inhomogeneity caused by ambient light. For example, for near-infrared band images, near-infrared standard radiation sources of specific wavelengths are used for calibration, so that the pixel values ​​of different areas in the image can accurately reflect the actual radiation intensity, laying the foundation for subsequent precise processing. Secondly, the image information is subjected to noise removal, as follows: an adaptive filtering algorithm is used to remove noise in multispectral images. The algorithm dynamically adjusts the filtering parameters according to the statistical characteristics of the local area of ​​the image. In the smooth area of ​​the image, a larger filtering window and a stronger filtering strength are used to effectively remove background noise such as Gaussian noise; at the edge and detail-rich area of ​​the image, the filtering window and strength are automatically reduced to retain important detail information such as cracks and structural changes. For example, for noise removal in visible light band images, the variance and mean of the local area of ​​the image are first calculated, and the filtering parameters are determined based on these statistics. While ensuring noise removal, the texture and damage detail information on the surface of the beam is retained to the greatest extent, thereby improving image quality.

[0067] Step S220: performing spectral band registration on the preprocessed image information.

[0068] Specifically, feature point extraction and matching are first performed, that is, the multi-scale Harris corner detection algorithm is used to extract feature points in images of different spectral bands. The algorithm can detect corner points in images at different scales, and these corner points have high stability and repeatability. Then, a feature matching method based on descriptors, such as SIFT (Scale Invariant Feature Transform) descriptors, is used to calculate the gradient information of the area around the feature points, generate feature description vectors, and match feature points in images of different spectral bands through the distance between feature vectors. For example, in near-infrared and visible light band images, the corner points at the edge of the beam, surface texture changes, etc. are detected respectively, and the corresponding relationship is found through SIFT descriptor matching, providing accurate matching point pairs for subsequent image registration. Then, image transformation and registration are performed, that is, according to the feature point matching results, an affine transformation model based on the least squares method is used for image registration. The affine transformation parameters that minimize the error between matching point pairs are calculated, including parameters such as translation, rotation, and scaling, and images of different spectral bands are mapped to the same coordinate system. In the registration process, robust estimation methods such as the RANSAC (Random Sample Consensus) algorithm are introduced to remove incorrectly matched point pairs and improve the accuracy and stability of the registration. For example, for the registration of thermal infrared band and visible light band images, the RANSAC algorithm is used to screen out the inliers (correctly matched point pairs), and the inliers are used to calculate the affine transformation parameters to achieve accurate registration of thermal infrared images and visible light images, ensuring the consistency of different spectral information in space.

[0069] Step S230: performing image fusion on the registered image information.

[0070] Specifically, first, weighted fusion is performed based on the pixel level. That is, the weighting coefficient is determined according to the information entropy and contrast characteristics of the pixels in the image in different spectral bands. In areas with rich details such as cracks and damages, a larger weighting coefficient is given to the spectral band image with higher detail resolution; in the smooth area of ​​the image, the weighting coefficient is assigned according to the noise level and radiation characteristics of each band image. As an example: when fusing visible light and near-infrared images to detect cracks on the surface of the beam, for the pixels at the edge of the crack, since the near-infrared image has an advantage in crack moisture detection, a higher weighting coefficient is given to the near-infrared image, so that the fused image is more accurate in crack edge detection; and in relatively smooth areas on the surface of the beam, the weights are reasonably assigned according to the noise and radiation conditions of the visible light and near-infrared images to obtain a high-quality fused image.

[0071] Secondly, the fusion based on regional segmentation is carried out, that is, the multispectral image is segmented into different homogeneous regions by using the image region segmentation method based on the watershed algorithm. For each region, the appropriate fusion rule is selected according to the spectral characteristic distribution of the pixels in the region. For example, in the steel bar corrosion detection of concrete beams, for the corrosion area (preliminarily determined by the thermal infrared band), the fusion method based on principal component analysis (PCA) is used to extract the principal component information related to corrosion from the images of different spectral bands for fusion, and enhance the characteristic display of the corrosion area; in the non-corroded area, the fusion method based on weighted average is used to integrate the image information of each band to improve the overall clarity and information content of the image.

[0072] Step S240, three-dimensionally reconstructing the fused image information to obtain a three-dimensional model of the beam of the bridge to be monitored.

[0073] Specifically, first, depth information is extracted, that is, the image information after multispectral image fusion is used to extract depth information. A method based on the principle of stereoscopic vision is used to match the image feature points under different viewing angles, calculate the disparity information, and then obtain the depth information. For areas with similar surface textures but different internal structures, multispectral information is used to assist depth calculation. For example, in areas where there are cavities or uneven distribution of steel bars inside the beam, the depth value of the feature point is more accurately calculated based on the difference in the penetration and reflection characteristics of the internal structure of the near-infrared band, combined with the surface texture information of the visible light image, to improve the accuracy and completeness of the depth information. Secondly, point cloud generation and optimization are performed, that is, the pixels in the image are converted into three-dimensional point cloud data based on the extracted depth information. In the process of point cloud generation, a method based on spatial indexing, such as octree indexing, is used to improve the storage and retrieval efficiency of point cloud data. Then, the generated point cloud is optimized, including removing outliers (by statistically analyzing the local density distribution of the point cloud) and smoothing the point cloud (using a smoothing algorithm based on the moving least squares method). For example, in the three-dimensional reconstruction of a bridge beam, the octree index is used to quickly locate and process point cloud data, remove outliers caused by noise or incorrect matching, and smooth the point cloud using the moving least squares method to make the reconstructed beam surface smoother and more realistic, thereby obtaining a high-precision three-dimensional point cloud model of the beam.

[0074] In an exemplary embodiment of the present application, the above step S300 includes:

[0075] Step S310 , matching and tracking feature points are performed based on the above beam body three-dimensional model and the historical beam body three-dimensional model, and local deformation parameter calculation is performed to obtain deformation data.

[0076] Specifically, feature point matching and tracking are first performed, that is, on the three-dimensional models of the beam reconstructed at different times, feature points are extracted using methods based on local feature description, such as 3D-SIFT (three-dimensional scale-invariant feature transform). These feature points can stably represent the geometric structural features of the beam, such as the edges and corners of the beam, key connection parts, etc. Then, the feature points in the models at different times are matched using the nearest neighbor search algorithm. In order to improve the accuracy of matching, a verification method based on geometric constraints is introduced, such as checking whether the distance and angle relationship between the matching point pairs conforms to the geometric deformation law of the beam.

[0077] For the successfully matched feature points, a tracking algorithm based on Kalman filtering is used to track them in time series. Kalman filtering can make the best estimate based on the state prediction of the previous moment and the observed value of the current moment, so as to obtain the precise position change of the feature points in different periods. In this way, the displacement information of the feature points in each local area of ​​the beam can be obtained, providing a data basis for subsequent deformation analysis.

[0078] Secondly, the local deformation parameters are calculated, that is, based on the displacement information of the characteristic points obtained by tracking, the curvature change of the local beam is calculated using the differential geometry method. For the curve on the surface of the beam, the curvature change is determined by calculating the change in its radius of curvature at different positions. For example, for the longitudinal axis curve of the beam, it is discretized into multiple small segments, and the curvature change of each small segment is calculated using the displacement difference of adjacent characteristic points. For the calculation of the degree of distortion, a method based on vector cross product is used. The normal vectors on the surface of the beam are compared at different times, and the degree of distortion is quantified by calculating the rotation angle and direction change of the normal vector. For example, in the cross-sectional area of ​​the beam, the cross product of the normal vectors at each point before and after the time is calculated to obtain a vector value representing the degree of distortion.

[0079] Step S320: Obtain deformation information of the bridge to be monitored based on the above deformation data and finite element analysis.

[0080] Specifically, first, the deformation data (including characteristic point displacement, local curvature change, distortion degree, etc.) obtained from multi-spectral 3D reconstruction is converted into a format that can be recognized by finite element analysis software. For example, the discrete characteristic point data on the beam surface is converted into the displacement boundary conditions of the finite element mesh nodes, and the local curvature and distortion information is converted into equivalent stress and strain initial state data.

[0081] During the data conversion process, a mapping method based on the physical model is used to ensure that the physical meaning of the deformation data in the finite element model is accurate. For example, according to parameters such as the elastic modulus and Poisson's ratio of the beam material, the deformation data is converted into equivalent load or initial strain in the finite element model, so that the finite element model can perform accurate mechanical analysis based on multi-spectral reconstruction data.

[0082] Secondly, multi-scale finite element modeling and analysis is carried out, that is, a multi-scale finite element model is established based on the structural characteristics of the bridge beam. At the macro scale, beam units and shell units are used to simulate the overall structural behavior of the beam, such as bending and torsion; at the micro scale, solid units are used for fine modeling of key parts of the beam (such as the bonding area between steel bars and concrete, areas near cracks, etc.) to accurately simulate local stress concentration and damage evolution.

[0083] The converted multi-spectral deformation data is input into the multi-scale finite element model for nonlinear finite element analysis. During the analysis, the nonlinear characteristics of the material (such as cracking of concrete, yielding of steel, etc.) and geometric nonlinearity (such as changes in the shape of the beam under large deformation) are considered. Through finite element analysis, detailed information such as the stress-strain distribution and internal force changes of the beam under actual load and deformation conditions are obtained, so as to comprehensively evaluate the structural safety of the bridge. For example, it is analyzed whether the stress in the key parts of the beam exceeds the allowable strength of the material under the combined action of uneven settlement and traffic load, and whether the overall stability of the beam is affected.

[0084] Step S330: obtaining the health monitoring result of the bridge to be monitored according to the deformation information and the preset health assessment model.

[0085] Specifically, the deformation information is input into a preset health evaluation model to obtain the health monitoring result of the bridge to be monitored. As an example: the health monitoring result of the bridge to be monitored may be in the form of an evaluation report, an evaluation score, and the like.

[0086] Please refer to Figure 2 As shown, the embodiment of the present application provides a drone-assisted bridge structure health monitoring system 100. For a detailed description of the system, please refer to the above method embodiment, which will not be repeated here. The system includes:

[0087] The information acquisition unit 110 is used to acquire image information of the bridge to be monitored based on the multi-spectral imager carried by the UAV.

[0088] The model acquisition unit 120 is used to perform three-dimensional reconstruction according to the above image information to obtain a three-dimensional model of the beam of the bridge to be monitored.

[0089] The monitoring unit 130 is used to perform health monitoring on the bridge to be monitored based on the three-dimensional model of the beam.

[0090] In an exemplary embodiment of the present application, the information acquisition unit 110 includes:

[0091] The first list acquisition subunit is used to determine the target critical loss type list corresponding to the bridge to be monitored according to the target bridge structure type of the bridge to be monitored; wherein the target bridge structure type is one of the preset bridge structure types; each preset bridge structure type has a corresponding critical loss type list; each critical loss type list includes several critical loss types; and each critical loss type has a corresponding critical area.

[0092] The second list acquisition subunit is used to determine a target monitoring area list of the bridge to be monitored according to the above target critical loss type list; wherein the target monitoring area list contains a corresponding critical area for each critical loss type in the target critical loss type list.

[0093] The information acquisition subunit is used to obtain the material properties and damage development stage corresponding to each target monitoring area.

[0094] The band determination subunit is used to set a corresponding target spectral band for the corresponding target monitoring area according to the material characteristics and damage development stage corresponding to each target monitoring area.

[0095] The image information acquisition subunit is used to acquire the image information corresponding to each target monitoring area according to the target spectral band corresponding to each target monitoring area.

[0096] In an exemplary embodiment of the present application, the above-mentioned band determination subunit includes:

[0097] The basic band determination component is used to determine the basic spectral band corresponding to each target monitoring area according to the material characteristics corresponding to each target monitoring area.

[0098] The target band determination component is used to determine the corresponding target spectral band within the corresponding basic spectral band according to the damage development stage corresponding to each target monitoring area.

[0099] In an exemplary embodiment of the present application, an electronic device capable of implementing the above method is also provided.

[0100] Those skilled in the art will appreciate that various aspects of the present application may be implemented as a system, method or program product. Therefore, various aspects of the present application may be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to as "circuit", "module" or "system" herein.

[0101] The electronic device according to this embodiment of the present application is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0102] The electronic device is presented in the form of a general-purpose computing device. The components of the electronic device may include, but are not limited to: the at least one processor mentioned above, the at least one storage device mentioned above, and a bus connecting different system components (including storage devices and processors).

[0103] The storage stores program codes, which can be executed by the processor, so that the processor executes the steps described in the above “Exemplary Method” section of this specification according to various exemplary embodiments of the present application.

[0104] The memory may include readable media in the form of volatile memory, such as random access memory (RAM) and / or cache memory, and may further include read only memory (ROM).

[0105] The storage may also include a program / utility having a set (at least one) of program modules, such program modules including but not limited to: an operating system, one or more application programs, other program modules and program data, each of which or some combination may include the implementation of a network environment.

[0106] The bus may represent one or more of several types of bus structures including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor, or a local bus using any of a variety of bus architectures.

[0107] The electronic device can also communicate with one or more external devices (such as keyboards, pointing devices, Bluetooth devices, etc.), and can also communicate with one or more devices that enable users to interact with the electronic device, and / or communicate with any device (such as routers, modems, etc.) that enables the electronic device to communicate with one or more other computing devices. This communication can be carried out through an input / output (I / O) interface. In addition, the electronic device can also communicate with one or more networks (such as local area networks (LANs), wide area networks (WANs) and / or public networks, such as the Internet) through a network adapter. As shown in the figure, the network adapter communicates with other modules of the electronic device through a bus. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with the electronic device, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0108] Through the description of the above implementation methods, it is easy for those skilled in the art to understand that the example implementation methods described here can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the implementation methods of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the implementation methods of the present application.

[0109] In an exemplary embodiment of the present application, a computer-readable storage medium is also provided, on which a program product capable of implementing the above method of the present specification is stored. In some possible implementations, various aspects of the present application can also be implemented in the form of a program product, which includes a program code. When the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps according to various exemplary implementations of the present application described in the above "Exemplary Method" section of the present specification.

[0110] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0111] Computer readable signal media may include data signals propagated in baseband or as part of a carrier wave, in which readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Readable signal media may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0112] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.

[0113] Program code for performing the operations of the present application may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0114] In addition, the above-mentioned figures are only schematic illustrations of the processes included in the method according to the exemplary embodiments of the present application, and are not intended to be limiting. It is easy to understand that the processes shown in the above-mentioned figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.

[0115] It should be noted that, although several modules or units of the equipment for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into being embodied by multiple modules or units.

[0116] The above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A method for monitoring bridge structure health assisted by an unmanned aerial vehicle, characterized in that: The method comprises: Obtain image information of the bridge to be monitored using the multispectral imager carried by the UAV; Performing three-dimensional reconstruction according to the image information to obtain a three-dimensional model of the beam of the bridge to be monitored; The health monitoring of the bridge to be monitored is performed according to the three-dimensional model of the beam body.

2. The method for monitoring bridge structure health using drones according to claim 1, characterized in that: The image information of the bridge to be monitored is obtained using the multispectral imager carried by the drone, including: According to the target bridge structure type of the bridge to be monitored, a target key loss type list corresponding to the bridge to be monitored is determined; wherein the target bridge structure type is one of the preset bridge structure types; each preset bridge structure type has a corresponding key loss type list; each key loss type list includes several key loss types; each key loss type has a corresponding key area; According to the target critical loss type list, a target monitoring area list of the bridge to be monitored is determined; wherein the target monitoring area list includes a corresponding critical area for each critical loss type in the target critical loss type list; Obtain the material properties and damage development stages corresponding to each target monitoring area; According to the material characteristics and damage development stage corresponding to each target monitoring area, a corresponding target spectral band is set for the corresponding target monitoring area; According to the target spectral band corresponding to each target monitoring area, the image information corresponding to each target monitoring area is obtained.

3. The method for monitoring bridge structure health using drones according to claim 2, characterized in that: According to the material properties and damage development stage of each target monitoring area, the corresponding target spectral band is set for the corresponding target monitoring area, including: According to the material characteristics corresponding to each target monitoring area, determine the basic spectral band corresponding to each target monitoring area; According to the damage development stage corresponding to each target monitoring area, the corresponding target spectral band is determined within the corresponding basic spectral band.

4. The method for monitoring bridge structure health using drones according to claim 3, characterized in that: Determining a corresponding target spectral band within a corresponding basic spectral band according to the damage development stage corresponding to each target monitoring area includes: If the damage development stage is the initial stage of damage, the change sensitive band corresponding to the corresponding material characteristic is determined as the corresponding target spectral band; If the damage development stage is the mid-stage of damage, the corresponding target spectral band is determined by the change sensitive band corresponding to the corresponding material property and the specific damage band; the mid-stage of damage is the damage characteristic change period determined according to the historical damage data; If the damage development stage is the late damage stage, the basic spectral band is determined as the corresponding target spectral band.

5. The method for monitoring bridge structure health using drones according to claim 2, characterized in that: After determining a target critical loss type list corresponding to the bridge to be monitored according to the target bridge structure type of the bridge to be monitored, the method further includes: According to the target critical loss type list, a non-target monitoring area list of the bridge to be monitored is determined; wherein the non-target monitoring area list includes all areas of the bridge to be monitored except for the corresponding critical areas for each critical loss type in the target critical loss type list; Obtain material properties corresponding to each non-target monitoring area; According to the material characteristics corresponding to each non-target monitoring area, a corresponding target spectral band is set for the corresponding non-target monitoring area; According to the target spectral band corresponding to each non-target monitoring area, image information corresponding to each non-target monitoring area is obtained.

6. The method for monitoring bridge structure health using drones according to any one of claims 1 to 5, characterized in that: The three-dimensional reconstruction is performed according to the image information to obtain a three-dimensional model of the beam of the bridge to be monitored, including: Preprocessing the image information; Perform spectral band registration on the preprocessed image information; Perform image fusion on the registered image information; The fused image information is three-dimensionally reconstructed to obtain a three-dimensional model of the bridge beam to be monitored.

7. The method for monitoring bridge structure health using drones according to any one of claims 1 to 5, characterized in that: The health monitoring of the bridge to be monitored according to the three-dimensional model of the beam body includes: Matching and tracking characteristic points of the beam body three-dimensional model and the historical beam body three-dimensional model, and calculating local deformation parameters to obtain deformation data; Obtaining deformation information of the bridge to be monitored based on the deformation data and finite element analysis; The health monitoring result of the bridge to be monitored is obtained according to the deformation information and a preset health assessment model.

8. A UAV-assisted bridge structure health monitoring system, characterized in that: The system comprises: An information acquisition unit, used for acquiring image information of the bridge to be monitored according to a multispectral imager carried by the UAV; A model acquisition unit, used for performing three-dimensional reconstruction according to the image information to obtain a three-dimensional model of the beam of the bridge to be monitored; The monitoring unit is used to perform health monitoring on the bridge to be monitored according to the three-dimensional model of the beam body.

9. The UAV-assisted bridge structure health monitoring system according to claim 8, characterized in that: The information acquisition unit comprises: The first list acquisition subunit is used to determine the target critical loss type list corresponding to the bridge to be monitored according to the target bridge structure type of the bridge to be monitored; wherein the target bridge structure type is one of the preset bridge structure types; each preset bridge structure type has a corresponding critical loss type list; each critical loss type list includes a number of critical loss types; each critical loss type has a corresponding critical area; The second list acquisition subunit is used to determine a target monitoring area list of the bridge to be monitored according to the target key loss type list; wherein the target monitoring area list includes a key area corresponding to each key loss type in the target key loss type list; An information acquisition subunit is used to obtain the material properties and damage development stages corresponding to each target monitoring area; The band determination subunit is used to set a corresponding target spectral band for the corresponding target monitoring area according to the material characteristics and damage development stage corresponding to each target monitoring area; The image information acquisition subunit is used to acquire the image information corresponding to each target monitoring area according to the target spectral band corresponding to each target monitoring area.

10. The UAV-assisted bridge structure health monitoring system according to claim 9, characterized in that: The band determination subunit comprises: A basic band determination component is used to determine the basic spectral band corresponding to each target monitoring area according to the material characteristics corresponding to each target monitoring area; The target band determination component is used to determine the corresponding target spectral band within the corresponding basic spectral band according to the damage development stage corresponding to each target monitoring area.

Images