A method for monitoring inter-story displacement of a building structure based on computer vision recognition

Abstract

The application relates to a computer vision recognition-based building structure interlayer displacement monitoring method, which comprises the following steps: installing an interlayer deformation monitoring device and performing image correction; acquiring a real-time interlayer displacement visual image frame sequence of a building floor to be monitored for pretreatment, and selecting a target area and a target time period; adopting a feature point detection algorithm to detect the target area in an initial image frame, obtaining initial frame feature point coordinates; selecting multiple feature points and performing feature tracking to obtain displacement data, screening and fusing the displacement data to obtain a visual recognition structure displacement curve; acquiring slab-column corner data, and obtaining a slab-column corner curve through filtering and resampling; time aligning and eliminating redundant parts of the two curves through curve peak value correlation, and calculating nominal interlayer displacement, harmless interlayer displacement and harmful interlayer displacement of a monitoring layer by using the two aligned curves. Compared with the prior art, the application has the advantages of being capable of accurately identifying harmful interlayer displacement.

Description

Technical Field

[0001] This invention relates to the field of building structure health monitoring technology, and in particular to a method for monitoring inter-story displacement of building structures based on computer vision recognition. Background Technology

[0002] To ensure structural safety and prevent accidents caused by structural failure during the normal service phase of a building, it is necessary to acquire dynamic response data, such as rotation and displacement, using on-site sensing systems and related analysis techniques. This data is then used for timely damage diagnosis and performance evaluation, achieving the goal of monitoring and assessing structural health. Generally, in the field of structural health monitoring, deformation data is often acquired using contact sensors such as accelerometers, strain gauges, or displacement meters. These sensors are mostly deployed in a limited number of locations within the structure, acquiring deformation information from the target area. This approach has significant limitations in terms of data acquisition methods, the quantity of data acquired, and the comprehensiveness of the data.

[0003] In structural design, controlling lateral displacement and deformation under horizontal loads is crucial. Chinese standards typically use the inter-story drift angle as the deformation control parameter for inter-story members, which is generally calculated as the ratio of the story's displacement difference to its height. This method of calculating inter-story deformation is more suitable for structures where lateral displacement is primarily due to overall shear deformation. However, it introduces significant errors for high-rise structures where bending deformation is dominant.

[0004] Inter-story displacements in building structures can be divided into harmful inter-story displacements and harmless inter-story displacements, with harmful inter-story displacements being the primary cause of structural damage. For high-rise structures, the proportion of harmless inter-story displacements in the nominal inter-story displacement increases with increasing floor height, while the proportion of harmful inter-story displacements decreases. Therefore, accurately identifying harmful inter-story displacements is crucial for the proper assessment of the service condition of building structures.

[0005] Harmful displacements in structures are typically identified by installing instruments such as laser displacement gauges, total stations, global navigation satellite systems, wire displacement gauges, and linear variable differential transformer displacement gauges on key floors. However, these measuring instruments are expensive and cumbersome to operate, and are often used in large building structures such as high-rise buildings, but are difficult to apply widely to conventional civil buildings. Summary of the Invention

[0006] The purpose of this invention is to provide a method for monitoring inter-story displacement of building structures based on computer vision recognition. The nominal inter-story displacement is obtained through visual recognition, the rotation time history of the floor slab during the cyclic loading process is measured using an inclinometer, and the harmless inter-story displacement is obtained by combining the floor height of the frame model. The harmful inter-story displacement is obtained by subtracting the nominal inter-story displacement from the harmless inter-story displacement.

[0007] The objective of this invention can be achieved through the following technical solutions:

[0008] A method for monitoring inter-story displacement of building structures based on computer vision recognition includes the following steps:

[0009] S1. Install an inter-story deformation monitoring device on the floor where the inter-story deformation is to be monitored, and use a calibration plate for image correction. The inter-story deformation monitoring device includes an inclinometer installed on the monitoring floor, an inclinometer installed on the key structural lateral force resisting components, a camera installed on the floor, and a tracking target point installed on the ceiling.

[0010] S2. Use the inter-story deformation monitoring device to obtain the inter-story displacement visual image frame sequence of the building floors to be monitored in real time, perform preprocessing, and select the target area including the tracking target point and the target time period covering the entire dynamic load process of the structure.

[0011] S3. Use a feature point detection algorithm to detect the target region in the initial frame of the image and obtain the coordinates of the feature points in the initial frame;

[0012] S4. Select multiple feature points and perform feature tracking. Calculate the motion trajectory of the feature points between adjacent frames. Repeat this process until the displacement data of all feature points in the entire target image frame sequence is obtained. Filter the obtained displacement data of multiple feature points to filter out non-target feature points and feature points that are misidentified. Fuse the three types of displacement data after filtering to obtain the displacement curve of the visual recognition structure fused by multiple feature points.

[0013] S5. Acquire the monitoring data of the inclinometer during the movement of the building structure, select the floor slab and column rotation angle data of the monitoring layer corresponding to the visual recognition, filter out noise and resample to the same frequency as the inter-story displacement visual image to obtain the slab and column rotation angle curve.

[0014] S6. The plate-column rotation curve and the visual identification structural displacement curve are time-aligned by the peak correlation of the curves and redundant parts are eliminated. The nominal inter-story displacement, harmless inter-story displacement and harmful inter-story displacement of the monitoring layer are calculated by using the aligned visual identification structural displacement curve and plate-column rotation curve.

[0015] The inclinometers on the floor slab and key structural components resisting lateral forces are rigidly fixed in the corners near the camera, and the tracking target point always falls within the camera's field of view.

[0016] The image correction using the calibration board specifically involves: before monitoring, using a camera to photograph the calibration board placed in the target area of ​​the tracking target to obtain an image of the calibration board, and identifying the conversion relationship between the pixel coordinates of the calibration board in the image and the actual coordinates. The conversion relationship is then used to eliminate the influence of lens distortion and correct the image.

[0017] The preprocessing in step S2 includes video stabilization, image denoising, and enhancement.

[0018] The feature points include Harris, Shi-Tomasi, and SURF.

[0019] In step S4, optical flow is used for feature tracking. Assuming constant brightness, continuous time, and consistent motion of points within the neighborhood, then:

[0020] I(x,y,z)=I(x+dx,y+dy,z+dz)=I(x,y,z)+I x dx+I y dy+I z dz

[0021] In the formula, I(x, y, z) represents the gray value of the image coordinates (x, y, z) at time t. a This represents the partial derivative of I with respect to the subscript a;

[0022] Then we have:

[0023] I x dx+I y dy+I t dt = 0

[0024] Let dx / dt = u and dy / dt = v, where (u, v) is the optical flow. Then, the basic equation of the optical flow method can be obtained:

[0025] I x u+I y v = -I t

[0026] Meanwhile, considering that points in the neighborhood of the optical flow tracking point have different weights, the displacement of the feature point is calculated by the least squares method, and the position of the feature point in the next frame image is determined.

[0027] The process of filtering the displacement data of multiple feature points adopts a random consistent sampling method to filter feature points that are not located on the target object, such as feature points on a static background or other moving objects. At the same time, the correlation between the remaining data is calculated, and the data with the highest sum of correlation coefficients with the remaining displacement time histories is used as the benchmark. Feature points whose correlation coefficient between the displacement time histories data and the benchmark is lower than a preset threshold are identified as non-target feature points and filtered out.

[0028] The filtering in step S5 uses an FIR filter.

[0029] The time alignment of the plate column rotation curve and the visual recognition structure displacement curve through the peak correlation of the curves is specifically as follows: by calculating the correlation estimate between two random sequences or the deterministic correlation between two deterministic signals, the time difference between the displacement curve and the rotation curve is obtained, and the adjustment is made according to the time difference to achieve time alignment.

[0030] The formulas for calculating the nominal inter-story displacement, harmless inter-story displacement, and harmful inter-story displacement of the monitoring layer using the aligned visual recognition structural displacement curve and the plate-column rotation curve are as follows:

[0031] Δu=Δ+htanα

[0032] Δv=Δu-htanθ

[0033] Where is the floor height, Δu is the nominal inter-story displacement, htanθ is the harmless inter-story displacement, Δv is the harmful inter-story displacement, Δ is the visually recognized inter-story structural displacement, α is the floor slab rotation angle, and θ is the column rotation angle.

[0034] Compared with the prior art, the present invention has the following beneficial effects:

[0035] (1) This invention introduces computer vision technology to obtain data on harmful interlayer deformation in a convenient way. It can accurately judge the health status of the structure from the structural health monitoring data and solve the current problems in the detection and monitoring of harmful displacement of building structures.

[0036] (2) The computer vision technology used in this invention is cheaper and easier to deploy and use than conventional vibration sensors.

[0037] (3) This invention uses computer vision technology to identify inter-story displacement, and further combines it with floor tilt data to obtain the harmful inter-story displacement of the target floor, so as to accurately determine the stress and deformation state of the key floors of the structure. Attached Figure Description

[0038] Figure 1 This is a flowchart of the method of the present invention;

[0039] Figure 2 This is a schematic diagram of the interlayer deformation monitoring device of the present invention;

[0040] Figure 3 This is a schematic diagram illustrating the relationship and solution method between the nominal inter-story displacement, harmless inter-story displacement, and harmful inter-story displacement of this invention.

[0041] The attached figures are labeled as follows: 1- Inclinometer installed on key structural components resisting lateral forces, 2- Inclinometer installed on the monitoring floor slab, 3- Camera, 4- Tracking target point. Detailed Implementation

[0042] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. These embodiments are based on the technical solution of the present invention and provide detailed implementation methods and specific operating procedures. However, the scope of protection of the present invention is not limited to the following embodiments.

[0043] This embodiment provides a method for monitoring inter-story displacement of building structures based on computer vision recognition. By combining data measured by an inclinometer with visually recognized structural displacement, the nominal inter-story displacement and harmful inter-story displacement of the building structure are obtained, accurately acquiring the inter-story stress and overall stress situation of the building structure. Figure 1 As shown, it includes the following steps:

[0044] S1. Install inter-layer deformation monitoring devices on the floors where inter-layer deformation is to be monitored, and use calibration plates to perform image correction.

[0045] like Figure 2 As shown, the inter-story deformation monitoring device includes an inclinometer 1 installed on key structural lateral force resisting components (such as shear walls, frame columns, etc.), an inclinometer 2 installed on the monitoring floor slab, a camera 3 installed on the floor slab, and a tracking target 4 installed on the ceiling. The inclinometers on the floor slab and the key structural lateral force resisting components are rigidly fixed at the corners near the camera 3, and the tracking target 4 always falls within the frame of the camera 3. The tracking target (4) has high contrast and clear boundary lines in both the horizontal and vertical directions in the image. The tracking target (4) is located within the frame of the camera (3) and the number of tracking target (4) is not less than 1.

[0046] Before monitoring, a camera is used to photograph the calibration board placed in the target area of ​​the tracking target to obtain an image of the calibration board. The image is then identified to obtain the conversion relationship between the pixel coordinates of the calibration board in the image and the actual coordinates. The conversion relationship is used to eliminate the influence of lens distortion and correct the image.

[0047] S2. The camera 3 of the inter-story deformation monitoring device acquires the inter-story displacement visual image frame sequence of the building floors to be monitored in real time, performs preprocessing, and selects the target area including the tracking target point and the target time period covering the entire dynamic load process of the structure.

[0048] In this embodiment, preprocessing includes video stabilization, image denoising, and enhancement.

[0049] S3. Use feature point detection algorithms such as Harris, Shi-Tomasi, and SURF to detect the target region (ROI) in the initial frame of the image, and obtain the feature point coordinates (x, y, y) of the initial frame. o y o ).

[0050] S4. Select multiple feature points and perform feature tracking using the Kanade-Lucas-Tomasi method. Calculate the motion trajectory of the feature points between adjacent frames. Repeat this process until the displacement data of all feature points in the entire target image frame sequence is obtained. Filter the obtained displacement data of multiple feature points. Use Random Sample Consensus (RANSAC) to filter out non-target feature points and misidentified feature point data. Fuse the three types of displacement data after filtering to obtain the visual recognition structure displacement curve of multi-feature point fusion.

[0051] Specifically, using optical flow for feature tracking, assuming constant brightness, continuous time, and consistent motion of points within the neighborhood, then:

[0052] I(x,y,z)=I(x+dx,y+dy,z+dz)=I(x,y,z)+I x dx+I y dy+I z dz

[0053] In the formula, I(x, y, z) represents the gray value of the image coordinates (x, y, z) at time t. a This represents the partial derivative of I with respect to the subscript a;

[0054] Then we have:

[0055] I x dx+I y dy+I t dt = 0

[0056] Let dx / dt = u and dy / dt = v, where (u, v) is the optical flow. Then, the basic equation of the optical flow method can be obtained:

[0057] I x u+I y v = -I t

[0058] Meanwhile, considering that points in the neighborhood of the optical flow tracking point have different weights, the displacement of the feature point is calculated by the least squares method, and the position of the feature point in the next frame image is determined.

[0059] The obtained displacement data of multiple feature points are filtered using a random consistent sampling method to filter feature points that are not located on the target object, such as feature points on a static background or other moving objects. At the same time, the correlation between the remaining data is calculated, and the data with the highest sum of correlation coefficients with the remaining displacement time histories is used as the benchmark. Feature points whose correlation coefficients with the displacement time histories data are lower than the benchmark are identified as non-target feature points and are filtered out.

[0060] The final result obtained is a multi-feature point fusion result, which has higher accuracy compared to the single-feature point tracking result.

[0061] S5. Acquire monitoring data of the inclinometer during the movement of the building structure, select floor slab and column rotation data (including floor slab rotation angle α and column rotation angle θ) corresponding to the monitoring layer of visual recognition, filter out the external noise influence received during the data acquisition process of the instrument through FIR filter, and resample to the same frequency as the inter-story displacement visual image to obtain the slab and column rotation curve.

[0062] S6. The plate-column rotation curve and the visual identification structural displacement curve are time-aligned by the peak correlation of the curves and redundant parts are eliminated. The nominal inter-story displacement, harmless inter-story displacement and harmful inter-story displacement of the monitoring layer are calculated by using the aligned visual identification structural displacement curve and plate-column rotation curve.

[0063] Specifically, by calculating the correlation estimate between two random sequences or the deterministic correlation between two deterministic signals, the time difference between the displacement curve and the turning curve is obtained, and adjustments are made based on the time difference to achieve time alignment.

[0064] like Figure 3 As shown, the formulas for calculating the nominal inter-story displacement, harmless inter-story displacement, and harmful inter-story displacement of the monitoring layer using the aligned visual recognition structural displacement curve and slab-column rotation curve are as follows:

[0065] Δu=Δ+htanα

[0066] Δv=Δu-htanθ

[0067] Where h is the floor height, Δu is the nominal inter-story displacement, htanθ is the harmless inter-story displacement, Δv is the harmful inter-story displacement, Δ is the visually recognized inter-story structural displacement, α is the floor slab rotation angle, and θ is the column rotation angle.

[0068] This embodiment uses feature point detection algorithms such as Harris, Shi-Tomasi, and SURF, optical flow feature tracing algorithm (Kanade-Lucas-Tomasi), Random Sample Consensus (RANSAC), and FIR filter. However, the method of this invention is not limited to using the above algorithms. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort should fall within the scope of protection of this invention.

[0069] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. A method for monitoring inter-story displacement of building structures based on computer vision recognition, characterized in that, Includes the following steps: S1. Install an inter-story deformation monitoring device on the floor where the inter-story deformation is to be monitored, and use a calibration plate for image correction. The inter-story deformation monitoring device includes an inclinometer installed on the monitoring floor, an inclinometer installed on the key structural lateral force resisting components, a camera installed on the floor, and a tracking target point installed on the ceiling. S2. Use the inter-story deformation monitoring device to obtain the inter-story displacement visual image frame sequence of the building floors to be monitored in real time, perform preprocessing, and select the target area including the tracking target point and the target time period covering the entire dynamic load process of the structure. S3. Use a feature point detection algorithm to detect the target region in the initial frame of the image and obtain the coordinates of the feature points in the initial frame; S4. Select multiple feature points and perform feature tracking. Calculate the motion trajectory of the feature points between adjacent frames. Repeat this process until the displacement data of all feature points in the entire target image frame sequence is obtained. Filter the obtained displacement data of multiple feature points to filter out non-target feature points and feature points that are misidentified. Fuse the three types of displacement data after filtering to obtain the displacement curve of the visual recognition structure fused by multiple feature points. S5. Acquire the monitoring data of the inclinometer during the movement of the building structure, select the floor slab and column rotation angle data of the monitoring layer corresponding to the visual recognition, filter out noise and resample to the same frequency as the inter-story displacement visual image to obtain the slab and column rotation angle curve. S6. The plate-column rotation curve and the visual recognition structural displacement curve are time-aligned by the peak correlation of the curves and redundant parts are eliminated. The nominal inter-story displacement, harmless inter-story displacement and harmful inter-story displacement of the monitoring layer are calculated by using the aligned visual recognition structural displacement curve and plate-column rotation curve. The formulas for calculating the nominal inter-story displacement, harmless inter-story displacement, and harmful inter-story displacement of the monitoring layer using the aligned visual recognition structural displacement curve and the plate-column rotation curve are as follows: in, h For floor height, This is the nominal inter-story displacement. This is a harmless inter-layer displacement. This is harmful inter-story displacement. This refers to the inter-story structural displacement as perceived visually. For floor corners, For the corner of the column.

2. The method for monitoring inter-story displacement of building structures based on computer vision recognition according to claim 1, characterized in that, The inclinometers on the floor slab and key structural components resisting lateral forces are rigidly fixed in the corners near the camera, and the tracking target point always falls within the camera's field of view.

3. The method for monitoring inter-story displacement of building structures based on computer vision recognition according to claim 1, characterized in that, The image correction using the calibration board specifically involves: before monitoring, using a camera to photograph the calibration board placed in the target area of ​​the tracking target to obtain an image of the calibration board, and identifying the conversion relationship between the pixel coordinates of the calibration board in the image and the actual coordinates. The conversion relationship is then used to eliminate the influence of lens distortion and correct the image.

4. The method for monitoring inter-story displacement of building structures based on computer vision recognition according to claim 1, characterized in that, The preprocessing in step S2 includes video stabilization, image denoising, and enhancement.

5. The method for monitoring inter-story displacement of building structures based on computer vision recognition according to claim 1, characterized in that, The feature points include Harris, Shi-Tomasi, and SURF.

6. The method for monitoring inter-story displacement of building structures based on computer vision recognition according to claim 1, characterized in that, In step S4, optical flow is used for feature tracking. Assuming constant brightness, continuous time, and consistent motion of points within the neighborhood, then: In the formula, Represents image coordinates In t grayscale value at time, express subscript a The partial derivatives; Then we have: set up , , Since it is optical flow, the basic equation of the optical flow method can be obtained: Meanwhile, considering that points in the neighborhood of the optical flow tracking point have different weights, the displacement of the feature point is calculated by the least squares method, and the position of the feature point in the next frame image is determined.

7. The method for monitoring inter-story displacement of building structures based on computer vision recognition according to claim 1, characterized in that, The process of filtering the displacement data of multiple feature points adopts a random consistent sampling method to filter feature points that are not located on the target object. At the same time, the correlation between the remaining data is calculated, and the data with the highest sum of correlation coefficients with the remaining displacement time histories is used as the benchmark. Feature points whose correlation coefficient between the displacement time histories data and the benchmark is lower than a preset threshold are identified as non-target feature points and are filtered out.

8. The method for monitoring inter-story displacement of building structures based on computer vision recognition according to claim 1, characterized in that, The filtering in step S5 uses an FIR filter.

9. A method for monitoring inter-story displacement of building structures based on computer vision recognition according to claim 1, characterized in that, The time alignment of the plate column rotation curve and the visual recognition structure displacement curve through the peak correlation of the curves is specifically as follows: by calculating the correlation estimate between two random sequences or the deterministic correlation between two deterministic signals, the time difference between the displacement curve and the rotation curve is obtained, and the adjustment is made according to the time difference to achieve time alignment.

Images