Method for automatically monitoring earthquake response of structure by using monitoring video

Abstract

The invention discloses a method for automatically monitoring structural seismic response by using a monitoring video, and relates to the technical field of structural health monitoring. The invention aims to solve the problems existing when a monitoring camera is used for earthquake response monitoring. The method comprises the following steps: respectively identifying natural feature points in each frame of image of a pre-earthquake video and an earthquake time video, and describing the natural feature points by adopting a BRIEF descriptor; matching the natural feature points in the pre-earthquake video and the post-earthquake video, and extracting two-dimensional pixel coordinates of the natural feature points during the earthquake; and based on the two-dimensional pixel coordinates of the natural feature points during the earthquake, the displacement of the monitoring equipment during the earthquake relative to the lower-layer ground is calculated by adopting a PnP algorithm, and the monitoring equipment is fixedly connected with the upper-layer floor, so that the structural earthquake response is represented.

Description

Technical Field

[0001] The invention belongs to the technical field of structural health monitoring. Background Art

[0002] In post-earthquake assessment, the determination of the state of structural damage is the core link, which requires accurate monitoring of the structure's response under external excitation. In the past, most monitoring methods relied on wired or wireless sensors to capture the response data of the structure. When evaluating structural integrity, the displacement of the structure and its inter-layer displacement ratio are particularly important. Traditional measurement methods include the use of linear variable differential transformers, potentiometers in series or dial indications, but these technologies often require preset fixed installation points, or rely on secondary integration of acceleration signals to obtain displacement, and the results of acceleration measurements often produce errors due to interference from low-frequency noise.

[0003] In order to break through the limitations of traditional monitoring technology, vision-based monitoring technology has emerged in recent years. This technology uses monitoring equipment as a sensing tool for displacement and strain measurement. This technology deploys cameras outside the structure to track preset markers or natural features on the structure to achieve accurate measurement of displacement and deformation. However, this type of professional industrial monitoring equipment is expensive and mostly independent of vibration test equipment, so its application in earthquake monitoring is limited. It is worth noting that the surveillance cameras commonly used in modern buildings originally served the purpose of security monitoring. They have the potential to be sensors without the need to add other auxiliary sensing equipment. Using these existing resources, expensive earthquake monitoring sensors can be replaced to a certain extent, thereby reducing dependence on traditional sensors and effectively saving manpower and material costs.

[0004] Although the monitoring equipment commonly installed in buildings can record videos containing a large amount of structural seismic response data when an earthquake occurs, this valuable information is often not effectively extracted. In order to extract this data, it is necessary to use surveillance cameras for monitoring, which requires relying on computer vision technology, image processing technology and graphics.

[0005] In the field of computer vision, extracting the feature points of targets in images is a crucial task, which is essential for applications such as object recognition, image matching, visual tracking and 3D reconstruction. Common feature point detection methods include contour feature-based algorithms, corner detection-based algorithms, SIFT scale-invariant feature transformation algorithms, etc. Although the contour feature algorithm can improve the influence of lighting and solve the rotation problem, it is sensitive to noise and requires precise contour extraction; the corner detection algorithm has no restrictions on the rotation angle and has high registration accuracy, but the effect is not ideal for images with less edge information; the SIFT algorithm effectively copes with image rotation, scaling and noise, but there is a mismatch problem.

[0006] In the field of image processing, point feature descriptors are crucial for image matching. BRIEF, as an efficient binary descriptor, generates binary strings by comparing pixel pairs around key points, which is fast and space-saving. In order to use these descriptors for matching, the BFMatcher algorithm uses a brute force search strategy to compare descriptor distances one by one to find the best match, but it may be slightly less efficient when processing large data.

[0007] In the field of computer vision and graphics, converting 2D pixel coordinates to 3D world space coordinates is a crucial task. Commonly used methods mainly include scale-based methods, which are a simple and intuitive way that relies on the proportional relationship between the size of objects in the image and the actual size of objects. However, the accuracy of this method is restricted by many factors, such as monitoring equipment distortion, target posture changes, etc., so it may have certain limitations in practical applications. Summary of the invention

[0008] The present invention aims to solve the problems existing in monitoring earthquake response using a surveillance camera, and now provides a method for automatically monitoring the earthquake response of a structure using surveillance video.

[0009] A method for automatically monitoring structural seismic response using surveillance video, comprising:

[0010] The natural feature points are identified in each frame of the video before and during the earthquake, and the BRIEF descriptor is used to describe each natural feature point.

[0011] Match the natural feature points in the videos before and during the earthquake, and then extract the two-dimensional pixel coordinates of the natural feature points during the earthquake;

[0012] Based on the two-dimensional pixel coordinates and three-dimensional spatial coordinates of the natural feature points during the earthquake, the PnP algorithm is used to calculate the relative position relationship between the monitoring equipment and the lower floor during the earthquake, thereby characterizing the structural seismic response.

[0013] Furthermore, the above-mentioned identifying natural feature points in each frame of the video before and during the earthquake includes:

[0014] Harris corner detection algorithm is used to identify natural feature points in each frame of the video before and during the earthquake.

[0015] Furthermore, the above Harris corner detection algorithm includes:

[0016] The sliding window variation function is used to find the pixel point with the largest overall pixel value variation in the sliding window as the natural feature point;

[0017] The sliding window variation function E(u,v) is expressed as follows:

[0018]

[0019] Among them, u and v represent the horizontal and vertical offsets of the window respectively, (x, y) represents the coordinates of the pixel points in the window, w(x, y) represents the window function, and I() represents the pixel intensity function.

[0020] Furthermore, the BRIEF descriptor is used to describe each natural feature point, including:

[0021] Multiple ordered point pairs are randomly selected in a preset area centered on the natural feature point. When the gray value of the first point in the ordered point pair is less than the gray value of the second point, the code of the point pair is recorded as 1, otherwise it is recorded as 0;

[0022] The codes of all ordered point pairs in the preset area are used to form a descriptor of the central natural feature point.

[0023] Furthermore, the above-mentioned matching of natural feature points in the videos before and during the earthquake includes:

[0024] The BF Matcher algorithm is used to match the natural feature points in the videos before and during the earthquake.

[0025] Furthermore, the above BF Matcher algorithm includes:

[0026] The Hamming distance between each natural feature point in the pre-earthquake video and each natural feature point in the earthquake video is calculated respectively, and the two natural feature points with the smallest Hamming distance are matched with each other.

[0027] Furthermore, the above-mentioned method of using the PnP algorithm to calculate the relative position relationship between the monitoring equipment and the lower floor during the earthquake based on the two-dimensional pixel coordinates and the three-dimensional spatial coordinates of the natural feature points during the earthquake includes:

[0028] The formula of the PnP algorithm includes:

[0029] m=PM,

[0030] Among them, P is the posture transformation matrix between the monitoring device coordinate system and the world coordinate system, and P = K[R|t], K and [R|t] represent the intrinsic parameter matrix and extrinsic parameter matrix of the monitoring device, respectively, R and t are the rotation matrix and translation vector of the monitoring device, respectively, and m and M are the two-dimensional pixel coordinates and three-dimensional spatial coordinates of the natural feature points during the earthquake, respectively.

[0031] Furthermore, before identifying the natural feature points, the monitoring equipment is calibrated.

[0032] Furthermore, the above-mentioned calibration operation on the monitoring device includes:

[0033] The monitoring equipment is calibrated using the checkerboard calibration plate and Zhang Zhengyou calibration method.

[0034] Furthermore, after the monitoring device is calibrated, a distortion correction process is performed on the imaging image of the monitoring device.

[0035] The method for automatically monitoring structural seismic response using surveillance video according to the present invention has the following beneficial effects:

[0036] 1. Strategy for converting civil monitoring equipment into earthquake sensing equipment: First, the monitoring equipment is accurately calibrated using a checkerboard and distortion correction is performed to ensure image accuracy. Next, feature point detection and matching technology is applied to extract the two-dimensional pixel coordinates of natural feature points from the image. Finally, combined with the PnP algorithm, the relative displacement problem between layers during an earthquake is transformed into a problem of solving the posture of the monitoring equipment, thereby achieving the acquisition of structural seismic response information.

[0037] 2. Automated matching method based on ROI screening and BRIEF descriptor: Pre-select the region of interest before an earthquake occurs to reduce redundant calculations. Use BRIEF binary descriptors to achieve low memory usage, accelerated matching, and near real-time computing efficiency. This method can automatically perform point feature matching and interlayer response extraction after an earthquake event occurs.

[0038] 3. High-precision displacement extraction and verification: This method can accurately extract displacement information at the sub-pixel level, ensuring accurate capture even if the target only shows slight movement. Therefore, it can provide high-precision data on inter-layer relative displacement under various peak ground acceleration (PGA) amplitudes. After verification in multiple scenarios, the accuracy of this method has always remained above 90%, demonstrating its high reliability and practicality. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 A flowchart of automatically monitoring the seismic response of a structure using surveillance video;

[0040] Figure 2 Provide modeling layout and renderings for simulation tests;

[0041] Figure 3 To simulate the test of natural feature point matching detection map;

[0042] Figure 4 This is a diagram of the vibration table test device;

[0043] Figure 5 Matching detection map of natural feature points for vibration table test;

[0044] Figure 6 It is a visual aid for the Perspective-n-Point algorithm;

[0045] Figure 7 This is the calculation result diagram of relative displacement between layers in the simulation test;

[0046] Figure 8 This is the result diagram of relative pixel calculation between layers in the simulation test;

[0047] Fig. 9 This is the calculation result diagram of relative displacement between layers in shaking table test;

[0048] Fig.10 This is the calculation result of relative pixels between layers in the shaking table test. DETAILED DESCRIPTION

[0049] The following will be combined with the accompanying drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work belong to the scope of protection of the present invention. It should be noted that the embodiments of the present invention and the features in the embodiments can be combined with each other without conflict.

[0050] In traditional earthquake damage monitoring, the measurement of inter-layer relative displacement often relies on contact measuring instruments, but these professional equipment are not common in ordinary civil buildings. Buildings are usually equipped with monitoring equipment, and the videos recorded during earthquakes contain rich structural seismic response data, but there is currently no reasonable method to effectively extract them.

[0051] In the field of computer vision, Harris corner detection, as a detection method based on image gradients, determines corners by calculating changes in pixel values ​​in local areas. It is insensitive to changes in brightness and contrast and is rotationally invariant. It performs well in natural corner detection and can accurately identify significant corner features in images.

[0052] In the field of image processing, approximate nearest neighbor search algorithms such as FLANN, and matching methods based on local region similarity in SIFT and SURF, which can demonstrate superior performance when dealing with scale and rotation changes, together constitute a rich system of image matching technology.

[0053] In the field of computer vision and graphics, the PnP algorithm, or Perspective-n-Point algorithm, has shown significant advantages in processing the relationship between known 3D spatial points and corresponding 2D points. By utilizing these point pair information, the PnP algorithm can effectively estimate the posture of the monitoring device, thereby achieving accurate conversion of 2D pixel coordinates to 3D world space coordinates. This algorithm is widely used in 6D posture estimation, visual SLAM (simultaneous localization and mapping) and other fields, and is known for its high accuracy and strong applicability.

[0054] In summary, in view of the fact that many civil buildings lack structural vibration response sensors during earthquakes, and these buildings are generally equipped with surveillance cameras, the earthquake videos recorded by them have not been effectively utilized. The purpose of this embodiment is to provide a technology for automatically monitoring the seismic response of structures using surveillance videos, so that the machine can automatically and efficiently analyze the surveillance videos and quickly obtain the data of relative displacement between layers. This embodiment enables the machine to quickly and automatically extract the relative displacement between structural layers through surveillance videos: the monitoring equipment is pre-calibrated through the chessboard calibration plate to obtain the participating distortion coefficients in the monitoring equipment for distortion correction; the post-earthquake feature points are quickly and automatically detected by pre-selecting the ROI area before the earthquake; the coordinates of natural feature points in the surveillance video are automatically extracted through efficient matching; the relative displacement between layers is quickly solved based on the posture estimation algorithm of the monitoring equipment.

[0055] Reference Figure 1 Specifically describing this embodiment, a method for automatically monitoring the seismic response of a structure using a surveillance video described in this embodiment includes:

[0056] Step 1: Use the checkerboard calibration board and Zhang Zhengyou calibration method to calibrate the monitoring device to extract the internal parameter information of the monitoring device.

[0057] The internal parameter matrix K of the monitoring device is:

[0058]

[0059] Among them, f x and f y They represent the focal lengths in the x and y directions, respectively, in millimeters; u0 and v0 represent the principal point coordinates (optical center position), that is, the coordinates of the intersection of the optical axis and the image plane on the image plane, in pixels.

[0060] Step 2: Use the distortion coefficients k1, k2, k3 contained in the internal reference information of the monitoring device to perform distortion correction processing on the imaging image of the monitoring device:

[0061]

[0062] Among them, (x dist ,ydist ) and (x undist ,y undist ) represent the coordinates of the same point in the distorted image and the image after distortion correction, r represents the coordinate point (x dist ,y dist ) to the center of the distorted image, and

[0063] The distortion of monitoring equipment is mainly for wide-angle monitoring equipment. It is necessary to distinguish and process radial distortion and tangential distortion, especially the barrel distortion in radial distortion. In order to effectively correct this type of distortion, the distortion coefficient obtained by calibration can be used for precise adjustment.

[0064] Step 3: In the pre-earthquake preparation stage, key areas containing natural feature points are pre-selected from the surveillance video, the Harris corner detection algorithm is used to identify natural feature points in the key areas, and the BRIEF descriptor is used to describe each natural feature point.

[0065] The above natural feature points are mainly selected from the easily observable areas on the surface of the structure, such as wall corners, window corners, etc. These corner points appear frequently in the monitoring screen and are easy to identify. Furthermore, by adopting the Harris corner detection algorithm, accurate identification of natural feature points can be achieved.

[0066] Harris corner detection algorithm uses sliding window method to analyze corner features. The key is to evaluate the change of overall pixel value in the window before and after the window moves in a certain direction. Corners are identified by locating the position where the sliding window change function E(u,v) changes significantly. The sliding window change function E(u,v) represents the sum of squares of the change of pixel grayscale value in the window after the window moves along the [u,v] direction. The expression of sliding window change function E(u,v) is:

[0067]

[0068] Among them, u and v represent the horizontal and vertical offsets of the window respectively. x and y represent the coordinates of the pixel within the window, and w(x,y) represents the window function, which contains weight information. Common forms include uniform weight (weight is 1) and binary Gaussian distribution weight, which is used to emphasize areas where pixel values ​​change significantly. I() represents the pixel intensity function, which is similar to the pixel value.

[0069] The BRIEF descriptor is an efficient binary feature description method, the core of which is to use the grayscale value characteristics of the image that are easy to obtain, rather than complex texture or color information. With the natural feature point as the center, BRIEF randomly selects 256 pairs of ordered point pairs in the preset area (that is, the order of m points and n points is fixed), and compares the grayscale value of each pair of points: if the grayscale value of point m is less than the grayscale value of point n, the code is recorded as 1, otherwise it is recorded as 0. The preset area is a preset local window or neighborhood around the natural feature point, usually a square area. The specific size may vary depending on the implementation and application scenario. In this implementation, a window with a default setting of 48x48 pixels is selected.

[0070] The encoding calculation is expressed as:

[0071]

[0072] Among them, τ() represents the encoding function and p() represents the grayscale value of the point.

[0073] The BRIEF descriptor is used to efficiently describe natural feature points. Thanks to its binary comparison characteristics, it not only takes up less memory but also has fast calculation speed, which can facilitate subsequent feature matching and seismic response analysis, thereby significantly improving the performance of the entire monitoring system.

[0074] Step 4: During the epicenter phase, video is recorded using monitoring equipment. The monitoring equipment is firmly installed on the upper floor slab, and the natural feature points on the lower floor slab are used as observation targets, so the relative displacement between the two floors can be calculated.

[0075] Step 5: In the post-earthquake stage, the earthquake surveillance video is analyzed frame by frame, and the Harris corner detection and BRIEF feature description technology are used to quickly extract the natural feature points in each frame of the earthquake video, aiming to compare them with the pre-earthquake pre-selected features. Then, the Brute-Force Matcher (BF Matcher) algorithm is used to accurately match the post-earthquake features with the pre-earthquake pre-selected features, and the two-dimensional pixel coordinates of the matching feature points are extracted, providing key data for the subsequent conversion into three-dimensional world space positions.

[0076] The BF Matcher algorithm is a simple and practical feature point matching tool that works based on the brute force matching principle: a feature point descriptor is selected from the first image and compared with the descriptors of all feature points in the second image one by one through the Hamming distance test to find the feature point with the smallest distance (i.e. the best match). This process is repeated until a match is found for all feature points. When used in conjunction with the BRIEF descriptor, BF Matcher can achieve efficient matching of large-scale data due to its simple implementation, wide range of applications, low memory usage due to the compact binary format, and fast Hamming distance calculation. Hamming distance is a measurement method used to compare the number of different characters between two equal-length strings (also called bit strings or binary strings).

[0077] Step 6: Based on the 2D pixel coordinates of natural feature points extracted from the surveillance video after the earthquake, the intrinsic parameter matrix measured by camera calibration before the earthquake, and the spatial relative relationship between natural feature points before the earthquake (to assist in establishing the world coordinate system), the PnP algorithm is used to calculate the motion of the surveillance equipment relative to the underlying ground. The number of pixel coordinates of natural feature points in each frame of the image needs to be greater than or equal to 4.

[0078] This is achieved by converting the relative displacement information between layers during earthquakes into solving the Perspective-n-Point (PnP) problem, which mainly involves processing the transformation relationship between three coordinate systems: the world coordinate system, the monitoring device coordinate system, and the pixel coordinate system. The process uses the coordinate transformation formula to map the point M in the three-dimensional space to the projection point m on the two-dimensional image plane.

[0079] Imaging projection formula: m = PM,

[0080]

[0081] Among them, P is the pose conversion matrix between the monitoring device coordinate system and the world coordinate system, and P = K[R|t], K and [R|t] represent the monitoring device internal parameter matrix and external parameter matrix respectively, R and t represent the monitoring device rotation matrix and translation vector respectively. m is the two-dimensional pixel coordinate of the natural feature point extracted from the monitoring video after the earthquake, denoted as (u m ,v m ). M is the world coordinate corresponding to the natural feature point, denoted as (x m ,y m ,z m ). (u m ,v m ) is the result obtained during the matching process and is a variable. m ,y m ,z m) is determined before the earthquake and is a constant. K is an internal parameter and is also a constant. In specific calculations, R and t can be solved by substituting the two-dimensional pixel coordinates of at least four feature points.

[0082] The PnP algorithm uses four or more pairs of pixel coordinates and world coordinates of natural feature points to solve the rotation matrix and translation vector of the monitoring device relative to the lower floor. Since the monitoring device is fixedly connected to the upper floor, the relative motion between the upper floor and the lower floor can be solved. That is, by inputting the calibrated monitoring device internal parameter matrix and the pixel coordinates of the feature points into PnP, the relative position of the monitoring device and the target can be calculated. The visual auxiliary diagram of the Perspective-n-Point algorithm is shown in the figure below. Figure 6 shown.

[0083] The smaller the amplitude, the greater the challenge to the accuracy of the visual measurement method. Therefore, this embodiment performs pixel-level time-course analysis on small-amplitude motion.

[0084] In order to verify the accuracy of the proposed method, two experiments were planned in this implementation: one was a simulation experiment using Blender software, and the other was a real experiment using a vibration table. The following is the preparation process for the two experiments:

[0085] Simulation test step 1: Build an indoor room model in the Blender environment and connect the monitoring equipment to the upper floor. And set detailed parameters of the monitoring equipment, including key indicators such as focal length, frame rate and resolution.

[0086] Step 2 of the simulation test: Use Ansys software to calculate a set of earthquake motion data, which provides displacement information of the floor and monitoring equipment.

[0087] Simulation test step three: Import the displacement information into Blender through the Python-API interface.

[0088] Simulation test step 4: Simulate the movement of monitoring equipment and structures along the earthquake trajectory in Blender.

[0089] Simulation test step five: Export the monitoring video recorded during the movement of the monitoring equipment and structure, and add appropriate Gaussian noise to generate monitoring video for earthquake simulation in order to be closer to the actual situation.

[0090] The modeling layout and effect diagram of the simulation test are as follows: Figure 2 As shown; the natural feature point matching detection diagram of the simulation experiment is shown in Figure 3 As shown in the figure; the calculation results of the relative displacement between layers in the simulation test are shown in Figure 7 As shown; the calculation results of relative pixels between layers in the simulation test are shown in Figure 8 shown.

[0091] Shaking table test step 1: Build a small three-layer frame structure model and install it firmly on the table of the shaking table. Deploy a monitoring device on the upper layer of the model to capture the dynamics during the experiment.

[0092] Shaking table test step 2: Set the excitation data of the shaking table to ensure that the experiment can simulate the required earthquake effect.

[0093] Shaking table test step three: Start the monitoring equipment and record the response of the entire frame structure when the shaking table simulates earthquake shaking, thereby generating a monitoring video of the shaking table experiment.

[0094] The device diagram of the vibration table test is as follows: Figure 4 As shown; the natural feature point matching detection diagram of the vibration table test is shown in Figure 5 As shown in the figure; the calculation results of the relative displacement between layers in the shaking table test are shown in Fig. 9 As shown in the figure; the calculation results of relative pixels between layers in the shaking table test are shown in Fig.10 shown.

[0095] In the vibration table test, the peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) are introduced to evaluate the quality of blurred videos. PSNR mainly measures the degree of image distortion, and a high value indicates good image quality and less distortion; while SSIM further evaluates the image structural similarity. The value range of SSIM is usually between 0 and 1. The closer the value is to 1, the better the image quality and the higher the structural similarity; and the closer the value is to 0, the poorer the image quality. The two are used together to quantify the degree of blurriness of the video.

[0096] After experimental verification, the average accuracy of the displacement response time-history data obtained by this method exceeds 93.2%, and sub-pixel measurement accuracy can be achieved.

[0097] The surveillance video recording covers surveillance video recording in a simulated environment and surveillance video capture during the vibration table experiment. These two types of videos are generated by:

[0098] The simulation monitoring video was produced using Blender software. The specific process is as follows: First, build an indoor room model in the Blender environment and connect the monitoring equipment to the upper floor. Then, set detailed parameters for the monitoring equipment, including key indicators such as focal length, frame rate, and resolution. Next, use Ansys software to calculate a set of ground motion data, which provides displacement information of the floor and monitoring equipment. Through the Python-API interface, this displacement information is imported into Blender, so that the monitoring equipment and structure can be simulated according to the movement trajectory of the actual earthquake. Finally, export the monitoring video recorded during the movement of the monitoring equipment and structure, and add appropriate Gaussian noise to generate a monitoring video for earthquake simulation in order to be closer to the actual situation.

[0099] The production process of the monitoring video of the shaking table experiment is summarized as follows: First, a small three-story frame structure model is built and firmly mounted on the table of the shaking table. Then, a monitoring device is deployed on the upper layer of the model to capture the dynamics during the experiment. Next, the excitation data of the shaking table is set to ensure that the experiment can simulate the desired earthquake effect. Finally, the monitoring device is started to record the response of the entire frame structure when the shaking table simulates earthquake shaking, thereby generating a monitoring video of the shaking table experiment.

[0100] In order to achieve the stable extraction of pixel coordinates of natural feature points in surveillance videos, the technical solution adopted in this implementation is as follows: first, in the pre-earthquake preparation stage, key areas containing natural feature points are pre-selected from the surveillance video, and the Harris corner detection algorithm is used to accurately identify these feature points, and the BRIEF point descriptor is used to efficiently describe each feature point. In the epicenter stage, the video is recorded by the surveillance equipment. In the post-earthquake stage, the surveillance video is parsed frame by frame, and each frame of the image is processed using Harris corner detection and BRIEF feature description technology. The post-earthquake features are accurately matched with the pre-earthquake pre-selected features through the Brute-Force Matcher algorithm to obtain the pixel coordinates of the feature points.

[0101] Although the present invention is described herein with reference to specific embodiments, it should be understood that these embodiments are merely examples of the principles and applications of the present invention. It should therefore be understood that many modifications may be made to the exemplary embodiments and that other arrangements may be devised without departing from the spirit and scope of the present invention as defined by the appended claims. It should be understood that the various dependent claims and features described herein may be combined in a manner different from that described in the original claims. It should also be understood that the features described in conjunction with a single embodiment may be used in other described embodiments.

Claims

1. A method for automatically monitoring structural seismic response using surveillance video, characterized in that: include: The natural feature points are identified in each frame of the video before and during the earthquake, and the BRIEF descriptor is used to describe each natural feature point. Match the natural feature points in the videos before and during the earthquake, and then extract the two-dimensional pixel coordinates of the natural feature points during the earthquake; Based on the two-dimensional pixel coordinates and three-dimensional spatial coordinates of the natural feature points during the earthquake, the PnP algorithm is used to calculate the relative position relationship between the monitoring equipment and the lower floor during the earthquake, thereby characterizing the structural seismic response.

2. The method of automatically monitoring structural seismic response using surveillance video according to claim 1, characterized in that: The identifying of natural feature points in each frame of the video before and during the earthquake includes: Harris corner detection algorithm is used to identify natural feature points in each frame of the video before and during the earthquake.

3. The method of automatically monitoring structural seismic response using surveillance video according to claim 2, characterized in that: The Harris corner detection algorithm includes: The sliding window variation function is used to find the pixel point with the largest overall pixel value variation in the sliding window as the natural feature point; The sliding window variation function E(u,v) is expressed as follows: Among them, u and v represent the horizontal and vertical offsets of the window respectively, (x, y) represents the coordinates of the pixel points in the window, w(x, y) represents the window function, and I() represents the pixel intensity function.

4. The method of automatically monitoring structural seismic response using surveillance video according to claim 1, characterized in that: The BRIEF descriptor is used to describe each natural feature point, including: Multiple ordered point pairs are randomly selected in a preset area centered on the natural feature point. When the gray value of the first point in the ordered point pair is less than the gray value of the second point, the code of the point pair is recorded as 1, otherwise it is recorded as 0; The codes of all ordered point pairs in the preset area are used to form a descriptor of the central natural feature point.

5. The method of automatically monitoring structural seismic response using surveillance video according to claim 1, characterized in that: The matching of natural feature points in the videos before and during the earthquake includes: The BF Matcher algorithm is used to match the natural feature points in the videos before and during the earthquake.

6. The method of automatically monitoring structural seismic response using surveillance video according to claim 5, characterized in that: The BF Matcher algorithm includes: The Hamming distance between each natural feature point in the pre-earthquake video and each natural feature point in the earthquake video is calculated respectively, and the two natural feature points with the smallest Hamming distance are matched with each other.

7. The method of automatically monitoring structural seismic response using surveillance video according to claim 1, characterized in that: The method of calculating the relative position relationship between the monitoring device and the lower floor during an earthquake using a PnP algorithm based on the two-dimensional pixel coordinates and the three-dimensional spatial coordinates of the natural feature points during an earthquake includes: The formula of the PnP algorithm includes: m=PM, Among them, P is the posture transformation matrix between the monitoring device coordinate system and the world coordinate system, and P = K[R|t], K and [R|t] represent the intrinsic parameter matrix and extrinsic parameter matrix of the monitoring device, respectively, R and t are the rotation matrix and translation vector of the monitoring device, respectively, and m and M are the two-dimensional pixel coordinates and three-dimensional spatial coordinates of the natural feature points during the earthquake, respectively.

8. The method of automatically monitoring structural seismic response using surveillance video according to claim 1, characterized in that: Before identifying natural feature points, the monitoring equipment must be calibrated first.

9. The method of automatically monitoring structural seismic response using surveillance video according to claim 8, characterized in that: The calibration operation on the monitoring device includes: The monitoring equipment is calibrated using the checkerboard calibration plate and Zhang Zhengyou calibration method.

10. A method for automatically monitoring structural seismic response using surveillance video according to claim 8 or 9, characterized in that: After the monitoring device is calibrated, a distortion correction process is performed on the imaging image of the monitoring device.

Images