A method and device for determining the displacement dynamic characteristics of a building structure based on video

Abstract

The present invention provides a method and device for determining the dynamic characteristics of building structure displacement based on video. Vibration video of the building structure to be identified, captured by a camera, can be used to identify and calibrate key points of the building structure. Conversion coefficients for the displacement calibration formula of the building structure to be identified based on the vibration video are obtained. The displacement of the building structure to be identified is determined based on pixel coordinates and the conversion coefficients. The obtained structural displacement is optimized to eliminate jagged errors, align peak points and zero points, and generate a corresponding dynamic characteristic diagram using the optimized displacement data. This application can capture vibration video of the building structure to be identified using low-cost cameras such as emergency monitoring cameras and mobile phone cameras, obtain the displacement of the building structure, and identify its dynamic characteristics. This avoids the need to install fixed sensors in traditional methods and addresses technical issues such as the complexity and fragility of existing monitoring systems and the high repair and maintenance costs.

Description

Technical Field

[0001] The present invention relates to the technical field of building structure displacement monitoring, and in particular to a method and device for determining dynamic characteristics of building structure displacement based on video. Background Art

[0002] With the rapid development of my country's economy and technology, and the increasing level of urbanization, the load-bearing capacity of building structures is declining year by year with increasing service life. In particular, the number of large structures, such as super-high-rise buildings and bridges, both completed and under construction, is increasing, and their structural forms are becoming increasingly diverse. In recent years, large-scale vibrations of these structures have become frequent, causing significant social impacts and economic losses. Therefore, monitoring the displacement and dynamic characteristics of building structures is of great urgency and significance.

[0003] However, on the one hand, traditional methods for monitoring displacement of building structures mostly use contact-type fixed sensors for monitoring. This type of monitoring method requires fixing several sensors, such as accelerometers, inclinometers, GPS, etc., at the locations to be measured on the structure, and acquiring monitoring data through centralized or distributed acquisition. However, for building structures, this type of contact-type fixed sensing monitoring requires the deployment of a large number of sensor measurement points. Due to factors such as site installation conditions, acquisition channel limitations, and cost constraints, sensors can generally only be installed at a few key measurement points to collect vibration information. There are also problems such as low spatial resolution of the generated monitoring, complex and fragile monitoring systems, and high repair and maintenance costs. On the other hand, due to the sudden nature of vibration and the lack or failure of some existing large-scale structural health monitoring systems, it is difficult to obtain vibration displacement and modal parameter information when vibration occurs, which makes vibration tracing difficult.

[0004] Therefore, it is of great significance to use surveillance videos of large-scale structural vibrations, vibration videos taken by mobile phones and other low-resolution videos under limited conditions to identify structural vibration displacements and modal parameters during structural vibrations as a supplementary and emergency method. Summary of the Invention

[0005] In view of this, the purpose of the present invention is to provide a method and device for determining the dynamic characteristics of building structure displacement based on video, which can accurately identify the displacement of the building structure through the vibration video of the building structure obtained by the camera, generate a corresponding dynamic characteristic diagram, and at the same time optimize the method of determining the video displacement signal, effectively improve the efficiency of identifying the displacement of the building structure, and have good universality, can meet the requirements of emergency monitoring of building structures, and at the same time, can also reduce the later maintenance cost of the building structure.

[0006] In a first aspect, an embodiment of the present invention provides a method for determining the displacement of a building structure, wherein the method includes: using a camera to obtain a vibration video of a building structure to be identified; calibrating key points of the building structure to be identified based on the vibration video, determining the pixel coordinates of the key points and the physical length of the key points mapped on the image plane of the vibration video; determining the conversion coefficient of the displacement calibration formula of the building structure to be identified based on the vibration video according to the pixel coordinates and the physical length; determining the displacement of the building structure to be identified based on the pixel coordinates and the conversion coefficient; determining the pixel displacement signal corresponding to the key point from the vibration video based on a target tracking algorithm; determining the physical displacement signal corresponding to the key point based on the pixel displacement signal and the conversion coefficient; generating a dynamic characteristic diagram of the displacement of the corresponding building structure to be identified based on the physical displacement signal; calibrating at least one key point of the building structure to be identified based on the vibration video, and using a random subspace method to identify the synchronous displacement signal corresponding to each key point, and obtaining the modal parameters of the building structure to be identified based on the vibration video, obtaining a refined vibration mode diagram, and judging the location of structural damage.

[0007] In combination with the first aspect, an embodiment of the present invention provides a first possible implementation method of the first aspect, wherein the step of generating a dynamic characteristic diagram of the displacement of the corresponding building structure to be identified based on the physical displacement signal includes: generating a first displacement signal time domain diagram corresponding to the physical displacement signal based on the physical displacement signal; using a resampling method to increase the sampling rate of the first displacement signal time domain diagram and obtain a corresponding second displacement signal time domain diagram; converting the second displacement signal time domain diagram into a corresponding first displacement signal frequency domain diagram through Fourier transform; using bandpass filtering to reduce noise on the first displacement signal frequency domain diagram, retaining the dominant mode of the first displacement signal frequency domain diagram to obtain a second displacement signal frequency domain diagram; converting the second displacement signal frequency domain diagram into a corresponding third displacement signal time domain diagram through inverse Fourier transform; performing peak point alignment and zero point alignment on the third displacement signal time domain diagram to obtain a fourth displacement signal time domain diagram; and downsampling the fourth displacement signal time domain diagram to obtain a dynamic characteristic diagram with the same frequency as the first displacement signal time domain diagram.

[0008] In combination with the first aspect, an embodiment of the present invention provides a second possible implementation method of the first aspect, wherein the step of determining the displacement of the building structure to be identified based on the pixel coordinates and the conversion coefficient includes: multiplying the pixel coordinates and the conversion coefficient to obtain the motion plane coordinates of the building structure to be identified in the building coordinate system; extracting the motion plane coordinate difference within a preset time period, wherein the motion plane coordinate difference is the difference between the first motion plane coordinate corresponding to the end time of the preset time period and the second motion plane coordinate corresponding to the starting time of the preset time period; and determining the displacement of the building structure to be identified based on the motion plane coordinate difference.

[0009] In combination with the first aspect, an embodiment of the present invention provides a third possible implementation of the first aspect, wherein, after using a camera to obtain a vibration video of the building structure to be identified, the method also includes: determining a pixel displacement signal of the building structure to be identified from the vibration video.

[0010] In combination with the first aspect, an embodiment of the present invention provides a fourth possible implementation of the first aspect, wherein the step of determining, based on pixel coordinates and physical length, a conversion coefficient for a displacement calibration formula for a building structure to be identified based on a vibration video includes: determining whether the image plane of the vibration video is parallel to the actual plane of the building structure to be identified; wherein the actual plane refers to the plane where the key point is located; if so, obtaining the conversion coefficient using the following formula:

[0011]

[0012] Where s is the conversion coefficient, X A , X B are the coordinates of the key points on the actual plane, I A and I B is the pixel coordinate of the key point mapping on the image plane of the vibration video, X A -X B is the physical length.

[0013] In combination with the fourth possible implementation of the first aspect, an embodiment of the present invention provides a fifth possible implementation of the first aspect, wherein the step of determining the conversion coefficient of the calibration formula for the displacement of the building structure based on the vibration video according to the pixel coordinates and the physical length also includes: if it is determined that the image plane of the vibration video is not parallel to the actual plane of the building structure to be identified, determining the angle between the optical axis of the camera lens of the camera and the normal direction of the actual plane; judging whether the focal length of the camera lens of the camera is a preset focal length; if so, determining the distance from the camera lens of the camera to the actual plane, and determining the conversion coefficient of the calibration formula for the displacement of the building structure to be identified based on the vibration video according to the pixel coordinates, the physical length, the distance from the camera lens of the camera to the actual plane and the angle.

[0014] In combination with the fifth possible implementation of the first aspect, an embodiment of the present invention provides a sixth possible implementation of the first aspect, wherein the conversion coefficient is obtained by the following formula:

[0015]

[0016] Where s is the conversion coefficient, X A , X B are the coordinates of the key points of the building structure to be identified on the actual plane, I A and I B is the pixel coordinate of the key point mapping on the image plane of the vibration video, are the distances from the camera lens to the actual plane, θ is the angle between the optical axis of the camera lens and the normal direction of the actual plane, and f is the focal length of the camera lens.

[0017] In combination with the fifth possible implementation of the first aspect, an embodiment of the present invention provides a seventh possible implementation of the first aspect, wherein, if the focal length of the camera lens of the camera is not a preset focal length, a laser rangefinder is used to measure the actual distance from the camera lens to the key point; a protractor is used to measure the actual angle between different lasers emitted by the laser rangefinder; the physical length is calculated based on the actual distance and the actual angle; and the conversion coefficient of the calibration formula for the displacement of the building structure to be identified based on the vibration video is determined through the pixel coordinates and the physical length.

[0018] In combination with the first aspect, an embodiment of the present invention provides an eighth possible implementation of the first aspect, wherein, if the pixel coordinates of the key points and the physical length of the key points mapped on the image plane of the vibration video cannot be determined, the conversion coefficient is obtained by the following formula:

[0019]

[0020] Among them, s is the conversion coefficient, l pixel is the pixel size of the camera, that is, the physical length of a single pixel of the camera, f is the focal length of the camera, Z is the distance from the camera lens to the actual plane, and θ is the angle between the optical axis of the camera lens and the normal direction of the actual plane.

[0021] In a second aspect, an embodiment of the present invention further provides a device for determining the displacement of a building structure, wherein the device includes: an acquisition module for acquiring a vibration video of a building structure to be identified using a camera; a first determination module for calibrating the key points of the building structure to be identified based on the vibration video, and determining the pixel coordinates of the key points of the building structure to be identified in the vibration video and the physical length of the key points of the building structure to be identified mapped on the image plane of the vibration video; a second determination module for determining, based on the pixel coordinates and the physical length, a conversion coefficient of a displacement calibration formula for the building structure to be identified based on the vibration video; a third determination module for determining the displacement of the building structure to be identified based on the pixel coordinates and the conversion coefficient; a fourth determination module for determining the pixel displacement signal corresponding to the key point from the vibration video based on a target tracking algorithm; a fifth determination module for determining the physical displacement signal corresponding to the key point based on the pixel displacement signal and the conversion coefficient; and a generation module for generating a corresponding dynamic characteristic diagram of the displacement of the building structure to be identified based on the physical displacement signal.

[0022] The embodiments of the present invention bring the following beneficial effects:

[0023] Embodiments of the present application provide a method, apparatus, and electronic device for determining the displacement of a building structure, wherein a vibration video of a building structure to be identified, captured by a camera, can be used to calibrate key points of the building structure to be identified, and the pixel coordinates of the key points and the physical length of the key points mapped on the image plane of the vibration video can be determined, thereby obtaining conversion coefficients for a calibration formula for the displacement of the building structure to be identified based on the vibration video. The displacement of the building structure to be identified is determined based on the pixel coordinates and the conversion coefficients, and a corresponding dynamic characteristic diagram is generated. The method for determining a video displacement signal is also optimized. The present application can capture a vibration video of the building structure to be identified through a camera, identify the displacement of the building structure based on data extracted from the video, determine the displacement of the building structure to be identified based on the pixel coordinates and the conversion coefficients, optimize the acquired structural displacement, eliminate jagged errors, align the peak points and the zero point, and generate a corresponding dynamic characteristic diagram using the optimized displacement data. This avoids the need to install fixed sensors in traditional methods, and can achieve the technical effect of using a targetless method to detect the displacement of the building structure to be identified using a consumer-grade camera, thereby solving the technical problems of existing monitoring systems such as complexity, fragility, and high repair and maintenance costs.

[0024] Other features and advantages of the present invention will be described in the following description, and in part will become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention are realized and obtained by the structures particularly pointed out in the description and the drawings.

[0025] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without paying any creative work.

[0027] Figure 1 A flowchart of a method for determining the dynamic characteristics of building structure displacement based on video provided by an embodiment of the present invention;

[0028] Figure 2 A flowchart of another method for determining the dynamic characteristics of building structure displacement based on video provided by an embodiment of the present invention;

[0029] Figure 3A flowchart of another method for determining the dynamic characteristics of building structure displacement based on video provided by an embodiment of the present invention;

[0030] Figure 4 A schematic diagram showing that an image plane of a vibration video provided by an embodiment of the present invention is parallel to the actual plane of a building structure to be identified;

[0031] Figure 5 A schematic diagram showing that an image plane of a vibration video provided by an embodiment of the present invention is not parallel to the actual plane of a building structure to be identified;

[0032] Figure 6 A schematic structural diagram of a device for identifying displacement of a building structure provided by an embodiment of the present invention;

[0033] Figure 7 A schematic structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0034] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work shall fall within the scope of protection of the present invention.

[0035] Considering that existing methods for identifying displacement of building structures often use the method of installing contact fixed sensors for monitoring, which has high requirements for site installation conditions and subsequent maintenance costs, the embodiments of the present invention provide a method, device and electronic equipment for identifying displacement of building structures. These methods can use consumer-grade cameras, especially low-cost emergency cameras such as surveillance cameras and mobile phone cameras, to obtain vibration videos of the building structure to be identified, calibrate key points, and identify the displacement of the building structure. By obtaining the vibration video, the displacement of the building structure can be monitored without installing contact sensors. The method has good universality, can meet the requirements of emergency monitoring of building structures, and reduces the subsequent maintenance costs of the building structure and the monitoring system.

[0036] This embodiment provides a method for determining the displacement of a building structure. Figure 1 The flowchart of a method for determining the displacement dynamic characteristics of a building structure based on video is shown, and the method specifically includes the following steps:

[0037] Step S102, using a camera to obtain a vibration video of the building structure to be identified;

[0038] In practical applications, when it is necessary to monitor the displacement of a certain building structure, a consumer-grade camera can be used to sample the video of the building structure to be identified and obtain a vibration video within a preset time. There is no specific restriction on the specific length of the preset time. By framing the building structure in the video, the specific position of the building structure in the vibration video can be determined, and the key points of the framed building structure can be selected. The selected key points can be the significant feature points of the building structure that require displacement monitoring, such as any number of stress points on the pillars of a large bridge, or any number of points near the roof of a large commercial building.

[0039] Step S104: calibrating key points of the building structure to be identified based on the vibration video to determine the pixel coordinates of the key points and the physical length of the key points mapped on the image plane of the vibration video;

[0040] After selecting the key points of the building structure, the determined key points are calibrated. A two-dimensional plane coordinate based on the sampled vibration video can be established, and the specific pixel coordinate positions of the determined key points are calibrated based on the two-dimensional plane coordinates. The physical length of each key point mapped on the image plane of the vibration video is determined, and the correspondence between the above pixel coordinates and the physical length is determined by determining the pixel coordinates of the above key points and the above physical length.

[0041] Step S106, determining a conversion coefficient of a displacement calibration formula for the building structure to be identified based on the vibration video according to the pixel coordinates and the physical length;

[0042] In practical applications, when a camera is used to sample a building structure, a correspondence between the above-mentioned pixel coordinates, the pixel coordinates of the key points of the building structure, and the physical length of the key points mapped on the image plane of the vibration video will be generated according to the imaging principle of the camera. This is the conversion coefficient of the above-mentioned displacement calibration formula for the identified building structure. The conversion coefficient can be used to determine the displacement of the building structure based on the change in the pixel coordinates of the key points of the building structure.

[0043] Step S108: determining the displacement of the building structure to be identified based on the pixel coordinates and the conversion coefficient.

[0044] The above process of determining the displacement of the building structure to be identified based on pixel coordinates and conversion coefficients can be implemented by steps A1 to A3:

[0045] Step A1: multiplying the pixel coordinates and the conversion coefficient to obtain the motion plane coordinates of the building structure to be identified in the building coordinate system;

[0046] Specifically, according to the process of solving the above conversion coefficient, it can be known that based on the optical imaging principle of the camera, the pixel coordinates of the key points of the building structure to be identified can be multiplied by the obtained conversion coefficient to obtain the corresponding motion plane coordinates of the building structure to be identified in the building coordinate system, and the displacement of the building structure to be identified can be determined based on the change of the above motion plane coordinates, that is, the difference before and after.

[0047] Step A2: extracting a motion plane coordinate difference within a preset time period, wherein the motion plane coordinate difference is a difference between a first motion plane coordinate corresponding to an end point of the preset time period and a second motion plane coordinate corresponding to a start point of the preset time period;

[0048] In practical applications, since cameras have different sampling frequencies, the difference in motion plane coordinates within a preset time period can be extracted. Alternatively, each frame of the video sampled by the camera can represent a sampling moment. In this way, the distance between two adjacent frames can be regarded as a preset time period, and the difference in motion plane coordinates can be used to determine the displacement of the building structure to be identified.

[0049] Step A3: determining the displacement of the building structure to be identified based on the motion plane coordinate difference.

[0050] Specifically, once the conversion coefficient is determined, the pixel coordinates obtained from video tracking are multiplied by the corresponding conversion coefficient to obtain the coordinates of the key point on the plane where the displacement occurs. The difference in pixel coordinates within a certain time period is the displacement within that time period. Therefore, a preset time period can be used to identify the key point, and the final building structure displacement can be determined based on the pixel coordinate difference within the preset time period and the conversion coefficient. At the same time, due to the sampling frequency constraint, each frame of the image represents a sampling moment. Here, the displacement at each sampling moment can be calculated based on the pixel coordinates obtained by the target tracking method, and a displacement time history diagram can be drawn to obtain a more intuitive building structure displacement result.

[0051] Step S110, determining pixel displacement signals corresponding to key points in the vibration video based on a target tracking algorithm;

[0052] After determining the displacement of the building structure to be identified using the above method, the pixel displacement signal of the building structure vibration can be obtained, and the physical displacement signal of the corresponding key point can be determined according to the above conversion coefficient.

[0053] Step S112, determining a physical displacement signal corresponding to the key point based on the pixel displacement signal and the conversion coefficient;

[0054] Step S114, generating a dynamic characteristic diagram of the corresponding building structure displacement based on the physical displacement signal;

[0055] In step S116, at least one key point of the building structure to be identified is calibrated based on the vibration video, and the synchronous displacement signal corresponding to each key point is identified using the random subspace method, and the modal parameters of the building structure based on the vibration video are obtained to obtain a refined vibration mode diagram and determine the location of structural damage.

[0056] At the same time, in an embodiment of the present invention, multiple key points of the building structure to be identified are calibrated based on the above-mentioned vibration video, and the random subspace method is used to identify the synchronous displacement signal corresponding to each key point, and the modal parameters of the building structure based on the vibration video are obtained.

[0057] In practical applications, in order to obtain more refined vibration mode information of the building structure, after obtaining the vibration displacement of the building structure through the above method, the stochastic subspace identification method (SSI method) can be used to identify the modal information of the building structure, and the target tracking method (Efficient Convolution Operators for Tracking, ECO method) can be used to obtain the pixel displacement signal of the vibration of a large bridge structure, thereby obtaining a more intuitive displacement image of the building structure.

[0058] The method in the embodiment of the present application can be used to simultaneously select and identify multiple target points, and the modal analysis can be performed on the displacement time domain information using the SSI method to obtain the dominant order mode. The refined normalized vibration shape values are calculated respectively, and the refined vibration shape can assist in the analysis of large-scale structural damage. The method for determining the displacement of a building structure provided in the embodiment of the present invention can use the vibration video of the building structure to be identified obtained by the camera to identify the displacement of the building structure, and determine the displacement of the building structure to be identified based on the pixel coordinates and the conversion coefficient, optimize the obtained structural displacement, eliminate the jagged error, align the peak point and the zero point, and use the optimized displacement data to generate the corresponding dynamic characteristic diagram. The displacement of the building structure can be identified without installing a contact fixed sensor, and the modal information of the building structure can be identified to obtain more refined vibration shape information and a more intuitive building structure displacement image, which solves the technical problems of the existing monitoring system being complex and fragile, and the high repair and maintenance cost.

[0059] This embodiment provides another method for identifying building structure displacement, which is implemented on the basis of the above embodiment; this embodiment focuses on describing the specific implementation method of generating the corresponding dynamic characteristic diagram of building structure displacement from the physical displacement signal. Figure 2 The flowchart of another method for determining the dynamic characteristics of building structure displacement based on video is shown. The method for determining the dynamic characteristics of building structure displacement based on video in this embodiment includes the following steps:

[0060] Step S202, using a camera to obtain a vibration video of the building structure to be identified;

[0061] Step S204: calibrate key points of the building structure to be identified based on the vibration video, and determine the pixel coordinates of the key points and the physical length of the key points mapped on the image plane of the vibration video;

[0062] Step S206, determining a conversion coefficient of a displacement calibration formula for the building structure to be identified based on the vibration video according to the pixel coordinates and the physical length;

[0063] Step S208: determining the displacement of the building structure to be identified based on the pixel coordinates and the conversion coefficient.

[0064] Step S210, determining pixel displacement signals corresponding to key points in the vibration video based on a target tracking algorithm;

[0065] Step S112, determining a physical displacement signal corresponding to the key point based on the pixel displacement signal and the conversion coefficient;

[0066] Step S214, generating a dynamic characteristic diagram of the corresponding building structure displacement based on the physical displacement signal;

[0067] Step S216, generating a first displacement signal time domain diagram corresponding to the physical displacement signal based on the physical displacement signal;

[0068] The above method can be used to generate the time domain diagram of the displacement signal corresponding to the displacement of the key point, that is, the first displacement signal time domain diagram. However, the displacement recognition result will produce obvious sawtooth errors, and the signals of different identification measurement points will be out of sync, and the peak points and zero points will not be aligned. Therefore, the following steps can be used to optimize the time domain diagram of the displacement signal.

[0069] Step S218, using a resampling method to increase the sampling rate of the first displacement signal time domain graph and obtain a corresponding second displacement signal time domain graph;

[0070] After using the target tracking algorithm to identify displacement in a vibration video of a building structure, a pixel displacement time-domain signal for the building structure is obtained. However, the conversion coefficients for different identification points vary, so target calibration is performed to obtain the conversion coefficients for each identification point. The pixel displacement time-domain signal is then converted using the conversion coefficients, converting it from pixel displacement to actual displacement, resulting in the displacement time-domain signal for the key point. To facilitate subsequent peak point alignment, the displacement time-domain signal can be upsampled from its original lower frequency, such as 30 Hz, to a higher frequency, such as 500 Hz. The specific frequency value is not restricted here. This results in a time-domain graph of the displacement signal with a higher sampling frequency, i.e., the second displacement signal time-domain graph.

[0071] Step S220, converting the second displacement signal time domain graph into the corresponding first displacement signal frequency domain graph through Fourier transform;

[0072] In practical applications, in order to further analyze the frequency structure of the displacement signal and perform noise reduction processing, it is necessary to convert the obtained time domain diagram of the displacement signal with a higher sampling frequency into the corresponding frequency domain diagram of the displacement signal, i.e., the first frequency domain diagram of the displacement signal, through Fourier transform.

[0073] Step S222: using bandpass filtering to reduce noise on the first displacement signal frequency domain graph, retaining the dominant mode of the first displacement signal frequency domain graph to obtain a second displacement signal frequency domain graph;

[0074] Since the lens cannot remain completely still during video shooting, inevitable shaking will occur. At the same time, due to the limited video resolution and frame rate, factors such as the robustness of the displacement recognition algorithm will interfere with the displacement time domain signal and cause errors. Therefore, the displacement time domain signal is processed by filtering to retain the true displacement signal and filter out other irrelevant interferences. There are several filtering methods for signal filtering, such as low-pass, high-pass, and band-pass. Taking into account the type of interference and the frequency of the interference, band-pass filtering can be used as the filtering method here, but there is no restriction on the specific filtering method. Band-pass filtering only retains signals within a specific frequency range, while effectively suppressing signals below the set frequency range and above the set frequency range. The signal processing method proposed in the embodiment of the present application is to retain the dominant mode and filter out the remaining modes other than the dominant mode, so as to ensure that the frequency domain of the displacement recognition signal is not interfered with by other irrelevant frequencies, so that the signal frequency domain diagram after noise reduction, i.e., the second displacement signal frequency domain diagram, can be obtained.

[0075] Step S224, converting the second displacement signal frequency domain graph into a corresponding third displacement signal time domain graph through inverse Fourier transform;

[0076] In practical applications, in order to convert the noise-reduced displacement signal frequency domain graph into a displacement signal time domain graph, an inverse Fourier transform is required to obtain the noise-reduced displacement signal time domain graph, ie, the third displacement signal time domain graph.

[0077] Step S226, performing peak point alignment and zero point alignment on the third displacement signal time domain graph to obtain a fourth displacement signal time domain graph;

[0078] In practical applications, the low resolution and frame rate of video captured by cameras can lead to asynchrony between signals at different key points, misalignment of peaks, and misalignment of zero points. This asynchrony can affect the modal analysis of the displacement signal, introducing phase errors that interfere with the determination of vibration mode values.

[0079] Therefore, the signal processing method provided in the embodiment of the present application is to delete the points in a small range near the peak point, delete the remaining peak points except the maximum peak point, and finally fill in the gaps by interpolation; and pick up the smallest point in multiple groups of data, delete the points in a small range near the minimum point, delete the remaining points except the minimum point, and finally fill in the gaps by interpolation. Through the above-mentioned peak point alignment and zero point alignment processing methods, the peak point and the zero point are aligned, ensuring the synchronization of the displacement time domain signal, and maximally retaining the original signal, and obtaining a displacement signal time domain diagram with both the peak point and the zero point aligned, i.e., the fourth displacement signal time domain diagram.

[0080] Step S228 : down-sampling the fourth displacement signal time domain graph to obtain a dynamic characteristic graph having the same frequency as the first displacement signal time domain graph.

[0081] Specifically, since the above steps upsample the displacement signal time domain graph from the original lower frequency, such as 30Hz, to a relatively high frequency, such as 500Hz, the displacement signal needs to be downsampled for frequency restoration, that is, a dynamic characteristic graph with the same frequency as the above-mentioned first displacement signal time domain graph is obtained. After the obtained dynamic characteristic graph is optimized, the jagged error disappears, and the peak point and the zero point are synchronized, which solves the problems caused by low video resolution and frame rate.

[0082] In step S230, at least one key point of the building structure to be identified is calibrated based on the vibration video, and the synchronous displacement signal corresponding to each key point is identified using the random subspace method, and the modal parameters of the building structure based on the vibration video are obtained to obtain a refined vibration mode diagram and determine the location of structural damage.

[0083] This embodiment provides another method for identifying building structure displacement, which is implemented on the basis of the above embodiment; this embodiment focuses on the specific implementation method of determining the conversion coefficient of the building structure displacement calibration formula based on vibration video. Figure 3 The flowchart of another method for determining the dynamic characteristics of building structure displacement based on video is shown. The method for identifying building structure displacement in this embodiment includes the following steps:

[0084] Step S302, using a camera to obtain a vibration video of the building structure to be identified;

[0085] Step S304: calibrate key points of the building structure to be identified based on the vibration video, and determine the pixel coordinates of the key points and the physical length of the key points mapped on the image plane of the vibration video;

[0086] Step S306, determining whether the image plane of the vibration video is parallel to the actual plane of the building structure to be identified; wherein the actual plane refers to the plane where the key point is located;

[0087] In actual applications, due to the influence of objective factors such as the site environment, the camera settings will also vary depending on the actual environmental conditions. Generally speaking, they can be divided into two situations: when the image plane of the vibration video is parallel or non-parallel to the actual plane of the building structure to be identified. Therefore, the process of solving the conversion coefficient is also different according to these two different situations. Therefore, before obtaining the conversion coefficient, it is necessary to first determine which of the above situations the current camera settings meet.

[0088] Step S308: If yes, the conversion coefficient is obtained by formula (1):

[0089]

[0090] Where s is the conversion coefficient, X A , X B are the coordinates of the key points on the actual plane, I A and I B is the pixel coordinate of the key point mapping on the image plane of the vibration video, X A -X B is the physical length.

[0091] Specifically, if Figure 3 The schematic diagram shows that the image plane of a vibration video is parallel to the actual plane of the building structure to be identified. It can be seen that according to the optical imaging principle of the camera, there is a certain correspondence between the pixel coordinates of the key points and the physical length of the key points mapped on the image plane of the vibration video, that is, the conversion coefficient, where the physical length can be represented by the actual length of the two key points on the image of the building structure in the vibration video, and the length of the pixel coordinate can be represented by the pixel size on the image (the physical length of a single pixel on the image plane), and thus can also be represented by the distance Z from the lens to the actual plane, the focal length f of the camera lens, and the pixel size l. pixel Expressed as follows, formula (1) can also be expressed as the following formula (2):

[0092]

[0093] Step S310: If it is determined that the image plane of the vibration video is not parallel to the actual plane of the building structure to be identified, determine the angle between the optical axis of the camera lens and the normal direction of the actual plane;

[0094] Specifically, if Figure 5 The schematic diagram shows that the image plane of the vibration video is not parallel to the actual plane of the building structure to be identified. It can be seen that the conversion coefficient cannot be determined by the method of step S208, so step S212 is executed.

[0095] Step S312: Determine whether the focal length of the camera lens is a preset focal length; if so, determine the distance between the camera lens and the actual plane, and determine the conversion coefficient of the vibration video-based building structure displacement calibration formula based on the pixel coordinates, physical length, and the distance and angle between the camera lens and the actual plane.

[0096] Among them, the conversion coefficient can be obtained by formula (3):

[0097]

[0098] Where s is the conversion coefficient, X A , X B are the coordinates of the key points of the building structure to be identified on the actual plane, I A and I B is the pixel coordinate of the key point mapping on the image plane of the vibration video, are the distances from the camera lens to the actual plane, θ is the angle between the optical axis of the camera lens and the normal direction of the actual plane, and f is the focal length of the camera lens.

[0099] If the pixel coordinates of the key points and the physical length of the key points mapped on the image plane of the vibration video cannot be determined, the conversion coefficient can be obtained by formula (4):

[0100]

[0101] Where s is the conversion coefficient, l pixel is the pixel size of the camera, that is, the physical length of a single pixel of the camera, f is the focal length of the camera, Z is the distance from the camera lens to the actual plane, and θ is the angle between the optical axis of the camera lens and the normal direction of the actual plane.

[0102] In practical applications, if the object is accessible, it is possible to directly measure the pixel coordinates of key points and the physical length of the key points mapped on the image plane of the vibration video. If direct measurement is not possible, the conversion coefficient can be obtained by looking up the parameters of the camera being used, obtaining the focal length and pixel size of the camera lens, and measuring the distance from the camera lens to the actual plane and the angle between the camera lens optical axis and the normal direction of the actual plane using the above formula.

[0103] Step S314: if the focal length of the camera lens is not the preset focal length, use a laser rangefinder to measure the actual distance from the camera lens to the key point;

[0104] In practical applications, due to limitations in the field of view and the distance between the measurement point and the camera, a lens with a fixed focal length may not meet the above conditions for solving the conversion coefficient. In this case, there is another way to obtain the conversion coefficient.

[0105] Step S316, using a protractor to measure the actual angle between the lasers emitted by the laser rangefinder;

[0106] In practical applications, a laser rangefinder can be used to measure the distance between two key points. After measuring the actual distances from the camera lens to the two key points, a protractor can be used to measure the angle between the two laser beams.

[0107] Step S318, calculating the physical length according to the actual distance and the actual angle;

[0108] Specifically, the actual distance from the camera lens to the two key points and the physical length will form a triangle, wherein the angle between the actual distance from the camera lens to the two key points is the angle measured by the protractor, and the physical length can be calculated according to the trigonometric theorem.

[0109] Step S320: determining the conversion coefficient of the building structure displacement calibration formula based on the vibration video through the pixel coordinates and the physical length.

[0110] Step S322, determining pixel displacement signals corresponding to key points in the vibration video based on a target tracking algorithm;

[0111] Step S324, determining a physical displacement signal corresponding to the key point based on the pixel displacement signal and the conversion coefficient;

[0112] Step S326: Generate a dynamic characteristic diagram of the corresponding building structure displacement based on the physical displacement signal.

[0113] In step S328, at least one key point of the building structure to be identified is calibrated based on the vibration video, and the synchronous displacement signal corresponding to each key point is identified using the random subspace method, and the modal parameters of the building structure based on the vibration video are obtained to obtain a refined vibration mode diagram and determine the location of structural damage.

[0114] Corresponding to the above method embodiment, the embodiment of the present invention provides a device for identifying displacement of a building structure. Figure 6 A schematic diagram of a device for identifying displacement of a building structure is shown in FIG. Figure 4 As shown, the device for identifying displacement of a building structure includes:

[0115] An acquisition module 501 is configured to acquire a vibration video of a building structure to be identified using a camera;

[0116] The first determination module 502 is configured to calibrate the key points of the building structure based on the vibration video, determine the pixel coordinates of the key points of the building structure to be identified in the vibration video, and determine the physical length of the key points of the building structure mapped on the image plane of the vibration video;

[0117] The second determination module 503 is used to determine the conversion coefficient of the building structure displacement calibration formula based on the vibration video according to the pixel coordinates and the physical length;

[0118] A third determination module 504 is configured to determine the displacement of the building structure to be identified based on the pixel coordinates and the conversion coefficient;

[0119] A fourth determining module 505 is configured to determine pixel displacement signals corresponding to key points in the vibration video based on a target tracking algorithm;

[0120] A fifth determining module 506 is configured to determine a physical displacement signal corresponding to a key point based on the pixel displacement signal and the conversion coefficient;

[0121] The generating module 507 is configured to generate a dynamic characteristic diagram of the corresponding building structure displacement based on the physical displacement signal.

[0122] The present application also provides an electronic device, such as Figure 7 , which is a schematic diagram of the structure of the electronic device, wherein the electronic device includes a processor 121 and a memory 120, the memory 120 stores computer executable instructions that can be executed by the processor 121, and the processor 121 executes the computer executable instructions to implement the above-mentioned method for identifying building structure displacement.

[0123] exist Figure 7 In the illustrated embodiment, the electronic device further includes a bus 122 and a communication interface 123 , wherein the processor 121 , the communication interface 123 and the memory 120 are connected via the bus 122 .

[0124] Among them, the memory 120 may include a high-speed random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 123 (which can be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. can be used. The bus 122 can be an ISA (Industry Standard Architecture, Industrial Standard Architecture) bus, a PCI (Peripheral Component Interconnect, Peripheral Component Interconnect Standard) bus or an EISA (Extended Industry Standard Architecture, Extended Industry Standard Architecture) bus, etc. The bus 122 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0125] The processor 121 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by hardware integrated logic circuits in the processor 121 or by software instructions. The above-mentioned processor 121 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of the present application can be directly embodied as being executed by a hardware decoding processor, or can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium mature in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, or electrically erasable programmable memory, registers, etc. The storage medium is located in the memory, and the processor 121 reads the information in the memory and completes the steps of the method for identifying building structure displacement in the aforementioned embodiment in combination with its hardware.

[0126] Embodiments of the present invention provide a method, apparatus, and computer program product for electronic equipment for determining displacement of a building structure, including a computer-readable storage medium storing program code. The program code includes instructions that can be used to execute the methods described in the previous method embodiments. For specific implementation, please refer to the method embodiments and will not be repeated here.

[0127] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described systems and devices can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0128] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0129] In the description of the present invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings and are intended solely to facilitate and simplify the description of the present invention. They are not intended to indicate or imply that the devices or components referred to must have, be constructed, or operate in a specific orientation, and therefore should not be construed as limitations on the present invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0130] Finally, it should be noted that the above embodiments are only specific implementation methods of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that any person skilled in the art can modify or easily conceive of changes to the technical solutions described in the above embodiments within the technical scope disclosed by the present invention, or replace some of the technical features therein with equivalents. Such modifications, changes or replacements do not deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A method for determining the dynamic characteristics of building structure displacement based on video, characterized in that: The method comprises: Acquire a vibration video of the building structure to be identified using a camera; wherein the camera is a consumer-grade camera; Calibrate key points of the building structure to be identified based on the vibration video, and determine the pixel coordinates of the key points and the physical length of the key points mapped on the image plane of the vibration video; Determining, according to the pixel coordinates and the physical length, a conversion coefficient of a displacement calibration formula for the building structure to be identified based on the vibration video; Determining the displacement of the building structure to be identified based on the pixel coordinates and the conversion coefficient; Determining pixel displacement signals corresponding to the key points from the vibration video based on a target tracking algorithm; Determine a physical displacement signal corresponding to the key point based on the pixel displacement signal and the conversion coefficient; generating a corresponding dynamic characteristic diagram of the displacement of the building structure to be identified based on the physical displacement signal; Based on the vibration video, at least one key point of the building structure to be identified is calibrated, and a random subspace method is used to identify the synchronous displacement signal corresponding to each key point, and modal parameters of the building structure to be identified based on the vibration video are obtained to obtain a refined mode shape diagram, and the location of structural damage is determined; The step of generating the corresponding dynamic characteristic diagram of the displacement of the building structure to be identified based on the physical displacement signal includes: generating a first displacement signal time domain diagram corresponding to the physical displacement signal based on the physical displacement signal; Using a resampling method to increase the sampling rate of the first displacement signal time domain graph and obtain a corresponding second displacement signal time domain graph; Converting the second displacement signal time domain graph into the corresponding first displacement signal frequency domain graph through Fourier transform; Using bandpass filtering to reduce noise on the first displacement signal frequency domain graph, retaining the dominant mode of the first displacement signal frequency domain graph to obtain a second displacement signal frequency domain graph; Converting the second displacement signal frequency domain graph into a corresponding third displacement signal time domain graph by inverse Fourier transform; Performing peak point alignment and zero point alignment on the third displacement signal time domain graph to obtain a fourth displacement signal time domain graph; Downsampling the fourth displacement signal time domain graph obtains the dynamic characteristic graph having the same frequency as the first displacement signal time domain graph.

2. The method according to claim 1, characterized in that The step of determining the displacement of the building structure to be identified based on the pixel coordinates and the conversion coefficient includes: Multiplying the pixel coordinates and the conversion coefficient to obtain the motion plane coordinates of the building structure to be identified in the building coordinate system; Extracting a motion plane coordinate difference within a preset time period, wherein the motion plane coordinate difference is a difference between a first motion plane coordinate corresponding to an end point of the preset time period and a second motion plane coordinate corresponding to a start point of the preset time period; The displacement of the building structure to be identified is determined based on the motion plane coordinate difference.

3. The method according to claim 1, characterized in that After acquiring the vibration video of the building structure to be identified using a camera, the method further includes: A pixel displacement signal of the building structure to be identified is determined from the vibration video.

4. The method according to claim 1, wherein The step of determining, according to the pixel coordinates and the physical length, a conversion coefficient of a displacement calibration formula for the building structure to be identified based on the vibration video comprises: Determining whether the image plane of the vibration video is parallel to the actual plane of the building structure to be identified; wherein the actual plane refers to the plane where the key point is located; If yes, the conversion coefficient is obtained by the following formula: in, is the conversion coefficient, , are the coordinates of the key points on the actual plane, and The pixel coordinates of the key points mapped on the image plane of the vibration video, is the physical length.

5. The method according to claim 4, characterized in that The step of determining, according to the pixel coordinates and the physical length, a conversion coefficient of a displacement calibration formula for the building structure to be identified based on the vibration video further includes: If it is determined that the image plane of the vibration video is not parallel to the actual plane of the building structure to be identified, determining the angle between the optical axis of the camera lens and the normal direction of the actual plane; Determining whether the focal length of the camera lens of the camera is a preset focal length; If so, determine the distance from the camera lens of the camera to the actual plane, and determine the conversion coefficient of the displacement calibration formula of the building structure to be identified based on the vibration video according to the pixel coordinates, the physical length, the distance from the camera lens of the camera to the actual plane and the angle.

6. The method according to claim 5, characterized in that The conversion coefficient is obtained by the following formula: in, is the conversion coefficient, , are the coordinates of the key points of the building structure to be identified on the actual plane, and The pixel coordinates of the key points mapped on the image plane of the vibration video, , are the distances from the camera lens of the camera to the actual plane, is the angle between the optical axis of the camera lens and the normal direction of the actual plane, is the focal length of the camera lens of the camera.

7. The method according to claim 5, characterized in that The method further comprises: If the focal length of the camera lens of the camera is not the preset focal length, measuring the actual distance from the camera lens of the camera to the key point using a laser rangefinder; Using a protractor to measure the actual angles between different lasers emitted by the laser rangefinder; Calculating the physical length according to the actual distance and the actual angle; The conversion coefficient of the displacement calibration formula of the to-be-identified building structure based on the vibration video is determined by the pixel coordinates and the physical length.

8. The method according to claim 4, characterized in that If the pixel coordinates of the key points and the physical length of the key points mapped on the image plane of the vibration video cannot be determined, the conversion coefficient is obtained by the following formula: in, is the conversion coefficient, is the pixel size of the camera, i.e. the physical length of a single pixel of the camera, is the focal length of the camera, is the distance from the camera lens of the camera to the actual plane, is the angle between the optical axis of the camera lens of the camera and the normal direction of the actual plane.

9. A device for determining the dynamic characteristics of building structure displacement based on video, characterized in that: The device comprises: An acquisition module, configured to acquire a vibration video of the building structure to be identified using a camera; wherein the camera is a consumer-grade camera; A first determination module is configured to calibrate the key points of the building structure to be identified based on the vibration video, determine the pixel coordinates of the key points of the building structure to be identified in the vibration video, and determine the physical length of the key points of the building structure to be identified mapped on the image plane of the vibration video; A second determination module is configured to determine, according to the pixel coordinates and the physical length, a conversion coefficient of a displacement calibration formula for the building structure to be identified based on the vibration video; A third determining module is configured to determine the displacement of the building structure to be identified based on the pixel coordinates and the conversion coefficient; a fourth determining module, configured to determine a pixel displacement signal corresponding to the key point from the vibration video based on a target tracking algorithm; a fifth determining module, configured to determine a physical displacement signal corresponding to the key point based on the pixel displacement signal and the conversion coefficient; A generating module, configured to generate a corresponding dynamic characteristic diagram of the displacement of the building structure to be identified based on the physical displacement signal; Among them, the generation module is specifically used to: generate a first displacement signal time domain diagram corresponding to the physical displacement signal based on the physical displacement signal; use a resampling method to increase the sampling rate of the first displacement signal time domain diagram and obtain a corresponding second displacement signal time domain diagram; convert the second displacement signal time domain diagram into a corresponding first displacement signal frequency domain diagram through Fourier transform; use bandpass filtering to reduce the noise of the first displacement signal frequency domain diagram, retain the dominant mode of the first displacement signal frequency domain diagram to obtain a second displacement signal frequency domain diagram; convert the second displacement signal frequency domain diagram into a corresponding third displacement signal time domain diagram through inverse Fourier transform; perform peak point alignment and zero point alignment on the third displacement signal time domain diagram to obtain a fourth displacement signal time domain diagram; downsample the fourth displacement signal time domain diagram to obtain the dynamic characteristic diagram with the same frequency as the first displacement signal time domain diagram.

Images