Displacement sensing apparatus and displacement sensing method for building structure, and storage medium

Abstract

The present application discloses a displacement sensing apparatus and displacement sensing method for a building structure, and a storage medium. The displacement sensing apparatus comprises: a fixing module comprising a fixing base and a fixing support, wherein the fixing base is configured to be coupled with a monitoring point for a structure under measurement to form a whole structure; and a projection module and a camera module, wherein the camera module is arranged in front of the projection module by means of the fixing support at an angle perpendicular to the field of view, and is configured to collect a laser image formed on the projection module by a laser emission module which is provided at a fixed point.

Description

Displacement sensing device, displacement sensing method and storage medium for building structure

[0001] Related applications

[0002] This application claims priority to Chinese patent application No. 202410022458.0 filed on January 8, 2024, the entire contents of which are incorporated by reference into this application. Technical Field

[0003] The present application relates to the field of building detection, and in particular to a displacement sensing device, a displacement sensing method, and a storage medium for building structures. Background Art

[0004] In the field of high-rise building measurement, compared to traditional contact measurement methods, vision-based vibration measurement technology can use cameras or sensors to capture the displacement information of the object's surface under vibration to achieve vibration measurement. Vision-based vibration measurement technology also has the advantages of high measurement accuracy, long monitoring distance, no need for direct contact with the measured object, and low monitoring costs.

[0005] In the related visual displacement measurement methods that combine laser projection technology, practical applications often involve an angle effect: in addition to translational displacement, the surface of the structure also has an angle. Due to the long projection distance of the laser light, even small angles are magnified by the projection distance, resulting in a large displacement change. The measured vibration displacement is the displacement caused by the coupling of the angle effect and the true translational displacement, ultimately causing the true translational information to be submerged. Therefore, the current visual displacement measurement method suffers from the problem of angle coupling.

[0006] The above content is only used to assist in understanding the technical solution of this application and does not constitute an admission that the above content is prior art. Summary of the Invention

[0007] The main purpose of this application is to provide a displacement sensing device for a building structure to solve the problem of angular coupling in the visual displacement measurement method in the prior art.

[0008] To achieve the above-mentioned objectives, the present application provides a method for sensing displacement of a building structure, which is applied to a displacement sensing device for a building structure. The displacement sensing device includes a fixing module, the fixing module including a fixing base and a fixing bracket, the fixing base being configured to couple with a monitoring point of a measured structure to form a structural whole; a projection module and a camera module, the camera module being disposed in front of the projection module at an angle perpendicular to the field of view via the fixing bracket and being configured to capture a laser image formed on the projection module by a laser emitting module disposed at a fixed point; and a processor module, the processor module being configured to acquire image data captured by the camera module and generate a displacement detection result based on the image data. The method for sensing displacement of a building structure includes:

[0009] Acquiring image data collected by the camera module and calibration data pre-stored in the processor module;

[0010] Determining a partial derivative array of the image data, and calculating a gradient magnitude matrix and a gradient direction matrix of the image data based on the partial derivative array;

[0011] Determine an amplitude mapping array corresponding to the gradient amplitude matrix, and determine a direction mapping array corresponding to the gradient direction matrix;

[0012] Based on the amplitude mapping array and the direction mapping array, selecting target calibration data with the largest mapping relationship value from the calibration data, and determining calibration parameters corresponding to the target calibration data;

[0013] Performing super-resolution reconstruction processing on the image data based on the calibration parameters to obtain reconstructed image data; and

[0014] The reconstructed image data is subjected to structural vibration amplification processing, and a displacement detection result of the structure to be measured is generated according to the target reconstructed image data after the structural vibration amplification processing.

[0015] In addition, to achieve the above-mentioned purpose, the present application also provides a displacement sensing device for a building structure, wherein the displacement sensing device for a building structure includes a memory, a processor, and a displacement sensing program for a building structure stored on the memory and executable on the processor. When the displacement sensing program for a building structure is executed by the processor, the steps of the displacement sensing method for a building structure as described above are implemented.

[0016] In addition, to achieve the above-mentioned purpose, the present application also provides a computer-readable storage medium, on which a displacement sensing program for a building structure is stored. When the displacement sensing program for a building structure is executed by a processor, the steps of the displacement sensing method for a building structure as described above are implemented.

[0017] The embodiments of the present application provide a displacement sensing device, a displacement sensing method, and a storage medium for a building structure. The displacement sensing device includes: a fixed module, the fixed module includes a fixed base and a fixed bracket, the fixed base is configured to couple with a monitoring point of the structure to be measured to form a structural whole, a projection module, and a camera module. The camera module is arranged in front of the projection module at an angle perpendicular to the field of view through the fixed bracket, and is configured to collect a laser image formed on the projection module by a laser emitting module arranged at a fixed point. It can be seen that the sensing device fixes the relative positions of modules such as the camera module and the projection module, and installs them at the vibration monitoring point of the structure; the laser emitting module is installed at a fixed point far away from the structure; because the light beam position translation caused by the slight rotation angle of the structure is related to the fixed height of the projection module, rather than the laser projection distance, and the fixed height of the projection module is much smaller than the laser projection distance, the problem of laser beam position translation caused by the slight rotation angle of the structure can be significantly reduced, solving the angle coupling problem existing in the existing "laser-vision" technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present application, and together with the specification, are used to explain the principles of the present application. In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for describing the embodiments. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without inventive work.

[0019] FIG1 is a schematic structural diagram of a displacement sensing device for a building structure of the present application;

[0020] FIG2 is a schematic diagram of a displacement recognition technology of a displacement sensing method for a building structure of the present application;

[0021] FIG3 is a flow chart of an embodiment of a method for sensing displacement of a building structure according to the present application;

[0022] FIG4 is a flow chart of an embodiment of a method for sensing displacement of a building structure according to the present application;

[0023] FIG5 is a flow chart of an embodiment of a method for sensing displacement of a building structure according to the present application;

[0024] FIG6 is a flow chart of an embodiment of a method for sensing displacement of a building structure according to the present application;

[0025] FIG7 is a schematic diagram of the terminal hardware structure of various embodiments of the method for sensing displacement of a building structure of the present application.

[0026] Explanation of Figure Numbers

[0027] The realization of the objectives, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0028] It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application.

[0029] Please refer to Figure 1. In one embodiment of the present application, a structural schematic diagram of a displacement sensing device for a building structure is proposed, which specifically includes: a fixed module 10, a projection module 20, a camera module 30, a laser emission module 40 and a processor module 50, wherein the fixed module 10 includes a fixed base 11 and a fixed bracket 12, and the fixed base 12 is used to couple with the monitoring point of the measured structure to form a structural whole. The camera module 30 is set in front of the projection module 20 at a vertical field of view through the fixed bracket, and is used to collect the laser image formed on the projection module 20 by the laser emission module 40 set at a fixed point. The camera module 30 uses a high-resolution camera.

[0030] In addition, referring to FIG. 1 , the displacement sensing device further includes a processor module 50 , and the processor module 50 is configured to obtain image data collected by the camera module 30 and generate a displacement detection result according to the image data.

[0031] It should be noted that the fixing module 10 fixes the projection module 20 and the camera module 30, and fixes the camera module 30 in front of the projection module 20 at an angle of 90° vertical field of view. By fixing the camera module 30 at an angle of 90° vertical field of view, the effect of the deflection angle on the vibration pixel imaging can be effectively avoided. After the fixing base 11 is mounted on the structure to be measured, the laser emitting module 40 is installed at an unfixed point away from the structure monitoring point, for example, it can be installed below the structure monitoring point, and emits laser light from the bottom to the top toward the projection module 20.

[0032] Based on this, when the structure under test experiences a tiny vibration, it drives the installed sensing device to vibrate along with it. At this point, the relative position of the device and the projected laser beam changes. Since the laser beam does not shift, the projection module experiences a movement of the laser beam in the opposite direction of the structure's tiny vibration, with the same amplitude. Furthermore, by fixing the camera module 30 in front of the projection module 20, the monitoring distance is significantly shortened compared to conventional methods that place the camera far from the structure. This significantly improves the resolution of the laser beam's movement image, enabling high-resolution sensing of tiny structural displacements.

[0033] Further, please refer to Figure 2. In the related "laser-vision" displacement measurement and identification technology, a laser light is usually installed on a building (i.e., the object to be measured), and the image data of the object to be measured when it vibrates is captured by a remote camera, and the object's tiny vibration displacement is then calculated. However, when measuring tiny displacements of the object to be measured, the measurement distance is relatively long, and the corresponding displacement change formula is as follows: Δx = L × Δθ. Based on this, due to the long measurement distance, the vibration measurement results are greatly affected by the angle change. In the sensing device of this embodiment, the fixed module is directly installed on the structure to be measured, and the laser emission module is installed at a distant fixed point. At this time, the light beam translation caused by the tiny angle of the structure has nothing to do with the laser projection distance, but is related to the distance between the camera module 30 and the projection module 20. When measuring tiny vibrations of the structure, the distance between the laser emitting module and the displacement sensing device is much greater than the distance between the camera module 30 and the projection module 20. Therefore, the position translation caused by the tiny rotation angle of the structure can be ignored. At this time, the problem of laser beam position translation caused by the tiny rotation angle of the structure can be significantly reduced, solving the angle coupling problem existing in the existing "laser-vision" technology.

[0034] In one embodiment, based on the displacement sensing device for a building structure described in the above embodiment, a displacement sensing method for a building structure is proposed. Referring to FIG. 3 , the displacement sensing method for a building structure of the present application includes:

[0035] Step S10, acquiring image data collected by the camera module and calibration data pre-stored in the processor module;

[0036] In this embodiment, calibration data refers to accurate data used for reference, measured indoors before the displacement sensing device is installed on the building structure to be measured. The calibration data includes laser beam imaging data at different resolutions and different blur conditions, as well as regression parameters corresponding to a linear regression model between the imaging data at different resolutions and the high-resolution laser beam imaging data under different blur conditions. The regression parameters are used to perform super-resolution reconstruction on the image data, and each imaging data at different resolutions and different blur conditions has a corresponding regression parameter. The calibration data is pre-stored in the processor module. The image data refers to the laser beam vibration video captured by the camera module. When processing the image data, processing begins with the first frame of the vibration video and continues sequentially until each frame is processed.

[0037] After the displacement sensing device is mounted on the structure to be measured and the laser emission module is controlled to emit a laser beam toward the projection module, the processor module of the displacement sensing device captures the image data of the beam movement formed by the laser beam on the projection module when the structure to be measured experiences slight vibrations. Structures to be measured include large buildings such as large office buildings, residential high-rise buildings, and bridges.

[0038] Step S20, performing super-resolution reconstruction processing on the image data according to the calibration data to obtain reconstructed image data;

[0039] In this embodiment, in order to improve the measurement accuracy of the micro-vibrations of the structure to be measured, it is necessary to calculate the corresponding super-resolution reconstructed image in a certain frame of the collected image based on the calibration parameters in the calibration data, so that the processing module can output a higher resolution image, thereby realizing the reconstruction processing of the collected image to improve the recognition and measurement accuracy of micro-vibrations. It should be noted that in the process of reconstructing the image data, it is necessary to calculate the gradient array of the image data, and then select the most appropriate calibration parameters in the calibration data for super-resolution reconstruction based on the image gradient array and the mapping array corresponding to the image gradient array. Therefore, please refer to Figure 4, and perform super-resolution reconstruction processing on the image data according to the calibration data to obtain reconstructed image data, which specifically includes:

[0040] Step S21, determining a partial derivative array of the image data, and calculating a gradient magnitude matrix and a gradient direction matrix of the image data based on the partial derivative array;

[0041] As an alternative implementation, the partial derivative array of the image data can be calculated using first-order finite differences: x ≈(S(x,y+1)-S(x,y)+S(x+1,y+1)-S(x+1,y)) / 2 D y ≈(S(x,y)-S(x+1,y)+S(x,y+1)-S(x+1,y+1)) / 2

[0042] Where Dx is the array of image partial derivatives along the x direction, and Dy is the array of image partial derivatives along the y direction.

[0043] The image gradient array is then calculated by combining the partial derivative arrays in the x and y directions: θ(x,y)=arctan(D y (x,y) / D x (x,y))

[0044] Among them, M represents the gradient amplitude and θ represents the gradient direction.

[0045] For example, the first frame of vibration image may be processed to obtain the gradient magnitude matrix and the gradient direction matrix of the frame of image in sequence.

[0046] Step S22, determining an amplitude mapping array corresponding to the gradient amplitude matrix, and determining a direction mapping array corresponding to the gradient direction matrix;

[0047] When selecting the most appropriate calibration parameters in the calibration data to perform super-resolution reconstruction on a frame of image data currently collected, it is necessary to comprehensively consider the weight ratios of the mapping relationship array of the gradient amplitude matrix and the mapping relationship array of the gradient direction matrix. Therefore, in this process, it is also necessary to determine the amplitude mapping relationship array and direction mapping relationship array corresponding to the amplitude matrix and direction matrix of the frame image in the calibration data.

[0048] In an optional implementation for determining an amplitude mapping array corresponding to a gradient magnitude matrix, the calibration data corresponding to the current resolution and the corresponding pre-stored gradient magnitude matrix can be determined. The pre-stored gradient magnitude matrix and the gradient magnitude matrix are then normalized. For each pixel position in the normalized gradient magnitude matrix, the corresponding mapping value in the normalized pre-stored gradient magnitude matrix and the amplitude mapping matrix corresponding to the mapping value are determined. Finally, the mapping relationships across all rows and columns of the amplitude mapping matrix are summed and averaged to obtain the amplitude mapping array. The current resolution refers to the resolution at which the camera module acquires image data from the projection module, i.e., the resolution used when acquiring the image data. Since the calibration data includes images with different blur conditions and resolutions, if the current camera resolution is 50 frames, the corresponding calibration data may be images between frames 45 and 55 of the calibration data. The selection of image data between frames 45 and 55 can be dynamically set based on the actual application scenario or the actual camera resolution. The images corresponding to the calibration data all contain corresponding gradient magnitude matrices and gradient direction matrices. The mapping relationship feature data at each position of the amplitude mapping array corresponds to calibration data with different blur conditions and different resolutions.

[0049] For example, the current acquisition resolution of the current camera module is 60 frames. At this time, it is necessary to select 55 frames and 65 frames of calibration images from the calibration data, and then use the pre-stored gradient amplitude matrix corresponding to these images as the matching template I1. The camera module is used to acquire the first frame of the structural micro-vibration beam image, that is, the gradient amplitude matrix corresponding to the currently processed image data is used as the tracking template T1. The two templates are then normalized to obtain T1' and I1'. The tracking template T1 is then slid on the matching template I1 from left to right and from top to bottom, moving one pixel position each time and calculating the mapping value of that position. Finally, the mapping matrix R1' is calculated. The mapping relationship values ​​of each row and each column of the mapping matrix R1' are summed and averaged to obtain the amplitude mapping relationship feature array {r'}.

[0050] It should be noted that the mapping feature values ​​at each position correspond to calibration images of different resolutions and different blur conditions. The left-to-right and top-to-bottom sliding process of matching the tracking template against the matching template is for illustrative purposes only and does not limit the matching operation.

[0051] In an optional implementation of determining a direction mapping array corresponding to a gradient direction matrix, a pre-stored gradient direction matrix corresponding to the calibration data corresponding to the current resolution can be determined, and then the pre-stored gradient direction matrix and the gradient direction matrix are normalized. Next, the mapping value corresponding to each pixel position in the normalized gradient direction matrix and the direction mapping matrix corresponding to the mapping value are determined. Finally, the mapping relationships of all rows and columns of the direction mapping matrix are summed and averaged to obtain the direction mapping array. It should be noted that in the process of determining the mapping value corresponding to each pixel position in the normalized gradient direction matrix and the pre-stored gradient direction matrix, a matching template is also selected and processed in the calibration data. This process is similar in principle to the process of obtaining the valued mapping array and will not be elaborated on here.

[0052] Step S23, based on the amplitude mapping array and the direction mapping array, selecting target calibration data with the largest mapping relationship value from the calibration data, and determining calibration parameters corresponding to the target calibration data;

[0053] Step S24: performing super-resolution reconstruction processing on the image data based on the calibration parameters to obtain the reconstructed image data.

[0054] As an optional implementation, after obtaining the amplitude mapping matrix and the direction mapping matrix, the indoor resolution calibration image Mr with the largest mapping relationship value can be selected from the two: Mr = α1{r} + α2{r'} Index = argmax(Mr)

[0055] Here, {r} and {r'} are the amplitude mapping relationship array and the direction mapping relationship array, respectively. α1 and α2 are the weight ratio parameters corresponding to the two arrays. The weight ratio parameters can be dynamically set according to the actual application scenario. After selecting the calibration image Mr, the collected beam motion image data is super-resolved and reconstructed based on the super-resolution regression parameters corresponding to Mr.

[0056] After super-resolution reconstruction of the image data, further processing can be performed through interpolation and reconstruction to improve image quality, thereby improving the measurement accuracy of the displacement detection results. Specifically, as an optional implementation method, the reconstructed image data can be interpolated, the interpolated reconstructed image data can be amplified, and all pixel coordinates of the amplified reconstructed image data can be determined. Then, based on the integer coordinates, decimal coordinates, and the weighted ratios of the integers and decimals of all the pixel coordinates, the target pixel coordinates of the amplified reconstructed image data can be calculated.

[0057] Exemplarily, after interpolation processing, the reconstructed image data is magnified to the target preset size, and then the output high-resolution image position (x, y) is calculated based on all positions (x', y') of the image, where x' and y' are integer multiples of x and y. Then, based on the integer parts xi and yi of x' and y', and the fractional parts dx = x'-xi and dy = y'-yi, the weighted proportions of the 16 pixels around each position are calculated. Then, based on the weights, the corresponding areas of the input image are weighted averaged to obtain the value of the pixel (x, y) in the output image:

[0058] Where O(x,y) is the pixel value of the output high-resolution image; I(x',y') is the pixel value at position (x',y') in the reconstructed image before equal magnification; and w(i,j) is the weight of pixel (xi+i,yj+j) to the output pixel (x,y). This improves the clarity of the image data, thereby increasing the measurement accuracy of displacement detection results during displacement monitoring and analysis.

[0059] It should be noted that the above parameters are only used for explanation and are not specific limitations of this application. That is, in calculating the weight ratio of multiple surrounding pixels, the selected number can be dynamically set according to the actual application scenario.

[0060] Step S30 , performing structural vibration amplification processing on the reconstructed image data, and generating a displacement detection result of the structure to be measured based on the reconstructed image data after the structural vibration amplification processing.

[0061] In this embodiment, after reconstructing each frame of the collected image data and obtaining the reconstructed image data, structural micro-vibration reconstruction is required to improve the measurement accuracy of the displacement detection results. As an alternative embodiment, spatial domain decomposition can be performed on each image in the time series direction of the super-resolution reconstructed laser beam vibration video, followed by temporal domain filtering. The filtered image data is then amplified and reconstructed to complete the structural vibration amplification of the reconstructed image data.

[0062] In an optional implementation method for generating a displacement detection result, a feature area corresponding to the target reconstructed image data can be determined, and a target image of the reconstructed image data after the second frame of the feature area can be obtained, where the feature area is the imaging area of ​​the projection module. The target image and the first frame image of the target reconstructed image data are then normalized to obtain a template image corresponding to the feature area and a vibration image corresponding to the first frame image. The mapping values ​​corresponding to the pixels of the vibration image when sliding in the template image and the mapping matrix corresponding to the mapping values ​​are then calculated. All frames of the reconstructed image data are then processed based on the position corresponding to the maximum value of the pixel in the mapping matrix to obtain a vibration time history signal of the structure to be measured, wherein the processor module of the displacement sensing device can generate the displacement detection result based on the vibration time history signal.

[0063] For example, the image data of the characteristic area of ​​the laser beam after the structural vibration amplification process is selected, and the images from the second frame to the last frame are selected as the template T, and the first frame image after the reconstructed interpolation process is used as the vibration template I. Then, the two images are normalized respectively to obtain the template image T′ and the vibration image I′ as shown in the following formula:

[0064] Next, the mapping relationship matrix R of the two images is calculated. The implementation process includes sliding the template image T′ on the vibration image I′ from left to right and from top to bottom, moving one pixel position each time and calculating the mapping value of the position. Finally, the mapping matrix R is calculated:

[0065] Where (x, y) is the coordinate of a point on the image to be matched; (x', y') is the coordinate of the template image; T(x, y) is the template image, with an image size of w × h; I is the image to be matched; a point (x, y) on the mapping matrix R(x, y) represents the correlation between T(x, y) and the image sub-block with (x, y) as the upper left corner in the image to be matched I and the same size as the template image T′(x, y). It should be noted that the direction of pixel sliding can be dynamically set according to the actual application scenario.

[0066] After obtaining the mapping matrix, since the maximum value reflects the similarity and consistency of the image, the position of the maximum value of the mapping matrix is ​​selected as the matching result, and the image of each frame in the reconstructed image data after interpolation and reconstruction, that is, the image after the first frame, is matched, and finally the vibration time history signal of the structure to be measured is obtained.

[0067] In the technical solution of this embodiment, during the process of detecting micro-displacements of the structure under test based on laser projection, indoor calibration, interpolation and reconstruction, and micro-vibration amplification processing are combined to improve the recognition resolution of the displacement sensing device in actual use based on the calibration data obtained in advance indoors. Interpolation and reconstruction and micro-vibration amplification processing are also performed to further improve the image clarity and measurement accuracy of the detection results. This integrated approach can significantly improve the recognition resolution of micro-vibrations at the monitoring position of the structure under test, achieve high-precision measurement of the micro-vibration displacement of the structure under test, and simultaneously improve the measurement accuracy of micro-vibration displacement.

[0068] Referring to FIG. 5 , in one embodiment, before step S10, the method further includes:

[0069] Step S40, collecting imaging data on the projection module based on different resolutions, and performing downsampling and blurring processing on the imaging data to obtain pre-stored imaging data, wherein the pre-stored imaging data includes the imaging data with different resolutions and different blurring conditions;

[0070] In this embodiment, before the displacement sensing device is put into use, a high-resolution camera can be used to collect high-resolution imaging data of the light beam, and then the super-resolution parameters of the laser beam data can be calibrated in advance, thereby effectively improving the recognition resolution of the device in actual use.

[0071] In an optional implementation of downsampling and blurring the imaging data, imaging data of the laser beam on the projection module can be collected based on different resolution photography conditions from low to high, obtaining multiple sets of laser beam projection imaging data at different resolutions. Subsequently, the collected images at different resolutions are downsampled to obtain laser beam imaging data at more resolutions. Gaussian convolution kernels with different scale factors are then used to blur the laser beam imaging data at different resolutions, further expanding the data to obtain laser beam imaging data at different resolutions at different blur scales. In one embodiment, multiple sets of different imaging data can also be collected from high to low resolutions.

[0072] For example, based on the imaging conditions of 30-120 frames, multiple sets of projection data of 30, 40, 50, ..., and 120 frames are collected. Then, these sets of projection data are downsampled multiple times to obtain laser beam imaging data of 31, 32, 33, and other frames. Gaussian convolution is then performed on the downsampled data to obtain imaging data of different blur scales.

[0073] Step S50, calculating and storing the gradient magnitude matrix and gradient direction matrix of the pre-stored imaging data based on first-order finite differences;

[0074] In this embodiment, the gradient magnitude matrix and gradient direction matrix of the image under each condition are calculated and stored by using first-order finite differences, so that the processor module can calculate the corresponding mapping array based on these pre-stored matrices, thereby improving the clarity of the image after super-resolution reconstruction.

[0075] Step S60, selecting target imaging data with the highest resolution among the imaging data, and determining a linear regression model between the imaging data with different resolutions and the target imaging data under different blurring conditions based on a least squares fitting algorithm;

[0076] Step S70: storing calibration parameters corresponding to the linear regression model in a processor module, and storing the imaging data as the calibration data in the processor module, wherein each calibration data corresponds to one calibration parameter.

[0077] In this embodiment, as an optional implementation, the highest-resolution imaging data is used as a reference. A least-squares fitting process is used to obtain a linear regression model between the high-resolution imaging data and other imaging data at different resolutions and under different blur conditions. The resulting regression parameters are then stored and used in the image resolution reconstruction process. This allows for the generation of different regression parameters corresponding to imaging data at different resolutions under different blur conditions.

[0078] In the technical solution disclosed in this embodiment, before the displacement sensing device is put into use, in a stable structure indoors, since the beam or spot image formed by the laser beam on the projection module is fixed, high-resolution imaging data of the beam is collected by a high-resolution camera, and the super-resolution parameters of the laser beam imaging data are calibrated in advance to obtain corresponding calibration data and calibration parameters, which can improve the recognition resolution of the displacement sensing device in actual use.

[0079] Referring to FIG. 6 , in one embodiment, the step of performing structural vibration amplification processing on the reconstructed image data specifically includes:

[0080] Step S31, performing multiple downsampling processes on the reconstructed image data to obtain a multi-layer image pyramid;

[0081] Step S32, based on the top image of the image pyramid, sequentially selecting two adjacent pyramid images of the image pyramid;

[0082] Step S33, performing upsampling processing on the matrix of the upper layer image in the two-order pyramid image, and subtracting the matrix of the lower layer image in the two-order pyramid image from the matrix of the upsampling upper layer image to obtain a Laplacian image pyramid;

[0083] Step S34 : filtering each pixel of each image of the Laplacian image pyramid in the time domain based on a preset filtering frequency, sequentially selecting bottom-level images of the Laplacian image pyramid after the filtering, and amplifying and upsampling the selected bottom-level images.

[0084] In this embodiment, during the process of reconstructing the vibration of the microstructure, it is necessary to perform spatial domain decomposition on each image in the time series direction of the vibration video of the laser beam after super-resolution reconstruction.

[0085] In one exemplary implementation, after obtaining a super-resolution reconstructed vibration video, each frame is processed sequentially. During this processing, the image is first downsampled β times to obtain a β+1-order image pyramid. For this β+1-order image pyramid, starting from the top layer, two adjacent pyramid images are sequentially selected. The upper image matrix of the selected two-order pyramid images is upsampled and then matrix-subtracted from the lower image matrix to obtain a Laplacian image pyramid. The image data of the Laplacian image pyramid is then filtered in the time domain. During this processing, each pixel in each image pyramid is filtered in the time domain using an ideal bandpass filter based on the [f1, f2] frequency band. Finally, the filtered image video is amplified and reconstructed. During this process, the filtered image pyramid is amplified and upsampled α times, starting from the bottom layer, and then superimposed on the image pyramid layer above, until it reaches the top layer. This completes the reconstruction of small vibrations, improving the accuracy of displacement detection.

[0086] In the technical solution disclosed in this embodiment, during the process of performing structural vibration amplification processing on the reconstructed image data, each image is decomposed in the spatial domain, then filtered based on the time domain, and finally amplified and reconstructed, thereby improving the measurement accuracy of small displacement detection of the structure to be measured.

[0087] Refer to Figure 7, which is a schematic diagram of the terminal structure of the hardware operating environment involved in the embodiment of the present application.

[0088] As shown in Figure 7, the terminal may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a network interface 1003, and a memory 1004. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The network interface 1003 may include a standard wired interface, a wireless interface (such as a Wireless-Fidelity (Wi-Fi interface)). The memory 1004 may be a high-speed RAM memory (Random Access Memory, RAM) or a stable non-volatile memory (Non-Volatile Memory, NVM), such as a disk memory. The memory 1004 may also be a storage device independent of the aforementioned processor 1001.

[0089] Those skilled in the art will understand that the terminal structure shown in FIG7 does not constitute a limitation on the terminal, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0090] As shown in FIG7 , the memory 1004 as a computer storage medium may include an operating system, a data storage module, a network communication module, and a displacement sensing program for a building structure.

[0091] In the terminal shown in FIG7 , the network interface 1003 is mainly used to connect to the backend server and perform data communication with the backend server; the processor 1001 can call the displacement sensing program of the building structure stored in the memory 1004 and perform the following operations:

[0092] Obtain image data collected by the camera module and calibration data pre-stored in the processor module;

[0093] Performing super-resolution reconstruction processing on the image data according to the calibration data to obtain reconstructed image data;

[0094] The reconstructed image data is subjected to structural vibration amplification processing, and a displacement detection result of the structure to be measured is generated according to the target reconstructed image data after the structural vibration amplification processing.

[0095] Furthermore, the processor 1001 may call the building structure displacement sensing program stored in the memory 1004 and perform the following operations:

[0096] Determining a partial derivative array of the image data, and calculating a gradient magnitude matrix and a gradient direction matrix of the image data based on the partial derivative array;

[0097] Determine an amplitude mapping array corresponding to the gradient amplitude matrix, and determine a direction mapping array corresponding to the gradient direction matrix;

[0098] Based on the amplitude mapping array and the direction mapping array, selecting target calibration data with the largest mapping relationship value from the calibration data, and determining calibration parameters corresponding to the target calibration data;

[0099] The image data is subjected to super-resolution reconstruction processing based on the calibration parameters to obtain the reconstructed image data.

[0100] Furthermore, the processor 1001 may call the building structure displacement sensing program stored in the memory 1004 and perform the following operations:

[0101] Determine the calibration data corresponding to the current resolution and the corresponding pre-stored gradient amplitude matrix;

[0102] performing normalization processing on the pre-stored gradient magnitude matrix and the gradient magnitude matrix;

[0103] Determining, for each pixel position of the normalized gradient magnitude matrix, a corresponding mapping value in the pre-stored normalized gradient magnitude matrix, and an amplitude mapping matrix corresponding to the mapping value;

[0104] The mapping relationships of all rows and columns of the amplitude mapping matrix are summed and averaged to obtain the amplitude mapping array, wherein the mapping relationship characteristic value of each position of the amplitude mapping array corresponds to the calibration data with different blur conditions and different resolutions.

[0105] Furthermore, the processor 1001 may call the building structure displacement sensing program stored in the memory 1004 and perform the following operations:

[0106] Determine a pre-stored gradient direction matrix corresponding to the calibration data corresponding to the current resolution;

[0107] performing normalization processing on the pre-stored gradient direction matrix and the gradient direction matrix;

[0108] Determine for each pixel position of the normalized gradient direction matrix, a corresponding mapping value in the normalized pre-stored gradient direction matrix, and a direction mapping matrix corresponding to the mapping value;

[0109] The mapping relationships of all rows and columns of the direction mapping matrix are summed and averaged to obtain the direction mapping array, wherein the mapping relationship characteristic value of each position of the direction mapping array corresponds to the calibration data with different blur conditions and different resolutions.

[0110] Furthermore, the processor 1001 may call the building structure displacement sensing program stored in the memory 1004 and perform the following operations:

[0111] Acquiring imaging data on a projection module based on different resolutions, and performing downsampling and blurring processing on the imaging data to obtain pre-stored imaging data, wherein the pre-stored imaging data includes the imaging data at different resolutions and different blurring conditions;

[0112] Calculating and storing the gradient magnitude matrix and the gradient direction matrix of the pre-stored imaging data based on first-order finite differences;

[0113] Selecting target imaging data with the highest resolution among the imaging data, and determining a linear regression model between the imaging data with different resolutions and the target imaging data under different blurring conditions based on a least squares fitting algorithm;

[0114] Calibration parameters corresponding to the linear regression model are stored in a processor module, and the imaging data is stored in the processor module as the calibration data, wherein each calibration data corresponds to one calibration parameter.

[0115] Furthermore, the processor 1001 may call the building structure displacement sensing program stored in the memory 1004 and perform the following operations:

[0116] performing interpolation processing on the reconstructed image data, amplifying the reconstructed image data after the interpolation processing, and determining all pixel coordinates of the reconstructed image data after the amplification processing;

[0117] Calculate target pixel coordinates of the reconstructed image data after the magnification process according to the integer coordinates, decimal coordinates, and weight ratios of the integers and decimals of all the pixel coordinates.

[0118] Furthermore, the processor 1001 may call the building structure displacement sensing program stored in the memory 1004 and perform the following operations:

[0119] Performing multiple downsampling processes on the reconstructed image data to obtain a multi-layer image pyramid;

[0120] Based on the top image of the image pyramid, sequentially selecting two adjacent pyramid images in the image pyramid;

[0121] performing upsampling processing on the matrix of the upper layer image in the two-order pyramid image, and subtracting the matrix of the lower layer image in the two-order pyramid image from the matrix of the upper layer image after the upsampling processing to obtain a Laplacian image pyramid;

[0122] Based on a preset filtering frequency, filtering is performed on each pixel of each image of the Laplacian image pyramid in the time domain, and bottom-level images of the Laplacian image pyramid after the filtering are selected in sequence, and the selected bottom-level images are amplified and upsampled.

[0123] Furthermore, the processor 1001 may call the building structure displacement sensing program stored in the memory 1004 and perform the following operations:

[0124] Determine a characteristic region corresponding to the target reconstructed image data, and obtain a target image of the reconstructed image data after the second frame in the characteristic region, wherein the characteristic region is an imaging region of a projection module;

[0125] performing normalization processing on the target image and the first frame image of the target reconstructed image data to obtain a template image corresponding to the feature area and a vibration image corresponding to the first frame image;

[0126] Calculating mapping values ​​corresponding to pixels of the vibration image when sliding in the template image, and a mapping matrix corresponding to the mapping values;

[0127] Based on the position corresponding to the maximum value of the pixel point in the mapping matrix, all frames of the target reconstructed image data are processed to obtain the vibration time history signal of the structure to be measured, wherein the processor module of the displacement sensing device can generate the displacement detection result based on the vibration time history signal.

[0128] Furthermore, those skilled in the art will appreciate that all or part of the steps in the method of the above-described embodiment can be implemented by instructing the relevant hardware through a computer program. The computer program includes program instructions, which can be stored in a storage medium that is computer-readable. The program instructions are executed by at least one processor in the control terminal to implement the steps of the above-described method embodiment.

[0129] Therefore, the present application also provides a computer-readable storage medium, which stores a displacement sensing program for a building structure. When the displacement sensing program for a building structure is executed by a processor, the various steps of the displacement sensing method for a building structure described in the above embodiment are implemented.

[0130] It should be noted that since the storage medium provided in the embodiments of this application is the storage medium used to implement the method of the embodiments of this application, based on the method described in the embodiments of this application, those skilled in the art will be able to understand the specific structure and deformation of the storage medium, and therefore will not be described in detail here. All storage media used in the method of the embodiments of this application fall within the scope of protection to be provided by this application.

[0131] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0132] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0133] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a product including an instruction device that implements the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0134] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0135] It should be noted that in the claims, any reference signs placed between parentheses shall not be construed as limiting the claims. The word "comprising" does not exclude the presence of components or steps not listed in the claim. The word "a" or "an" preceding a component does not exclude the presence of a plurality of such components. The present application may be implemented by means of hardware comprising several different components and by means of a suitably programmed computer. In a unit claim enumerating several means, several of these means may be embodied by one and the same item of hardware. The use of the words first, second, and third etc. does not indicate any order. These words may be interpreted as names.

[0136] Although the optional embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the optional embodiments and all changes and modifications that fall within the scope of the present application.

[0137] Obviously, those skilled in the art may make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalents, this application is intended to include these modifications and variations.

[0138] The above are merely optional embodiments of the present application and do not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A method for sensing displacement of a building structure, wherein, A displacement sensing device applied to a building structure. The displacement sensing device includes a fixing module, and the fixing module includes a fixing base and a fixing bracket. The fixing base is configured to be coupled with a monitoring point of the structure to be measured to form an integral structure; a projection module and a camera module. The camera module is disposed in front of the projection module at an angle of a vertical field of view through the fixing bracket, and is configured to collect a laser image formed by a laser emission module disposed at a fixed point on the projection module. A processor module configured to obtain the image data collected by the camera module and generate a displacement detection result based on the image data. The displacement sensing method for the building structure includes: Obtaining the image data collected by the camera module and the calibration data pre-stored in the processor module; Determining a partial derivative array of the image data, and calculating a gradient amplitude matrix and a gradient direction matrix of the image data based on the partial derivative array; Determining an amplitude mapping array corresponding to the gradient amplitude matrix, and determining a direction mapping array corresponding to the gradient direction matrix; Based on the amplitude mapping array and the direction mapping array, selecting target calibration data with the largest mapping relationship value in the calibration data, and determining calibration parameters corresponding to the target calibration data; Performing super-resolution reconstruction processing on the image data based on the calibration parameters to obtain reconstructed image data; and Performing structural vibration amplification processing on the reconstructed image data, and generating a displacement detection result of the structure to be measured based on the target reconstructed image data after the structural vibration amplification processing.

2. The displacement sensing method of the building structure according to claim 1, wherein, The step of determining the amplitude mapping array corresponding to the gradient amplitude matrix includes: Determining the calibration data corresponding to the current resolution and the pre-stored gradient amplitude matrix corresponding thereto; Performing normalization processing on the pre-stored gradient amplitude matrix and the gradient amplitude matrix; Determining the mapping value corresponding to each pixel position of the gradient amplitude matrix after normalization processing in the pre-stored gradient amplitude matrix after normalization processing, and the amplitude mapping matrix corresponding to the mapping value; and Performing summation and averaging processing on the mapping relationships of all rows and columns of the amplitude mapping matrix to obtain the amplitude mapping array, wherein the mapping relationship characteristic value at each position of the amplitude mapping array corresponds to the calibration data under different blurring conditions and different resolutions.

3. The displacement sensing method of the building structure according to claim 1, wherein, The step of determining the direction mapping array corresponding to the gradient direction matrix includes: Determining the pre-stored gradient direction matrix corresponding to the calibration data corresponding to the current resolution; Performing normalization processing on the pre-stored gradient direction matrix and the gradient direction matrix; Determining the mapping value corresponding to each pixel position of the gradient direction matrix after normalization processing in the pre-stored gradient direction matrix after normalization processing, and the direction mapping matrix corresponding to the mapping value; and Performing summation and averaging processing on the mapping relationships of all rows and columns of the direction mapping matrix to obtain the direction mapping array, wherein the mapping relationship characteristic value at each position of the direction mapping array corresponds to the calibration data under different blurring conditions and different resolutions.

4. The displacement sensing method for a building structure according to claim 1, wherein, Before the steps of obtaining the image data collected by the camera module and the calibration data pre-stored in the processor module, the following steps are further included: Collect the imaging data on the projection module based on different resolutions, and perform downsampling processing and blurring processing on the imaging data to obtain pre-stored imaging data, where the pre-stored imaging data includes the imaging data with different resolutions and different blurring conditions; Calculate and store the gradient magnitude matrix and gradient direction matrix of the pre-stored imaging data based on the first-order finite difference; Select the target imaging data with the highest resolution in the imaging data, and determine the linear regression model between the imaging data with different resolutions under different blurring conditions and the target imaging data based on the least squares fitting algorithm; and Store the calibration parameters corresponding to the linear regression model in the processor module, and store the imaging data as the calibration data in the processor module, where each calibration data corresponds to one calibration parameter.

5. The displacement sensing method for a building structure according to claim 1, wherein, After the step of performing super-resolution reconstruction processing on the image data based on the calibration parameters to obtain the reconstructed image data, the following steps are further included: Perform interpolation processing on the reconstructed image data, perform magnification processing on the interpolated reconstructed image data, and determine all pixel coordinates of the magnified reconstructed image data; Calculate the target pixel coordinates of the magnified reconstructed image data according to the integer coordinates, decimal coordinates, and the weight ratio of the integer and the decimal of all the pixel coordinates.

6. The displacement sensing method of the building structure according to claim 1, wherein, The step of generating the displacement detection result of the structure to be measured according to the target reconstructed image data amplified by the structural vibration includes: Determine the characteristic region corresponding to the target reconstructed image data, and obtain the target image after the second frame of the reconstructed image data in the characteristic region, where the characteristic region is the imaging region of the projection module; Perform normalization processing on the target image and the first frame image of the target reconstructed image data to obtain the template image corresponding to the characteristic region and the vibration image corresponding to the first frame image; Calculate the mapping value corresponding to the pixel points of the vibration image sliding in the template image, and the mapping matrix corresponding to the mapping value; and Process all frames of the target reconstructed image data based on the position corresponding to the maximum value of the pixel points in the mapping matrix to obtain the vibration time history signal of the structure to be measured, where the processor module of the displacement sensing device can generate the displacement detection result based on the vibration time history signal.

7. The displacement sensing method of the building structure according to claim 1, wherein, The calibration data is accurate data for reference measured indoors before installing the displacement sensing device on the building structure to be measured.

8. The displacement sensing method of the building structure according to claim 1, wherein, The determination of the partial derivative array of the image data includes: Calculate the partial derivative array of the image data through the first-order finite difference to obtain the image partial derivative arrays along the x and y directions respectively.

9. The displacement sensing method of the building structure according to claim 8, wherein, The calculation of the gradient magnitude matrix and gradient direction matrix of the image data according to the partial derivative array includes: Combined with the image partial derivative arrays along the x and y directions, calculate the image gradient array. Based on the processing of the vibration image corresponding to the first frame of image, sequentially obtain the gradient magnitude matrix and gradient direction matrix of this frame of image.

10. The displacement sensing method of the building structure according to claim 2, wherein, The current resolution is the resolution corresponding to the camera module when collecting the image data of the projection module.

11. The displacement sensing method for the building structure according to claim 1, wherein, The generating the displacement detection result of the structure to be measured based on the reconstructed image data after structural vibration amplification processing includes: Performing spatial domain decomposition and time domain filtering processing on each image in the time series direction of the vibration video of the laser beam after super-resolution reconstruction; and amplifying and reconstructing the image data after time domain filtering processing.

12. The displacement sensing method of the building structure according to claim 2, wherein, The downsampling processing and blurring processing of the imaging data include: Respectively collecting the imaging data of the laser beam on the projection module based on different resolution photography conditions from low to high, and obtaining multiple sets of projection imaging data of the laser beam with different resolutions.

13. The displacement sensing method of the building structure according to claim 12, wherein, After obtaining the multiple sets of projection imaging data of the laser beam with different resolutions, the method further includes: Performing downsampling processing on each of the multiple sets of projection imaging data of the laser beam with different resolutions collected, to obtain laser beam imaging data under more resolution conditions.

14. The displacement sensing method of the building structure according to claim 13, wherein, The downsampling processing and blurring processing of the imaging data further include: using Gaussian convolution kernels with different scale factors to perform blurring processing on the laser beam imaging data with different resolutions, and expanding to obtain the laser beam imaging data with different resolutions under different blurring scales.

15. The displacement sensing method of the building structure according to claim 1, wherein, Performing structural vibration amplification processing on the reconstructed image data includes: performing multiple downsampling processes on the reconstructed image data to obtain a multi-layer image pyramid.

16. The displacement sensing method of the building structure according to claim 15, wherein, After obtaining the multi-layer image pyramid, the method further includes: based on the top layer image of the image pyramid, sequentially selecting two adjacent pyramid images in the image pyramid.

17. The displacement sensing method for the building structure according to claim 16, wherein, After sequentially selecting two adjacent pyramid images in the image pyramid based on the top layer image of the image pyramid, the method further includes: Performing upsampling processing on the matrix of the upper layer image in the two pyramid images, and subtracting the matrix of the lower layer image in the two pyramid images from the matrix of the upsampled upper layer image, to obtain a Laplacian image pyramid.

18. The displacement sensing method for the building structure according to claim 17, wherein, After obtaining the Laplacian image pyramid, the method further includes: Based on a preset filtering frequency, performing filtering processing on each pixel point of each image in the Laplacian image pyramid in the time domain, and sequentially selecting the bottom layer image of the Laplacian image pyramid after filtering processing, and performing amplification and upsampling processing on the selected bottom layer image.

19. A displacement sensing device for a building structure, wherein, The displacement sensing device for a building structure includes: a memory, a processor, and a displacement sensing program for a building structure stored on the memory and executable on the processor. When the displacement sensing program for a building structure is executed by the processor, it implements a displacement sensing method for a building structure, and the method includes: Obtaining the image data collected by the camera module, and the calibration data pre-stored in the processor module; Performing super-resolution reconstruction processing on the image data according to the calibration data to obtain reconstructed image data; and Performing structural vibration amplification processing on the reconstructed image data, and generating a displacement detection result of the structure to be measured according to the target reconstructed image data after the structural vibration amplification processing.

20. A computer-readable storage medium, wherein, A displacement sensing program of a building structure is stored on the computer-readable storage medium. When the displacement sensing program of the building structure is executed by a processor, a displacement sensing method of a building structure is implemented. The method includes: Obtaining image data collected by a camera module and calibration data pre-stored in a processor module; Performing super-resolution reconstruction processing on the image data according to the calibration data to obtain reconstructed image data; and Performing structural vibration amplification processing on the reconstructed image data, and generating a displacement detection result of the structure to be measured according to the target reconstructed image data after the structural vibration amplification processing.

Images