Building seismic displacement monitoring method and system based on super-resolution reconstruction

Abstract

The application discloses a kind of based on super-resolution reconstruction's building earthquake displacement monitoring method and system, method includes: obtaining target low-resolution earthquake monitoring video, and determine structure feature area and ground reference area;According to structure feature area and ground reference area, determine to be handled area;According to the resolution of to-be-processed area, area category and target magnification call matching super-resolution generation model, obtain super-resolution image;Building structure features and ground reference features are detected in super-resolution image respectively, obtain local pixel coordinates;Local pixel coordinates are reflected to original low-resolution video coordinate system, and obtain reflected image point coordinates;According to reflected image point coordinates, camera calibration parameter and corresponding space height constraint, calculate the earthquake displacement of building structure relative to ground;Based on earthquake displacement, the quality of the calculation result of earthquake displacement is judged, and when the determination result meets preset condition, the earthquake displacement time course is output.

Description

Technical Field

[0001] This invention belongs to the fields of structural health monitoring, computer vision, earthquake engineering, and image super-resolution reconstruction, and particularly relates to a method and system for monitoring building seismic displacement based on super-resolution reconstruction. Background Technology

[0002] While current super-resolution-based structural visual displacement measurement methods can improve the texture quality and displacement measurement accuracy of low-resolution images to some extent, they still have the following shortcomings when used for seismic displacement monitoring of outdoor buildings: (1) Effective motion features in low-resolution seismic videos are difficult to recover stably, and the accuracy and robustness of displacement measurement are still limited. Outdoor monitoring videos are affected by factors such as camera hardware resolution, monitoring distance, changes in lighting, and compression during multiple transmissions. Features such as building edges, corners, and surface textures are easily blurred or lost, leading to drift, flat segments, step-like changes, or tracking failures during feature detection and continuous tracking. Existing super-resolution methods mainly aim to improve the visual quality of images or the accuracy of general structural displacement measurement, and have not yet fully developed a dedicated processing flow for the recovery and stable tracking of small-amplitude, rapid, and continuous motion features in building seismic responses.

[0003] (2) Existing super-resolution processing methods struggle to simultaneously balance measurement accuracy, computational efficiency, and coordinate consistency. Full-frame super-resolution reconstruction simultaneously processes background areas unrelated to displacement measurement, such as the sky, distant views, and roads, resulting in significant computational and storage requirements. Under the verification conditions of this application, a 4x reconstruction of a single low-resolution full image takes approximately 60 seconds, which is insufficient for rapid processing of seismic videos. While partial Region of Interest (ROI) super-resolution schemes can reduce the processing area, they still require restoring or stitching the super-resolution ROI to the complete image before measurement, increasing image storage, interpolation, and stitching steps. For fixed or dynamic ROIs, existing schemes rarely systematically consider consistent transformations between local coordinates, cropping origin, super-resolution magnification, original video coordinates, and camera calibration coordinates, easily introducing additional coordinate errors or spurious displacements due to ROI position changes.

[0004] (3) Existing methods do not adequately consider the co-motion of cameras in outdoor earthquake scenarios. Under urban outdoor monitoring conditions, cameras are usually installed on adjacent buildings, towers, or other carriers. When an earthquake occurs, the camera and its mounting carrier may move simultaneously with the structure under test. If only the structural features of the building under test are tracked, the obtained pixel motion may simultaneously include components of the actual structural motion and the co-motion of the camera. Existing super-resolution structural displacement measurement schemes are mostly aimed at single structural targets, artificial targets, or controlled test environments. They have not yet formed a complete technical chain that simultaneously acquires the structural features of the building and the ground reference features under the same imaging model and calculates the seismic displacement of the building relative to the ground through the relative motion of the two.

[0005] (4) Scene-specific model training, local loss adaptation, and processing quality closed-loop are still imperfect. There may be domain differences between general super-resolution training data and specific monitoring perspectives, building textures, imaging distances, and video compression degradation methods. Directly using a general model can easily generate details that are inconsistent with the real structure and texture, while large-scale collection of scene-specific high and low resolution samples will increase engineering costs. In addition, after directly transferring the loss weights used in the training of the complete image to a small-sized ROI, the magnitude of each loss term may change, resulting in insufficient regularization constraints or pixel consistency constraints, which in turn produces noise, artifacts, and color shifts, affecting subsequent feature tracking. Existing solutions also rarely incorporate information such as the number of effective features, tracking errors, feature out-of-bounds, ROI range, processing time, and model version into a unified quality control and result traceability mechanism, making it difficult to dynamically adjust the image region according to the processing quality and verify the complete coordinate chain from image reconstruction to physical displacement output. Summary of the Invention

[0006] To address the aforementioned technical issues, this invention proposes a method and system for monitoring building seismic displacement based on super-resolution reconstruction. By performing joint structure-ground ROI local reconstruction, loss function rebalancing, and local coordinate mapping, the accuracy of displacement identification and processing efficiency are improved.

[0007] To achieve the above objectives, the present invention provides a method for monitoring building seismic displacement based on super-resolution reconstruction, comprising: Acquire low-resolution seismic monitoring video of the target and determine the structural feature area and ground reference area within the same field of view of the monitoring video; The area to be processed is determined based on the structural feature area and the ground reference area; Based on the resolution, region category, and target magnification of the region to be processed, a matching super-resolution generation model is invoked to perform super-resolution reconstruction on the region to be processed, thereby obtaining a super-resolution image; In the super-resolution image, building structural features and ground reference features are detected respectively, and the local pixel coordinates of the building structural features and ground reference features in the super-resolution image are obtained. Based on the local pixel coordinates, the actual cropping origin of the area to be processed in the original low-resolution video, and the target magnification, the local pixel coordinates are mapped back to the original low-resolution video coordinate system to obtain the mapping point coordinates. Based on the coordinates of the echo point, camera calibration parameters, and corresponding spatial height constraints, the seismic displacement of the building structure relative to the ground is calculated. Based on the building structure characteristics and ground reference characteristics used to calculate the seismic displacement, the quality of the calculated seismic displacement is judged, and the seismic displacement time history of the building structure relative to the ground is output when the judgment result meets the preset conditions.

[0008] Optionally, the following steps may be taken before acquiring the target low-resolution seismic monitoring video: Obtain high-resolution representative images and low-resolution representative images; Geometric registration is performed on the high-resolution representative image and the low-resolution representative image, and the training region is simultaneously cropped from the registered image. High-resolution and low-resolution paired training samples are constructed by synchronous rotation, mirroring and sliding window sampling with sliding window step size set according to the target magnification. The super-resolution generation model is trained using the paired training samples, and then trained using adversarial training to obtain the trained super-resolution generation model.

[0009] Optionally, training a super-resolution generation model using the paired training samples and then training the super-resolution generation model using adversarial training to obtain a trained super-resolution generation model includes: Based on the paired training samples, a super-resolution generative adversarial network containing a generator and a discriminator is constructed. The generator is first pre-trained with pixel consistency loss, and then adversarial training is performed by alternating updates of the generator and discriminator to obtain a trained super-resolution generative model.

[0010] Optionally, the region to be processed includes a complete video frame, a continuous joint region of interest that simultaneously covers the structural feature region and the ground reference region, or multiple separate regions of interest that respectively correspond to the structural feature region and the ground reference region; Based on the distribution of effective features in the structural feature region and the ground reference region, the expected seismic displacement range, the edge allowance, the computing power of the equipment, and the processing time limit, the size and location of the region to be processed are determined, and the actual clipping origin and region size are recorded frame by frame when the region of interest method is used.

[0011] Optionally, detecting building structure features and ground reference features in the super-resolution image, and obtaining the local pixel coordinates of the building structure features and ground reference features in the super-resolution image includes: In the super-resolution image, building edge intersections, window corners, exterior wall textures, ground markings, or stable gray-scale gradient features are detected as building structural features and ground reference features, and corner detection and pyramid optical flow are used for frame-by-frame subpixel tracking. Feature points that are out of bounds, occluded, have excessive forward and backward tracking errors, insufficient texture gradients, are continuously lost, or are outliers of the same type of displacement are removed. The local pixel coordinates of the selected building structure features and ground reference features are recorded respectively.

[0012] Optionally, obtaining the echo point coordinates includes: The local pixel coordinates are scaled according to the target magnification factor, and then superimposed with the actual cropping origin of the region to be processed in the original low-resolution video to map the local pixel coordinates back to the original low-resolution video coordinate system and obtain the mapping point coordinates. When a fixed region of interest is used, the cropping origin remains unchanged, and when a dynamic region of interest is used, the cropping origin updated for each frame is used.

[0013] Optionally, the seismic displacement of the building structure relative to the ground is calculated based on the echo point coordinates, camera calibration parameters, and corresponding spatial height constraints, including: Substitute the back projection point coordinates, camera projection matrix, and corresponding height constraints into the back projection relationship to obtain the physical coordinates of the building structural features and ground reference features, wherein the height of the ground reference features is zero, and the height of the structural features is the height value determined by on-site measurement or design data. Robust aggregation of the physical coordinates of multiple building structural features or multiple ground reference features is performed to obtain the aggregated location of the building structural features and the aggregated location of the ground reference features. The seismic displacement of the building structure relative to the ground is calculated based on the difference between the aggregated location of the building structure features and the aggregated location of the ground reference features relative to the initial time.

[0014] Optionally, a quality assessment is performed based on the building structure characteristics and ground reference characteristics, and the seismic displacement time history of the building structure relative to the ground is output when the assessment result meets preset conditions, including: The quality score is calculated based on the effective number of building structural features and ground reference features, forward and backward tracking errors, distance to the boundary, number of consecutive lost frames, dispersion of similar displacements, and single-frame processing time, respectively. When both the building structural features and the ground reference features meet the quality threshold, the seismic displacement time history is output, and the video time information, model version, magnification, actual cropping origin, number of effective features, tracking error and processing time are associated and saved. If any type of feature fails to meet the quality threshold, perform feature re-detection, region of interest adjustment, or missing detection labeling.

[0015] The present invention also provides a building seismic displacement monitoring system based on super-resolution reconstruction, comprising: The video acquisition module is used to acquire low-resolution seismic monitoring videos of the target and determine the structural feature area and the ground reference area within the same field of view of the monitoring video. The region to be processed determination module is used to determine the region to be processed based on the structural feature region and the ground reference region; The super-resolution reconstruction module is used to call a matching super-resolution generation model based on the resolution, region category and target magnification of the region to be processed, and to perform super-resolution reconstruction on the region to be processed to obtain a super-resolution image. The feature tracking module is used to detect building structure features and ground reference features in the super-resolution image, and obtain the local pixel coordinates of the building structure features and ground reference features in the super-resolution image. The coordinate mapping module is used to map the local pixel coordinates back to the original low-resolution video coordinate system based on the local pixel coordinates, the actual cropping origin of the area to be processed in the original low-resolution video, and the target magnification, and obtain the mapping point coordinates. The displacement calculation module is used to calculate the seismic displacement of the building structure relative to the ground based on the coordinates of the echo point, camera calibration parameters, and corresponding spatial height constraints. The quality assessment module assesses the quality of the earthquake displacement calculation results based on the building structure characteristics and ground reference characteristics used to calculate the earthquake displacement, and outputs the earthquake displacement time history of the building structure relative to the ground when the assessment results meet preset conditions.

[0016] Optionally, the system further includes: The representative image acquisition module is used to acquire high-resolution and low-resolution representative images. The sample construction module is used to perform geometric registration on the high-resolution representative image and the low-resolution representative image, and synchronously extract the training region in the registered image. Through synchronous rotation, mirroring and sliding window sampling with a sliding window step size set according to the target magnification, high-resolution and low-resolution paired training samples are constructed. The model training module is used to train the super-resolution generation model using the paired training samples, and to train the super-resolution generation model using alternating adversarial training to obtain the trained super-resolution generation model.

[0017] Compared with the prior art, the present invention has the following advantages and technical effects: (1) Improve the accuracy of seismic displacement identification in low-resolution videos.

[0018] By using super-resolution generative models to recover high-frequency information such as structural edges, corners, and surface textures that are missing or weakened in low-resolution videos, drift, flat segments, and step-like distortions that occur during feature tracking can be reduced, improving the consistency between building seismic displacement measurements and the actual response. In shaking table experiments, under the condition of PGA of 0.2g and tracking natural features, the RMSE of the displacement results decreased from 1.2904mm to 0.1630mm, the correlation coefficient increased from 0.7435 to 0.9791, and R² increased from -1.7140 to 0.9567.

[0019] (2) Reduce the computational cost of super-resolution reconstruction.

[0020] This invention can perform super-resolution reconstruction on complete video frames, or preferably only process local ROIs containing structural features and ground reference features, avoiding redundant calculations for irrelevant background areas such as the sky, distant views, and roads. In the verification environment, single-frame full... Figure 4 The average time for super-resolution reconstruction is about 60 seconds, while the average time for ROI local reconstruction is about 1.4 seconds, which is nearly 43 times faster.

[0021] (3) Improve the reliability of building-to-ground seismic displacement measurement.

[0022] This invention simultaneously tracks building structural features and ground reference features within the same surveillance video. By calculating the seismic displacement of the building structure relative to the ground through their relative motion, it reduces the impact of the camera's movement along with the installed building. For ROI local reconstruction, the local feature coordinates can be directly mapped back to the original video coordinate system based on the ROI cropping origin and super-resolution magnification, eliminating the need to re-stitch the ROI into a complete super-resolution image and avoiding false displacements caused by dynamic ROI movement.

[0023] (4) It has good training feasibility, stability and scalability.

[0024] This invention utilizes a small number of high- and low-resolution representative images with approximately the same viewpoint to construct a large number of paired training sub-blocks through registration, synchronized rotation, mirroring, and sliding window sampling, reducing reliance on large-scale field data and manual annotation. To address the limited pixel count of ROIs, the weights of regularization and pixel consistency terms are adjusted to reduce noise, artifacts, and color shifts in local reconstruction. Furthermore, this invention is applicable to complete video frames, single ROIs, continuous joint ROIs, or multiple separate ROIs, and can be extended to employ different super-resolution models and visual feature tracking methods. Attached Figure Description

[0025] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings: Figure 1 This is a flowchart of a building seismic displacement monitoring method based on super-resolution reconstruction according to an embodiment of the present invention; Figure 2 This is a structural diagram of a building seismic displacement monitoring system based on super-resolution reconstruction according to an embodiment of the present invention; Figure 3 This is a schematic diagram illustrating the training principle of the SRGAN network used in the embodiments of the present invention; Figure 4 This is a schematic diagram of the generator network architecture used in the embodiments of the present invention; Figure 5 This is a schematic diagram of the discriminator network architecture used in the embodiments of the present invention; Figure 6 This is a schematic diagram of geometric transformations of training image rotation and mirroring according to an embodiment of the present invention, wherein (a) is the original image, (b) is the mirror image of the original image, (c) is the original image rotated by 90°, (d) is the original image rotated by 90° and mirrored, (e) is the original image rotated by 180°, (f) is the original image rotated by 180° and mirrored, (g) is the original image rotated by 270°, and (h) is the original image rotated by 270° and mirrored. Figure 7 This is a schematic diagram of synchronous sliding window sampling of high-resolution and low-resolution images according to an embodiment of the present invention; Figure 8 This is a schematic diagram of the simulated monitoring video and region of interest configuration in an embodiment of the present invention, wherein (a) is a schematic diagram of the simulated working condition and (b) is a schematic diagram of the region of interest capture; Figure 9 This is a comparison chart of displacement monitoring results before and after the 360p full-screen super-resolution reconstruction to 2k in an embodiment of the present invention; Figure 10 This is a comparison image of displacement monitoring results before and after the 360p region of interest super-resolution reconstruction to a 2k region of interest according to an embodiment of the present invention; Figure 11 This is a schematic diagram of the region of interest segmentation from a low-resolution small vibration table experimental video according to an embodiment of the present invention. Figure 12 This is a comparison chart of displacement monitoring results of natural features from a small shaking table experiment video when the peak ground acceleration is 0.2g, according to an embodiment of the present invention. Detailed Implementation

[0026] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0027] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0028] This embodiment proposes a method for monitoring building seismic displacement based on super-resolution reconstruction, such as... Figure 1 As shown, the specific steps include: Acquire low-resolution seismic monitoring video of the target and determine the structural feature area and ground reference area within the same field of view of the monitoring video; The area to be processed is determined based on the structural feature area and the ground reference area; Based on the resolution, region category, and target magnification of the region to be processed, a matching super-resolution generation model is invoked to perform super-resolution reconstruction on the region to be processed, thereby obtaining a super-resolution image; In the super-resolution image, building structural features and ground reference features are detected respectively, and the local pixel coordinates of the building structural features and ground reference features in the super-resolution image are obtained. Based on the local pixel coordinates, the actual cropping origin of the area to be processed in the original low-resolution video, and the target magnification, the local pixel coordinates are mapped back to the original low-resolution video coordinate system to obtain the mapping point coordinates. Based on the coordinates of the echo point, camera calibration parameters, and corresponding spatial height constraints, the seismic displacement of the building structure relative to the ground is calculated. Based on the building structure characteristics and ground reference characteristics used to calculate the seismic displacement, the quality of the calculated seismic displacement is judged, and the seismic displacement time history of the building structure relative to the ground is output when the judgment result meets the preset conditions.

[0029] Specifically, (1) Video and task area determination. Acquire target low-resolution seismic monitoring video, determine the structural feature area used to characterize the movement of the building structure and the ground reference area used to characterize the joint movement of the camera or mounting carrier within the same video field of view, and set the area to be processed as a complete video frame, a structure-ground continuous joint ROI, or multiple ROIs corresponding to structural features and ground reference features respectively.

[0030] (2) Super-resolution reconstruction of task-related regions. Based on the video resolution, the category of the region to be processed, and the target magnification, a matching super-resolution generation model is invoked to perform super-resolution reconstruction on the complete video frame or ROI. When using the ROI method, the generated super-resolution ROI is directly used for subsequent feature detection and tracking, without requiring it to be restored or stitched into a complete super-resolution video frame.

[0031] (3) Cooperative detection and tracking of structural and ground features. Architectural structural features and ground reference features are detected in the reconstructed image region. The local pixel coordinates of the two types of features in the super-resolution image or super-resolution ROI are obtained frame by frame. Features that are out of bounds, occluded, have excessive tracking errors, insufficient texture, or abnormal displacement are removed.

[0032] (4) Local coordinate mapping. Based on the actual cropping origin and super-resolution magnification of the ROI in the original low-resolution complete video frame, the local feature coordinates in the super-resolution ROI are transformed to the original low-resolution complete video coordinate system, so that the changes in the position or size of the ROI are not included in the actual movement of the building structure.

[0033] (5) Calculation of building displacement relative to the ground. The reflected structural feature image points and ground reference feature image points are combined with the camera calibration parameters and corresponding spatial constraints to obtain the physical coordinate changes of the structural features and ground reference features. The seismic displacement of the building structure relative to the ground is then calculated based on the relative difference between the structural motion and the ground reference motion.

[0034] (6) Quality judgment and result output. The quality judgment is based on the effective number of structural and ground features, tracking error, boundary distance, number of consecutive lost frames, and processing time. When both types of features meet the quality conditions, the displacement time history and quality information are output. When either type of feature does not meet the quality conditions, feature re-detection, ROI adjustment, or missing measurement marking is performed.

[0035] The above six technical features together form a closed loop from low-resolution seismic video input to building displacement relative to the ground output. Among them, "structure-ground dual-class features", "ROI super-resolution direct tracking", "cropping origin and magnification jointly participate in coordinate mapping" and "subtraction of structural motion and ground reference motion" constitute the combination relationship that should be retained in the independent claims of the method.

[0036] Furthermore, prior to acquiring low-resolution seismic monitoring video of the target, the following steps are included: Obtain high-resolution representative images and low-resolution representative images; Geometric registration is performed on the high-resolution representative image and the low-resolution representative image, and the training region is simultaneously cropped from the registered image. High-resolution and low-resolution paired training samples are constructed by synchronous rotation, mirroring and sliding window sampling with sliding window step size set according to the target magnification. The super-resolution generation model is trained using the paired training samples, and then trained using adversarial training to obtain the trained super-resolution generation model.

[0037] Specifically, S1, acquire high-resolution and low-resolution representative images. Acquire at least one high-resolution (HR) representative image and at least one low-resolution (LR) representative image from a perspective similar to that of the target LR surveillance video. The HR image provides high-frequency details of building edges, corners, surface textures, and ground reference textures, and can be obtained from a high-resolution camera, mobile device, or UAV imaging system. The LR image characterizes the resolution, blur, and compression degradation features of the target video and can be obtained directly from the target surveillance camera or from the HR image through downsampling and compression degradation. The two types of images do not need to be acquired from the same device or in the exact same pose, but their perspective difference should be within the registration range and should at least cover the structural feature area; when it is necessary to measure the displacement of the structure relative to the ground, the ground reference feature area should also be covered.

[0038] S2. Perform geometric registration on the representative images and determine the training region. Select common reference points, natural feature points, or artificial markers from the HR and LR representative images, and complete the registration using homography transformation, affine transformation, feature matching, or camera calibration mapping. If the structure and ground regions belong to different spatial planes or a single homography relationship is insufficient, perform separate regional registration. Calculate the reference point residuals after registration; if the residuals exceed a preset threshold, reselect points, switch to regional registration, or reacquire the image. Depending on the subsequent processing method, retain the complete registered image, or simultaneously crop the structural ROI, ground ROI, or continuous joint ROI.

[0039] S3, construct HR-LR paired training samples. Simultaneously perform 0°, 90°, 180°, and 270° rotations and corresponding mirroring on the registered complete image or ROI, forming 8 pairs of paired enhanced images. Let the magnification ratio be... The length, width, and sliding window step size of the HR training sub-block are respectively taken from the corresponding LR sub-block. To ensure that each pair of samples represents the same physical region, the ratio is increased by 1:1. When reconstructing by 4x, 256×256 pixel HR sub-blocks and 64×64 pixel LR sub-blocks are preferred. The training set and validation set can be divided in an 8:2 ratio. Related samples formed by geometric transformation of the same original location are preferably assigned to the same set to avoid leakage of data from the same source.

[0040] S4. Construct and pre-train a super-resolution generation model. Construct an SRGAN consisting of a generator and a discriminator. The generator sequentially includes shallow feature extraction, deep residual learning, pixelShuffle upsampling, and image reconstruction output; preferably, it uses 9×9 shallow convolutions, 16 residual blocks, two sets of pixelShuffle modules (each magnified by 2x), and a 9×9 output convolution. The discriminator distinguishes between real HR images and generated SR images through multi-level convolutions and classification units. The generator mapping relationship is shown in the following equation: (1); Further, the super-resolution generation model is trained using the paired training samples, and then trained using adversarial training to obtain the trained super-resolution generation model, including: Based on the paired training samples, a super-resolution generative adversarial network containing a generator and a discriminator is constructed. The generator is first pre-trained with pixel consistency loss, and then adversarial training is performed by alternating updates of the generator and discriminator to obtain a trained super-resolution generative model.

[0041] Specifically, in S5, alternating adversarial training is performed and the generator model is solidified. Pre-trained parameters are loaded, and the generator and discriminator are updated alternately. The adversarial objective is shown in the equation: (2); The full image mode and ROI mode adopt the following formula: (3); Composite loss; considering changes in ROI sample pixel count, texture density, and background ratio, the ROI mode correspondingly increases the weights of the regularization and pixel consistency terms to suppress local high-frequency reconstruction noise, artifacts, and color shifts. After training reaches the preset number of epochs or the validation set quality requirements, the generator parameters are saved, along with the applicable viewpoint, input resolution, magnification, region category, training sub-block size, loss configuration, number of training epochs, learning rate, and model version; the discriminator is only used for offline training.

[0042] Furthermore, the region to be processed includes a complete video frame, a continuous joint region of interest that simultaneously covers the structural feature region and the ground reference region, or multiple separate regions of interest that respectively correspond to the structural feature region and the ground reference region; Based on the distribution of effective features in the structural feature region and the ground reference region, the expected seismic displacement range, the edge allowance, the computing power of the equipment, and the processing time limit, the size and location of the region to be processed are determined, and the actual clipping origin and region size are recorded frame by frame when the region of interest method is used.

[0043] Specifically, in S6, camera calibration is completed and low-resolution seismic video is integrated. Lens distortion correction is performed on the monitoring camera, and the LR seismic video to be processed is integrated. Preferably, at least six spatially non-coplanar control points are selected to obtain their three-dimensional physical coordinates and corresponding two-dimensional pixel coordinates. The camera projection matrix is ​​obtained using the direct linear transformation method and singular value decomposition. Non-coplanar arrangement is used to avoid equation configuration degradation. After video integration, resolution, frame rate, frame number, total number of frames, and timestamp are recorded. When camera position, focal length, or lens status changes, the calibration or mapping relationship is re-established.

[0044] S7, determine the video region to be processed. Based on the spatial distribution of structural and ground features, select to process a complete frame, a continuous joint ROI, or a structure-ground separated ROI. It is preferred to use a fixed ROI, and set reserved boundaries according to the initial feature position, expected seismic displacement range, optical flow search window, and edge safety distance; the cropping origin and size of the fixed ROI remain unchanged in each frame. Optionally, the ROI is translated, expanded, or re-initialized according to the effective feature position of the previous frame. At this time, the actual cropping origin and region size are recorded frame by frame. The change of ROI only changes the local input range of the generator, does not change the original LR complete frame coordinate system, and preferably satisfies the pixel budget and processing time constraints of Equation (4): (4); S8 performs super-resolution inference on complete frames or ROIs. It calls the matching generator based on video resolution, region category, and target magnification. For the complete frame method, it outputs a complete SR image frame by frame; for the ROI method, it first extracts structural ROIs and ground ROIs from the complete LR frame, then inputs them separately or jointly into the generator to obtain the SR-ROI. The local reconstruction results are directly used for feature detection and tracking, eliminating the need to stitch back to a complete SR frame, thus reducing irrelevant background reconstruction, complete image storage, and secondary interpolation; its speed improvement compared to the full-image method can be calculated using the following formula. (5); Further, detecting building structure features and ground reference features in the super-resolution image, and obtaining the local pixel coordinates of the building structure features and ground reference features in the super-resolution image includes: In the super-resolution image, building edge intersections, window corners, exterior wall textures, ground markings, or stable gray-scale gradient features are detected as building structural features and ground reference features, and corner detection and pyramid optical flow are used for frame-by-frame subpixel tracking. Feature points that are out of bounds, occluded, have excessive forward and backward tracking errors, insufficient texture gradients, are continuously lost, or are outliers of the same type of displacement are removed. The local pixel coordinates of the selected building structure features and ground reference features are recorded respectively.

[0045] Specifically, S9 detects and tracks building structural features and ground reference features. In the initial SR image, it detects building edge intersections, window corners, exterior wall textures, ground markings, or other stable gray-level gradient features, preferably using Shi-Tomasi corner detection and pyramid LK optical flow for frame-by-frame sub-pixel tracking. A multi-scale pyramid transforms larger displacements in the original image into smaller motions in lower-resolution layers, which are then corrected from coarse to fine. Feature points that are out of bounds, occluded, have excessive forward or backward errors, insufficient texture gradients, are continuously lost, or are significantly outliers of the same type of displacement are removed. The local SR image point time histories of structural and ground features are recorded separately.

[0046] Furthermore, obtaining the coordinates of the echo point includes: The local pixel coordinates are scaled according to the target magnification factor, and then superimposed with the actual cropping origin of the region to be processed in the original low-resolution video to map the local pixel coordinates back to the original low-resolution video coordinate system and obtain the mapping point coordinates. When a fixed region of interest is used, the cropping origin remains unchanged, and when a dynamic region of interest is used, the cropping origin updated for each frame is used.

[0047] Specifically, in S10, the local super-resolution coordinates are mapped back to the original full-frame coordinates. For the full-frame SR coordinates, the magnification is adjusted accordingly. Scale to the original LR coordinates; for the local coordinates of the SR-ROI, first press... Scaling, and then overlaying the actual cropping origin of the frame's ROI in the original full LR frame, as shown in the formula: (6); When using a fixed ROI, the clipping origin remains unchanged; when using a dynamic ROI, the clipping origin updated in each frame is used to avoid mistaking the movement of the ROI window for structural movement.

[0048] Furthermore, based on the echo point coordinates, camera calibration parameters, and corresponding spatial height constraints, the seismic displacement of the building structure relative to the ground is calculated, including: Substitute the back projection point coordinates, camera projection matrix, and corresponding height constraints into the back projection relationship to obtain the physical coordinates of the building structural features and ground reference features, wherein the height of the ground reference features is zero, and the height of the structural features is the height value determined by on-site measurement or design data. Robust aggregation of the physical coordinates of multiple building structural features or multiple ground reference features is performed to obtain the aggregated location of the building structural features and the aggregated location of the ground reference features. The seismic displacement of the building structure relative to the ground is calculated based on the difference between the aggregated location of the building structure features and the aggregated location of the ground reference features relative to the initial time.

[0049] Specifically, S11 calculates the physical coordinates and the seismic displacement of the building relative to the ground. The mapped 2D image points, camera projection matrix, and corresponding height constraints are substituted into the back-projection relationship. The ground reference feature height is taken as... The structural feature height is determined by on-site measurements or design data. ; Obtain the robust aggregate value of physical coordinates of the same type according to formula (7), and then calculate the structure-ground relative displacement relative to the initial time according to formula (8).

[0050] (7); (8); Multiple similar features can be aggregated using the mean, median, truncated average, or weighted by tracking residuals; in controlled experiments, calibrated pixel-physical ratios or planar homography can also be used instead of three-dimensional back projection.

[0051] Furthermore, based on the building structure characteristics and ground reference characteristics, a quality assessment is performed, and when the assessment result meets preset conditions, the seismic displacement time history of the building structure relative to the ground is output, including: The quality score is calculated based on the effective number of building structural features and ground reference features, forward and backward tracking errors, distance to the boundary, number of consecutive lost frames, dispersion of similar displacements, and single-frame processing time, respectively. When both the building structural features and the ground reference features meet the quality threshold, the seismic displacement time history is output, and the video time information, model version, magnification, actual cropping origin, number of effective features, tracking error and processing time are associated and saved. If any type of feature fails to meet the quality threshold, perform feature re-detection, region of interest adjustment, or missing detection labeling.

[0052] Specifically, in S12, a quality assessment is performed and the monitoring results are output. The effective number of structural and ground features, forward and backward tracking errors, boundary distances, number of consecutively lost frames, dispersion of similar displacements, reconstruction quality, and single-frame processing time are evaluated respectively. The quality score can be calculated using equation (9). (9); When both quality categories meet the threshold, the system outputs the displacement time history, peak displacement, main frequencies, or other response indicators of the building structure relative to the ground, and saves video time information, model version, magnification, ROI cropping origin and size, number of effective features, tracking error, and processing time. If either quality category fails to meet the requirements, features are re-detected, the ROI is adjusted, or a missing measurement marker is set. Dynamic ROI is an optional method and does not constitute a necessary condition for the main implementation of this invention.

[0053] This invention also provides a building seismic displacement monitoring system based on super-resolution reconstruction, such as... Figure 2 As shown, it includes: The video acquisition module is used to acquire low-resolution seismic monitoring videos of the target and determine the structural feature area and the ground reference area within the same field of view of the monitoring video. The region to be processed determination module is used to determine the region to be processed based on the structural feature region and the ground reference region; The super-resolution reconstruction module is used to call a matching super-resolution generation model based on the resolution, region category and target magnification of the region to be processed, and to perform super-resolution reconstruction on the region to be processed to obtain a super-resolution image. The feature tracking module is used to detect building structure features and ground reference features in the super-resolution image, and obtain the local pixel coordinates of the building structure features and ground reference features in the super-resolution image. The coordinate mapping module is used to map the local pixel coordinates back to the original low-resolution video coordinate system based on the local pixel coordinates, the actual cropping origin of the area to be processed in the original low-resolution video, and the target magnification, and obtain the mapping point coordinates. The displacement calculation module is used to calculate the seismic displacement of the building structure relative to the ground based on the coordinates of the echo point, camera calibration parameters, and corresponding spatial height constraints. The quality assessment module assesses the quality of the earthquake displacement calculation results based on the building structure characteristics and ground reference characteristics used to calculate the earthquake displacement, and outputs the earthquake displacement time history of the building structure relative to the ground when the assessment results meet preset conditions.

[0054] Furthermore, the system also includes: The representative image acquisition module is used to acquire high-resolution and low-resolution representative images. The sample construction module is used to perform geometric registration on the high-resolution representative image and the low-resolution representative image, and synchronously extract the training region in the registered image. Through synchronous rotation, mirroring and sliding window sampling with a sliding window step size set according to the target magnification, high-resolution and low-resolution paired training samples are constructed. The model training module is used to train the super-resolution generation model using the paired training samples, and to train the super-resolution generation model using alternating adversarial training to obtain the trained super-resolution generation model.

[0055] Specifically, (1) represents the image acquisition, registration, and sample construction module. It corresponds to S1 to S3. This module acquires HR and LR representative images with approximately the same viewpoint, completes the selection of common reference points, geometric registration, and registration residual checks; then, it determines the training area according to the complete image, structure ROI, ground ROI, or structure-ground joint ROI, and constructs spatially strictly corresponding HR-LR training sub-blocks through synchronous rotation, mirroring, and sliding window sampling.

[0056] (2) Model Training and Management Module. Corresponding to S4 and S5. This module constructs an SRGAN consisting of a generator and a discriminator. First, the generator is pre-trained with pixel consistency loss, and then the generator and discriminator are trained adversarially. After training, the generator weights, applicable viewpoints, resolutions, region categories, magnification, loss configurations, and version information are saved; the discriminator does not enter the video inference stage.

[0057] (3) Video calibration and region management module. Corresponding to S6 and S7. This module completes lens distortion correction, camera projection matrix calibration, and LR seismic video access, and determines the complete frame or ROI based on the location of structural and ground features, expected displacement range, edge allowance, and computing resources. When using dynamic ROI, the actual cropping origin and region size are recorded frame by frame to ensure that changes in local windows do not alter the original video coordinate system.

[0058] (4) Super-resolution reconstruction and feature tracking module. Corresponds to S8 and S9. This module calls a generator that matches the resolution, region category and magnification to reconstruct the entire frame or local ROI; then it detects building structure features and ground reference features, and preferably uses pyramid LK optical flow for continuous tracking and anomalous feature removal.

[0059] (5) Coordinate mapping and displacement calculation module. Corresponding to S10 and S11. This module maps the local SR coordinates back to the original LR complete frame coordinates according to the super-resolution magnification and the ROI clipping origin of the current frame. Then, it obtains the physical coordinates by combining the camera projection matrix and feature height constraints. After robust aggregation of similar features, it calculates the seismic displacement of the building structure relative to the ground.

[0060] (6) Quality evaluation and result output module. Corresponds to S12. This module evaluates the quality of the results by comprehensively considering the number of effective features, tracking residuals, boundary distances, number of consecutive lost frames, and processing time; it performs feature re-detection, ROI adjustment, or missing measurement marking for unqualified moments, and outputs displacement time history and corresponding quality and traceability information for qualified moments.

[0061] The following is a detailed description of this embodiment: Example 1: Full-image and ROI super-resolution reconstruction under simulated surveillance video: (1) Simulation scene and video configuration: An outdoor seismic monitoring simulation scenario was constructed, capable of simultaneously observing key components of the building under monitoring and a ground reference area, ensuring that the monitoring image simultaneously includes both building structural features and ground reference features. Seismic actions with peak ground acceleration (PGA) of 0.2g, 0.4g, and 0.6g were applied, and low-resolution videos with resolutions of 960×540 pixels, 720×480 pixels, and 640×360 pixels were output, hereinafter referred to as 540p, 480p, and 360p videos, respectively. The simulation conditions under different PGA and video resolutions are as follows: Figure 8 As shown in (a).

[0062] During the simulation, the displacement time history of the building structure relative to the ground was obtained in advance and used as the true value for comparison with the visual monitoring results. The frame rate, camera parameters, structural texture, and ground texture of the low-resolution video were kept consistent in the same set of comparative experiments.

[0063] For full-image super-resolution processing, the complete low-resolution video frame is used as the image region to be processed.

[0064] For ROI local super-resolution processing, joint ROIs that simultaneously cover structural features and ground reference features are extracted from videos at three different resolutions. The extraction locations of the ROIs in the videos at the three resolutions are as follows: Figure 8 As shown in (b), with the top left corner of the original video as the origin of the coordinate system, the ROI example configuration is as follows: The top-left corner of the ROI for the 540p video is (100, 20), with a horizontal width of 512 pixels and a vertical height of 128 pixels. The top-left corner coordinates of the ROI of the 480p video are (88, 18), the horizontal width is 424 pixels, and the vertical height is 113 pixels; The top-left corner of the ROI of the 360p video is (60, 0), with a horizontal width of 360 pixels and a vertical height of 132 pixels.

[0065] The above-described ROI location and dimensions are for illustrative purposes only and do not constitute a limitation on the shape, location, or area of ​​the ROI. In practical applications, the ROI can be determined based on the location of structural features, ground reference location, expected displacement range, and processing time.

[0066] (2) Training sample construction: High-resolution and low-resolution representative images similar to the three low-resolution video perspectives are acquired respectively, and geometric registration of the high- and low-resolution representative images is performed through a common reference point and homography transformation.

[0067] For the full-image super-resolution model, 0°, 90°, 180°, and 270° rotations and corresponding mirroring operations are simultaneously performed on the registered high- and low-resolution images, forming 8 pairs of specific geometric transformations for image enhancement, as follows: Figure 6 As shown in (a)-(h). Then, a synchronous sliding window method is used as follows: Figure 7 As shown, 256×256 pixel sub-blocks are cropped from high-resolution images, and 64×64 pixel sub-blocks are cropped from corresponding low-resolution images to construct training sample sets for 540p, 480p, and 360p videos, respectively. The example sample counts for the three full-image models are 3840 pairs, 4032 pairs, and 3480 pairs, respectively.

[0068] For the ROI local model, the structure corresponding to the video ROI—the ground joint ROI—is first extracted from the registered high- and low-resolution images, and then rotation, mirroring, and sliding window sampling are performed simultaneously. The number of high- and low-resolution samples for the three ROI models are 4104 pairs, 3240 pairs, and 3247 pairs, respectively.

[0069] The training and validation samples were divided into groups in an 8:2 ratio. The positions and step sizes of the high- and low-resolution sliding windows were set according to the super-resolution magnification to ensure that each pair of high- and low-resolution sub-blocks represented the same physical region.

[0070] (3) Model training configuration: This embodiment trains a full-image SRGAN model and a ROI-SRGAN model, each magnified by 4 times in both width and height. The training principle of the SRGAN model is as follows: Figure 3 As shown, it includes a generator and a discriminator, where the generator is used to reconstruct a low-resolution image into a super-resolution image, such as... Figure 4 As shown, the discriminator is used during the training phase to distinguish between real high-resolution images and generated images, such as... Figure 5 As shown.

[0071] During training, the generator is first pre-trained for 100 epochs with pixel mean squared error loss and an example pre-training learning rate of 0.0001. Then, the pre-trained weights are loaded and adversarial training is performed by alternating updates of the generator and discriminator.

[0072] Example of adversarial training parameters for the full-graph model: Batch Size is 16, generator learning rate is 0.001, discriminator learning rate is 0.0001, and the number of training epochs for 540p, 480p and 360p videos are 43, 50 and 36 respectively.

[0073] Example of adversarial training parameters for the ROI model: Batch Size is 16, generator learning rate is 0.0001, discriminator learning rate is 0.00005, and the number of training epochs corresponding to the three resolutions are 139, 152 and 185 respectively.

[0074] To address the issue of the change in the magnitude of the loss term due to the reduction in the number of ROI pixels, the ROI model adopts the composite loss rebalancing method shown in Equation (3) to increase the weight of the regularization term and the pixel consistency term, so as to reduce noise, artifacts and color shift in local reconstruction.

[0075] After training, the generator parameters, magnification, applicable video resolution, training region categories, loss parameters, and model version are saved. The discriminator is mainly used during the training phase, and the trained generator is called during the video processing phase.

[0076] (4) Super-resolution reconstruction and displacement processing: For the full-image processing method, each low-resolution video frame is analyzed, and the complete video frame is input into the corresponding full-image SRGAN generator to obtain a complete super-resolution image magnified by 4 times in both width and height. The feature coordinates in the super-resolution image are then transformed back to the original low-resolution video coordinate system according to the magnification ratio.

[0077] For ROI processing, low-resolution ROIs are cropped according to the ROI boundaries recorded in each frame and input into the corresponding ROI-SRGAN generator to obtain super-resolution ROIs. There is no need to re-stitch the super-resolution ROIs into the complete image; structural features and ground reference features are directly detected and tracked within the super-resolution ROIs.

[0078] The local image point time histories of structural features and ground reference features are obtained using the pyramid LK optical flow method, and feature points that are out of bounds, occluded, have excessive forward and backward tracking errors, or have insufficient texture quality are removed. According to Equation (6), the local coordinates are mapped back to the original low-resolution video coordinate system using the actual cropping origin of the ROI and the super-resolution magnification factor; then, combined with Equations (7) and (8), the seismic displacement of the building structure relative to the ground is calculated.

[0079] (5) Results and technical effects: The displacement monitoring results of the original low-resolution video, the full-image super-resolution video, and the ROI super-resolution video were compared with the simulated true displacement values, and the root mean square error (RMSE), Pearson correlation coefficient (r), peak relative error (PRF), and coefficient of determination (R²) were used for evaluation.

[0080] The displacement monitoring results before and after the 360p full-screen super-resolution reconstruction to 2K are as follows: Figure 9As shown, the displacement monitoring results before and after the 360p region of interest was super-reconstructed into a 2k region of interest are as follows: Figure 10 As shown, the results indicate that both full-image SRGAN and ROI-SRGAN processing can improve the traceability of structural textures and features in low-resolution videos, significantly reducing drift and flat segments in the displacement curve.

[0081] Regarding the ROI processing method, under the condition of PGA 0.2g, the R² of the three original LR-ROI results were negative, while the R² of the corresponding SR-ROI results all reached above 0.72; under the conditions of PGA 0.4g and 0.6g, the R² of the SR-ROI results all reached above 0.88, the correlation coefficients were all not less than 0.95, and the RMSE was all less than 5mm.

[0082] In this embodiment, using an RTX4060 GPU and the same software environment, the average time for 4x super-resolution reconstruction of a single low-resolution complete image is about 60 seconds, and the average time for reconstructing a single LR-ROI into an SR-ROI is about 1.4 seconds. The ROI scheme improves the inference speed by about 43 times compared to the full-image scheme.

[0083] Based on an estimated frame rate of 1.4 seconds, performing ROI super-resolution inference on a 20-second video segment with a frame rate of 50 FPS and a total of 1000 frames would take approximately 1400 seconds, or about 23 minutes. This timeframe is applicable to offline batch processing or phased assessments after an earthquake and does not constitute real-time video stream processing in the strict sense. The above times are exemplary test results under specific hardware and software conditions and do not constitute a limitation on the processing speed or hardware configuration of this invention.

[0084] Example 2: Video verification using a small vibration table: (1) Experiment and Sample: Three sets of seismic excitation conditions with PGA values ​​of 0.2g, 0.44g, and 0.65g were set on a small precision vibration table. The vibration process was recorded using a motion camera at a resolution of 960×540 pixels and a frame rate of 60 FPS.

[0085] Extract a joint structure-tabletop Region of Interest (ROI) from the low-resolution video. Using the top-left corner of the original video as the origin, the top-left corner of the ROI is located at (142, 72), with a horizontal width of 878 pixels and a vertical height of 298 pixels. The ROI simultaneously includes the artificial checkerboard pattern on the structure side, the natural texture of the structure, and the reference features on the shaking table surface side.

[0086] Laser displacement sensors are placed on the shaking table surface and the top layer of the test model, respectively. The displacement time history of the top layer of the structure relative to the shaking table surface is obtained by the difference in displacement measured by the two sensors, and it is used as the true value for comparison of visual displacement measurement.

[0087] (2) Training samples and model configuration: Acquire a 4K high-resolution representative image and a 540p low-resolution representative image with a similar viewpoint to the shaking table video. Perform geometric registration on the two images and extract their corresponding joint ROIs. Figure 11 ).

[0088] The paired ROIs were simultaneously rotated at 0°, 90°, 180° and 270° and mirrored accordingly to form 8 pairs of enhanced images; then, high-resolution sub-blocks of 256×256 pixels and low-resolution sub-blocks of 64×64 pixels were simultaneously cropped to construct a total of 3744 pairs of training samples.

[0089] The training and validation sets are divided in an 8:2 ratio. The generator is first pre-trained for 100 epochs using mean squared error loss with a learning rate of 0.0001. During the adversarial training phase, the batch size is 16, the generator learning rate is 0.0001, the discriminator learning rate is 0.00005, and the training epochs are 137. The above parameters are only for feasible examples.

[0090] (3) Feature tracking and results: The low-resolution ROIs under three PGA conditions were input frame by frame into the trained 4x SRGAN generator to obtain the corresponding super-resolution ROIs. The pyramid LK optical flow method was used to track the artificial checkerboard and natural features in the original LR-ROI and the generated SR-ROI, respectively, and the displacement of the top layer of the structure relative to the platform was calculated based on the relative motion between the structural features and the platform reference features.

[0091] To simplify the accompanying drawings and highlight the natural texture and technical effects under low-amplitude seismic loading, the measurement results of natural features at a PGA value of 0.2g are selected as representative results, such as... Figure 12 As shown. Figure 12 The data includes displacement monitoring curves from the original LR-ROI video, displacement monitoring curves from the SR-ROI video after SRGAN processing, and true displacement curves measured by the laser displacement sensor.

[0092] Under the three PGA conditions, the Pearson correlation coefficients of the SR-ROI measurement results were all no less than 0.95, the R² was no less than 0.87, and the RMSE was less than 0.7 mm.

[0093] Under natural characteristic conditions with PGA of 0.2g, the RMSE of the LR-ROI measurement results was 1.2904mm, the correlation coefficient was 0.7435, the peak relative error was 34.7567%, and the R² was -1.7140; the RMSE of the SR-ROI measurement results decreased to 0.1630mm, the correlation coefficient increased to 0.9791, the peak relative error decreased to 3.4728%, and the R² increased to 0.9567.

[0094] In the checkerboard feature measurement under the same PGA condition, RMSE decreased from 0.4783 mm to 0.1442 mm, and R² increased from 0.6271 to 0.9671.

[0095] The above results show that, under the experimental conditions of this embodiment, ROI super-resolution reconstruction can improve the tracking stability of artificial targets and natural features in low-resolution videos and improve the measurement accuracy of the displacement of the structure relative to the platform.

[0096] Example 3: Dynamic ROI and Quality Closed Loop This embodiment illustrates a method for dynamically updating the ROI when the building vibration amplitude is large, the monitoring screen exhibits slow drift, or the fixed ROI cannot continuously cover effective features. This embodiment is used to illustrate the control logic and is not limited to specific experimental results.

[0097] In the initial video frame, structural feature regions and ground reference regions are determined separately, and edge margins are set outside the effective feature bounding boxes. The category, top-left crop origin, horizontal width, vertical height, and super-resolution magnification of each ROI are recorded.

[0098] After completing feature tracking for each frame, the number of candidate features, the number of effective features, the tracking residual, the feature bounding box, and the minimum distance from the feature to the ROI boundary are counted in the structure and ground ROI, and the tracking quality scores of the structure ROI and ground ROI are calculated according to Equation (9).

[0099] Dynamic ROI is updated as follows: 1. When the tracking quality score is not lower than the preset threshold and the effective features are not close to the ROI boundary, the current ROI will continue to be used in the next frame; 2. When the tracking quality score is not lower than the preset threshold, but the effective features are close to the ROI boundary, shift the ROI in the next frame according to the effective feature bounding box so that the features are located in the middle of the ROI again; 3. When the tracking quality score falls below a preset threshold, expand the ROI range and re-detect features; 4. When a structural ROI or ground ROI fails to obtain sufficient effective features in multiple consecutive video frames, the corresponding video frames are marked as missing and displacement values ​​that have not been quality verified are not output.

[0100] After each ROI update, the actual cropping origin and ROI size of the current frame are saved. Local image points in the super-resolution ROI are still mapped back to their coordinates according to Equation (4), so that the ROI update only changes the local image range fed into the generation model, without changing the original video coordinate system.

[0101] When the tracking quality of both structural features and ground reference features meets the preset conditions, the seismic displacement of the building structure relative to the ground is calculated according to Equations (5) and (6); when the tracking quality of any category does not meet the conditions, ROI adjustment, feature re-initialization, or missing measurement markers are executed.

[0102] Alternative implementation methods: 1. Image region to be processed. The region to be processed can be a complete low-resolution video frame, a structural ROI, a structure-ground continuous joint ROI, or multiple separate ROIs. When using separate ROIs, record the category, cropping origin, and size of each ROI separately.

[0103] 2. Super-resolution models. Super-resolution generation models can employ SRGAN, ESRGAN, Real-ESRGAN, or other models capable of establishing a mapping relationship between low-resolution and super-resolution images. Different models should be adapted to the training samples, magnification, and coordinate transformation relationships.

[0104] 3. Feature detection and tracking. Shi-Tomasi corner detection, pyramid LK optical flow, SIFT feature matching, template matching, digital image correlation, or other methods that can obtain the time history of feature points can be used.

[0105] 4. High- and low-resolution image registration. Homography transformation, feature matching, registration with known landmarks, camera calibration relationships, or other methods that can establish the correspondence between physical regions of high- and low-resolution images can be used.

[0106] 5. Pixel-to-Physical Coordinate Transformation. This can be achieved using DLT projection matrices and height constraints, planar homography, known scale factors, or other calibrated pixel-to-physical coordinate mapping methods. When using structural and ground reference features, the relative motion of these features can reduce the impact of camera co-motion.

[0107] 6. ROI Update Method. The ROI can remain fixed, or it can be updated frame by frame, updated according to a preset time window, or triggered when the tracking quality falls below a threshold. After the ROI is updated, the updated actual cropping origin should be used for local coordinate mapping.

[0108] 7. Calculate constraints and quality records. The total ROI area, edge allowance, and model magnification can be set according to the equipment's computing power, processing time limit, feature distribution, and expected displacement range. The model version, processing area, clipping origin, processing time, number of effective features, and tracking error are recorded to support result verification.

[0109] The above are merely preferred embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for monitoring building seismic displacement based on super-resolution reconstruction, characterized in that, include: Acquire low-resolution seismic monitoring video of the target and determine the structural feature area and ground reference area within the same field of view of the monitoring video; The area to be processed is determined based on the structural feature area and the ground reference area; Based on the resolution, region category, and target magnification of the region to be processed, a matching super-resolution generation model is invoked to perform super-resolution reconstruction on the region to be processed, thereby obtaining a super-resolution image; In the super-resolution image, building structural features and ground reference features are detected respectively, and the local pixel coordinates of the building structural features and ground reference features in the super-resolution image are obtained. Based on the local pixel coordinates, the actual cropping origin of the area to be processed in the original low-resolution video, and the target magnification, the local pixel coordinates are mapped back to the original low-resolution video coordinate system to obtain the mapping point coordinates. Based on the coordinates of the echo point, camera calibration parameters, and corresponding spatial height constraints, the seismic displacement of the building structure relative to the ground is calculated. Based on the building structure characteristics and ground reference characteristics used to calculate the seismic displacement, the quality of the calculated seismic displacement is judged, and the seismic displacement time history of the building structure relative to the ground is output when the judgment result meets the preset conditions.

2. The method for monitoring building seismic displacement based on super-resolution reconstruction according to claim 1, characterized in that, Before acquiring low-resolution earthquake monitoring video of the target, the following steps are required: Obtain high-resolution representative images and low-resolution representative images; Geometric registration is performed on the high-resolution representative image and the low-resolution representative image, and the training region is simultaneously cropped from the registered image. High-resolution and low-resolution paired training samples are constructed by synchronous rotation, mirroring and sliding window sampling with sliding window step size set according to the target magnification. The super-resolution generation model is trained using the paired training samples, and then trained using adversarial training to obtain the trained super-resolution generation model.

3. The method for monitoring building seismic displacement based on super-resolution reconstruction according to claim 2, characterized in that, The process of training a super-resolution generation model using the paired training samples and then training the super-resolution generation model using adversarial training to obtain the trained super-resolution generation model includes: Based on the paired training samples, a super-resolution generative adversarial network containing a generator and a discriminator is constructed. The generator is first pre-trained with pixel consistency loss, and then adversarial training is performed by alternating updates of the generator and discriminator to obtain a trained super-resolution generative model.

4. The method for monitoring building seismic displacement based on super-resolution reconstruction according to claim 1, characterized in that, The region to be processed includes a complete video frame, a continuous joint region of interest that simultaneously covers the structural feature region and the ground reference region, or multiple separate regions of interest that respectively correspond to the structural feature region and the ground reference region. Based on the distribution of effective features in the structural feature region and the ground reference region, the expected seismic displacement range, the edge allowance, the computing power of the equipment, and the processing time limit, the size and location of the region to be processed are determined, and the actual clipping origin and region size are recorded frame by frame when the region of interest method is used.

5. The method for monitoring building seismic displacement based on super-resolution reconstruction according to claim 1, characterized in that, Detecting building structure features and ground reference features in the super-resolution image, and obtaining the local pixel coordinates of the building structure features and ground reference features in the super-resolution image, includes: In the super-resolution image, building edge intersections, window corners, exterior wall textures, ground markings, or stable gray-scale gradient features are detected as building structural features and ground reference features, and corner detection and pyramid optical flow are used for frame-by-frame subpixel tracking. Feature points that are out of bounds, occluded, have excessive forward and backward tracking errors, insufficient texture gradients, are continuously lost, or are outliers of the same type of displacement are removed. The local pixel coordinates of the selected building structure features and ground reference features are recorded respectively.

6. The method for monitoring building seismic displacement based on super-resolution reconstruction according to claim 1, characterized in that, Obtaining the coordinates of the echo point includes: The local pixel coordinates are scaled according to the target magnification factor, and then superimposed with the actual cropping origin of the region to be processed in the original low-resolution video to map the local pixel coordinates back to the original low-resolution video coordinate system and obtain the mapping point coordinates. When a fixed region of interest is used, the cropping origin remains unchanged, and when a dynamic region of interest is used, the cropping origin updated for each frame is used.

7. The method for monitoring building seismic displacement based on super-resolution reconstruction according to claim 1, characterized in that, Based on the echo point coordinates, camera calibration parameters, and corresponding spatial height constraints, the seismic displacement of the building structure relative to the ground is calculated as follows: Substitute the back projection point coordinates, camera projection matrix, and corresponding height constraints into the back projection relationship to obtain the physical coordinates of the building structural features and ground reference features, wherein the height of the ground reference features is zero, and the height of the structural features is the height value determined by on-site measurement or design data. Robust aggregation of the physical coordinates of multiple building structural features or multiple ground reference features is performed to obtain the aggregated location of the building structural features and the aggregated location of the ground reference features. The seismic displacement of the building structure relative to the ground is calculated based on the difference between the aggregated location of the building structure features and the aggregated location of the ground reference features relative to the initial time.

8. The method for monitoring building seismic displacement based on super-resolution reconstruction according to claim 1, characterized in that, The quality is determined based on the building structure characteristics and ground reference characteristics, and the seismic displacement time history of the building structure relative to the ground is output when the determination result meets preset conditions, including: The quality score is calculated based on the effective number of building structural features and ground reference features, forward and backward tracking errors, distance to the boundary, number of consecutive lost frames, dispersion of similar displacements, and single-frame processing time, respectively. When both the building structural features and the ground reference features meet the quality threshold, the seismic displacement time history is output, and the video time information, model version, magnification, actual cropping origin, number of effective features, tracking error and processing time are associated and saved. If any type of feature fails to meet the quality threshold, perform feature re-detection, region of interest adjustment, or missing detection labeling.

9. A building seismic displacement monitoring system based on super-resolution reconstruction, used to implement the method as described in any one of claims 1-8, characterized in that, include: The video acquisition module is used to acquire low-resolution seismic monitoring videos of the target and determine the structural feature area and the ground reference area within the same field of view of the monitoring video. The region to be processed determination module is used to determine the region to be processed based on the structural feature region and the ground reference region; The super-resolution reconstruction module is used to call a matching super-resolution generation model based on the resolution, region category and target magnification of the region to be processed, and to perform super-resolution reconstruction on the region to be processed to obtain a super-resolution image. The feature tracking module is used to detect building structure features and ground reference features in the super-resolution image, and obtain the local pixel coordinates of the building structure features and ground reference features in the super-resolution image. The coordinate mapping module is used to map the local pixel coordinates back to the original low-resolution video coordinate system based on the local pixel coordinates, the actual cropping origin of the area to be processed in the original low-resolution video, and the target magnification, and obtain the mapping point coordinates. The displacement calculation module is used to calculate the seismic displacement of the building structure relative to the ground based on the coordinates of the echo point, camera calibration parameters, and corresponding spatial height constraints. The quality assessment module assesses the quality of the earthquake displacement calculation results based on the building structure characteristics and ground reference characteristics used to calculate the earthquake displacement, and outputs the earthquake displacement time history of the building structure relative to the ground when the assessment results meet preset conditions.

10. A building seismic displacement monitoring system based on super-resolution reconstruction according to claim 9, characterized in that, The system also includes: The representative image acquisition module is used to acquire high-resolution and low-resolution representative images. The sample construction module is used to perform geometric registration on the high-resolution representative image and the low-resolution representative image, and synchronously extract the training region in the registered image. Through synchronous rotation, mirroring and sliding window sampling with a sliding window step size set according to the target magnification, high-resolution and low-resolution paired training samples are constructed. The model training module is used to train the super-resolution generation model using the paired training samples, and to train the super-resolution generation model using alternating adversarial training to obtain the trained super-resolution generation model.

Images