Portable monitoring method and system for structural body deformation based on handheld imaging, terminal and storage medium

Abstract

The invention relates to the technical field of structural health monitoring, and discloses a portable structural body deformation monitoring method and system based on handheld imaging, a terminal and a storage medium, the method comprises the following steps: in a target monitoring area, arranging a cooperative mark point array along the axis of a monitoring structure, the cooperative mark point array comprising stable mark points and mark points to be measured; the method comprises the following steps: acquiring an initial state image and a monitoring state image of a monitoring area through handheld imaging equipment, and extracting respective vertical pixel coordinates of a stable mark point and a mark point to be detected from the initial state image and the monitoring state image; respectively calculating the pixel variation of the stable mark point and the pixel variation of the mark point to be measured according to the respective vertical pixel coordinates; and inputting the respective pixel variation, the camera internal reference of the handheld imaging equipment and the respective initial coordinates of the stable mark point and the mark point to be measured into a displacement correction model for solving to obtain the real deformation of the mark point to be measured. According to the invention, the structural body deformation monitoring precision is improved.

Description

Technical Field

[0001] This invention relates to the field of structural health monitoring technology, and in particular to a portable monitoring method, system, terminal, and computer-readable storage medium for structural deformation based on handheld imaging. Background Technology

[0002] Modern handheld imaging devices are widely equipped with high-resolution CMOS (Complementary Metal-Oxide-Semiconductor) sensors, optical image stabilization modules, and strong on-chip computing capabilities. Their image quality is sufficient to meet the basic requirements for structural deformation observation under short to medium working distances. Furthermore, handheld imaging devices are highly accessible, portable, and easy to deploy, requiring no complex installation or debugging procedures. This makes them a potential supplementary acquisition terminal to professional monitoring systems, enabling the acquisition of high-frequency, distributed structural deformation information and providing a technological foundation for addressing the pain points of traditional monitoring methods.

[0003] Existing representative technical solutions for visual displacement or deformation monitoring include: main-secondary dual-camera motion compensation method, multi-camera collaboration, network measurement method, displacement transfer serial camera network method. However, none of the above-mentioned existing visual monitoring technologies can solve the scale coupling problem between camera self-motion and structural deformation under the condition of "single camera, unstable platform (handheld)", and are difficult to adapt to the lightweight and portable monitoring requirements of handheld imaging devices.

[0004] Therefore, existing technologies still need to be improved and developed. Summary of the Invention

[0005] The main objective of this invention is to provide a portable method, system, terminal, and computer-readable storage medium for monitoring structural deformation based on handheld imaging. This aims to solve the problems in the prior art where visual deformation monitoring technology cannot be adapted to unstable observation platforms such as handheld imaging devices, and where scale coupling errors caused by camera self-motion and structural deformation result in insufficient accuracy in monitoring structural deformation.

[0006] To achieve the above objectives, the present invention provides a portable method for monitoring structural deformation based on handheld imaging, the method comprising the following steps: In the target monitoring area, a cooperative marker array is deployed along the axis of the monitoring structure. The cooperative marker array includes stable markers and markers to be measured. The initial state image and the monitoring state image of the target monitoring area are acquired by a handheld imaging device. The vertical pixel coordinates of the stable marker point and the marker point to be measured are extracted from the initial state image and the monitoring state image. The pixel change of the stable marker point and the marker point to be measured is calculated based on the vertical pixel coordinates. The pixel change, the camera intrinsic parameters of the handheld imaging device, and the initial coordinates of the stable marker and the marker to be measured are input into a preset displacement correction model. The displacement correction model is solved by the least squares method to obtain the true deformation of the marker to be measured.

[0007] Optionally, in the portable structural deformation monitoring method based on handheld imaging, the cooperative marker array includes at least two stable markers and at least one marker to be measured. The stable markers are placed on the monitoring structure at positions assumed to be free from displacement, while the markers to be measured are placed at structural locations where deformation is to be monitored.

[0008] Optionally, the portable structural deformation monitoring method based on handheld imaging, wherein acquiring an initial state image and a monitoring state image of the target monitoring area using a handheld imaging device, and extracting the vertical pixel coordinates of the stable marker point and the marker point to be measured from the initial state image and the monitoring state image, specifically includes: The handheld imaging device's built-in level and grid line functions are used to assist in adjusting the shooting posture, and the camera aperture, ISO sensitivity, and exposure time parameters of the handheld imaging device are locked. Once the aperture, ISO sensitivity, and exposure time parameters are locked, the handheld imaging device acquires the initial state image and the monitoring state image of the target monitoring area, respectively. Identify the first preset auxiliary pattern on the stable marker point and the second preset auxiliary pattern on the marker point to be tested from the initial state image and the monitoring state image, respectively; The first coordinate value of the center point of the first preset auxiliary pattern in the vertical direction in the image coordinate system and the second coordinate value of the center point of the second preset auxiliary pattern in the vertical direction in the image coordinate system are obtained by using a sub-pixel level positioning algorithm. The first coordinate value is used as the vertical pixel coordinate of the stable marker point, and the second coordinate value is used as the vertical pixel coordinate of the marker point to be tested.

[0009] Optionally, in the portable monitoring method for structural deformation based on handheld imaging, the displacement correction model is constructed based on the pinhole imaging model and includes correction terms; The displacement correction model is used to jointly model and solve the constraints of the cooperative marker array, so as to decouple the self-motion parameters of the handheld imaging device from the actual deformation of the structure. The correction term is used to correct the imaging scale error caused by the pose change of the handheld imaging device.

[0010] Optionally, the portable structural deformation monitoring method based on handheld imaging, wherein inputting the respective pixel changes, the camera intrinsic parameters of the handheld imaging device, and the initial coordinates of the stable marker point and the marker point to be measured into a preset displacement correction model specifically includes: The pixel changes, camera intrinsic parameters of the handheld imaging device, and initial coordinates of the stable marker and the marker to be measured are input into a preset displacement correction model to establish a mapping relationship between pixel coordinate changes and actual structural settlement. ; in, This indicates a correction term for errors caused by movement when shooting with a handheld camera. This represents the pixel change of the stable marker point or the marker point to be measured. and These represent the magnification factors at the initial time and the observation time, respectively. This indicates the initial coordinates of the stable marker point or the marker point to be measured. The image principal point ordinate represents the intrinsic parameters of the camera. This indicates the vertical structural displacement of the marker point to be measured. This represents the vertical displacement component of the camera. This represents the distance along the optical axis from the camera to the target marker at the initial moment; This indicates the camera's own pitch angle change.

[0011] Optionally, the portable structural deformation monitoring method based on handheld imaging, wherein solving the displacement correction model using the least squares method to obtain the true deformation of the target marker specifically includes: Based on the mapping relationship established by the displacement correction model, equations are constructed for all the stable marker points and the marker points to be measured, and they are combined to form a set of equations with the self-motion parameters of the handheld imaging device and the structural vertical displacement of the marker points to be measured as unknowns. The equations are solved using the least squares method, and the self-motion parameters of the handheld imaging device and the vertical displacement of the target point are obtained simultaneously. The vertical displacement of the structure is then used as the actual deformation of the target point.

[0012] Optionally, the portable structural deformation monitoring method based on handheld imaging further includes, after obtaining the actual deformation of the target marker, the following steps: In the same portable structural deformation monitoring task, multiple independent deformation results of multiple test markers are obtained and output.

[0013] Furthermore, to achieve the above objectives, the present invention also provides a portable structural deformation monitoring system based on handheld imaging, wherein the portable structural deformation monitoring system based on handheld imaging includes: The measuring point layout module is used to lay out a cooperative marker point array along the axis of the monitoring structure in the target monitoring area. The cooperative marker point array includes stable marker points and marker points to be measured. The image acquisition and calculation module is used to acquire the initial state image and the monitoring state image of the target monitoring area through a handheld imaging device, extract the vertical pixel coordinates of the stable marker point and the marker point to be measured from the initial state image and the monitoring state image, and calculate the pixel change of the stable marker point and the marker point to be measured respectively based on the vertical pixel coordinates. The deformation calculation module is used to input the respective pixel changes, the camera intrinsic parameters of the handheld imaging device, and the initial coordinates of the stable marker point and the marker point to be measured into a preset displacement correction model, and solve the displacement correction model by the least squares method to obtain the true deformation of the marker point to be measured.

[0014] In addition, to achieve the above objectives, the present invention also provides a terminal, wherein the terminal includes: a memory, a processor, and a portable monitoring program for structural deformation based on handheld imaging stored in the memory and executable on the processor, wherein when the portable monitoring program for structural deformation based on handheld imaging is executed by the processor, it implements the steps of the portable monitoring method for structural deformation based on handheld imaging as described above.

[0015] Furthermore, to achieve the above objectives, the present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a portable monitoring program for structural deformation based on handheld imaging, and when the portable monitoring program for structural deformation based on handheld imaging is executed by a processor, it implements the steps of the portable monitoring method for structural deformation based on handheld imaging as described above.

[0016] In this invention, a cooperative marker array is deployed along the axis of the monitored structure within the target monitoring area. This array includes stable markers and markers to be measured. Initial and monitoring state images of the monitoring area are acquired using a handheld imaging device. The vertical pixel coordinates of the stable and markers to be measured are extracted from these images, and the pixel changes of each marker are calculated based on their respective coordinates. The pixel changes, the camera intrinsic parameters of the handheld imaging device, and the initial coordinates of the stable and markers to be measured are input into a preset displacement correction model. The displacement correction model is solved using the least squares method to obtain the actual deformation of the marker to be measured. This invention can control the camera attitude using low-cost auxiliary means such as grid lines and levels integrated into the handheld imaging device, eliminating the need for a stable fixed platform and complex hardware configuration, while maintaining millimeter-level measurement accuracy. It overcomes the dependence of traditional technologies on stable platforms and specialized equipment, achieving lightweight, portable, and low-cost deformation monitoring, and providing a feasible path for building a public-participatory, collaborative perception model for structural safety. Attached Figure Description

[0017] Figure 1 This is a flowchart of a preferred embodiment of the portable monitoring method for structural deformation based on handheld imaging of the present invention; Figure 2 This is a flowchart of the single-camera monitoring process of the portable monitoring method for structural deformation based on handheld imaging according to the present invention. Figure 3 This is a schematic diagram of the single-camera pose change principle of the portable monitoring method for structural deformation based on handheld imaging of the present invention; Figure 4 This is a structural diagram of a preferred embodiment of the portable structural deformation monitoring system based on handheld imaging of the present invention; Figure 5 This is a structural diagram of a preferred embodiment of the terminal of the present invention. Detailed Implementation

[0018] This application provides a portable monitoring method, system, and terminal for structural deformation based on handheld imaging. To make the objectives, technical solutions, and effects of this application clearer and more explicit, the following detailed description is provided with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only for explaining this application and are not intended to limit this application.

[0019] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless specifically defined as herein.

[0020] Furthermore, if the embodiments of this invention involve descriptions such as "first" or "second," these descriptions are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first" or "second" may explicitly or implicitly include at least one of those features. Additionally, the technical solutions of the various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. If the combination of technical solutions is contradictory or impossible to implement, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed by this invention.

[0021] The portable structural deformation monitoring method based on handheld imaging described in the preferred embodiment of the present invention, such as... Figure 1 and Figure 2 As shown, the portable structural deformation monitoring method based on handheld imaging includes the following steps: Step S10: In the target monitoring area, a cooperative marker array is set up along the axis of the monitoring structure. The cooperative marker array includes stable markers and markers to be measured.

[0022] Specifically, the cooperative marker array includes at least two stable markers and at least one marker to be measured; wherein the stable markers are placed on the monitoring structure at positions assumed to be non-displaceable, and the markers to be measured are placed on structural parts to be monitored for deformation.

[0023] In this embodiment, a target detection area is selected, and a cooperative marker array containing stable points is deployed along the monitoring structure axis. Stable marker points (reference points) are designated, and the rest are marker points to be measured (i.e., in the monitoring area, monitoring points are deployed along a certain line, which is the monitoring structure axis).

[0024] It is understood that the cooperative marker array must include at least two stable markers and at least two markers to be measured. Specifying at least two stable markers is to provide sufficient and necessary constraints when using the displacement correction model for subsequent calculations. For unsolvable problems involving multi-point measurements, introducing at least two stable markers transforms the system into a solvable state through constraints, enabling the synchronous calculation of camera motion parameters and actual settlement. Figure 3 As shown, where , , These represent the directions of the three coordinate axes in the camera coordinate system; Indicates the coordinates of the principal point; Indicates the initial time; Indicates the observation time; For point from Time's up The settlement value at time, of which for exist Location at any given moment for exist Current location; u and v These represent the directions of the two coordinate axes in the pixel coordinate system. This ensures that the self-motion parameters of the handheld imaging device during the shooting process can be uniquely calculated. Specifying at least two test marker points is to meet the needs of effective and practical deformation monitoring of structures, enabling simultaneous monitoring of multiple key components.

[0025] Furthermore, the stability markers must be placed on structural locations that are assumed to remain displacement throughout the entire monitoring period. This is the physical basis and logical premise for setting the measured vertical displacement to zero in the subsequent mathematical model for constraint solving, while the markers to be measured must be placed on the target structural locations where deformation needs to be monitored.

[0026] It should be noted that in other implementations, actively emitting markers (such as LED cross markers) can be used to replace the stable markers in this invention, in order to improve the imaging quality and feature extraction accuracy in nighttime or low-light environments, and to expand the applicable time period of the method.

[0027] Step S20: Acquire the initial state image and monitoring state image of the target monitoring area using a handheld imaging device; extract the vertical pixel coordinates of the stable marker point and the marker point to be measured from the initial state image and the monitoring state image; and calculate the pixel change of the stable marker point and the marker point to be measured based on their respective vertical pixel coordinates.

[0028] The step of acquiring initial state images and monitoring state images of the target monitoring area using a handheld imaging device, and extracting the vertical pixel coordinates of the stable marker points and the marker points to be measured from the initial state images and the monitoring state images, specifically includes: The handheld imaging device's built-in level and grid line functions are used to assist in adjusting the shooting posture, and the camera aperture, ISO sensitivity, and exposure time parameters of the handheld imaging device are locked. Once the aperture, ISO sensitivity, and exposure time parameters are locked, the handheld imaging device acquires the initial state image and the monitoring state image of the target monitoring area, respectively. Identify the first preset auxiliary pattern on the stable marker point and the second preset auxiliary pattern on the marker point to be tested from the initial state image and the monitoring state image, respectively; The first coordinate value of the center point of the first preset auxiliary pattern in the vertical direction in the image coordinate system and the second coordinate value of the center point of the second preset auxiliary pattern in the vertical direction in the image coordinate system are obtained by using a sub-pixel level positioning algorithm. The first coordinate value is used as the vertical pixel coordinate of the stable marker point, and the second coordinate value is used as the vertical pixel coordinate of the marker point to be tested.

[0029] Understandably, this embodiment utilizes the level and grid line functions built into the handheld imaging device to assist in adjusting the shooting posture. This is to minimize systematic errors caused by significant phone tilt or composition shifts at the subjective operational level, ensuring that the initial and monitored shooting angles are essentially aligned. Locking the aperture, ISO sensitivity, and exposure time parameters, from the imaging hardware level, strictly guarantees that the two images have completely consistent photometric conditions (brightness, contrast, and noise levels). This is a crucial prerequisite for subsequent image processing algorithms to stably and repeatedly identify and locate the same landmark point, preventing the brightness center of the same physical point from shifting in the image due to exposure differences.

[0030] Furthermore, identifying pre-defined auxiliary patterns from the two images demonstrates that the system relies on pre-placed physical markers with specific geometric shapes (such as circles or checkerboard patterns). Using the first and second pre-defined auxiliary patterns shows that the object to be identified is a manually designed pattern on the marker, easily and stably detected by the image algorithm, rather than the natural texture of a structural surface, thus improving the accuracy of feature extraction. The sub-pixel algorithm overcomes the integer limitation of physical pixels, improving positioning accuracy to below the pixel level.

[0031] Obtaining the vertical coordinates of the center point of the auxiliary pattern in the image coordinate system is to define a stable and unique representative point, avoiding the use of locations such as edges that are easily affected by imaging conditions. Finally, the obtained coordinates are used as the vertical pixel coordinates of the stable marker point and the marker point to be measured, respectively, completing the transformation from the image pixel domain to the measurement data domain.

[0032] Furthermore, the pixel changes of the stable marker points and the marker points to be tested are calculated based on their respective vertical pixel coordinates. Specifically, for each stable marker point, the vertical pixel coordinates extracted from the monitoring state image are subtracted from the vertical pixel coordinates of the same stable marker point extracted from the initial state image; the difference is the pixel change of that stable marker point. For each marker point to be tested, the vertical pixel coordinates extracted from the monitoring state image are subtracted from the vertical pixel coordinates of the same marker point to be tested extracted from the initial state image; the difference is the pixel change of that marker point to be tested. The pixel changes of all stable marker points and all marker points to be tested are recorded and stored as input data for subsequent displacement correction model calculations.

[0033] Understandably, in this embodiment, for each stable marker point and the marker point to be measured, the vertical pixel coordinates in the monitoring state image are subtracted from the corresponding coordinates in the initial state image, and the difference is the pixel change for each. This calculation process quantifies the vertical pixel displacement of each marker point between the two imaging sessions. The pixel change of the stable marker point is considered a pseudo-displacement caused entirely by the camera's own motion, while the pixel change of the marker point to be measured is a mixed signal resulting from both camera motion and actual structural deformation.

[0034] It should be noted that in other implementations, a self-supervised learning feature stabilization method can be used to replace the traditional template matching algorithm. For example, non-cooperative markers such as natural textures can be used to improve the robustness of marker detection and tracking in harsh environments such as drastic changes in lighting, partial occlusion, or complex background textures, thereby enhancing the environmental adaptability of the method.

[0035] Step S30: Input the respective pixel change, the camera intrinsic parameters of the handheld imaging device, and the initial coordinates of the stable marker point and the marker point to be measured into a preset displacement correction model, and solve the displacement correction model by the least squares method to obtain the true deformation of the marker point to be measured.

[0036] like Figure 3As shown, a single-camera deformation measurement model based on pose change correction is constructed based on the principle of single-camera pose change. Specifically, the displacement correction model is constructed based on the pinhole imaging model and includes a correction term; the displacement correction model is used to jointly model and solve the constraints of the cooperative marker array to decouple the self-motion parameters of the handheld imaging device from the actual deformation of the structure; the correction term is used to correct the imaging scale error caused by the pose change of the handheld imaging device.

[0037] It should be noted that in another practical approach, the inertial sensing information from a handheld imaging device's IMU (accelerometer, gyroscope) can also be additionally fused and added as a priori constraints to the correction model. This can improve the model's adaptability to complex attitude disturbances such as pitch, roll, and yaw angles, and reduce spurious displacement errors introduced by instantaneous jitter.

[0038] In this embodiment, based on the vertical pixel changes of the marker points in the initial and monitored states, and combined with the pinhole imaging model and simplified assumptions, a displacement correction model is constructed to achieve accurate mapping between pixel changes and actual vertical displacement. This correction model includes core influencing parameters (focal length, initial distance, actual settlement, camera vertical displacement, pitch angle change, etc.), which can effectively separate the coupled effects of camera motion and structural settlement. The physical meaning of each parameter in the model is clearly defined to ensure the interpretability and rationality of the solution.

[0039] The step of inputting the respective pixel changes, the camera intrinsic parameters of the handheld imaging device, and the initial coordinates of the stable marker point and the marker point to be measured into a preset displacement correction model specifically includes: The pixel changes, camera intrinsic parameters of the handheld imaging device, and initial coordinates of the stable marker and the marker to be measured are input into a preset displacement correction model to establish a mapping relationship between pixel coordinate changes and actual structural settlement. ; in, This indicates a correction term for errors caused by movement when shooting with a handheld camera. This represents the pixel change of the stable marker point or the marker point to be measured. and These represent the magnification factors at the initial time and the observation time, respectively. This indicates the initial coordinates of the stable marker point or the marker point to be measured. The image principal point ordinate represents the intrinsic parameters of the camera. This indicates the vertical structural displacement of the marker point to be measured. This represents the vertical displacement component of the camera. This represents the distance along the optical axis from the camera to the target marker at the initial moment; This indicates the camera's own pitch angle change.

[0040] It is understood that this invention establishes a mapping relationship between three-dimensional spatial points and two-dimensional image plane points based on an ideal pinhole imaging model, clarifying the correlation logic between pixel coordinates and world coordinates. Through analysis of the core equations of the imaging relationship, key parameters affecting vertical settlement measurement are extracted. These core parameters include scale factor, camera intrinsic parameters (equivalent focal length, principal point position, etc.), and extrinsic parameters (rotation matrix describing attitude, translation vector describing position).

[0041] The core principle lies in the fact that handheld imaging devices undergo three-dimensional translation and rotation during handheld shooting. Mechanism analysis is used to identify key motion parameters that significantly affect vertical settlement measurement, while ignoring the influence of secondary motion parameters to simplify subsequent model calculations. A simplified rotation matrix is ​​constructed based on the selected results, focusing on the impact of core motion parameters on imaging. For example, camera displacement along the optical axis (Z-axis direction) leads to a change in imaging magnification. This change is highly coupled with the pixel-level variation in structural vertical settlement and is the core cause of settlement estimation distortion.

[0042] Furthermore, the step of solving the displacement correction model using the least squares method to obtain the true deformation of the target point specifically includes: Based on the mapping relationship established by the displacement correction model, equations are constructed for all the stable marker points and the marker points to be measured, and they are combined to form a set of equations with the self-motion parameters of the handheld imaging device and the structural vertical displacement of the marker points to be measured as unknowns. The equations are solved using the least squares method, and the self-motion parameters of the handheld imaging device and the vertical displacement of the target point are obtained simultaneously. The vertical displacement of the structure is then used as the actual deformation of the target point.

[0043] In this embodiment, the modified model is transformed into a standard matrix equation form, and the least squares method is used to solve the equation system to decouple the camera's self-motion parameters from the actual settlement, thereby obtaining the actual settlement value of each measuring point.

[0044] Furthermore, after obtaining the actual deformation of the target marker, the process further includes: In the same portable structural deformation monitoring task, multiple independent deformation results of multiple test markers are obtained and output.

[0045] Understandably, after completing the model solution, deformation results from multiple measured marker points at different locations are aggregated. Since structural deformation may exhibit spatial distribution characteristics, and the measurement errors at each measuring point are independent, this sample set contains richer monitoring information and possesses inherent cross-validation capabilities. Subsequently, statistical analysis can be performed on multiple independent deformation results, particularly calculating their arithmetic mean. Mean calculation, as a fundamental data fusion method, effectively suppresses random errors (such as accidental errors caused by image noise and instantaneous vibrations), smooths inconsistencies in the data from different measuring points, and thus obtains a more stable and reliable estimate of the overall or local deformation trend of the structure. Finally, this statistically derived mean is used as the optimized final deformation output, completing the transformation from raw solution values ​​to purified data usable for decision-making.

[0046] Furthermore, this invention can also integrate the entire process of "image acquisition - camera calibration - feature extraction - model solving - result output" into a handheld imaging device APP, and add shooting guidance (such as posture deviation prompts), automatic marker recognition, data cloud upload and management functions to achieve fully automated operation, reduce the threshold for public participation, and meet the needs of large-scale engineering applications. All automated process improvements that integrate this method into a mobile APP are within the protection scope of this invention.

[0047] The present invention has the following beneficial effects: (1) Multi-measurement point collaborative modeling and stable point constraint mechanism: A decoupling strategy of multi-measurement point joint modeling and stable point constraint is proposed. By using a small number of stable points, the unsolvable system is transformed into a solvable state, realizing the synchronous solution of camera self-motion parameters and real settlement without the need for additional attitude sensors. By multi-measurement point joint modeling and introducing stable point constraints, the camera self-motion parameters and the real deformation of the structure are accurately decoupled, effectively overcoming the measurement deviation caused by motion artifacts in traditional methods.

[0048] (2) Lightweight Deployment Architecture for Handheld Imaging Devices: The system is compatible with handheld imaging device platforms, requiring no stable platform or professional equipment, and supports handheld operation by non-professionals. This architecture has strong environmental adaptability, maintaining sub-millimeter accuracy under complex natural conditions, significantly reducing technical barriers and deployment costs. Combined with the built-in functions of mobile phones, it achieves low-cost attitude control and accurate feature extraction, balancing measurement accuracy and portability, and can be directly transferred to public participation monitoring scenarios.

[0049] (3) Low-cost posture self-constraint method: The shooting posture is autonomously constrained by the built-in sensor of the mobile phone, eliminating the need for external sensors, and ensuring the accuracy requirements of the simplified assumptions of the model while controlling costs.

[0050] In summary, this invention achieves lightweight, low-cost, and high-precision settlement monitoring, and can aggregate multi-source observation data for cloud-based cross-validation, providing technical support for building a public-participatory, collectively intelligent perception model, and effectively filling the technical gap between traditional professional monitoring and public perception.

[0051] Furthermore, such as Figure 4 As shown, based on the above-described portable monitoring method for structural deformation based on handheld imaging, the present invention also provides a portable monitoring system for structural deformation based on handheld imaging, wherein the portable monitoring system for structural deformation based on handheld imaging includes: The measuring point layout module 51 is used to lay out a cooperative marker point array along the axis of the monitoring structure in the target monitoring area. The cooperative marker point array includes stable marker points and marker points to be measured. The image acquisition and calculation module 52 is used to acquire the initial state image and the monitoring state image of the target monitoring area through a handheld imaging device, extract the vertical pixel coordinates of the stable marker point and the marker point to be measured from the initial state image and the monitoring state image, and calculate the pixel change of the stable marker point and the marker point to be measured respectively based on the vertical pixel coordinates. The deformation calculation module 53 is used to input the respective pixel changes, the camera intrinsic parameters of the handheld imaging device, and the initial coordinates of the stable marker point and the marker point to be measured into a preset displacement correction model, and solve the displacement correction model by the least squares method to obtain the true deformation of the marker point to be measured.

[0052] Furthermore, such as Figure 5 As shown, based on the above-mentioned portable monitoring method and system for structural deformation based on handheld imaging, the present invention also provides a terminal, which includes a processor 10, a memory 20 and a display 30. Figure 5 Only some of the terminal components are shown; however, it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0053] In some embodiments, the memory 20 may be an internal storage unit of the terminal, such as a hard disk or memory. In other embodiments, the memory 20 may be an external storage device of the terminal, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc. Further, the memory 20 may include both internal and external storage devices. The memory 20 is used to store application software and various types of data installed on the terminal, such as program code installed on the terminal. The memory 20 can also be used to temporarily store data that has been output or will be output. In one embodiment, the memory 20 stores a portable structural deformation monitoring program 40 based on handheld imaging, which can be executed by the processor 10 to implement the portable structural deformation monitoring method based on handheld imaging in this application.

[0054] In some embodiments, the processor 10 may be a central processing unit (CPU), a microprocessor, or other data processing chip, used to run program code stored in the memory 20 or process data, such as executing the portable monitoring method for structural deformation based on handheld imaging.

[0055] In some embodiments, the display 30 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. The display 30 is used to display information on the terminal and to display a visual user interface. The components of the terminal communicate with each other via a system bus.

[0056] In one embodiment, when the processor 10 executes the portable monitoring program 40 for structural deformation based on handheld imaging stored in the memory 20, the following steps are performed: In the target monitoring area, a cooperative marker array is deployed along the axis of the monitoring structure. The cooperative marker array includes stable markers and markers to be measured. The initial state image and the monitoring state image of the target monitoring area are acquired by a handheld imaging device. The vertical pixel coordinates of the stable marker point and the marker point to be measured are extracted from the initial state image and the monitoring state image. The pixel change of the stable marker point and the marker point to be measured is calculated based on the vertical pixel coordinates. The pixel change, the camera intrinsic parameters of the handheld imaging device, and the initial coordinates of the stable marker and the marker to be measured are input into a preset displacement correction model. The displacement correction model is solved by the least squares method to obtain the true deformation of the marker to be measured.

[0057] The cooperative marker array includes at least two stable markers and at least one marker to be measured. The stable markers are placed on the monitoring structure at positions assumed to be free from displacement, while the markers to be measured are placed at structural locations where deformation is to be monitored.

[0058] The step of acquiring initial state images and monitoring state images of the target monitoring area using a handheld imaging device, and extracting the vertical pixel coordinates of the stable marker point and the marker point to be measured from the initial state images and the monitoring state images, specifically includes: The handheld imaging device's built-in level and grid line functions are used to assist in adjusting the shooting posture, and the camera aperture, ISO sensitivity, and exposure time parameters of the handheld imaging device are locked. Once the aperture, ISO sensitivity, and exposure time parameters are locked, the handheld imaging device acquires the initial state image and the monitoring state image of the target monitoring area, respectively. Identify the first preset auxiliary pattern on the stable marker point and the second preset auxiliary pattern on the marker point to be tested from the initial state image and the monitoring state image, respectively; The first coordinate value of the center point of the first preset auxiliary pattern in the vertical direction in the image coordinate system and the second coordinate value of the center point of the second preset auxiliary pattern in the vertical direction in the image coordinate system are obtained by using a sub-pixel level positioning algorithm. The first coordinate value is used as the vertical pixel coordinate of the stable marker point, and the second coordinate value is used as the vertical pixel coordinate of the marker point to be tested.

[0059] The displacement correction model is constructed based on the pinhole imaging model and includes correction terms; The displacement correction model is used to jointly model and solve the constraints of the cooperative marker array, so as to decouple the self-motion parameters of the handheld imaging device from the actual deformation of the structure. The correction term is used to correct the imaging scale error caused by the pose change of the handheld imaging device.

[0060] Specifically, inputting the respective pixel changes, the camera intrinsic parameters of the handheld imaging device, and the initial coordinates of the stable marker point and the marker point to be measured into a preset displacement correction model includes: The pixel changes, camera intrinsic parameters of the handheld imaging device, and initial coordinates of the stable marker and the marker to be measured are input into a preset displacement correction model to establish a mapping relationship between pixel coordinate changes and actual structural settlement. ; in, This indicates a correction term for errors caused by movement when shooting with a handheld camera. This represents the pixel change of the stable marker point or the marker point to be measured. and These represent the magnification factors at the initial time and the observation time, respectively. This indicates the initial coordinates of the stable marker point or the marker point to be measured. The image principal point ordinate represents the intrinsic parameters of the camera. This indicates the vertical structural displacement of the marker point to be measured. This represents the vertical displacement component of the camera. This represents the distance along the optical axis from the camera to the target marker at the initial moment; This indicates the camera's own pitch angle change.

[0061] Specifically, solving the displacement correction model using the least squares method to obtain the true deformation of the target marker point includes: Based on the mapping relationship established by the displacement correction model, equations are constructed for all the stable marker points and the marker points to be measured, and they are combined to form a set of equations with the self-motion parameters of the handheld imaging device and the structural vertical displacement of the marker points to be measured as unknowns. The equations are solved using the least squares method, and the self-motion parameters of the handheld imaging device and the vertical displacement of the target point are obtained simultaneously. The vertical displacement of the structure is then used as the actual deformation of the target point.

[0062] After obtaining the actual deformation of the target marker, the method further includes: In the same portable structural deformation monitoring task, multiple independent deformation results of multiple test markers are obtained and output.

[0063] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a portable monitoring program for structural deformation based on handheld imaging, and the portable monitoring program for structural deformation based on handheld imaging, when executed by a processor, implements the steps of the portable monitoring method for structural deformation based on handheld imaging as described above.

[0064] In summary, this invention provides a portable monitoring method, system, terminal, and storage medium for structural deformation based on handheld imaging. The method includes: deploying a cooperative marker array along the axis of the monitored structure in the target monitoring area, the cooperative marker array including stable markers and markers to be measured; acquiring initial state images and monitoring state images of the monitoring area using a handheld imaging device; extracting the vertical pixel coordinates of the stable markers and the markers to be measured from the initial state images and monitoring state images; calculating the pixel change of each of the stable markers and the markers to be measured based on their respective vertical pixel coordinates; inputting the pixel change, the camera intrinsic parameters of the handheld imaging device, and the initial coordinates of the stable markers and the markers to be measured into a preset displacement correction model; solving the displacement correction model using the least squares method to obtain the true deformation of the markers to be measured. This invention can control the camera attitude by combining low-cost auxiliary means such as grid lines and levels built into handheld imaging devices, without the need for a stable fixed platform and complex hardware configuration, and maintains millimeter-level measurement accuracy. It breaks through the dependence of traditional technologies on stable platforms and professional equipment, and realizes lightweight, portable and low-cost deformation monitoring, providing a feasible path for building a public participation model for the intelligent perception of structural safety.

[0065] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal that includes that element.

[0066] Of course, those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided by this invention can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0067] It should be understood that the application of the present invention is not limited to the examples above. Those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the protection scope of the appended claims.

Claims

1. A portable method for monitoring structural deformation based on handheld imaging, characterized in that, The portable structural deformation monitoring method based on handheld imaging includes: In the target monitoring area, a cooperative marker array is deployed along the axis of the monitoring structure. The cooperative marker array includes stable markers and markers to be measured. The initial state image and the monitoring state image of the target monitoring area are acquired by a handheld imaging device. The vertical pixel coordinates of the stable marker point and the marker point to be measured are extracted from the initial state image and the monitoring state image. The pixel change of the stable marker point and the marker point to be measured is calculated based on the vertical pixel coordinates. The pixel change, the camera intrinsic parameters of the handheld imaging device, and the initial coordinates of the stable marker and the marker to be measured are input into a preset displacement correction model. The displacement correction model is solved by the least squares method to obtain the true deformation of the marker to be measured.

2. The portable structural deformation monitoring method based on handheld imaging according to claim 1, characterized in that, The cooperative marker array includes: at least two of the stable markers and at least one of the markers to be measured; The stable markers are placed on the monitoring structure at positions assumed to be free from displacement, while the markers to be measured are placed at structural locations where deformation is to be monitored.

3. The portable structural deformation monitoring method based on handheld imaging according to claim 1, characterized in that, The step of acquiring initial state images and monitoring state images of the target monitoring area using a handheld imaging device, and extracting the vertical pixel coordinates of the stable marker points and the marker points to be measured from the initial state images and the monitoring state images, specifically includes: The handheld imaging device's built-in level and grid line functions are used to assist in adjusting the shooting posture, and the camera aperture, ISO sensitivity, and exposure time parameters of the handheld imaging device are locked. Once the aperture, ISO sensitivity, and exposure time parameters are locked, the handheld imaging device acquires the initial state image and the monitoring state image of the target monitoring area, respectively. Identify the first preset auxiliary pattern on the stable marker point and the second preset auxiliary pattern on the marker point to be tested from the initial state image and the monitoring state image, respectively; The first coordinate value of the center point of the first preset auxiliary pattern in the vertical direction in the image coordinate system and the second coordinate value of the center point of the second preset auxiliary pattern in the vertical direction in the image coordinate system are obtained by using a sub-pixel level positioning algorithm. The first coordinate value is used as the vertical pixel coordinate of the stable marker point, and the second coordinate value is used as the vertical pixel coordinate of the marker point to be tested.

4. The portable structural deformation monitoring method based on handheld imaging according to claim 1, characterized in that, The displacement correction model is constructed based on the pinhole imaging model and includes correction terms; The displacement correction model is used to jointly model and solve the constraints of the cooperative marker array, so as to decouple the self-motion parameters of the handheld imaging device from the actual deformation of the structure. The correction term is used to correct the imaging scale error caused by the pose change of the handheld imaging device.

5. The portable structural deformation monitoring method based on handheld imaging according to claim 1, characterized in that, The step of inputting the respective pixel changes, the camera intrinsic parameters of the handheld imaging device, and the initial coordinates of the stable marker point and the marker point to be measured into a preset displacement correction model specifically includes: The pixel changes, camera intrinsic parameters of the handheld imaging device, and initial coordinates of the stable marker and the marker to be measured are input into a preset displacement correction model to establish a mapping relationship between pixel coordinate changes and actual structural settlement. ； in, This indicates a correction term for errors caused by movement when shooting with a handheld camera. This represents the pixel change of the stable marker point or the marker point to be measured. and These represent the magnification factors at the initial time and the observation time, respectively. This indicates the initial coordinates of the stable marker point or the marker point to be measured. The image principal point ordinate represents the intrinsic parameters of the camera. This indicates the vertical structural displacement of the marker point to be measured. This represents the vertical displacement component of the camera. This represents the distance along the optical axis from the camera to the target marker at the initial moment; This indicates the camera's own pitch angle change.

6. The portable structural deformation monitoring method based on handheld imaging according to claim 5, characterized in that, The step of solving the displacement correction model using the least squares method to obtain the true deformation of the target marker point specifically includes: Based on the mapping relationship established by the displacement correction model, equations are constructed for all the stable marker points and the marker points to be measured, and they are combined to form a set of equations with the self-motion parameters of the handheld imaging device and the structural vertical displacement of the marker points to be measured as unknowns. The equations are solved using the least squares method, and the self-motion parameters of the handheld imaging device and the vertical displacement of the target point are obtained simultaneously. The vertical displacement of the structure is then used as the actual deformation of the target point.

7. The portable structural deformation monitoring method based on handheld imaging according to claim 2, characterized in that, After obtaining the actual deformation of the target marker, the method further includes: In the same portable structural deformation monitoring task, multiple independent deformation results of multiple test markers are obtained and output.

8. A portable structural deformation monitoring system based on handheld imaging, characterized in that, The portable structural deformation monitoring system based on handheld imaging includes: The measuring point layout module is used to lay out a cooperative marker point array along the axis of the monitoring structure in the target monitoring area. The cooperative marker point array includes stable marker points and marker points to be measured. The image acquisition and calculation module is used to acquire the initial state image and the monitoring state image of the target monitoring area through a handheld imaging device, extract the vertical pixel coordinates of the stable marker point and the marker point to be measured from the initial state image and the monitoring state image, and calculate the pixel change of the stable marker point and the marker point to be measured respectively based on the vertical pixel coordinates. The deformation calculation module is used to input the respective pixel changes, the camera intrinsic parameters of the handheld imaging device, and the initial coordinates of the stable marker point and the marker point to be measured into a preset displacement correction model, and solve the displacement correction model by the least squares method to obtain the true deformation of the marker point to be measured.

9. A terminal, characterized in that, The terminal includes: a memory, a processor, and a portable structural deformation monitoring program based on handheld imaging stored in the memory and executable on the processor. When the portable structural deformation monitoring program based on handheld imaging is executed by the processor, it implements the steps of the portable structural deformation monitoring method based on handheld imaging as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a portable monitoring program for structural deformation based on handheld imaging, which, when executed by a processor, implements the steps of the portable monitoring method for structural deformation based on handheld imaging as described in any one of claims 1-7.

Images