Bridge deformation monitoring method and system based on image intelligent analysis

Abstract

The embodiment of the application provides a bridge deformation monitoring method and system based on image intelligent analysis, and belongs to the technical field of bridge deformation monitoring. The method unifies and connects the target pixel observation of each frame, the anchor point selection result, the pose solvable state, the pose solution result, the unsolvable time period record, the pixel drift compensation amount and the structure displacement output through a space target coordinate anchor chain. When the pose is unsolvable, the pixel drift of the unsolvable time period is constrained and regressed through anchor chain closure correction, so as to restore the continuity of the structure displacement output and reduce the gap error diffusion. Therefore, by constructing the space target coordinate anchor chain and performing the anchor chain closure correction in the gap stage when the pose is unsolvable, the bridge deformation result has continuity, determinability and traceability, so that the special scene of long-distance oblique view, strong shaking, strong light reflection and short-time shielding can be adapted.

Description

Technical Field

[0001] This invention relates to the field of bridge deformation monitoring technology, and specifically to a bridge deformation monitoring method and system based on intelligent image analysis. Background Technology

[0002] Bridge deformation monitoring is a crucial foundation for bridge structural safety assessment, operation and maintenance, and disaster early warning. Common bridge deformation monitoring methods in existing engineering projects include total station measurement, displacement gauge measurement, GNSS measurement, and image measurement. Among these, target-based image measurement offers advantages such as being non-contact, allowing for long-distance deployment, and covering large-span structures. It typically involves deploying identifiable targets at key locations on the bridge, acquiring images of the targets using cameras, and calculating changes in the spatial position of the targets to determine changes in bridge structural displacement or deflection.

[0003] However, in certain application scenarios, traditional target-based image measurement methods often fail to work stably. For example, in long-distance oblique-view monitoring of long-span bridges, cameras are usually fixed on bridge towers, high points on the bank, or maintenance platforms, resulting in long shooting distances and significant tilting of the viewing angle. At the same time, high-frequency micro-vibrations caused by vehicles passing on the bridge surface can cause inter-frame jitter in the images, while nighttime lighting or headlight reflections can lead to local overexposure, and fog, rain, snow, or low clouds can cause a decrease in contrast. More importantly, reference targets located at the edge or far end of the field of view may experience short-term occlusion, reflection distortion, or recognition failure, resulting in unstable visibility of the reference targets in consecutive image frames.

[0004] In the aforementioned scenarios, traditional methods typically rely on a sufficient number of reference targets to calculate the camera pose in each frame, and then calculate the spatial coordinates of the structural targets within that pose. If there are insufficient, poorly distributed, or degraded reference targets within a certain timeframe, the camera pose becomes unsolvable or unreliable, interrupting the structural displacement calculation and ultimately leading to fractures, jumps, or cumulative error propagation in the bridge deformation results. Since this problem occurs during the brief periods when reference targets are unavailable, and this period is often accompanied by complex lighting and jitter, simple interpolation or empirical corrections usually cannot balance continuity and accuracy, making it difficult to meet the requirements for continuous, traceable, and verifiable deformation curves under bridge operational conditions.

[0005] Therefore, how to maintain the continuous output of bridge deformation results and constrain and close the error propagation in unsolvable time periods under monocular long-distance squint monitoring and when the reference target is unavailable for a short time is a technical problem that urgently needs to be solved in the field of bridge target image measurement. Summary of the Invention

[0006] The purpose of this invention is to provide a bridge deformation monitoring method based on image intelligent analysis, so as to at least solve the problem that the camera pose cannot be solved and the structural displacement output is interrupted in the monocular long-distance squint monitoring scenario due to the short-term invisibility of the reference target or the unstable observation quality.

[0007] To achieve the above objectives, the first aspect of the present invention provides a bridge deformation monitoring method based on intelligent image analysis. The method includes: acquiring a bridge monitoring video frame sequence, camera calibration parameters, and a target spatial reference table, and establishing a target grouping relationship based on the target spatial reference table; performing target region detection on each frame image based on the bridge monitoring video frame sequence to generate a candidate target region set; performing target confirmation and pixel observation extraction based on the candidate target region set to form a target pixel observation set including target identifier, target pixel coordinates, and observation quality; constructing a spatial target coordinate anchor chain based on the target pixel observation set and the target grouping relationship, and outputting a pose solvable flag and an anchor point set; when the pose solvable flag indicates solvability, calculating the camera pose based on the camera calibration parameters and the anchor point set, and performing spatial displacement calculation on the structural target under the camera pose to output the bridge deformation result; when the pose solvable flag indicates insolvability, performing anchor chain correction on the time period to which the insolvable frame belongs based on the spatial target coordinate anchor chain, and outputting the bridge deformation result based on the correction result.

[0008] Optionally, the process involves acquiring bridge monitoring video frame sequences, camera calibration parameters, and a target spatial reference table, and establishing target grouping relationships based on the target spatial reference table. This includes: acquiring the camera intrinsic parameter matrix and distortion parameters as the camera calibration parameters; acquiring the target spatial reference table; wherein the target spatial reference table includes target identifiers, target types, initial spatial coordinates of the targets, unit vectors of structural target displacement directions, and target physical dimensions; and dividing the targets into a reference target set and a structural target set based on the target types. The reference target set is used for camera pose calculation, and the structural target set is used for bridge deformation output.

[0009] Optionally, target region detection is performed on each frame of the bridge monitoring video frame sequence to generate a set of candidate target regions. This includes: calling a target detection model to output a set of candidate boxes for each frame of the image; wherein the set of candidate boxes includes candidate box position parameters and candidate box confidence; performing confidence threshold filtering on the set of candidate boxes to retain candidate boxes that meet a preset confidence threshold; performing pixel scale consistency verification on the filtered candidate boxes based on the target physical size in the target space reference table to remove candidate boxes that do not meet a preset scale range; and mapping the candidate boxes that pass the confidence threshold filtering and pixel scale consistency verification to a set of candidate target regions.

[0010] Optionally, target confirmation and pixel observation extraction are performed based on the candidate target region set to form a target pixel observation set containing target identifier, target pixel coordinates, and observation quality. This includes: performing target confirmation processing in each candidate target region to output a target identifier and a set of target corner pixel coordinates or target center pixel coordinates; generating target pixel coordinates based on the set of target corner pixel coordinates or the target center pixel coordinates; generating observation quality for each target pixel coordinate; wherein the observation quality is obtained by combining at least two processing results from whether decoding or fitting is successful, corner consistency or fitting residual, and consistency between the candidate region area and a preset range; and summarizing the target identifier, target pixel coordinates, and observation quality to form a target pixel observation set.

[0011] Optionally, a spatial target coordinate anchor chain is constructed based on the target pixel observation set and the target grouping relationship, and a pose solvable flag and an anchor point set are output, including: generating a baseline target visibility record based on the target pixel observation set and the baseline target set, and generating a structural target visibility record based on the target pixel observation set and the structural target set; performing sorting and filtering based on observation quality in the baseline target visibility record, and generating an anchor point set according to the rule that the number of anchor points is not less than a preset threshold and the dispersion of anchor points in the image plane meets a preset condition; generating a pose solvable flag based on the number of anchor point sets and the observation quality threshold, and writing the pose solvable flag and the anchor point set into the spatial target coordinate anchor chain.

[0012] Optionally, if the pose solvable flag indicates that the pose is solvable, the camera pose is calculated based on the camera calibration parameters and the anchor point set, including: obtaining the initial spatial coordinates of the corresponding reference target based on the anchor point set, and obtaining the corresponding reference target pixel coordinates based on the target pixel observation set, to construct a set of corresponding point pairs of two-dimensional pixel coordinates and three-dimensional spatial coordinates; performing pose calculation and outlier culling functions on the corresponding point pairs based on the camera intrinsic parameter matrix and distortion parameters to output the camera pose and inlier set; calculating the reprojection error based on the camera pose and generating a pose error evaluation value; and writing the camera pose, the inlier set, and the pose error evaluation value into the spatial target coordinate anchor chain.

[0013] Optionally, the spatial displacement calculation of the structural target under the camera pose to output the bridge deformation result includes: obtaining the initial spatial coordinates and displacement direction unit vector of the structural target based on the target spatial reference table, and obtaining the corresponding structural target pixel coordinates based on the target pixel observation set; constructing a one-dimensional displacement parameterized spatial point along the displacement direction based on the initial spatial coordinates and displacement direction unit vector of the structural target; calculating the zero displacement projection point and the micro-displacement projection point based on the camera pose and camera intrinsic parameter matrix, and calculating the pixel sensitivity vector; calculating the one-dimensional displacement of the structural target based on the pixel sensitivity vector, the structural target pixel coordinates and the zero displacement projection point, and outputting the one-dimensional displacement as the bridge deformation.

[0014] Optionally, if the pose solvable flag indicates that the frame is unsolvable, anchor chain closure correction is performed on the time period to which the unsolvable frame belongs based on the spatial target coordinate anchor chain, and the bridge deformation result is output based on the closure correction result. This includes: identifying continuous unsolvable time periods based on the pose solvable flag sequence in the spatial target coordinate anchor chain, determining the continuous unsolvable time periods as gap time periods, and recording the start and end times of the gap time periods and the nearest solvable times at both ends of the gap; selecting a fixed background template region in the image of the last solvable time before the start of the gap, and processing each frame image within the gap time period on the fixed background template. Template matching is performed near the region to generate a sequence of pixel drift compensation amounts; the actual pixel equivalent drift at the first solvable moment after the gap end is obtained, and the closure residual is calculated based on the actual pixel equivalent drift and the predicted pixel drift compensation amount; the closure residual is distributed to each frame in the gap time period according to the time weight to generate the closed-corrected pixel drift compensation amount; the pixel domain correction of the structural target pixel coordinates is performed based on the camera pose at the last solvable moment before the gap start and the closed-corrected pixel drift compensation amount, and the spatial displacement calculation is performed based on the corrected structural target pixel coordinates to output the bridge deformation result.

[0015] Optionally, the bridge deformation monitoring method based on image intelligent analysis further includes: generating a deformation result confidence level based on the observation quality in the target pixel observation set, the pose error evaluation value in the camera pose calculation process, and the closure residual in the anchor chain closure correction process; wherein, the deformation result confidence level increases with the increase of observation quality, decreases with the increase of pose error evaluation value, and decreases with the increase of closure residual; the bridge deformation result and the deformation result confidence level are jointly output, and an alarm event is output when the bridge deformation result exceeds a preset displacement threshold and the deformation result confidence level exceeds a preset alarm threshold.

[0016] A second aspect of the present invention provides a bridge deformation monitoring system based on intelligent image analysis. The system is used to execute the aforementioned bridge deformation monitoring method based on intelligent image analysis. The system includes: an establishment unit, used to acquire a bridge monitoring video frame sequence, camera calibration parameters, and a target spatial reference table, and to establish target grouping relationships based on the target spatial reference table; a generation unit, used to perform target region detection on each frame of the bridge monitoring video frame sequence to generate a set of candidate target regions; and an extraction unit, used to perform target confirmation and pixel observation extraction based on the set of candidate target regions to form a result including target identifiers and target pixel coordinates. The system includes: a target pixel observation set with observation quality; a construction unit for constructing a spatial target coordinate anchor chain based on the target pixel observation set and the target grouping relationship, and outputting a pose solvable flag and an anchor point set; a solution unit for calculating the camera pose based on the camera calibration parameters and the anchor point set when the pose solvable flag indicates solvability, and performing spatial displacement calculation on the structural target under the camera pose to output the bridge deformation result; and a correction unit for performing anchor chain closure correction on the time period to which the unsolvable frame belongs based on the spatial target coordinate anchor chain when the pose solvable flag indicates unsolvability, and outputting the bridge deformation result.

[0017] Through the above technical solution, this invention proposes a data organization and correction mechanism centered on a spatial target coordinate anchor chain. This mechanism uses the spatial target coordinate anchor chain to uniformly connect the observation of target pixels in each frame, anchor point selection results, pose solvable states, pose calculation results, records of unsolvable time periods, pixel drift compensation, and structural displacement output. When an unsolvable pose time period occurs, anchor chain closure correction constrains and regresses the pixel drift during that period, thereby restoring the continuity of structural displacement output and reducing gap error propagation. Therefore, by constructing a spatial target coordinate anchor chain and performing anchor chain closure correction during the pose-unsolvable gap phase, the bridge deformation results possess continuity, determinability, and traceability, thus adapting to special scenarios such as long-distance squint, strong vibration, strong light reflection, and short-term occlusion.

[0018] Other features and advantages of the embodiments of the present invention will be described in detail in the following detailed description section. Attached Figure Description

[0019] The accompanying drawings are provided to further illustrate embodiments of the present invention and form part of the specification. They are used together with the following detailed description to explain the embodiments of the present invention, but do not constitute a limitation thereof. In the drawings:

[0020] Figure 1 This is a flowchart illustrating the steps of a bridge deformation monitoring method based on image intelligent analysis according to one embodiment of the present invention.

[0021] Figure 2 This is a system structure diagram of a bridge deformation monitoring system based on image intelligent analysis provided by one embodiment of the present invention. Detailed Implementation

[0022] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0023] like Figure 1 As shown, this invention provides a bridge deformation monitoring method based on image intelligent analysis, the method comprising:

[0024] Step S1: Obtain the bridge monitoring video frame sequence, camera calibration parameters, and target spatial reference table, and establish target grouping relationships based on the target spatial reference table.

[0025] In this embodiment of the invention, by acquiring a bridge monitoring video frame sequence and establishing a target grouping relationship consistent with a spatial reference, subsequent image processing steps and spatial coordinate calculation steps have a unified reference basis. In one implementation, step S1 includes the following execution sub-steps:

[0026] Step S101: Obtain camera calibration parameters. Specifically, step S101 is used to obtain the camera's imaging geometric constraints, so that the relationship between pixel coordinates and spatial projection can be determined.

[0027] Camera calibration parameters include at least the camera intrinsic parameter matrix and distortion parameters. The camera intrinsic parameter matrix describes the relationship between intrinsic parameters such as focal length and principal point, while the distortion parameters describe the impact of lens distortion on pixel coordinates. By introducing these calibration parameters into subsequent projection or back-projection processing, pixel measurement errors and lens distortion errors can be distinguished, reducing systematic biases caused by distortion under long-distance telephoto conditions.

[0028] Optionally, camera calibration parameters can be obtained through a calibration board during the installation and commissioning phase, or imported from existing calibration files in the project. In another possible implementation, preset calibration parameters can be used as initial values ​​based on the lens model and imaging resolution, and then the projection residual can be lightly checked through a stable benchmark target during the operation phase to improve deployment efficiency.

[0029] Step S102: Obtain the target spatial reference table. Specifically, step S102 is used to obtain the static reference information of the target, so that subsequent observations can be interpreted under the same bridge coordinate reference.

[0030] The target spatial reference table includes at least: target identifier, target type, initial spatial coordinates of the target, unit vector of the displacement direction of the structural target, and physical dimensions of the target. Specifically, the target identifier establishes a correspondence between target observations in the image and entries in the reference table; the target type distinguishes between the reference target and the structural target; the initial spatial coordinates of the target provide a three-dimensional reference point for pose calculation; the unit vector of the displacement direction constrains the deformation degrees of freedom of the structural target, ensuring the solvability of displacement calculations under monocular imaging conditions; and the physical dimensions of the target are used for pixel scale consistency verification during the image detection stage to suppress false detections. It should be noted that the target spatial reference table can be generated through measurement import, design coordinate import, or manual annotation and input.

[0031] Step S103: Establish target grouping relationships. Specifically, step S103 is used to divide the targets into a baseline target set and a structural target set, and to clarify the different roles of the two in the subsequent processing chain.

[0032] In this embodiment, the reference target is preferably placed on the stable components of the bridge abutment or tower, or in areas where the deflection of the main beam does not change significantly, to reflect the imaging changes caused by camera line-of-sight drift or support micro-movements. The structural target is preferably placed at the mid-span, critical sections, or locations of components prone to deformation, to output the bridge deformation results. Through the above grouping, subsequent pose calculations can preferentially rely on the reference target, and structural displacement calculations can be performed under conditions of pose stability or compensable drift, thereby avoiding misjudging camera drift as bridge deformation.

[0033] Optionally, the number of reference targets can be set to no less than four to meet the basic geometric conditions for pose calculation; the number of structural targets can be set to one or more based on the number of monitoring sections to output multi-point deformation curves. The above quantity settings can be adjusted according to the available deployment conditions on site and are not limited to specific values.

[0034] Step S2: Target region detection is performed on each frame of the bridge monitoring video frame sequence to generate a set of candidate target regions.

[0035] In this embodiment of the invention, in scenarios involving long-distance strabismus, complex backgrounds, significant lighting changes, and frequent local occlusion, coarse localization of the target region is first performed on the entire frame image. This limits subsequent target confirmation and fine localization to the candidate region, reducing the probability of false detection and minimizing unnecessary full-image search overhead. In one implementation, step S2 includes the following sub-steps:

[0036] Step S201: Generate a set of candidate boxes. Specifically, step S201 is used to quickly locate regions that may contain the target from the entire frame image. In practice, a mature object detection model can be invoked to output the set of candidate boxes and their confidence scores. This mature object detection model can be a general detection model that can be directly deployed in existing projects or a detection model trained for the target category.

[0037] It should be noted that this invention only uses the mature model for candidate region generation and does not require the output of precise corner points or precise center coordinates in this stage.

[0038] Step S202: Confidence Threshold Filtering. Specifically, step S202 is used to perform reliability filtering on the candidate box set. In actual implementation, the confidence level of the candidate boxes can be compared with a preset threshold, and only candidate boxes that meet the threshold condition can be retained.

[0039] Optionally, the confidence threshold can be adaptively set according to lighting conditions, rain and fog conditions, and lens focal length. For example, the threshold can be appropriately reduced in foggy or low-contrast conditions at night to improve recall, while the threshold can be appropriately increased in strong reflective or complex background conditions to reduce false detections.

[0040] Step S203: Pixel scale consistency verification. Specifically, step S203 is used to perform secondary constraints on the candidate bounding boxes using the target physical dimensions in the target space reference table.

[0041] In practice, a reasonable pixel scale range can be preset based on the physical size of the target and the imaging conditions of the camera. The pixel width and height of the candidate box are then judged to be consistent with this range. Candidate boxes that do not meet the scale range are eliminated to suppress false detection of similar textures or strong reflective spots.

[0042] Step S204: Form and output a set of candidate target regions. Specifically, step S204 maps the set of candidate boxes that have passed the filtering and verification to a set of candidate target regions and outputs it to step S30. Through this output relationship, step S30 does not need to perform a target search on the entire image, thereby improving processing stability.

[0043] S3: Based on the candidate target region set, perform target confirmation and pixel observation extraction to form a target pixel observation set containing target identifier, target pixel coordinates and observation quality.

[0044] In this embodiment of the invention, the obtained candidate regions are converted into structured observation data that can be used for spatial calculation. This observation data includes not only target pixel coordinates representing the location of the target, but also observation quality representing the reliability of the observation, so that subsequent anchor point selection, pose calculation, and gap determination can make interpretable decisions based on quality information. In one implementation, step S3 includes the following sub-steps:

[0045] Step S301: Target Confirmation and Identification. Specifically, step S301 is used to confirm whether a target actually exists within the candidate region and to establish a correspondence between the identifier and the target spatial reference table. In practice, when the target is an coded target, a mature encoding / decoding module can be called to output the target identifier and the set of corner pixel coordinates; when the target is a circular or reflective target, brightness segmentation and contour extraction can be performed on the candidate region, and the target center pixel coordinates can be output through ellipse fitting.

[0046] It should be noted that the target form is not limited to coded targets or circular targets. Any target that can output a stable target identifier and geometric feature points for subsequent localization within the candidate area can be used as an optional implementation of the present invention.

[0047] Step S302: Generate target pixel coordinates. Specifically, step S302 is used to uniformly convert corner points or center geometric features into a single target pixel coordinate representation so that subsequent pose and displacement calculations can use a unified input.

[0048] For example, when generating target pixel coordinates using a set of corner pixel coordinates, the expression is as follows: Where u represents the target pixel coordinates; This represents the pixel coordinates of the k-th corner point. For example, when generating target pixel coordinates using the center pixel coordinates, the target pixel coordinates are directly taken from the fitted center.

[0049] With the unified representation method described above, subsequent steps can perform anchor point selection and spatial calculation without distinguishing between target types, thereby reducing process complexity.

[0050] Step S303: Generate observation quality. Specifically, step S303 is used to generate observation quality for the coordinates of each target pixel, which serves as the basis for subsequent anchor point selection and gap determination.

[0051] In this embodiment, the observation quality can be obtained by combining at least two types of evaluation information, including: whether decoding or fitting is successful, the geometric consistency of corner points or the size of the fitting residual, the consistency between the candidate region area and the preset range, and the edge sharpness or local contrast index of the candidate region. By introducing observation quality, the impact of such observations on subsequent pose calculations can be actively reduced in cases of strong reflections, local overexposure, rain and fog blurring, or corner point jitter caused by heat wave disturbances, thus avoiding pose divergence triggered by erroneous observations.

[0052] Optionally, the combination of observation quality can be weighted summation, segmented rule judgment, or grade scoring, without limiting the specific form; however, it should be ensured that the observation quality reflects the basic law that the reliability of observations increases with the degree of identifiability.

[0053] Step S304: Form and output the target pixel observation set. Specifically, step S304 is used to summarize the target identifier, target pixel coordinates, and observation quality to form a target pixel observation set, which is then output to steps S40 and S50. Through the output of this set, subsequent steps can simultaneously acquire the observation results of the reference target and the structural target within the same frame, and select and control the target based on the observation quality.

[0054] Step S4: Construct a spatial target coordinate anchor chain based on the target pixel observation set and the target grouping relationship, and output the pose solvable flag and the anchor point set.

[0055] In this embodiment of the invention, by organizing frame-by-frame observations into a continuously interpretable chain structure and forming a determination result on whether the pose can be reliably solved in each frame, clear branching conditions are provided for direct solution of subsequent solvable frames and for gap closure of unsolvable frames. In one embodiment, step S4 includes the following sub-steps:

[0056] Step S401: Generate a visibility record. Specifically, step S401 records the reference target and structural target whose pixel observations can be successfully obtained in each frame. In practice, the target pixel observation set can be split into a reference target visibility record and a structural target visibility record based on the target identifier and target type. This visibility record helps to clarify whether the current frame can be used for camera pose calculation, whether it can be used for structural displacement calculation, and which processing path should be entered when the structural target or reference target is missing.

[0057] Step S402: Generate an anchor point set. Specifically, step S402 is used to select an anchor point set for pose calculation from the visible reference targets. In practice, the reference targets can first be sorted according to observation quality, prioritizing targets with higher observation quality; then, constraints can be applied based on the dispersion of the anchor points' distribution on the image plane to ensure that the anchor points cover a larger image area as much as possible, thereby reducing the risk of geometric degradation. For example, in a distant oblique view scene, if the anchor points are concentrated in a small area on one side of the bridge tower, the pose calculation is more sensitive to pixel noise and prone to instability; dispersion constraints can make the anchor point distribution more uniform, thereby improving the robustness of the pose calculation.

[0058] It should be noted that the threshold for the number of anchor points and the threshold for dispersion can be set according to the number of targets deployed on site, the field of view, and the resolution, and there is no limit to the specific values.

[0059] Step S403: Generate a pose-solvable flag and write it to the anchor chain. Specifically, step S403 is used to determine whether the current frame has reliable pose calculation conditions by comprehensively considering the number of anchor points and the anchor point observation quality. In actual implementation, the pose-solvable flag can be set to solvable when the number of anchor points is not less than a preset threshold and the anchor point observation quality meets the preset requirements; otherwise, it is set to unsolvable. By explicitly generating the pose-solvable flag, the divergence caused by the traditional method forcibly calculating the pose when there are insufficient anchor points can be avoided, and boundary conditions are provided for subsequent gap closure correction.

[0060] Step S404: Write and output the spatial target coordinate anchor chain. Specifically, step S404 is used to write visibility records, anchor point sets, pose solvability flags, and other information into the spatial target coordinate anchor chain, and output the spatial target coordinate anchor chain and pose solvability status. It should be noted that the implementation of the fields of the spatial target coordinate anchor chain is not limited; in one optional implementation, it may include at least: frame number, reference target visibility record, structural target visibility record, anchor point set, pose solvability flag, pose calculation result, pose error evaluation value, gap time period record, pixel drift compensation amount, and structural displacement output record. Through this anchor chain, any output displacement point can be traced back to its dependent observation, screening, and correction source, thereby improving engineering usability.

[0061] S5: When the pose solvable flag indicates that the pose is solvable, calculate the camera pose based on the camera calibration parameters and the set of anchor points, and perform spatial displacement calculation on the structural target under the camera pose to output the bridge deformation result.

[0062] In this embodiment of the invention, when the pose solvability flag indicates that the pose is solvable, the camera pose is calculated based on the anchor point set, and spatial displacement calculation is performed on the structural target under the camera pose to output the bridge deformation result. In one embodiment, step S5 includes the following sub-steps:

[0063] Step S501: Construct pixel-spatial correspondence pairs. Specifically, step S501 establishes a correspondence between image observations and spatial references, providing input for subsequent camera pose calculation. In practice, the initial spatial coordinates of the corresponding reference target can be read based on the anchor point set, and the corresponding reference target pixel coordinates can be read from the target pixel observation set, thus forming a set of one-to-one correspondence pairs between two-dimensional pixel coordinates and three-dimensional spatial coordinates. Through this set of corresponding point pairs, the camera pose calculation problem can be transformed into a standard two-dimensional-three-dimensional correspondence problem, facilitating subsequent calculation.

[0064] Step S502: Calculate the camera pose and output the inlier set. Specifically, step S502 is used to robustly calculate the camera pose even in the presence of occlusion, reflection, or local observation anomalies. In practice, based on the camera intrinsic parameter matrix and distortion parameters, a mature pose calculation and outlier culling function (such as a pose calculation function based on random sampling consistency) can be called on the corresponding point pair set to output the camera pose and inlier set.

[0065] Step S503: Calculate the reprojection error and write it to the anchor chain. Specifically, step S503 is used to evaluate the quality of the pose calculation results and write the evaluation results to the spatial target coordinate anchor chain to support subsequent confidence and backtracking. In actual implementation, the reprojection error can be calculated based on the camera pose, satisfying the following formula:

[0066] ;

[0067] Where e represents the reprojection error; u represents the pixel coordinates of the reference target; K represents the camera intrinsic parameter matrix; [R|t] represents the combination of the rotation matrix and translation vector of the camera pose; X represents the initial spatial coordinates of the reference target; π(·) represents the projection normalization operator; and ||·|| represents the Euclidean norm.

[0068] In practice, the reprojection errors corresponding to the inlier set can be aggregated to form a pose error evaluation value. The camera pose, inlier set, and pose error evaluation value are then written into the spatial target coordinate anchor chain. Even if gap closure correction occurs later, the reliability of the pose at both ends of the gap can be clearly determined.

[0069] Step S504: Solve the one-dimensional displacement of the structural target and output the bridge deformation results. Specifically, step S504 is used to output the solvable displacement of the structural target under monocular conditions. In actual implementation, the initial spatial coordinates and displacement direction unit vector of the structural target can be obtained based on the target spatial reference table, and the corresponding structural target pixel coordinates can be obtained from the target pixel observation set; then, a one-dimensional displacement parameterized spatial point along the displacement direction is constructed, for example, the product of the initial spatial coordinates of the structural target, the displacement direction unit vector, and the one-dimensional displacement along the displacement direction is used as the displacement parameterized spatial point.

[0070] Furthermore, to solve for the one-dimensional displacement, the pixel sensitivity vector can be calculated using the finite difference method. Specifically, the zero-displacement projection point and the micro-displacement projection point can be calculated based on the camera pose and the camera intrinsic parameter matrix, and the pixel sensitivity vector can be calculated, satisfying the following formula: ; ; ;in, Indicates the zero-displacement projection point; Δ represents the micro-displacement projection point; Δ represents the finite difference step size; s represents the pixel sensitivity vector.

[0071] After obtaining the pixel sensitivity vector, the one-dimensional displacement can be calculated based on the pixel coordinates of the structural target and the zero-displacement projection point, satisfying the following formula: Where T represents the vector transpose; ε represents a positive number to prevent the denominator from being zero.

[0072] Using the above method, the structural target displacement can be stably output in the pose-solvable frame, thereby forming a reliable segment of the bridge deformation curve.

[0073] It should be noted that the unit vector of displacement direction can be set according to the monitoring target. For example, when the monitoring target is the vertical deflection of the main beam, the unit vector of displacement direction can be set to the vertical direction; when the monitoring target is the lateral displacement or the relative displacement in a specific direction, it can also be set to the corresponding direction.

[0074] S6: If the pose solvable flag indicates that the pose is unsolvable, perform anchor chain correction on the time period to which the unsolvable frame belongs based on the spatial target coordinate anchor chain, and output the bridge deformation result based on the correction result.

[0075] In this embodiment of the invention, when the pose solvability flag indicates that the pose is unsolvable, anchor chain closure correction is performed on the time period to which the unsolvable frame belongs based on the spatial target coordinate anchor chain, and the bridge deformation result is output based on the closure correction result. It should be noted that when the pose solvability flag indicates that the pose is unsolvable, it usually means that the number of reference targets is insufficient, the distribution is degraded, or the observation quality is significantly reduced. If the pose is still forcibly solved in this case, noise is easily introduced into the displacement output and causes sudden jumps in the curve.

[0076] Therefore, this invention explicitly marks the continuous unsolvable time period as a gap time period, and uses the solvable frames at both ends of the gap to establish a closed constraint, correcting and writing back the pixel drift compensation amount within the gap, thereby restoring the continuity of the displacement output. In one embodiment, the pose unsolvable branch includes the following sub-steps:

[0077] Step S601: Identify the gap time period and record the anchor chain nodes. Specifically, step S601 is used to identify continuous unsolvable time periods in the spatial target coordinate anchor chain and clarify their boundary conditions. In actual implementation, continuous unsolvable frame segments can be identified based on the pose solvable flag sequence, which are then determined as gap time periods. At the same time, the start and end times of the gap time period and the nearest solvable times at both ends of the gap are recorded.

[0078] Therefore, by recording the gap boundary, the alignment time between the drift and the actual drift can be clearly predicted during subsequent calculation of the closed residual, and the data segment can be clearly marked in the output results as obtained after closed correction, thereby improving engineering interpretability.

[0079] Step S602: Generate a pixel drift compensation sequence. Specifically, step S602 is used to obtain a pixel drift compensation sequence reflecting camera imaging drift within the gap time period. In actual implementation, a fixed background template region can be selected in the image of the last solvable moment before the start of the gap, and template matching can be performed on each frame image within the gap time period near this template region to generate a pixel drift compensation sequence. The fixed background template region is preferably a region that does not significantly deform with the main beam, such as the fixed components of the bridge tower, the stable texture area of ​​the guardrail, or the surface texture area of ​​the bridge abutment, so that the template matching result mainly reflects the imaging drift caused by camera shake, support micro-motion, or atmospheric disturbance.

[0080] Optionally, the template matching method can be normalized cross-correlation matching, squared difference matching, or other mature template matching methods; the template search range can be preset according to the camera shake amplitude to avoid mismatches caused by an excessively large search range. The above implementations do not constitute a limitation of the present invention.

[0081] Step S603: Obtain the true pixel equivalent drift and calculate the closure residual. Specifically, step S603 is used to construct the closure constraint at the first solvable moment after the gap endpoint. In actual implementation, the true pixel equivalent drift can be constructed at this solvable moment using the average difference between the observed pixel coordinates of the reference target and the pixel coordinates obtained based on the solvable pose projection. The calculation of this true pixel equivalent drift can satisfy the following form: ;in, This represents the actual pixel equivalent shift; M represents the number of baseline targets involved in the calculation. Represents the observed pixel coordinates of the j-th reference target; K represents the camera intrinsic parameter matrix; [R|t] represents the camera pose at the solvable time step; Let represent the initial spatial coordinates of the j-th reference target. Further, the pixel drift compensation amount extending from template matching to the solvable time point within the gap time interval can be used as the predicted pixel drift compensation amount, and the closure residual can be calculated accordingly. Specifically, the closure residual can be characterized by the difference between the actual pixel equivalent drift and the predicted pixel drift compensation amount. Through the above closure residual, the deviation between the template matching drift continuation and the reference target pose anchoring result can be quantitatively characterized, providing a basis for subsequent write-back correction.

[0082] Step S604: Closure correction and write-back of pixel drift compensation. Specifically, step S604 is used to smoothly distribute the closure residual to each frame within the gap time period to avoid abrupt jumps at the end of the gap. In actual implementation, the closure residual can be distributed to each frame within the gap time period according to time weight to generate the pixel drift compensation amount after closure correction. Specifically, the pixel drift compensation amount after closure correction can be represented by subtracting the product of the time weight and the closure residual from the pixel drift compensation amount generated by template matching. The time weight can be represented by dividing the difference between the current time and the start time of the gap time period by the difference between the end time of the gap time period and the start time of the gap time period.

[0083] Therefore, by using this smooth distribution method, the closure error can be evenly spread across the entire gap, thus making the displacement curve continuous and closable at the gap boundary.

[0084] Step S605: Pixel-domain correction and recovery of displacement output during the gap time period. Specifically, step S605 is used to recover the structural displacement output during the gap time period without reintroducing unreliable pose calculations. In actual implementation, pixel-domain translation correction can be performed on the structural target pixel coordinates based on the pixel drift compensation amount after closure correction. Specifically, the corrected structural target pixel coordinates can be characterized by the difference between the original structural target pixel coordinates and the pixel drift compensation amount after closure correction.

[0085] Subsequently, using the camera pose at the last solvable moment before the gap start as a reference pose, the corrected structural target pixel coordinates are substituted into the aforementioned one-dimensional displacement calculation process of the structural target to obtain the structural displacement output within the gap time period. This output, along with the closure correction information, is then written into the spatial target coordinate anchor chain.

[0086] It should be noted that the present invention does not limit the use of the last solvable pose before the gap start point as the reference pose; in another optional embodiment, the first solvable pose after the gap end point or the reference pose formed by interpolation of the poses at both ends of the gap can also be used as the reference condition. As long as it can be used in conjunction with the pixel domain drift compensation amount to achieve continuous displacement output, it should fall within the protection scope of the present invention.

[0087] In a preferred embodiment, after outputting the bridge deformation results, a confidence level for the deformation results can be generated based on the observation quality, pose error evaluation value, and closure residual, and the bridge deformation results and confidence level are output together. This confidence level is used to distinguish the reliability difference between the direct solution results of solvable frames and the gap closure correction results, and can be used for subsequent alarm threshold control.

[0088] Furthermore, alarm events can be associated with their corresponding spatial target coordinate anchor chain segment index, so that any alarm can be traced back to the corresponding candidate target area set, target pixel observation set, anchor point set, camera pose, gap record and drift compensation source, thereby improving interpretability and verifiability in operation and maintenance scenarios.

[0089] like Figure 2 As shown, this invention provides a bridge deformation monitoring system based on intelligent image analysis. The system is used to execute the aforementioned bridge deformation monitoring method based on intelligent image analysis. The system includes: an establishment unit, used to acquire a bridge monitoring video frame sequence, camera calibration parameters, and a target spatial reference table, and to establish target grouping relationships based on the target spatial reference table; a generation unit, used to perform target region detection on each frame of the bridge monitoring video frame sequence to generate a set of candidate target regions; and an extraction unit, used to perform target confirmation and pixel observation extraction based on the set of candidate target regions to form a result containing target identifiers and target pixel coordinates. The system includes: a target pixel observation set with observation quality; a construction unit for constructing a spatial target coordinate anchor chain based on the target pixel observation set and the target grouping relationship, and outputting a pose solvable flag and an anchor point set; a solution unit for calculating the camera pose based on the camera calibration parameters and the anchor point set when the pose solvable flag indicates solvability, and performing spatial displacement calculation on the structural target under the camera pose to output the bridge deformation result; and a correction unit for performing anchor chain correction on the time period to which the unsolvable frame belongs based on the spatial target coordinate anchor chain when the pose solvable flag indicates unsolvability, and outputting the bridge deformation result based on the correction result.

[0090] Those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing related hardware. This program is stored in a storage medium and includes several instructions to cause a microcontroller, chip, or processor to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0091] The optional embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the embodiments of the present invention are not limited to the specific details described above. Within the scope of the technical concept of the embodiments of the present invention, various simple modifications can be made to the technical solutions of the embodiments of the present invention, and these simple modifications all fall within the protection scope of the embodiments of the present invention. It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. To avoid unnecessary repetition, the embodiments of the present invention will not further describe the various possible combinations.

[0092] Furthermore, various different embodiments of the present invention can be combined in any way, as long as they do not violate the spirit of the embodiments of the present invention, they should also be regarded as the content disclosed by the embodiments of the present invention.

Claims

1. A bridge deformation monitoring method based on image intelligent analysis, characterized in that, The method includes: Obtain bridge monitoring video frame sequences, camera calibration parameters, and target spatial reference tables, and establish target grouping relationships based on the target spatial reference tables; Target region detection is performed on each frame of the bridge monitoring video frame sequence to generate a set of candidate target regions. Based on the candidate target region set, target confirmation and pixel observation extraction are performed to form a target pixel observation set containing target identifier, target pixel coordinates, and observation quality; Based on the target pixel observation set and the target grouping relationship, a spatial target coordinate anchor chain is constructed, and the pose solvable flag and anchor point set are output. When the pose solvable flag indicates that the pose is solvable, the camera pose is calculated based on the camera calibration parameters and the set of anchor points, and the spatial displacement of the structural target is calculated under the camera pose to output the bridge deformation result. When the pose solvable flag indicates that the pose is unsolvable, anchor chain closure correction is performed on the time period to which the unsolvable frame belongs based on the spatial target coordinate anchor chain, and the bridge deformation result is output; specifically including: Based on the pose solvable flag sequence in the spatial target coordinate anchor chain, a continuous unsolvable time period is identified, the continuous unsolvable time period is determined as a gap time period, and the start and end times of the gap time period and the nearest solvable time at both ends of the gap are recorded. In the image of the last solvable moment before the gap start, a fixed background template region is selected, and template matching is performed on each frame image within the gap time period near the fixed background template region to generate a pixel drift compensation sequence. The true pixel equivalent drift is obtained at the first solvable time after the end of the gap, and the closure residual is calculated based on the true pixel equivalent drift and the predicted pixel drift compensation amount. The closure residual is distributed to each frame within the gap time period according to the time weight to generate the pixel drift compensation amount after closure correction; Pixel domain correction is performed on the structural target pixel coordinates based on the camera pose at the last solvable moment before the gap start point and the pixel drift compensation amount after closure correction, and spatial displacement calculation is performed based on the corrected structural target pixel coordinates to output the bridge deformation result.

2. The bridge deformation monitoring method based on image intelligent analysis according to claim 1, characterized in that, Acquire bridge monitoring video frame sequences, camera calibration parameters, and a target spatial reference table, and establish target grouping relationships based on the target spatial reference table, including: The camera intrinsic parameter matrix and distortion parameters are obtained as the camera calibration parameters; Obtain the target space reference table; wherein, the target space reference table includes target identifier, target type, initial spatial coordinates of the target, unit vector of structural target displacement direction, and target physical size; Based on the target type, the targets are divided into a baseline target set and a structural target set; wherein, the baseline target set is used for camera pose calculation, and the structural target set is used for bridge deformation output.

3. The bridge deformation monitoring method based on image intelligent analysis according to claim 1, characterized in that, Target region detection is performed on each frame of the bridge monitoring video frame sequence to generate a set of candidate target regions, including: For each frame of image, the object detection model is called to output a set of candidate boxes; wherein, the set of candidate boxes includes candidate box position parameters and candidate box confidence scores; Perform confidence threshold filtering on the candidate box set to retain candidate boxes that meet the preset confidence threshold; Pixel scale consistency checks are performed on the filtered candidate boxes based on the target physical size in the target space reference table to remove candidate boxes that do not meet the preset scale range. The candidate boxes filtered by the confidence threshold and verified by the pixel scale are mapped to a set of candidate target regions.

4. The bridge deformation monitoring method based on image intelligent analysis according to claim 3, characterized in that, Based on the candidate target region set, target confirmation and pixel observation extraction are performed to form a target pixel observation set containing target identifiers, target pixel coordinates, and observation quality, including: Perform target confirmation processing within each candidate target region to output the target identifier and the set of target corner pixel coordinates or the target center pixel coordinates; Target pixel coordinates are generated based on the target corner pixel coordinate set or the target center pixel coordinates; For each target pixel coordinate, an observation quality is generated; wherein, the observation quality is obtained by combining at least two of the following processing results: whether decoding or fitting is successful, corner consistency or fitting residual, and consistency between the candidate region area and the preset range; The target identifier, target pixel coordinates, and observation quality are combined to form a target pixel observation set.

5. The bridge deformation monitoring method based on image intelligent analysis according to claim 1, characterized in that, Based on the target pixel observation set and the target grouping relationship, a spatial target coordinate anchor chain is constructed, and the pose solvable flag and anchor point set are output, including: A baseline target visibility record is generated based on the target pixel observation set and the baseline target set, and a structural target visibility record is generated based on the target pixel observation set and the structural target set. In the benchmark target visibility record, sorting and filtering are performed based on observation quality, and a set of anchor points is generated according to the rule that the number of anchor points is not less than a preset threshold and the dispersion of the anchor points in the image plane meets preset conditions. Based on the number of anchor points and the observation quality threshold, a pose solvable flag is generated, and the pose solvable flag and the anchor point set are written into the spatial target coordinate anchor chain.

6. The bridge deformation monitoring method based on image intelligent analysis according to claim 1, characterized in that, When the pose solvable flag indicates that the pose is solvable, the camera pose is calculated based on the camera calibration parameters and the set of anchor points, including: The initial spatial coordinates of the corresponding benchmark target are obtained based on the anchor point set, and the corresponding benchmark target pixel coordinates are obtained based on the target pixel observation set, so as to construct a set of corresponding point pairs of two-dimensional pixel coordinates and three-dimensional spatial coordinates. Based on the camera intrinsic parameter matrix and distortion parameters, the pose calculation and outlier culling functions are performed on the corresponding point pair set to output the camera pose and the inlier set; Calculate the reprojection error based on the camera pose and generate a pose error evaluation value; The camera pose, the set of interior points, and the pose error evaluation value are written into the spatial target coordinate anchor chain.

7. The bridge deformation monitoring method based on image intelligent analysis according to claim 6, characterized in that, Performing spatial displacement calculations on the structural target under the camera pose to output bridge deformation results includes: The initial spatial coordinates and displacement direction unit vector of the structural target are obtained based on the target spatial reference table, and the corresponding structural target pixel coordinates are obtained based on the target pixel observation set. Construct a one-dimensional displacement parameterized spatial point along the displacement direction based on the initial spatial coordinates of the structural target and the unit vector of the displacement direction. Calculate the zero-displacement projection point and the micro-displacement projection point based on the camera pose and the camera intrinsic parameter matrix, and calculate the pixel sensitivity vector; The one-dimensional displacement of the structural target is calculated based on the pixel sensitivity vector, the pixel coordinates of the structural target, and the zero displacement projection point, and the one-dimensional displacement is output as the bridge deformation.

8. The bridge deformation monitoring method based on image intelligent analysis according to claim 1, characterized in that, The method further includes: The confidence level of the deformation result is generated based on the observation quality in the target pixel observation set, the pose error evaluation value in the camera pose calculation process, and the closure residual in the anchor chain closure correction process; wherein, the confidence level of the deformation result increases with the increase of the observation quality, decreases with the increase of the pose error evaluation value, and decreases with the increase of the closure residual; The bridge deformation results and their confidence levels are output together, and an alarm event is output when the bridge deformation results exceed a preset displacement threshold and the confidence level of the deformation results exceeds a preset alarm threshold.

9. A bridge deformation monitoring system based on image intelligent analysis, characterized in that, The system is used to execute the bridge deformation monitoring method based on image intelligent analysis as described in any one of claims 1-8, and the system comprises: A unit is established to acquire bridge monitoring video frame sequences, camera calibration parameters, and target spatial reference tables, and to establish target grouping relationships based on the target spatial reference tables. The generation unit is used to perform target region detection on each frame of the bridge monitoring video frame sequence to generate a set of candidate target regions. The extraction unit is used to perform target confirmation and pixel observation extraction based on the candidate target region set, so as to form a target pixel observation set containing target identifier, target pixel coordinates and observation quality; The construction unit is used to construct a spatial target coordinate anchor chain based on the target pixel observation set and the target grouping relationship, and output the pose solvable flag and the anchor point set. The calculation unit is used to calculate the camera pose based on the camera calibration parameters and the set of anchor points when the pose solvable flag indicates that it is solvable, and to perform spatial displacement calculation on the structural target under the camera pose to output the bridge deformation result. The correction unit is used to perform anchor chain correction on the time period to which the unsolvable frame belongs based on the spatial target coordinates anchor chain when the pose solvable flag indicates that it is unsolvable, and output the bridge deformation result based on the correction result; specifically including: Based on the pose solvable flag sequence in the spatial target coordinate anchor chain, a continuous unsolvable time period is identified, the continuous unsolvable time period is determined as a gap time period, and the start and end times of the gap time period and the nearest solvable time at both ends of the gap are recorded. In the image of the last solvable moment before the gap start, a fixed background template region is selected, and template matching is performed on each frame image within the gap time period near the fixed background template region to generate a pixel drift compensation sequence. The true pixel equivalent drift is obtained at the first solvable time after the end of the gap, and the closure residual is calculated based on the true pixel equivalent drift and the predicted pixel drift compensation amount. The closure residual is distributed to each frame within the gap time period according to the time weight to generate the pixel drift compensation amount after closure correction; Pixel domain correction is performed on the structural target pixel coordinates based on the camera pose at the last solvable moment before the gap start point and the pixel drift compensation amount after closure correction, and spatial displacement calculation is performed based on the corrected structural target pixel coordinates to output the bridge deformation result.

Images