A method for locating obstructed piers and identifying and monitoring project progress in railway construction based on UAV monocular vision.

Abstract

This invention provides a method for locating obstructed railway construction piers and identifying and monitoring project progress based on UAV monocular vision, relating to the field of UAV railway engineering inspection based on computer vision target detection. The method uses a UAV to collect video and record attitude and position data; it utilizes YOLOv5 and DeepSORT to detect and track various components; when a pier is determined to be obstructed, it uses a monocular vision imaging model to infer its ground clearance from the beam width, calculating multiple frames of original latitude and longitude samples of the beam joints; it employs the RANSAC algorithm to remove outliers from the multi-frame samples and optimizes the solution to obtain the optimal latitude and longitude, which serves as the final geographic coordinates of the obstructed pier. This invention eliminates the need for additional equipment such as LiDAR or RTK base stations, relying solely on the visual features of the beam to infer the center position of the obstructed pier and accurately calculate its geographic latitude and longitude, achieving automatic location of obstruction points and determination of construction status, thus improving inspection efficiency.

Description

Technical Field

[0001] This invention relates to the field of UAV railway engineering inspection based on computer vision target detection, and particularly to a method for locating obstructed piers and identifying and monitoring project progress in railway construction based on UAV monocular vision. Background Technology

[0002] Railway bridge construction is characterized by numerous points along the line, strong continuity of procedures, and spatially dispersed construction status. Traditional methods relying on manual inspection are inefficient, costly, and pose significant safety risks, making them unsuitable for the refined management requirements of modern railway engineering. In recent years, automated inspection technology based on drone aerial photography and visual inspection has gradually become the industry mainstream. This technology uses visible light cameras to collect video of the construction line, combines YOLO series target detection algorithms to identify key construction components such as foundation pits, abutments, piers, and beams, and then utilizes multi-target tracking algorithms such as DeepSORT to maintain target consistency. Finally, monocular visual positioning methods are used to calculate the geographical latitude and longitude of each component, thereby achieving automatic assessment of construction progress and visualized management of the entire line's status. This type of technology offers advantages such as high inspection speed, wide coverage, and objective, traceable data, effectively replacing manual labor in large-scale, long-distance railway construction line inspections.

[0003] However, in actual railway construction inspections, existing technologies generally suffer from a common problem: once the beams are erected at a construction site, the piers are completely obscured by the beams, leaving no directly detectable visual features in aerial images. This prevents algorithms from identifying the piers and locating their geographical coordinates, resulting in a lack of assessment of the construction status at that point. To compensate for this data gap, current engineering practices still require manual on-site verification to supplement pier location information, which not only increases manpower and operational risks but also disrupts the continuity and real-time nature of automated inspections. Some high-precision positioning solutions use lidar or 3D point cloud reconstruction to obtain the spatial location of piers, but such equipment is expensive, data processing is complex, and it places extremely high demands on the payload and computing power of UAV platforms, making it difficult to promote its application in lightweight, routine inspection tasks.

[0004] Meanwhile, existing railway construction inspection technologies mostly focus on the detection and location of unobstructed targets, lacking an effective reverse-engineering mechanism for pier locations under beam-obstructed scenarios, and failing to systematically integrate target detection, multi-target tracking, occlusion reasoning, and monocular visual geolocation. Most methods can only achieve passive state recognition—"if it's visible, it's measurable; if it's invisible, it's missing"—unable to actively infer the center position and geographical coordinates of obstructed piers based on beam structural characteristics. This results in incomplete construction progress assessments and discontinuous location information, making it difficult to truly meet the engineering requirements of full-process, comprehensive, and automated monitoring of railway construction.

[0005] Therefore, developing a construction progress detection method that can achieve precise positioning of obstructed piers using only visible light monocular vision and lightweight algorithms has significant practical implications and engineering application value. Summary of the Invention

[0006] Purpose of the invention: In view of the above-mentioned engineering background, the present invention provides a method for locating obstructed piers and identifying and monitoring the progress of railway construction based on monocular vision of unmanned aerial vehicles (UAVs). It does not require additional equipment such as lidar or RTK base stations. It only relies on video images collected by a visible light camera to infer the center position of the obstructed pier through the visual features of the beam and accurately calculate its geographical latitude and longitude. This achieves automatic location of obstruction points and determination of construction status, reduces inspection costs, improves inspection efficiency, and meets the needs of automated progress monitoring for the entire process of railway construction with full coverage and precision.

[0007] This invention proposes a method for locating obstructed piers and identifying and monitoring project progress in railway construction based on monocular vision from unmanned aerial vehicles (UAVs), comprising the following steps:

[0008] Collect video image data during drone inspections and simultaneously record the corresponding location information, flight attitude information, and camera shooting parameters; The video image data is input into a pre-trained target detection model, which outputs the target category and pixel detection box of each construction component in the current frame image. Cross-frame association is performed on the pixel detection boxes, a unique tracking ID is assigned to the same construction component, and a stable target sequence containing tracking ID, target category and pixel detection boxes is obtained; The components at each construction point are occluded according to the stable target sequence. If it is determined that the pier is completely occluded by the beam, the occluded pier reverse positioning process is initiated. In the process of reverse positioning of the shielded pier, the height above the ground of the target plane is calculated by pixel detection box in combination with the preset physical width of the beam, position information and camera shooting parameters; Valid frames that meet the preset screen height range are selected from the video image data. Based on the beam joint pixels in the valid frames and the ground height, multi-frame target latitude and longitude samples are calculated frame by frame. The random sampling consensus algorithm is used to robustly optimize the latitude and longitude samples of the target in the multiple frames corresponding to the same tracking ID, remove abnormal outliers, and obtain the optimal latitude and longitude coordinates of the obstructing pier. The optimal latitude and longitude coordinates of the shielding pier are spatially matched and correlated with the preset coordinates of the route design points to comprehensively determine the current construction status of the construction points.

[0009] As a preferred embodiment, the location information includes the latitude and longitude of the UAV, its relative flight altitude, and its absolute altitude; the flight attitude information includes the gimbal heading angle, pitch angle, and roll angle; and the camera shooting parameters include the camera intrinsic parameter matrix and digital zoom ratio.

[0010] As a preferred embodiment, before inputting the video image data into the pre-trained object detection model, the following steps are also included: The training dataset was obtained by acquiring drone inspection images covering different construction stages and labeling four types of targets: foundation pit, pile cap, pier, and beam. When labeling beams, the upper and lower edges of the rectangular detection boxes used for labeling were placed close to the gap between adjacent beams. The target detection model is pre-trained using the training dataset.

[0011] As a preferred approach, shallow feature fusion weights are added to the backbone network of the target detection model; and the CIoU loss function is used to optimize the bounding box regression.

[0012] As a preferred embodiment, cross-frame association of the pixel detection boxes specifically includes: The coordinates of the pixel detection box are converted into the center coordinates and width and height information of the construction component. The position of the construction component in the next frame is predicted by a Kalman filter to establish a motion model. The region where the pixel detection box is located is feature-encoded using a feature extraction network to generate a feature vector that characterizes the appearance of the target. By combining Mahalanobis distance and cosine similarity to construct a matching cost matrix, the Hungarian algorithm is used to achieve the optimal association between the detected target in the current frame and the historical tracking trajectory, thereby outputting a stable target sequence.

[0013] As a preferred embodiment, the occlusion determination of components at each construction point is performed based on the stable target sequence, specifically including: If a beam is detected at the current construction site, and no pier is detected at the same construction site, and the pier is confirmed to have been constructed based on the line topology, and this is consistent across multiple consecutive tracking frames, then the pier is confirmed to be completely obscured by the beam.

[0014] As a preferred embodiment, the calculation of the ground clearance of the target plane specifically includes: Based on the camera shooting parameters and the pixel detection box of the beam, the image coordinates of the two ends of the lower boundary of the pixel detection box are converted into physical coordinates of the image plane; Using the attitude angles in the flight attitude information, a rotation matrix is ​​constructed to transform from the camera coordinate system to the northeast-northeast coordinate system; Combining the physical coordinates of the image plane and the rotation matrix, calculate the scaling factor of the ray direction vectors corresponding to the two endpoints in the northeast-northeast coordinate system, and calculate the Euclidean distance between the two scaling factors; The ground clearance of the target plane is calculated based on the relative flight altitude, the preset physical width of the beam, and the Euclidean distance.

[0015] As a preferred embodiment, the step of calculating multi-frame target latitude and longitude samples frame by frame specifically includes: For each valid frame, the corresponding actual physical focal length and pixel physical size are calculated, the image coordinates of the beam joint pixels are converted into the physical coordinates of the joint image plane, and the camera coordinate system ray vector is constructed. The camera coordinate system ray vector is transformed to the northeast-northeast coordinate system using the rotation matrix. The ray parameters are obtained by combining the ground clearance and the relative flight altitude. The local offset of the target point relative to the UAV ground projection point is then calculated. The local offset is converted into latitude and longitude increments, and then superimposed with the latitude and longitude of the UAV to calculate the target latitude and longitude sample corresponding to a single frame.

[0016] As a preferred embodiment, obtaining the optimal latitude and longitude coordinates of the shielding pier specifically includes: Construct a latitude and longitude sample set consisting of the target latitude and longitude samples from the multi-frame sample set, and randomly extract a preset number of samples from the latitude and longitude sample set to obtain candidate reference latitude and longitude positions; Calculate the ground plane distance from all sample points in the latitude and longitude sample set to the candidate reference latitude and longitude location, and count the number of inliers whose ground plane distance is less than a preset distance threshold; If the number of inliers in the current iteration is greater than the number of historical best inliers, then update the best baseline coordinates and save the current best inlier sample set, and then iterate in a loop until the preset maximum number of iterations is reached. After the iteration is completed, extract all latitude and longitude samples of the interior points in the optimal interior point sample set, calculate the arithmetic mean of the longitude and latitude respectively, and finally obtain the optimal latitude and longitude coordinates.

[0017] As a preferred solution, the current construction status of the construction site is comprehensively determined, and the determination rules specifically include: If only the foundation pit is detected and multi-frame tracking is stable, then the current construction status is determined to be the foundation pit construction stage; If the foundation is detected but the pier and beam are not detected and the tracking is stable across multiple frames, then the current construction status is determined to be the foundation construction stage. If the pier is detected but the beam is not detected and the tracking is stable across multiple frames, then the current construction status is determined to be the pier construction completion stage. If the beam is detected but the pier is not detected, and the optimal latitude and longitude coordinates of the obstructing pier are successfully calculated and the results are stable across multiple frames, then the current construction status is determined to be that the pier has been completed and the beam has been erected.

[0018] Compared with the prior art, the present invention has at least the following beneficial effects: (1) This invention achieves pier occlusion positioning and automatic construction progress detection by using UAV monocular vision and beam feature back-inference. It fundamentally solves the industry pain point that piers are completely obscured after beam erection in railway construction, making them undetectable and unpositionable. It breaks through the technical limitation of traditional visual inspection that "if it is not visible, it cannot be measured", and achieves full coverage of construction points, no omissions, and full-cycle status determination, ensuring complete and continuous construction progress management. The solution can complete the entire process calculation using only a visible light monocular camera, without the need for expensive equipment such as lidar, RTK base stations, and 3D point clouds. It has low hardware costs, low computing power requirements, and convenient deployment, making it suitable for long-distance, large-scale, and routine railway engineering inspection.

[0019] (2) This invention significantly reduces positioning errors caused by shooting angle, inter-frame jitter, and target truncation through multiple optimization strategies, including standard beam width substitution calculation, pixel coordinate distortion correction, optimal area selection of 1 / 3 to 4 / 5 of the image, and multi-frame latitude and longitude weighted averaging. The accuracy and stability of the pier geographic coordinate calculation are high. At the same time, the entire process from data acquisition, component detection, target tracking, occlusion judgment, pier back-inference, latitude and longitude calculation to progress comprehensive judgment is automated, eliminating the need for manual review and on-site supplementary measurement. This greatly improves inspection efficiency, reduces manpower input and on-site operation risks, and ensures unified detection rules, clear judgment logic, quantifiable and traceable results.

[0020] (3) This solution is technically mature and highly adaptable to engineering projects. It has been successfully implemented in railway projects and can adapt to complex construction scenarios with different sections, beam types, and aerial photography heights and angles. The inspection results are accurate and the status judgment is reliable. It can directly provide data support for construction units, supervisors and owners for progress management, quality verification and visual scheduling. It has good engineering practical value and application prospects. Attached Figure Description

[0021] Figure 1 This is an example diagram showing the shooting angle of the drone according to the present invention.

[0022] Figure 2 This is a schematic diagram illustrating the markings for four different component categories according to the present invention.

[0023] Figure 3 This is a schematic diagram of the tracking results.

[0024] Figure 4 A schematic diagram for detecting the two ends of the joint in the beam.

[0025] Figure 5 This is a schematic diagram showing the beam width and the two ends of the joint. Detailed Implementation

[0026] In the following description, numerous specific details are set forth in order to provide a more thorough understanding of the invention. However, it will be apparent to those skilled in the art that the invention can be practiced without one or more of these details. In other instances, certain technical features well-known in the art have not been described in order to avoid obscuring the invention.

[0027] The embodiments are described in detail below with reference to the accompanying drawings.

[0028] This embodiment discloses a method for automatically determining the construction progress of the entire line by inferring the position of the piers and calculating their latitude and longitude based on the visual features of the beams when the beams have been erected and the piers are completely obscured. The specific implementation steps are as follows: Step 1: Collect drone inspection data: The drone is controlled to collect inspection data along the railway construction line. Operations require clear skies, ample sunlight, moderate cloud cover, and no wind or light winds (wind speed below level 5). Environmental factors such as insufficient sunlight, strong winds, and fog / haze should be avoided as they can cause image blurring, feature loss, or unstable flight. During flight, the drone maintains a constant speed and straight flight path, utilizing a three-axis gimbal stabilization technology to ensure clear, stable, and stabilizing footage without significant shaking or sudden directional changes, guaranteeing the accuracy and reliability of subsequent target detection, feature extraction, and localization calculations. In this solution, the drone's shooting angles are described in [reference needed]. Figure 1 As shown.

[0029] The flight altitude is controlled within 80 to 120 meters above the ground surface, with a typical operational relative altitude of 100 meters. This altitude is flexibly adjusted according to the size of the construction components on site to ensure sufficient resolution of the target in the frame. The shooting angle adopts a tilted view of 45°-70° off-vertical, and the flight direction strictly follows the increasing direction of the railway line's station numbers. The drone remains directly above the railway line, without deviating from the line or shooting to the side, ensuring full coverage of the construction area without blind spots or omissions. Video acquisition uses a DJIM 4E 3× medium-long telephoto camera with a video resolution of 3840×2160 and a fixed digital zoom ratio of 1.12x. This ensures that the width of the construction target occupies no less than 1 / 4 of the frame, providing clear and detectable visual features for key components such as foundation pits, foundations, piers, and beams, avoiding detection failure or positioning errors due to targets being too small. Typical shooting parameters include: ISO 180, shutter speed 1 / 1600.0, aperture f / 2.8, and 35mm equivalent focal length 78.40mm.

[0030] During data acquisition, the UAV's SRT caption data recording function must be enabled to record and save in real time the UAV's position information (WGS84 latitude and longitude, relative flight altitude H, absolute altitude), flight attitude information (gimbal yaw, pitch, roll), camera shooting parameters (ISO, shutter speed, aperture, focal length, digital zoom ratio), and camera intrinsic parameter matrix corresponding to each frame of image. The camera intrinsic parameters used in this method are fixed as follows: fx=10060.427069, fy=10049.790588, cx=1920, cy=1200; The distortion coefficients are: k1=-0.183604, k2=0.632180, p1=0.003162, p2=-0.000860, k3=-0.034516.

[0031] Step 2: Construction component inspection based on YOLOv5: Before model testing, a dedicated dataset for railway construction components was first constructed. UAV inspection images were collected under different lighting conditions, angles, and construction stages. Four types of targets—foundation pits, pile caps, piers, and beams—were precisely labeled manually using the PASCAL VOC or YOLO standard format, including the target category and bounding box coordinates. For beam targets, the labeling had to meet strict specifications: the upper and lower edges of the rectangular detection boxes used for labeling must tightly fit the gaps between adjacent beams, ensuring that the selected area completely and accurately corresponds to the actual area of ​​a single beam, avoiding incomplete gap coverage or boundary offsets, and ensuring the accuracy of subsequent beam width calculations, joint location, and pier back-calculation. The dataset was randomly divided into training, validation, and test sets in a ratio of 7:2:1 to ensure a balanced distribution of samples across categories, used for model training, hyperparameter tuning, and performance evaluation.

[0032] A construction component detection model was built based on the YOLOv5 object detection framework. Using YOLOv5s as the base network, the model was optimized for the characteristics of small targets, long distance, and oblique shooting in railway scenes: shallow feature fusion weights were added to the backbone network to improve the feature representation ability of small targets; during training, Mosaic data augmentation, random scaling and cropping, and tone saturation adjustment were used to improve the model's generalization ability; and the CIoU loss function was used to optimize bounding box regression to improve localization accuracy.

[0033] The model training employs an adaptive learning rate strategy, using the SGD optimizer for parameter updates. After sufficient iterative convergence on the training set, the optimal weights are selected through a validation set. Following training, the model's accuracy is evaluated to ensure that the recognition precision and recall rates for foundation pits, pile caps, piers, and beams meet inspection requirements.

[0034] The trained model is then used for actual video stream detection: Each frame of video image undergoes size normalization and distortion correction preprocessing, before being input into the YOLOv5 model for forward inference. The output includes component categories (foundation pit, pile cap, pier, beam), target pixel detection boxes (x1, y1, x2, y2), and category confidence scores, providing reliable detection results for subsequent target tracking and occluded pier back-reasoning. See [link to relevant documentation] in this scheme. Figure 2 As shown, (a) is a pier column, (b) is a pile cap, (c) is a foundation pit, and (d) is a schematic diagram of the beam.

[0035] Step 3: Multi-target tracking based on DeepSORT: After obtaining the component detection results output by the YOLOv5 model, to ensure the consistency of the identity of the same construction target in the video sequence and avoid duplicate detection and ID jumps, the DeepSORT algorithm is used to perform multi-target tracking on four types of targets: foundation pits, pile caps, piers, and beams. The tracking process uses the detection bounding box as input, combining appearance features and motion features to achieve stable cross-frame association of targets. See [link / details]. Figure 3 As shown, each beam is represented within a rectangle, the blue line in the middle represents the target's trajectory, and the target's tracking ID and category are displayed in the upper left corner.

[0036] First, the detection bounding box undergoes coordinate transformation to obtain the target's center coordinates and width and height information. Then, a Kalman filter is used to predict the target's position in the next frame, establishing a target motion model to filter out sudden positional changes caused by detection jitter and local occlusion. Subsequently, DeepSORT's built-in deep feature extraction network is used to encode features in the target detection bounding box region, generating feature vectors that characterize the target's appearance and are used to distinguish different targets.

[0037] During the inter-frame matching stage, a matching cost matrix is ​​constructed by combining Mahalanobis distance and cosine similarity. The Hungarian algorithm is used to achieve the optimal association between the detected target in the current frame and the historical tracking trajectory. A globally unique tracking ID is assigned to each stable construction component to ensure that the same target always maintains the same identifier in continuous video. For targets that are missed or temporarily occluded, the position is predicted by Kalman filtering to maintain tracking, and the trajectory is removed only after multiple consecutive frames of loss.

[0038] After tracking and processing, a stable target sequence containing the following information is output: (1) Globally unique tracking ID; (2) Target category; (3) Real-time pixel coordinates of the target; (4) Tracking confidence and tracking status.

[0039] Stable tracking results can effectively avoid repeated positioning and reduce false detection interference, providing a reliable and continuous target basis for subsequent pier occlusion judgment, beam joint extraction, and latitude and longitude calculation.

[0040] Step 4: Construction Status and Obstruction Assessment: Combining the continuous distribution characteristics of the lines with the uniqueness of the tracking ID, each tracking target is assigned to the corresponding construction point of the line, forming an attribution relationship of "one point corresponding to one or more construction components".

[0041] This step is only used to identify whether the pier is obscured by the beam, in order to determine whether to initiate the obscuration reverse calculation process. The determination rules are as follows: (1) The beam is detected and no pier is detected at the same construction point. At the same time, the pier is confirmed to have been constructed based on the line topology relationship. → It is determined that the pier is blocked by the beam and the process of reverse positioning of the blocked pier is automatically entered.

[0042] (2) Other cases (only the foundation pit, only the pile cap, only the pier, and the beam and pier are visible at the same time) → are judged as unobstructed, no need to start reverse calculation, and directly enter the subsequent latitude and longitude calculation and progress judgment stage.

[0043] During the state determination process, consistency verification is performed using the tracking results of multiple consecutive frames to filter out state jitter caused by false detections or missed detections in a single frame. The final state is confirmed only when the detection results of multiple frames are consistent, thus ensuring the reliability of the judgment.

[0044] Step 5: Determine the design height of the beam from its width: Figure 4 The diagram shows the detection of the two ends of the joint in the beam. Figure 5 The diagram shows the beam width and the two ends of the joint.

[0045] The top width of the railway double-track simply supported box girder shall comply with national and industry standards and specifications. According to the "Railway Bridge and Culvert Design Specification" (TB10002-2017) and the general box girder drawings for high-speed / passenger dedicated railways, the top width of the standard double-track box girder shall be uniformly taken as 12.6m.

[0046] Based on the monocular vision imaging model, the height of the beam relative to the ground is calculated from the pixel width of the beam detection box. The detailed calculation process is as follows: (1) Input parameters drone relative ground height (Unit: meters): Obtained from relative altitude data provided by the UAV flight control system; Two pixels on the image and And the actual horizontal distance d (in meters) between these two points, which is obtained through on-site measurement or calibration with a known reference object; in this embodiment , These represent the coordinates of the two lower points of the rectangular frame detected by the beam.

[0047] Camera intrinsic parameters: image width W, height H (unit: pixels), principal point coordinates Pixel physical size (Unit: meters / pixel), where , The physical width of the sensor (meters). Digital zoom magnification; actual physical focal length f (unit: meters); Camera attitude angle: yaw angle (North by east is positive), pitch angle (Horizontal is 0, downward is negative), roll angle is set to 0.

[0048] (2) Output results The height of the target plane above the ground (Unit: meters)

[0049] (3) Definition of intermediate parameters : The physical coordinates (in meters) of the image point on the image plane; : The direction vector of light rays in the camera coordinate system; R: Rotation matrix for transforming from the camera coordinate system to the NED (North-East-Ground) coordinate system; : The ray direction vector in the NED coordinate system, where ; , Dimensionless proportionality coefficient; : Euclidean distance between two points with proportionality coefficients.

[0050] (4) Calculation steps 1) Calculate the physical coordinates of the image plane:

[0051] 2) Construct the ray direction vector in the camera coordinate system:

[0052] 3) Transform the vector to the NED (North-East-Earth) coordinate system using the rotation matrix R (without roll):

[0053]

[0054] 4) Calculate the dimensionless proportionality coefficient:

[0055] 5) Calculate the proportional coefficient distance between two points:

[0056] 6) Target height inversion:

[0057] Step Six: Positioning of Beam Joints and Calculation of Pier Latitude and Longitude After obtaining the ground clearance of the beam target through step five, the geographical latitude and longitude of the beam joint feature points and the center point of the obscured pier can be calculated. When the UAV flies at a constant speed along the line for inspection, the image coordinates of the same construction target change regularly in consecutive video frames. Affected by flight attitude jitter, gimbal angle deviation, target detection pixel box offset, lens residual distortion, and monocular imaging model error, the latitude and longitude of the same target calculated in different effective frames have discrete fluctuations and are accompanied by some outlier samples. Therefore, this method first adopts a screening strategy in the range of 1 / 3 to 4 / 5 of the image height to remove images. For video frames with large imaging errors at the top and invalid target truncation at the bottom, a set of valid frames with reliable imaging conditions is retained. Then, the pixels of the beam joint are substituted frame by frame into the monocular vision geolocation model to calculate the original latitude and longitude samples of the target for each valid frame. Instead of directly using the mean or weighted average for the final positioning result, all latitude and longitude samples from multiple frames of the same target are uniformly fed into subsequent step seven. The RANSAC algorithm is used for outlier removal and robust fitting to obtain the unique, accurate, and reliable final geographic coordinates of the pier and beam joint, effectively suppressing random errors and outlier interference in single-frame positioning. The detailed calculation process of substituting the joint pixels into the monocular vision geolocation model to calculate the latitude and longitude of the target location is as follows: (1) Input parameters Image coordinates (pixels), the origin is located at the top left corner of the image, u points to the right, and v points downwards; Camera optical equivalent focal length (millimeters), digital zoom magnification ; Drone location: Latitude (degrees), longitude (degrees), absolute altitude (meters), relative ground height (rice); Gimbal attitude: Yaw angle (degrees, 0 for true north, positive for east), pitch angle (degrees, horizontal is 0, downward is negative), roll angle (Degree, set to 0 in this embodiment); Image dimensions: Width W (pixels), Height H (pixels); Sensor physical dimensions: width (millimeters), height (millimeters); target point height above the ground (meters, or 0 if the target is on the ground).

[0058] (2) Output results Longitude of target point (degrees), latitude (degrees), altitude (rice).

[0059] (3) Definition of intermediate parameters f: Actual physical focal length (meters), calculated from the equivalent focal length and crop factor; Pixel physical size (meters / pixel), corrected for digital zoom; Image plane physical coordinates (meters); : The direction vector of light rays in the camera coordinate system; R: Rotation matrix from the camera coordinate system to the NED (North-East-Ground) coordinate system; : Ray direction vector in NED coordinate system; : Vertical distance (meters) from the optical center of the camera to the horizontal plane of the target; Light parameters; The target point's offset (in meters) relative to the UAV's ground projection point in the north and east directions. Meter: Earth's radius (semi-major axis of the WGS84 ellipsoid).

[0060] (4) Calculation steps 1) Calculate the actual physical focal length

[0061] 2) Calculate the physical size of the pixel (meters / pixel) First, crop the sensor size according to the aspect ratio of the target image to obtain the effective sensor width. (meters), then:

[0062] 3) Convert the image coordinates to physical coordinates (meters) on the image plane.

[0063] 4) Construct the ray vectors in the camera coordinate system

[0064] 5) Calculate the rotation matrix (Camera → NED, no roll) set up , (When the optical axis is lowered) ):

[0065] 6) Convert the ray direction to the NED coordinate system

[0066] 7) Solve for the intersection parameters of the light ray and the target horizontal plane.

[0067] 8) Calculate the northward and eastward offsets (meters).

[0068] 9) Convert local offsets into latitude and longitude increments (spherical approximation)

[0069] 10) Calculate the absolute coordinates of the target point.

[0070]

[0071] Step 7: Robust optimization solution of multi-frame target latitude and longitude based on RANSAC algorithm In step six, after selecting 1 / 3 to 4 / 5 of the effective range of the same construction target, multiple sets of latitude and longitude results can be obtained from multiple frames independently calculated. Affected by factors such as drone flight jitter, slight deviations in gimbal attitude, pixel shifts in target detection boxes, residual lens distortion, and geometric errors in monocular imaging, the latitude and longitude of the same physical target calculated in different video frames exhibit discrete deviations, and some outlier samples exist. If the arithmetic mean or weighted average is directly used to calculate the final coordinates, it is easily interfered with by outliers, reducing the positioning accuracy of the joint between the pier and the beam.

[0072] To eliminate outliers in multi-frame latitude and longitude samples, retain reliable inlier samples, and fit the globally optimal and robust true geographic coordinates of the target, this invention introduces the RANSAC random sampling consensus algorithm to robustly optimize the multi-frame latitude and longitude sequence of the same target, replacing the traditional simple averaging strategy, and further improving the positioning accuracy and stability of the locations of obstructed piers and beam joints.

[0073] (1) Construction of latitude and longitude sample set For the construction target corresponding to the same tracking ID, the positioning results obtained from all valid video frames are collected to construct a two-dimensional latitude and longitude sample set:

[0074] In the formula: lon is the target longitude, lat is the target latitude, and n is the number of effective positioning frames.

[0075] (2) RANSAC algorithm parameter initialization Model definition: The stationary piers and beam joints are considered as a two-dimensional fixed-point model. The minimum number of samples required to fit the model is... ; Distance threshold T: Convert the difference between latitude and longitude into Euclidean distance on the ground plane (unit: m), set a distance threshold, and points less than the threshold are identified as inliers, while points greater than the threshold are identified as outliers. Maximum number of iterations K: A preset fixed number of iterations to balance positioning accuracy and algorithm computation time; Optimal interior point set: Initialize an empty set to store the reliable sample with the most interior points and its corresponding reference coordinates during the iteration process.

[0076] (3) RANSAC iterative optimization calculation steps Step 1: Random sampling: Randomly sample from the latitude and longitude sample set S. For each set of samples, the center coordinates of two points are obtained and used as the latitude and longitude coordinates of the current iteration candidate reference. Step 2: In-point statistics: Calculate the ground plane distance from all sample points to the candidate reference location, and count the distances less than a threshold. The total number of interior points; Step 3 Model Update: If the number of inliers in the current iteration is greater than the number of historical best inliers, then update the optimal baseline coordinates and save the current best inlier sample set; Step 4: Iterate repeatedly: Repeat the random sampling, interior point statistics, and model update process until the preset maximum number of iterations is reached. ; Step 5: Interior Point Fitting: After iteration, extract all optimal interior point latitude and longitude samples selected by RANSAC, and calculate the arithmetic mean of longitude and latitude respectively to obtain the final robust optimal latitude and longitude of the target.

[0077]

[0078] In the formula: N is the optimal number of interior point samples. , These are the longitude and latitude of each reliable interior point after RANSAC filtering.

[0079] (4) Results output and business relevance The unique optimal latitude and longitude obtained through RANSAC algorithm optimization is used as the final geographic coordinates of the obstructed pier and beam joint corresponding to the tracking ID, replacing the single-frame discrete positioning results and ordinary mean results. The optimized accurate latitude and longitude is then fed into the subsequent comprehensive construction progress judgment stage, where it is spatially matched and correlated with the station coordinates of the route design. This eliminates multi-frame positioning discrete errors and outlier interference, ensuring that the reverse positioning of the obstructed pier is error-free and that the construction status judgment logic is more rigorous and reliable.

[0080] Step 8: Comprehensive Assessment of Construction Progress After completing the center positioning, distortion correction, multi-frame optimal area selection and latitude and longitude calculation of the obscured pier, the construction status and overall progress of each point are comprehensively judged and the results are output in combination with the design coordinates of the construction points along the entire line.

[0081] The calculated latitude and longitude coordinates of components such as piers, beams, pile caps, and foundation pits are spatially matched and associated with the coordinates of the design points along the route to confirm the design station number and construction point corresponding to each detection target. Combining the construction procedure logic and the consistency verification results of multi-frame tracking, the final construction status is determined according to the following rules: (1) Only the foundation pit was detected and the multi-frame tracking was stable → it was determined to be the foundation pit construction stage; (2) The foundation was detected, but the piers and beams were not detected, and the multi-frame tracking was stable → it was determined to be the foundation construction stage; (3) Pier columns were detected, but beams were not detected, and multi-frame tracking was stable → it was determined that the pier column construction was completed; (4) The beam was detected but the pier was not detected. The latitude and longitude of the pier were successfully deduced from the midpoint of the lower frame of the beam detection frame, and the results of multiple frames were stable. → It was determined that the pier had been completed and the beam had been erected. (5) If the beam is detected but the joint cannot be effectively identified or the latitude and longitude of the pier cannot be calculated, it is determined to be an abnormal construction or the beam is not erected in place and is included in the abnormal point reminder.

[0082] After determining the completion status of all construction sites along the entire line, a structured construction progress report is generated, including: design station number, site latitude and longitude, component type, current construction status, positioning completion status, anomaly markers, data collection time, and UAV flight parameters. All results support visualization, data export, and historical comparison, providing complete, objective, and traceable data support for railway construction management, progress statistics, and quality verification.

[0083] The underlying logic of the method for locating obstructed piers and identifying and monitoring project progress in railway construction based on UAV monocular vision disclosed in the above embodiments can be implemented entirely or partially through software, hardware, firmware, or other arbitrary combinations. When implemented using software, the above embodiments can be implemented entirely or partially in the form of a computer program product. A computer program product includes one or more computer instructions or computer programs.

[0084] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0085] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0086] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments have been described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification. Although the present invention has been shown and described with reference to specific preferred embodiments, it should not be construed as limiting the invention itself. Various changes in form and detail may be made without departing from the spirit and scope of the invention as defined in the appended claims.

Claims

1. A method for locating obstructed piers and identifying and monitoring project progress in railway construction based on monocular vision from unmanned aerial vehicles (UAVs), characterized in that, Includes the following steps: Collect video image data during drone inspections and simultaneously record the corresponding location information, flight attitude information, and camera shooting parameters; The video image data is input into a pre-trained target detection model, which outputs the target category and pixel detection box of each construction component in the current frame image. Cross-frame association is performed on the pixel detection boxes, a unique tracking ID is assigned to the same construction component, and a stable target sequence containing tracking ID, target category and pixel detection boxes is obtained; The components at each construction point are occluded according to the stable target sequence. If it is determined that the pier is completely occluded by the beam, the occluded pier reverse positioning process is initiated. In the process of reverse positioning of the shielded pier, the height above the ground of the target plane is calculated by pixel detection box in combination with the preset physical width of the beam, position information and camera shooting parameters; Valid frames that meet the preset screen height range are selected from the video image data. Based on the beam joint pixels in the valid frames and the ground height, multi-frame target latitude and longitude samples are calculated frame by frame. The random sampling consensus algorithm is used to robustly optimize the latitude and longitude samples of the target in the multiple frames corresponding to the same tracking ID, remove abnormal outliers, and obtain the optimal latitude and longitude coordinates of the obstructing pier. The optimal latitude and longitude coordinates of the shielding pier are spatially matched and correlated with the preset coordinates of the route design points to comprehensively determine the current construction status of the construction points.

2. The method for locating obstructed piers and identifying and monitoring project progress in railway construction based on UAV monocular vision, as described in claim 1, is characterized in that... The location information includes the latitude and longitude of the UAV, its relative flight altitude, and its absolute altitude; the flight attitude information includes the gimbal heading angle, pitch angle, and roll angle; and the camera shooting parameters include the camera intrinsic parameter matrix and digital zoom ratio.

3. The method for locating obstructed piers and identifying and monitoring project progress in railway construction based on UAV monocular vision, as described in claim 1, is characterized in that... Before inputting the video image data into the pre-trained object detection model, the following steps are also included: The training dataset was obtained by acquiring drone inspection images covering different construction stages and labeling four types of targets: foundation pit, pile cap, pier, and beam. When labeling beams, the upper and lower edges of the rectangular detection boxes used for labeling were placed close to the gap between adjacent beams. The target detection model is pre-trained using the training dataset.

4. The method for locating obstructed piers and identifying and monitoring project progress in railway construction based on UAV monocular vision, as described in claim 3, is characterized in that... Shallow feature fusion weights are added to the backbone network of the target detection model; the CIoU loss function is used to optimize the bounding box regression.

5. The method for locating obstructed piers and identifying and monitoring project progress in railway construction based on UAV monocular vision, as described in claim 1, is characterized in that... Cross-frame association of the pixel detection boxes specifically includes: The coordinates of the pixel detection box are converted into the center coordinates and width and height information of the construction component. The position of the construction component in the next frame is predicted by a Kalman filter to establish a motion model. The region where the pixel detection box is located is feature-encoded using a feature extraction network to generate a feature vector that characterizes the appearance of the target. By combining Mahalanobis distance and cosine similarity to construct a matching cost matrix, the Hungarian algorithm is used to achieve the optimal association between the detected target in the current frame and the historical tracking trajectory, thereby outputting a stable target sequence.

6. The method for locating obstructed piers and identifying and monitoring project progress in railway construction based on UAV monocular vision, as described in claim 1, is characterized in that... Based on the stable target sequence, the occlusion of components at each construction point is determined, specifically including: If a beam is detected at the current construction site, and no pier is detected at the same construction site, and the pier is confirmed to have been constructed based on the line topology, and this is consistent across multiple consecutive tracking frames, then the pier is confirmed to be completely obscured by the beam.

7. The method for locating obstructed piers and identifying and monitoring project progress in railway construction based on UAV monocular vision, as described in claim 2, is characterized in that... The calculation of the target's altitude above the ground specifically includes: Based on the camera shooting parameters and the pixel detection box of the beam, the image coordinates of the two ends of the lower boundary of the pixel detection box are converted into physical coordinates of the image plane; Using the attitude angles in the flight attitude information, a rotation matrix is ​​constructed to transform from the camera coordinate system to the northeast-northeast coordinate system; Combining the physical coordinates of the image plane and the rotation matrix, calculate the scaling factor of the ray direction vectors corresponding to the two endpoints in the northeast-northeast coordinate system, and calculate the Euclidean distance between the two scaling factors; The ground clearance of the target plane is calculated based on the relative flight altitude, the preset physical width of the beam, and the Euclidean distance.

8. The method for locating obstructed piers and identifying and monitoring project progress in railway construction based on UAV monocular vision, as described in claim 7, is characterized in that... The step-by-frame calculation to obtain multi-frame target latitude and longitude samples specifically includes: For each valid frame, the corresponding actual physical focal length and pixel physical size are calculated, the image coordinates of the beam joint pixels are converted into the physical coordinates of the joint image plane, and the camera coordinate system ray vector is constructed. The camera coordinate system ray vector is transformed to the northeast-northeast coordinate system using the rotation matrix. The ray parameters are obtained by combining the ground clearance and the relative flight altitude. The local offset of the target point relative to the UAV ground projection point is then calculated. The local offset is converted into latitude and longitude increments, and then superimposed with the latitude and longitude of the UAV to calculate the target latitude and longitude sample corresponding to a single frame.

9. The method for locating obstructed piers and identifying and monitoring project progress in railway construction based on monocular vision of unmanned aerial vehicles, as described in claim 1, is characterized in that... Obtaining the optimal latitude and longitude coordinates of the shielding pier specifically includes: Construct a latitude and longitude sample set consisting of the target latitude and longitude samples from the multi-frame sample set, and randomly extract a preset number of samples from the latitude and longitude sample set to obtain candidate reference latitude and longitude positions; Calculate the ground plane distance from all sample points in the latitude and longitude sample set to the candidate reference latitude and longitude location, and count the number of inliers whose ground plane distance is less than a preset distance threshold; If the number of inliers in the current iteration is greater than the number of historical best inliers, then update the best baseline coordinates and save the current best inlier sample set, and then iterate in a loop until the preset maximum number of iterations is reached. After the iteration is completed, extract all latitude and longitude samples of the interior points in the optimal interior point sample set, calculate the arithmetic mean of the longitude and latitude respectively, and finally obtain the optimal latitude and longitude coordinates.

10. A method for locating obstructed piers and identifying and monitoring project progress in railway construction based on monocular vision of unmanned aerial vehicles, as described in claim 1, is characterized in that... The current construction status of the construction sites is comprehensively determined, and the determination rules specifically include: If only the foundation pit is detected and multi-frame tracking is stable, then the current construction status is determined to be the foundation pit construction stage; If the foundation is detected but the pier and beam are not detected and the tracking is stable across multiple frames, then the current construction status is determined to be the foundation construction stage. If the pier is detected but the beam is not detected and the tracking is stable across multiple frames, then the current construction status is determined to be the pier construction completion stage. If the beam is detected but the pier is not detected, and the optimal latitude and longitude coordinates of the obstructing pier are successfully calculated and the results are stable across multiple frames, then the current construction status is determined to be that the pier has been completed and the beam has been erected.

Images