Unmanned aerial vehicle positioning method and system based on total station measurement network and visual collaboration

Abstract

The invention discloses an unmanned aerial vehicle positioning method and system based on a total station measurement network and visual collaboration. The method comprises the following steps: determining a space reference of the total station based on a ground control point; determining the spatial position of the unmanned aerial vehicle by using the total station after determining the spatial reference; capturing a space image corresponding to the unmanned aerial vehicle by using a camera; and projecting a positioning mark corresponding to the unmanned aerial vehicle on the captured space image based on the determined space position of the unmanned aerial vehicle. Therefore, efficient labeling, excellent consistency, cross-device / cross-camera unified output, precision verifiability, traceability, flexible deployment and excellent expandability can be realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of unmanned aerial vehicle positioning, and more particularly to a method and system for unmanned aerial vehicle positioning based on a total station measurement network and vision cooperation. BACKGROUND

[0002] Recently, there is an increasing demand for external positioning and data labeling of unmanned aerial vehicles in complex urban environments and the like.

[0003] However, external positioning and data labeling of unmanned aerial vehicles in complex urban environments and the like have long relied on manual frame-by-frame or semi-automatic labeling. Such methods have three outstanding problems: first, the efficiency is low and the cycle is long, making it difficult to support large-scale data production of multiple machines, multiple scenes, and long time series; second, the consistency is poor and the subjectivity is strong, with significant deviations between different labeling personnel and different batches, and high rework costs; third, there is a lack of traceable measurement reference and rigid spatial constraints, making it difficult to objectively evaluate and cross-device reuse the labeling results. The "rigid spatial constraint" refers to anchoring the scene in a unified coordinate system with a measurable, low-drift absolute coordinate reference, avoiding the scale drift and overall deformation that may occur when relying only on image features. Although traditional pure visual external positioning can assist labeling to some extent, it still belongs to weak constraint calculation due to the lack of such rigid constraints and device-level hardware timing, and the camera external parameters, time offset, and rolling shutter line time delay rely on weak priori self-estimation, and the time and space alignment error is easy to accumulate, and the engineering precision usually stays at the decimeter level, which is difficult to serve as a high-confidence labeling true value. SUMMARY

[0004] The purpose of the present application is to provide a method and system for unmanned aerial vehicle positioning that can achieve efficient labeling, excellent consistency, cross-device / cross-camera unified output, verifiable precision, traceability, flexible deployment, and excellent scalability.

[0005] According to one embodiment of the present application, a method for unmanned aerial vehicle positioning based on a total station measurement network and vision cooperation is provided, the method comprising: determining a spatial reference of a total station based on a ground control point; determining a spatial position of an unmanned aerial vehicle using the total station after the spatial reference is determined; capturing a spatial image corresponding to the unmanned aerial vehicle using a camera; and projecting a positioning mark corresponding to the unmanned aerial vehicle on the captured spatial image based on the determined spatial position of the unmanned aerial vehicle.

[0006] Optionally, the step of determining the spatial reference of the total station based on the ground control points comprises: deploying a plurality of ground control points; obtaining initial values of spatial positions of each of the plurality of ground control points using global navigation satellite system static measurement or network real-time kinematic; measuring measured values of spatial positions of each of the plurality of ground control points using the total station; performing network adjustment on the measured values of spatial positions of each of the plurality of ground control points based on least squares free network adjustment and Halmos constraint using the initial values of spatial positions of each of the plurality of ground control points; determining the pose and covariance corresponding to the spatial reference of the total station based on the results of the network adjustment.

[0007] Optionally, the step of determining the spatial position of the unmanned aerial vehicle using the total station with the determined spatial reference comprises: measuring a position vector of the unmanned aerial vehicle in the station coordinate system corresponding to the slant range from the total station to the unmanned aerial vehicle, the azimuth angle from the horizontal reference of the total station to the unmanned aerial vehicle, and the pitch angle from the horizontal plane of the total station to the unmanned aerial vehicle using the total station with the determined spatial reference; determining a position vector of the unmanned aerial vehicle in the engineering local coordinate system as the spatial position of the unmanned aerial vehicle based on the measured position vector of the unmanned aerial vehicle in the station coordinate system, a three-dimensional rotation matrix from the station coordinate system to the engineering local coordinate system, and a position vector of the origin of the station coordinate system in the engineering local coordinate system.

[0008] Optionally, the position vector of the unmanned aerial vehicle in the engineering local coordinate system refers to a position vector of a reference point of a prism or passive reflector mounted to the unmanned aerial vehicle in the engineering local coordinate system, and the step of determining the spatial position of the unmanned aerial vehicle using the total station with the determined spatial reference further comprises: determining a position vector of the center of mass of the unmanned aerial vehicle in the engineering local coordinate system as the spatial position of the unmanned aerial vehicle based on a lever arm vector of the reference point of the prism or passive reflector mounted to the unmanned aerial vehicle to the center of mass of the unmanned aerial vehicle in the unmanned aerial vehicle body coordinate system, a three-dimensional rotation matrix from the unmanned aerial vehicle body coordinate system to the engineering local coordinate system, and the position vector of the reference point of the prism or passive reflector mounted to the unmanned aerial vehicle in the engineering local coordinate system.

[0009] Optionally, before the step of capturing a spatial image corresponding to the unmanned aerial vehicle using the camera, the method further comprises: deploying a center-symmetric high-contrast marker; measuring a position vector of the center of the center-symmetric high-contrast marker using the total station with the determined spatial reference; extracting a sub-pixel center of the center-symmetric high-contrast marker using the camera; establishing a point-image constraint of the camera based on the position vector and the sub-pixel center of the center of the center-symmetric high-contrast marker.

[0010] Optionally, before the step of capturing a spatial image corresponding to the UAV using the camera, the method further comprises: time synchronizing the total station and the camera based on a global navigation satellite system; segmenting and time calibrating a measurement window of the total station; and frame / line two-stage time calibrating the camera to establish a rolling shutter time model of frame rate scale, time offset and line time delay.

[0011] Optionally, the method further comprises: determining the extrinsic parameters of the camera and aligning the time of the camera with the time of the total station based on the position vector and covariance of the center-symmetric high-contrast mark provided by the total station and the time synchronization provided by the global navigation satellite system, by taking the extrinsic parameters of the camera and the frame rate scale, time offset and line time delay of the rolling shutter time model as joint variables.

[0012] According to an embodiment of the present inventive concept, there is provided a system for UAV positioning based on total station measurement network and visual collaboration, the system comprising: a ground control point for determining a spatial reference of a total station; a total station configured to determine a spatial position of a UAV; a camera configured to capture a spatial image corresponding to the UAV; and a host configured to: receive the determined spatial position of the UAV and the captured spatial image, and project a positioning mark corresponding to the UAV on the captured spatial image based on the determined spatial position of the UAV.

[0013] According to an embodiment of the present inventive concept, there is provided a computer readable storage medium storing instructions which, when executed by at least one processor, cause the at least one processor to perform the method for UAV positioning based on total station measurement network and visual collaboration.

[0014] According to an embodiment of the present inventive concept, there is provided a computer program product comprising computer instructions which, when executed by at least one processor, implement the method for UAV positioning based on total station measurement network and visual collaboration.

[0015] According to the embodiment of the present inventive concept, the total station is taken as a "true value label" to realize "labeling while collecting", to generate a reference trajectory on site, to significantly reduce the cost of manual frame-by-frame labeling and review, to improve the labeling throughput and consistency, and to realize efficient labeling and excellent consistency; through a closed loop of "unified space-time reference - reference transmission - unified calculation", different cameras share the same rigid coordinate and time anchor, to realize cross-camera relay, continuous trajectory and consistent coordinate output, and to realize unified output across devices / cameras; rolling shutter explicit modeling and device-level timing, combined with covariance propagation and end-to-end error budget, provide auditable precision and confidence interval, and realize precision verifiability and traceability; support single total station rapid landing, can be smoothly expanded to multi-total station network to cover a large range and complex occlusion, and seamlessly cooperate with existing city cameras, and realize flexible deployment and excellent scalability. BRIEF DESCRIPTION OF DRAWINGS

[0016] The above and / or other aspects of the present inventive concept will become apparent and more readily appreciated from the following detailed description, taken in conjunction with the accompanying drawings.

[0017] Figure 1 is a flowchart of a method for unmanned aerial vehicle positioning based on a total station measurement network and visual cooperation according to an embodiment of the present inventive concept.

[0018] Figure 2 is a flowchart of a method for determining a space reference of a total station according to an embodiment of the present inventive concept.

[0019] Figure 3 is a flowchart of a method for establishing a point-image constraint of a camera according to an embodiment of the present inventive concept.

[0020] Figure 4 is a flowchart of a method for determining a time reference for unmanned aerial vehicle positioning according to an embodiment of the present inventive concept.

[0021] Figure 5 is a block diagram of a system for unmanned aerial vehicle positioning based on a total station measurement network and visual cooperation according to an embodiment of the present inventive concept.

[0022] Throughout the drawings and the detailed description, unless otherwise described or provided, the same drawing reference labels will be understood to refer to the same elements, features, and structures. The drawings can not be to scale, and the relative dimensions, proportions, and depiction of elements in the drawings can be exaggerated for clarity, illustration, and convenience. DETAILED DESCRIPTION

[0023] The following detailed description is presented to aid the reader in gaining a comprehensive understanding of the methods, apparatuses, and / or systems described herein. However, various changes, modifications, and equivalents can be used, and thus particular embodiments described herein are not intended as being exhaustive of what the present disclosure can provide. For example, although sequences of operations can be described, the sequence of operations should not be construed as limiting, and is not intended to indicate that these operations are to be performed in the precise order stated. Unless otherwise specified, operations described herein can be performed in any order. Moreover, certain features can be omitted from the description, as modifying such features can be understood by those of ordinary skill in the art.

[0024] The features described herein can be implemented in different forms and should not be construed as limited to the examples described herein. Rather, these described examples have been provided as an example of carrying out the methods, apparatuses, and / or systems described herein as would be obvious to one of ordinary skill in the art, upon understanding the present disclosure.

[0025] Throughout the specification, where assemblies are described as "connected to" or "coupled to" other assemblies, it will be understood that the assemblies can be directly connected or coupled to the other assemblies, or intervening assemblies can be present. Conversely, where an element is described as being "directly connected to" or "directly coupled to" another element, it will be understood that no intervening assembly is present. Also, the same can apply with respect to the use of like terminology (e.g., "between" versus "immediately between" and "adjacent to" versus "immediately adjacent to"). As used herein, the term "and / or" includes any one of the listed items, or any combination of two or more of the listed items.

[0026] Although terms such as "first", "second", and "third" can be used herein to describe various elements, components, regions, layers or sections, these elements, components, regions, layers or sections should not be limited by these terms. Rather, these terms are only used to distinguish one element, component, region, layer or section from another element, component, region, layer or section. Thus, the element, component, region, layer or section referred to as the first element, the first component, the first region, the first layer or the first section in the examples described herein can also be referred to as the second element, the second component, the second region, the second layer or the second section without departing from the teachings of the examples.

[0027] The terminology used herein is for the purpose of describing various examples only and is not intended to be limiting of the disclosure. Singular forms are intended to include the plural forms, unless the context clearly indicates otherwise. The terms "comprises", "comprising", and "having" are intended to be inclusive and to mean that there can be additional elements, including ones that are not specifically mentioned. The terms "comprise", "comprising", "comprises", and "comprising" specify the presence of stated features, integers, operations, components, elements, and / or portions, but do not preclude the presence or addition of one or more other features, integers, operations, components, elements, and / or portions thereof.

[0028] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art in the field of the disclosure to which this application belongs. Unless explicitly defined herein, otherwise, terms such as those defined in commonly used dictionaries are to be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and the disclosure of this application, and are not to be interpreted in an idealized or overly formal sense. The use of the term "may" herein (for example, as to what an example or embodiment can include or implement) indicates that there are at least one example or embodiment for which the feature is included or implemented, and all examples are not limited thereto.

[0029] Embodiments of the inventive concept will be described in detail below with reference to the accompanying drawings.

[0030] Figure 1 is a flowchart of a method of unmanned aerial vehicle positioning based on total station measurement network and visual synergy according to embodiments of the inventive concept.

[0031] Referring to Figure 1 , the method of unmanned aerial vehicle positioning based on total station measurement network and visual synergy according to embodiments of the inventive concept can include steps S100 to S400.

[0032] In step S100, a spatial reference of a total station can be determined based on ground control points (GCPs). In the following, details of step S100 will be described with reference to Figure 2 .

[0033] In step S200, a spatial position of an unmanned aerial vehicle can be determined using the total station after the spatial reference is determined. In one embodiment, step S200 of determining a spatial position of an unmanned aerial vehicle using the total station after the spatial reference is determined can include two conversion operations (e.g., a first conversion operation and a second conversion operation after the first conversion operation).

[0034] For example, in the first conversion operation, a slant range (or, distance) (e.g., unit: meter) from the total station to the unmanned aerial vehicle, an azimuth (e.g., unit: radian) of a horizontal reference of the total station to the unmanned aerial vehicle, and a pitch (e.g., unit: radian) of a horizontal plane of the total station to the unmanned aerial vehicle can be measured using the total station after the spatial reference is determined. (e.g., unit: meter) corresponding to the unmanned aerial vehicle in a station coordinate system can be determined. For example, the first conversion operation can be represented by the following Equation 1, where, , and is a position vector of the unmanned aerial vehicle in the station coordinate system The three-dimensional coordinates (e.g., in meters).

[0035] [Formula 1]

[0036] For example, in the second transformation operation, the measured position vector of the UAV in the station coordinate system can be used as a basis. The three-dimensional rotation matrix from the station coordinate system to the local engineering coordinate system (in the following text, the East-North-Sky (ENU) coordinate system is used as an example). (For example, the size is:) (Unit: dimensionless), and the position vector of the origin of the station coordinate system in the local engineering coordinate system. (For example, in meters), determine the position vector of the UAV in the local engineering coordinate system. (For example, in meters) as the spatial location of the drone. For example, the second conversion operation can be represented by Equation 2 below.

[0037] [Formula 2]

[0038] Accordingly, in this embodiment, the covariance of the total station's observation noise of the distance and angle of the UAV in the polar coordinate domain is... (For example, the units are meters) 2 ,radian 2 ,radian 2 The position vector of the UAV can be converted using the following formula 3. Covariance of coordinates in the station coordinate system (For example, the unit is: meter) 2 ).

[0039] [Formula 3]

[0040] Formula 3 It is the Jacobian matrix from the polar coordinates of the total station to the Cartesian coordinates of the total station, and can be expressed by the following Equation 4, where the partial derivatives in Equation 4 are: , , , , , , , , .

[0041] [Formula 4]

[0042] Next, the position vector of the UAV in the station coordinate system. Covariance of coordinates in the station coordinate system The position vector of the UAV in the local engineering coordinate system can be converted using Formula 5 below. Covariance in the local coordinate system of the project (For example, the unit is: meter) 2 )in, It is the covariance matrix of the uncertainty of the station pose (position and orientation) (the dimension of the covariance matrix depends on the dimension of the parameters). It is the position vector of the UAV in the local coordinate system of the engineering project. Jacobian for minor perturbations to the station's pose (e.g., small-angle rotational perturbations), sign This represents the covariance composition operation of uncertainties from different sources under first-order linearization.

[0043] [Formula 5]

[0044] Therefore, using formulas 3 to 5 above, the covariance of the observation noise of the total station for the distance and angle of the UAV can be expressed in the polar coordinate domain of the total station. Propagate to the local coordinate system of the project.

[0045] In one embodiment, the position vector of the UAV in the local engineering coordinate system... This could refer to the position vector of the reference point of the prism or passive reflector mounted on the UAV in the local engineering coordinate system. In this case, step S200, which uses a total station after determining the spatial reference, may include additional coordinate transformation operations (e.g., a third transformation operation following a second transformation operation).

[0046] For example, in the third transformation operation, the lever arm vector from the reference point of the prism or passive reflector mounted on the UAV to the center of mass of the UAV in the UAV body coordinate system can be used. (For example, in meters), the three-dimensional rotation matrix from the UAV body coordinate system to the engineering local coordinate system. And the position vector of the reference point of the prism or passive reflector mounted on the drone in the local engineering coordinate system. (For example, the unit is: meter) (i.e., Determine the position vector of the UAV's centroid in the local engineering coordinate system. (For example, in meters) as the spatial location of the drone. For example, the third conversion operation can be represented by Equation 6 below.

[0047] [Formula 6]

[0048] Accordingly, in this embodiment, the covariance of the UAV's centroid position... (Unit: meters) 2 The result can be obtained through Formula 7, which represents the synthesis path of the end-to-end error budget, where... It is the covariance of the position of the reference point of the prism or passive reflector mounted on the drone (unit: meters). 2 ), It is the covariance of the uncertainty of the installation angle (rotation) (unit: radians) 2 (corresponding to the minimum rotation vector or Euler perturbation). It is the covariance of the measurement uncertainty of the lever arm (unit: meter) 2 ), yes The Jacobian matrix for small-angle rotational perturbations (e.g., of size: ).

[0049] [Formula 7]

[0050] In step S300, a camera can be used to capture a spatial image corresponding to the drone. In step S400, based on the determined spatial position of the drone, a positioning marker corresponding to the drone can be projected onto the captured spatial image.

[0051] In one embodiment, the total station and / or the host computer can output the UAV's position covariance at each timestamp. and / or attitude small perturbation covariance Calculate, for example, the equivalent radius of a 95% confidence ellipsoid. (in, , It is a square root function, and It is used to determine the location covariance. The function of the largest eigenvalue is used as the quality summary index, and the reprojection root mean square error (RMSE), normalized estimate square error (NEES) / normalized innovation square (NIS) consistency statistics and PTP jitter are output for acceptance and traceability.

[0052] Therefore, through the above steps S100 to S400, the total station can be used as the "truth label" to achieve "simultaneous sampling and labeling", generating reference trajectories on site, significantly reducing the cost of manual frame-by-frame labeling and verification, improving labeling throughput and consistency, thereby achieving efficient labeling and excellent consistency.

[0053] Figure 2This is a flowchart of a method for determining the spatial reference of a total station according to an embodiment of the present invention.

[0054] Reference Figure 2 The method for determining the spatial reference of a total station according to an embodiment of the present invention (i.e., step S100) may include steps S110 to S150.

[0055] In step S110, multiple GCPs can be deployed. For example, at least four geometrically well-distributed GCPs can be deployed in the work area. Optionally, in one embodiment, elevation densification and occlusion analysis can be performed on each of the multiple GCPs to ensure that the station visibility redundancy and elevation angle distribution meet the requirements.

[0056] In step S120, initial values ​​of the spatial location of each of the multiple GCPs can be obtained using Global Navigation Satellite System (GNSS) static measurements or Network Real-Time Kinematic (RTK, also known as Real-Time Dynamic Positioning or Carrier Phase Differential Positioning). In one embodiment, the initial values ​​of the spatial locations of each of the multiple GCPs can be unified to (e.g., converted to) initial values ​​in the ENU coordinate system, and the seven-parameter (or four-parameter) transformation parameters with the national coordinate system (e.g., CGCS2000 / ITRF) can be recorded to obtain a low-drift, traceable "rigid" spatial frame.

[0057] In step S130, a total station can be used to measure the spatial location of each of the multiple GCPs. For example, in one example, multiple redundant distance / angle measurements can be performed on each of the multiple GCPs using (e.g., one or more) total stations.

[0058] In step S140, the initial values ​​of the spatial locations of each of the multiple GCPs can be used to perform network adjustment on the measurements of the spatial locations of each of the multiple GCPs based on least squares free network adjustment and Helmert model constraints.

[0059] In step S150, the pose and covariance corresponding to the spatial reference of the total station can be determined based on the results of the network adjustment. For example, in one example, the free network / resection adjustment method can be used to solve for the pose and covariance of each station.

[0060] In one embodiment, one or more network robustness metrics can be output based on the results of network adjustment. For example, network robustness metrics may include, but are not limited to, redundancy, condition number, maximum baseline, stereo scatter angle, residual histogram, and chi-square test p-value. Furthermore, in one embodiment, the deployment and elevation angle distribution of multiple GCPs can be adjusted based on network robustness metrics. For example, when the condition number in the network robustness metrics deteriorates or the precision factor (DOP) is too large, diagonal baselines can be increased or station deployment and elevation angle distribution can be optimized.

[0061] Through steps S110 to S150, the spatial reference of the total station and GCP can be unified, resulting in a low-drift, traceable, and "rigid" spatial framework. With a single total station, the trajectory and uncertainty of the UAV with high confidence can be quickly obtained. With multiple total stations networked together, based on measurement quality scoring and inverse covariance weighting, relayable continuous tracking and robust fusion can be achieved, supporting stable relay and unified coordinate output across cameras. Therefore, steps S110 to S150 support rapid deployment of a single total station, which can be smoothly expanded into a multi-total-station network to cover large areas and complex obstructions, seamlessly cooperating with existing urban cameras, thus achieving flexible deployment and excellent scalability.

[0062] Figure 3 This is a flowchart of a method for establishing point-image constraints for a camera according to an embodiment of the present invention.

[0063] Prior to step S300, which involves capturing a spatial image corresponding to the drone using a camera, the drone localization method based on a total station measurement network and visual collaboration, according to an embodiment of the present invention, may further include establishing point-image constraints for the camera. (Refer to...) Figure 3 The method for establishing point-image constraints of a camera according to an embodiment of the present invention may include steps S510 to S540.

[0064] In step S510, centrally symmetrical high-contrast markers may be deployed. In one example, the centrally symmetrical high-contrast markers may be centrally symmetrical high-contrast circular markers and / or centrally symmetrical high-contrast four-quadrant markers, and are not limited thereto.

[0065] In step S520, the position vector of the center of the center-symmetric high-contrast marker can be measured using a total station after determining the spatial reference. In one embodiment, in addition to measuring the position vector of the center of the center-symmetric high-contrast marker, the covariance between the total station output after determining the spatial reference and the measured position vector of the center of the center-symmetric high-contrast marker can also be used.

[0066] In step S530, the subpixel center of the centrally symmetrical high-contrast mark can be extracted using a camera.

[0067] In step S540, point-image constraints of the camera can be established based on the position vector of the center of the centrally symmetric high-contrast marker and the sub-pixel center.

[0068] In one embodiment, prior to step S300, which uses a camera to capture a spatial image corresponding to the UAV, the UAV localization method based on a total station measurement network and visual collaboration according to an embodiment of the present invention may further include performing intrinsic parameter calibration and distortion compensation on the camera. For example, calibration maps can be acquired at multiple distances and orientations using a checkerboard / ring target to calculate the camera's intrinsic parameter matrix and distortion parameters. Next, the robustness of the calculation results can be improved using the Random Sample Consensus (RANSAC) algorithm and sub-pixel corner points (or ellipse fitting), and the intrinsic parameter covariance and reprojection error statistics can be output. Then, a temperature drift / focus change model of the camera can be established to perform online focus fine-tuning for zoom lenses or long-term operation (e.g., introducing small perturbation factors and weak priors in a sliding window).

[0069] In one embodiment, when centrally symmetric high-contrast markers are not visible or insufficient in number, sparse natural features can be combined with short-term static assumptions for hybrid calibration.

[0070] Through steps S510 to S540, the determined spatial reference of the total station can be transferred to the camera. Therefore, through the closed loop of "unified spatiotemporal reference - reference transfer - unified calculation", different cameras share the same rigid coordinates and time anchor, realizing cross-camera relay, continuous trajectory and consistent coordinate output, thereby achieving unified output across devices / cameras.

[0071] Figure 4 This is a flowchart of a method for determining a time reference for UAV positioning according to an embodiment of the present invention.

[0072] Prior to step S300, which involves capturing a spatial image corresponding to the drone using a camera, the drone localization method based on a total station measurement network and visual coordination, according to an embodiment of the present invention, may further include determining a time reference for drone localization. (Refer to...) Figure 4 The method for determining a time reference for UAV positioning according to an embodiment of the present invention may include steps S610 to S630.

[0073] In step S610, the total station and camera can be timed based on GNSS. For example, in one example, a GNSS timing module can be deployed to provide Coordinated Universal Time (UTC) and pulses per second (1PPS) for the edge master clock; next, the total station (e.g., the total station control terminal), the camera (e.g., the camera acquisition terminal), and the computing node (e.g., the host) can achieve device-level timing via Precision Time Protocol (PTP) / IEEE 1588 / 802.1AS, or via a combination of 1PPS and Time Messages (ToD); simultaneously, clock level, link delay, and jitter can be recorded.

[0074] In step S620, the measurement window of the total station can be calibrated with segmented delays. For example, in one example, the single-cycle measurement process of the total station can be decomposed into four segments: "request transmission - measurement execution - response transmission - dead zone", and (e.g., offline) the delay of each segment can be calibrated to obtain the equivalent center time; then, (e.g., online) drift compensation can be performed based on temperature / humidity / link load, etc.

[0075] In step S630, the camera can undergo frame / line two-level time calibration to establish a rolling shutter time model with frame rate scale, time offset, and line delay. For example, in one example, LED / electronic shutter triggers driven by a unified clock can be arranged within the camera's field of view to generate a known trigger sequence according to UTC time scale, and the camera can record the external trigger interruption time; next, the frame index of the trigger event can be located in the image. and row index This is used to establish a time model. For example, in one example, the rolling shutter time model. It can be represented by the following formula 8, where the rolling shutter time model The image represents the first The first frame Exposure time per row pixel (e.g., in seconds), frame index It is a frame number starting from zero (e.g., in dimensionless units), and a row index. It is the pixel row index (e.g., in dimensionless units). It is a frame rate scaling factor and characterizes a small deviation from the nominal frame period (e.g., in seconds per frame). It is a fixed offset of the camera time from a uniform time scale (e.g., in seconds), and This is expressed as the line readout delay for the rolling shutter (e.g., in seconds per line).

[0076] [Formula 8]

[0077] Therefore, through the above steps S610 to S630, the time reference of the total station and the camera can be unified.

[0078] Furthermore, in one embodiment, the camera's extrinsic parameters can be determined and its time aligned with the total station's time by using the frame rate scale, time offset, and line delay of the rolling shutter time model as joint variables, based on the position vector and covariance of the centrally symmetric high-contrast markers provided by the total station and the time synchronization provided by GNSS. For example, the operation of determining the camera's extrinsic parameters and aligning the camera's time with the total station's time can be implemented in the initial solution and bundle adjustment (BA) operation of the camera's perspective n-point (PnP), and robust least squares / Huber loss can be used to jointly estimate the frame rate scale. Time deviation Line delay The system provides statistics on frame-level and line-level alignment errors (e.g., frame-level ≤ 1 ms, line-level ≤ 0.2 ms).

[0079] Because the exposure time is written in the form of a rolling shutter time model established by frame rate scale, time offset, and line delay, and this rolling shutter time model embeds the camera's temporal sequence into a unified time scale, the camera and total station can be anchored to the same coordinates and the same time scale, and support linkage estimation with extrinsic parameters during optimization. Furthermore, calculating the camera pose at the corresponding time minimizes the reprojection error of the objective function, and introducing the total station coordinate covariance as a weight eliminates the line delay caused by the rolling shutter. This allows all cameras to share a set of metrologically traceable "hard anchor points," and provides a consistent pose reference and covariance-weighted fusion basis for cross-camera (cross-lens / cross-viewpoint) tracking and seamless coordinate relay.

[0080] Furthermore, in one embodiment, the extrinsic and temporal parameters of the camera can be fine-tuned via a sliding window during system operation, and time loopback testing, measurement window timing correction, residual threshold, and 3D modeling are provided. The system includes online self-checking mechanisms such as threshold alarms. Therefore, long-term stability and traceability can be ensured.

[0081] In addition, in one embodiment, when PTP is unavailable, alternative synchronization of 1PPS+ToD, optical LED events, or audio pulses can be supported to ensure that the time alignment error is no more than 10 ms.

[0082] According to embodiments of the present invention, a total station is used as a "truth label" to achieve "on-the-spot labeling," generating reference trajectories on-site. This significantly reduces the cost of manual frame-by-frame labeling and verification, improves labeling throughput and consistency, thereby achieving efficient labeling and excellent consistency. Through a closed loop of "unified spatiotemporal reference - reference transfer - unified solution," different cameras share the same rigid coordinates and time anchors, enabling cross-camera relay, continuous trajectories, and consistent coordinate output, thus achieving unified output across devices / cameras. Explicit modeling with rolling shutter and device-level time synchronization, combined with covariance propagation and end-to-end error budgeting, provides auditable accuracy and confidence intervals (e.g., typical indicators: horizontal ≤1 cm, vertical ≤1-2 cm, trigger time alignment ≤10ms), thereby achieving verifiable and traceable accuracy. It supports rapid deployment of a single total station and can be smoothly expanded into a multi-total-station network to cover large areas and complex occlusions, seamlessly cooperating with existing urban cameras, thus achieving flexible deployment and excellent scalability.

[0083] Figure 5 This is a block diagram of a UAV positioning system based on a total station measurement network and visual collaboration, according to an embodiment of the present invention.

[0084] Reference Figure 5 The system 100 may include a GCP 110, a total station 120, a camera 130, and a host 140.

[0085] According to an embodiment of the present invention, GCP 110 can be used to determine the spatial reference of the total station. The total station 120 can be configured to determine the spatial position of the UAV and can perform operations corresponding to steps S100 (including steps S110 to S150) and S200. The camera 130 can be configured to capture a spatial image corresponding to the UAV and can perform an operation corresponding to step S300. The host 140 can be configured to receive the determined spatial position of the UAV and the captured spatial image, and based on the determined spatial position of the UAV, project a positioning marker corresponding to the UAV onto the captured spatial image. The host 140 can perform operations corresponding to steps S400, S510 to S540, and S610 to S630. In one embodiment, host 140 can provide a data interface with traceability fields, where observations and status are indexed by project reference time and include device identifier (ID), firmware version, temperature / humidity, timing level, link latency and jitter. It also supports JavaScript Object Notation (JSON) / comma-separated (CSV) / Robot Operating System 2 (ROS2) message packaging, historical replay, and threshold over-limit alarms.

[0086] According to embodiments of the present invention, a total station is used as a "truth label" to achieve "on-the-spot labeling," generating reference trajectories on-site. This significantly reduces the cost of manual frame-by-frame labeling and verification, improves labeling throughput and consistency, thereby achieving efficient labeling and excellent consistency. Through a closed loop of "unified spatiotemporal reference - reference transfer - unified solution," different cameras share the same rigid coordinates and time anchors, enabling cross-camera relay, continuous trajectories, and consistent coordinate output, thus achieving unified output across devices / cameras. Explicit modeling with rolling shutter and device-level time synchronization, combined with covariance propagation and end-to-end error budgeting, provides auditable accuracy and confidence intervals (e.g., typical indicators: horizontal ≤1 cm, vertical ≤1-2 cm, trigger time alignment ≤10ms), thereby achieving verifiable and traceable accuracy. It supports rapid deployment of a single total station and can be smoothly expanded into a multi-total-station network to cover large areas and complex occlusions, seamlessly cooperating with existing urban cameras, thus achieving flexible deployment and excellent scalability.

[0087] The UAV localization method based on total station surveying network and vision collaboration according to embodiments of the present invention can be programmed into a computer program and stored on a computer-readable storage medium. When executed by a processor, the computer program implements the UAV localization method based on total station surveying network and vision collaboration as described above. Examples of computer-readable storage media include: read-only memory (ROM), random access programmable read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, non-volatile memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD-ROM, DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-R LTH, BD-RE, Blu-ray or optical disc storage, hard disk drive (HDD), solid-state drive (SSD), card storage (such as multimedia cards, secure digital (SD) cards, or ultra-fast digital (XD) cards), magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid-state drive, and any other device configured to store computer programs and any associated data, data files, and data structures in a non-transitory manner and to provide the computer programs and any associated data, data files, and data structures to a processor or computer so that the processor or computer can execute the computer programs. In one example, the computer programs and any associated data, data files, and data structures are distributed across a networked computer system, such that the computer programs and any associated data, data files, and data structures are stored, accessed, and executed in a distributed manner through one or more processors or computers.

[0088] According to embodiments of the present invention, a total station is used as a "truth label" to achieve "on-the-spot labeling," generating reference trajectories on-site. This significantly reduces the cost of manual frame-by-frame labeling and verification, improves labeling throughput and consistency, thereby achieving efficient labeling and excellent consistency. Through a closed loop of "unified spatiotemporal reference - reference transfer - unified solution," different cameras share the same rigid coordinates and time anchors, enabling cross-camera relay, continuous trajectories, and consistent coordinate output, thus achieving unified output across devices / cameras. Explicit modeling with rolling shutter and device-level time synchronization, combined with covariance propagation and end-to-end error budgeting, provides auditable accuracy and confidence intervals (e.g., typical indicators: horizontal ≤1 cm, vertical ≤1-2 cm, trigger time alignment ≤10ms), thereby achieving verifiable and traceable accuracy. It supports rapid deployment of a single total station and can be smoothly expanded into a multi-total-station network to cover large areas and complex occlusions, seamlessly cooperating with existing urban cameras, thus achieving flexible deployment and excellent scalability.

[0089] Although the invention has been specifically shown and described with reference to exemplary embodiments thereof, those skilled in the art will understand that various changes in form and detail may be made therein without departing from the spirit and scope of the invention as defined by the claims.

Claims

1. A method for UAV positioning based on total station surveying network and visual collaboration, characterized in that, The method comprises: determining a spatial reference of a total station based on ground control points; determining a spatial position of a UAV using the total station after determining the spatial reference; capturing a spatial image corresponding to the UAV using a camera; projecting a positioning mark corresponding to the UAV on the captured spatial image based on the determined spatial position of the UAV.

2. The method of claim 1, wherein, The step of determining the spatial reference of the total station based on the ground control points comprises: deploying a plurality of ground control points; acquiring an initial value of the spatial position of each of the plurality of ground control points using a global navigation satellite system static measurement or network real-time kinematic; measuring a measured value of the spatial position of each of the plurality of ground control points using the total station; performing network adjustment on the measured value of the spatial position of each of the plurality of ground control points based on least squares free network adjustment and Hertz model constraint using the initial value of the spatial position of each of the plurality of ground control points; determining a pose and a covariance corresponding to the spatial reference of the total station based on the result of the network adjustment.

3. The method of claim 1, wherein, The step of determining the spatial position of the UAV using the total station after determining the spatial reference comprises: measuring a position vector of the UAV in a station coordinate system corresponding to the slant range from the total station to the UAV, the azimuth angle from the horizontal reference of the total station to the UAV, and the pitch angle from the horizontal plane of the total station to the UAV using the total station after determining the spatial reference; determining a position vector of the UAV in an engineering local coordinate system as the spatial position of the UAV based on the measured position vector of the UAV in the station coordinate system, a three-dimensional rotation matrix from the station coordinate system to the engineering local coordinate system, and a position vector of the origin of the station coordinate system in the engineering local coordinate system.

4. The method of claim 3, wherein, The position vector of the UAV in the engineering local coordinate system refers to a position vector of a reference point of a prism or a passive reflector mounted to the UAV in the engineering local coordinate system, wherein the step of determining the spatial position of the UAV using the total station after determining the spatial reference further comprises: determining a position vector of the center of mass of the UAV in the engineering local coordinate system as the spatial position of the UAV based on a lever arm vector of the reference point of the prism or the passive reflector mounted to the UAV to the center of mass of the UAV in a UAV body coordinate system, a three-dimensional rotation matrix from the UAV body coordinate system to the engineering local coordinate system, and the position vector of the reference point of the prism or the passive reflector mounted to the UAV in the engineering local coordinate system.

5. The method of claim 1, wherein, Before the step of capturing the spatial image corresponding to the UAV using the camera, the method further comprises: deploying a center-symmetric high-contrast mark; measuring a position vector of the center of the center-symmetric high-contrast mark using the total station after determining the spatial reference; extracting a sub-pixel center of the center-symmetric high-contrast mark using the camera; establishing a point-image constraint of the camera based on the position vector and the sub-pixel center of the center of the center-symmetric high-contrast mark.

6. The method of claim 1, wherein, Before the step of capturing the spatial image corresponding to the UAV using the camera, the method further comprises: time-synchronizing the total station and the camera based on a global navigation satellite system; segmenting and time-delay calibrating a measurement window of the total station; Frame / line two-level time calibration of the camera is performed to establish a rolling shutter time model of frame rate scale, time offset and line latency.

7. The method of claim 6, wherein, The method further comprises: determining the camera's extrinsic parameters and aligning the camera's time with the total station's time based on the position vector and covariance of the center-symmetric high-contrast mark provided by the total station and the time service provided by the global navigation satellite system by taking the camera's extrinsic parameters and the frame rate scale, time offset and line latency of the rolling shutter time model as joint variables.

8. A system for unmanned aerial vehicle positioning based on total station measurement network and visual collaboration, characterized in that, The system comprises: a ground control point for determining a spatial reference of the total station; a total station configured to determine a spatial position of the drone; a camera configured to capture a spatial image corresponding to the drone; a host configured to receive the determined spatial position of the drone and the captured spatial image and project a positioning mark corresponding to the drone on the captured spatial image based on the determined spatial position of the drone.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium stores instructions which, when executed by at least one processor, cause the at least one processor to perform the method of drone positioning based on total station measurement network and visual collaboration according to any one of claims 1 to 7.

10. A computer program product comprising computer instructions, characterized in that, The computer instructions, when executed by at least one processor, implement the method of drone positioning based on total station measurement network and visual collaboration according to any one of claims 1 to 7.

Images