Unmanned aerial vehicle depth measurement method based on monocular vision positioning

Abstract

The application is a kind of unmanned aerial vehicle depth measurement method based on monocular vision positioning, relates to the technical field of sonar depth processing, and comprises the following steps: S01, obtaining the exterior orientation element bias correction number; S02, creating a three-dimensional model in a local coordinate system with the acoustic center of the sonar probe on the towed body as the origin, and accurately determining the three-dimensional coordinates of a plurality of coded markers arranged on the upper surface of the towed body in the local coordinate system; S03, detecting by the unmanned aerial vehicle towing the towed body, obtaining the exterior orientation elements of the monocular camera at the time of shooting each frame of image after correction by step S01; S04, solving the three-dimensional position and three-dimensional attitude of the towed body in the global coordinate system at each moment by space forward intersection; S05, obtaining the three-dimensional coordinates of the underwater terrain points in the global coordinate system. The application uses the flight of the unmanned aerial vehicle to drive the sonar to measure the underwater terrain and uses the technology of monocular vision to position the three-dimensional coordinates and attitude of the sonar to obtain the accurate coordinates of the measured points.

Description

Technical Field

[0001] This invention relates to the field of sonar depth sounding technology, and in particular to a UAV depth sounding method based on monocular vision positioning. Background Technology

[0002] Traditional underwater topographic surveying mainly relies on various ships equipped with sonar devices combined with GNSS. Currently, some also use aircraft equipped with depth-sounding lidar for detection, but both of these methods have certain limitations.

[0003] First, there's the ship-based sonar measurement method. Whether manned or unmanned, sonar detection requires entering the relevant waters, but due to various limitations, not all waters are suitable for ship deployment. Furthermore, ship operations are greatly affected by wind and waves, resulting in a relatively small working window. Taking the Qiantang River tidal bore area as an example, for safety reasons, ships can generally only conduct marine mapping operations after the tidal bore and before low tide, with operation time typically not exceeding four hours. Mapping efficiency in special areas is extremely low.

[0004] Secondly, there is the depth-sounding lidar method. This method has high requirements for water clarity. If the water body to be measured has even a slight sediment content, the laser signal will be severely interfered with, making it impossible to obtain effective data. Therefore, its application is quite limited.

[0005] To address the above problems, this paper proposes a depth sounding method based on towed sonar from an unmanned aerial vehicle (UAV). This method involves deploying a towed sonar into the water body using a UAV, and utilizing the UAV's flight to drive the sonar for underwater topographic measurement. Simultaneously, a monocular vision-based technique is employed to determine the sonar's three-dimensional coordinates and attitude. Finally, post-processing is used to calculate the precise coordinates of the measured points.

[0006] In summary, traditional underwater topographic surveying methods are inefficient and difficult to implement in complex waters because the operating platform is severely limited by the aquatic environment and water visibility, as well as by the risks and window periods of ship operations, and the stringent requirements of lidar for water clarity. Summary of the Invention

[0007] To address the aforementioned technical problems, the present invention provides a UAV depth measurement method based on monocular vision positioning, the method comprising the following steps:

[0008] S01. Determine the interior orientation elements and distortion parameters of the monocular camera of the UAV, and calibrate the relative position and attitude relationship between the UAV's GNSS and attitude measurement system and the monocular camera to obtain the exterior orientation element deviation correction number.

[0009] S02. Create a three-dimensional model in a local coordinate system with the acoustic center of the sonar probe on the towed body as the origin, and accurately measure the three-dimensional coordinates of multiple coded marks arranged on the upper surface of the towed body in the local coordinate system.

[0010] S03. The UAV tows the towed body to conduct detection, and simultaneously acquires the sequence of images containing the towed body captured by the monocular camera, the depth measurement data acquired by the sonar probe on the towed body, and the exterior orientation elements of the monocular camera when capturing each frame of the image, which are obtained by the UAV's GNSS and attitude measurement system and corrected by step S01.

[0011] S04. Process the sequence of images obtained in step S03, identify and extract the image coordinates of each coded marker, and combine them with the parameters calibrated in step S01 to calculate the three-dimensional position and three-dimensional attitude of the towed body in the global coordinate system at each moment through spatial forward intersection.

[0012] S05. Based on the calculated three-dimensional position and three-dimensional attitude of the towed body, and combined with the coordinates of the sonar probe in the local coordinate system of the towed body established in step S02, the depth measurement data obtained in step S03 is reduced to the three-dimensional coordinates of the underwater topographic points in the global coordinate system.

[0013] Preferably, in step S01, the determination of the intrinsic orientation elements and distortion parameters of the monocular camera adopts the Zhang Zhengyou calibration method. The corner pixel coordinates are extracted by taking pictures of a chessboard calibration board of known size from multiple perspectives, and the intrinsic parameter matrix and distortion coefficient model of the camera are solved by direct linear transformation and least squares method.

[0014] Preferably, the calibration of the relative position and attitude relationship between the GNSS and attitude measurement system and the monocular camera in step S01 includes:

[0015] The drone flew in a calibration field with known ground control points and acquired multiple synchronized images and corresponding UAV GNSS position and attitude angle data.

[0016] The precise exterior orientation elements of each image are calculated by spatial back intersection;

[0017] The calculated precise exterior orientation elements are subtracted from the corresponding position and attitude angle data recorded by the UAV to obtain the position deviation. and attitude angle deviation ;

[0018] The average of multiple sets of deviations is taken as the system deviation correction value to correct the UAV GNSS position and attitude angle data acquired in real time in step S03, so as to obtain the true exterior orientation elements of the monocular camera.

[0019] Preferably, the encoding mark in step S02 is a QR code;

[0020] The precise measurement involves acquiring detailed color point cloud data of the towed body using a total station or high-resolution laser scanner, establishing a local coordinate system based on the detailed color point cloud data, and determining the three-dimensional coordinates of each QR code corner point.

[0021] Preferably, step S02 further includes establishing a proportional relationship model between the physical size of the QR code and the pixel size of the camera image. Specifically, this is achieved by capturing the QR code with a monocular camera at different distances, based on the formula:

[0022] ;

[0023] in, Focal length It can be obtained through measurement using a total station or similar instruments. It can be read from the image. Given a known value, k is obtained by taking pictures of the QR code at different distances with a monocular camera. Multiple equations can be established, calculated, and then the average value is taken.

[0024] Preferably, step S04, which involves processing the image coordinates of the extracted sequence images and the extracted coded markers, includes:

[0025] S41. Using the distortion parameters described in step S01, the original image of each frame in the sequence image is distorted to obtain the distorted image.

[0026] S42. Using the FLANN-based fast nearest neighbor search algorithm, the corrected image is matched with the pre-established drag body template image to locate the region of the drag body in the image.

[0027] S43. The Canny operator based on edge detection segments a local image of the towed body from the region where the towed body is located in the image;

[0028] S44. In the segmented local image of the towed body, a CNN-based QR code detector is used to identify and decode each QR code, and the image pixel coordinates of the center point of each QR code are extracted and defined as the image coordinates of the encoded marker.

[0029] In step S42, the FLANN algorithm is used, and the index parameters of the FLANN algorithm are configured to prioritize the search of the k-means tree algorithm, and to filter and exclude incorrect matching points through k-nearest neighbor matching and Lowe's ratio test.

[0030] The Canny operator edge detection used in step S43 performs at least the following processing on the image in the region where the dragging body is located in the image: Gaussian filtering to remove noise, calculating gradient magnitude and direction, performing non-maximum suppression on the gradient magnitude, and detecting and connecting edges using a double threshold method.

[0031] Preferably, the step S04, which involves calculating the three-dimensional position and three-dimensional attitude of the towed body in the global coordinate system at each moment by spatial forward intersection, includes substituting the image coordinates of at least three non-collinear coded markers extracted in step S44 and their three-dimensional coordinates in the local coordinate system determined in step S02 into the collinearity equation, and calculating the coordinates of these marker points in the global coordinate system by spatial forward intersection.

[0032] Based on the coordinate sets of these marker points in the global coordinate system and the local coordinate system of the towed body, the optimal spatial transformation parameters between the two coordinate systems are calculated by the least squares method. The transformation parameters include a three-dimensional translation vector representing the position of the towed body and a rotation matrix representing the attitude of the towed body.

[0033] Preferably, the three-dimensional coordinates of the underwater topographic points in step S05 It can be calculated using the following formula:

[0034] ;

[0035] in, The calculated coordinates of the dragged body center in the global coordinate system. The rotation matrix from the local coordinate system to the global coordinate system of the towed body, calculated in step S04. The fixed coordinates of the sonar probe in the local coordinate system of the towed body.

[0036] The present invention has at least the following beneficial effects:

[0037] 1. Using drones towed sonar for operations eliminates the need for survey vessels to enter the water, fundamentally overcoming the difficulties of traditional methods in complex waters such as tidal flats, shoals, and dangerous rapids, where vessel deployment is impossible or the operational window is extremely short. Drones are less affected by wind and waves, allowing them to operate safely and flexibly for longer periods, greatly improving the efficiency and feasibility of surveying in special environments.

[0038] 2. Through innovative monocular vision positioning technology, the 3D position and attitude of the towed vehicle can be reconstructed with high precision using only a single ordinary camera on the drone to photograph the coded markings on the towed vehicle and then processed by post-processing algorithms. This avoids integrating expensive GNSS and inertial navigation units onto the towed vehicle, significantly reducing hardware costs, system weight, and power consumption, thereby improving the drone's endurance.

[0039] 3. Compared to ships, drones fly faster and have more flexible path planning, enabling them to quickly cover target waters. By combining the high maneuverability of drones with the reliable depth-sounding capabilities of sonar, and through visual positioning and post-processing, accurate three-dimensional coordinates of underwater terrain points are ultimately output. Attached Figure Description

[0040] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0041] Figure 1 A flowchart provided for Embodiment 1 of the present invention;

[0042] Figure 2 This is a flowchart of the towed body image data processing provided in Embodiment 1 of the present invention;

[0043] Figure 3 This is a diagram of a chessboard calibration board provided in Embodiment 1 of the present invention. Detailed Implementation

[0044] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0045] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0046] Example 1

[0047] This embodiment provides a UAV depth measurement method based on monocular vision positioning, including an apparatus for implementing this method, comprising:

[0048] The drone is equipped with a GNSS receiver, an attitude measurement unit, and a monocular camera.

[0049] A small towed vehicle that connects to the drone via a cable. The upper surface of the towed vehicle is equipped with multiple coded markings and a single-beam sonar probe.

[0050] The data processing unit receives and stores image data from a monocular camera, depth measurement data from a sonar probe, and position and attitude data from a GNSS receiver and attitude measurement unit, and outputs the three-dimensional coordinates of underwater topographic points.

[0051] The above method includes the following steps, such as Figure 1 As shown:

[0052] S01. Determine the interior orientation elements and distortion parameters of the monocular camera of the UAV, and calibrate the relative position and attitude relationship between the UAV's GNSS and attitude measurement system and the monocular camera to obtain the exterior orientation element deviation correction number.

[0053] Furthermore, the aforementioned determination of the monocular camera's interior orientation elements and distortion parameters employs the Zhang Zhengyou calibration method. Corner pixel coordinates are extracted by photographing a known-sized checkerboard calibration board from multiple perspectives, and the camera's intrinsic parameter matrix and distortion coefficient model are solved using direct linear transformation and the least squares method. Specifically, the interior orientation elements are parameters describing the relative position between the image center and the image, including the principal distance (f) and the principal point coordinates (u0, v0), which are important parameters in photogrammetry; while distortion primarily corrects radial and tangential distortion. Using the Zhang Zhengyou calibration method to obtain the monocular camera's interior orientation elements and distortion model, the interior orientation elements and camera distortion parameters can be represented by matrix A:

[0054] ;

[0055] In the above formula, Represents the principal point coordinates. This includes parameters that incorporate distortion and focal length.

[0056] First, prepare the chessboard grid marking board ( Figure 3 The size of the chessboard grid on the calibration board is preset.

[0057] Next, an image of the checkerboard calibration board with a coordinate system is obtained, and the coordinates of each corner point are detected.

[0058] Then calculate the coordinates of each corner point in the calibrated coordinate system, denoted as (u, v, w=0), and then perform the following calculations:

[0059] ;

[0060] ;

[0061] ;

[0062] in, , The above matrix is ​​a homogeneous matrix with 8 unknown elements. Each corner point on the calibration board can provide two constraint equations. Only 4 corner points are needed to find matrix M for an image. When an image has more than 4 corner points, the least squares method can be used to find the optimal M.

[0063] Next, we obtain the intrinsic parameter matrix A:

[0064] ;

[0065] Given that the two columns of a rotation matrix have the property of being orthogonal to each other, i.e.:

[0066] ;

[0067] Depend on , It can be known that:

[0068] ;

[0069] ;

[0070] remember Then there is

[0071] ;

[0072] ;

[0073] ;

[0074] remember ,in

[0075] Then we have:

[0076] ;

[0077] The above is recorded as:

[0078] ;

[0079] Each calibration board image can provide more than one constraint relationship, which contains two constraint equations. Since vector b has 6 parameters, three calibration board images are needed to solve for vector b. When there are more than 3 images, the least squares method can be used to obtain vector b, then matrix B, and finally the intrinsic parameter matrix A.

[0080] ;

[0081] ;

[0082] ;

[0083] ;

[0084] ;

[0085] Thus, the intrinsic parameter matrix was obtained.

[0086] Furthermore, the relative position and attitude relationship between the aforementioned calibrated GNSS and attitude measurement system and the monocular camera includes:

[0087] The drone flew in a calibration field with known ground control points and acquired multiple synchronized images and corresponding UAV GNSS position and attitude angle data.

[0088] The precise exterior orientation elements of each image are calculated by spatial back intersection;

[0089] The calculated precise exterior orientation elements are subtracted from the corresponding position and attitude angle data recorded by the UAV to obtain the position deviation. and attitude angle deviation ;

[0090] The average of multiple sets of deviations is taken as the system deviation correction value to correct the UAV GNSS position and attitude angle data acquired in real time in step S03, so as to obtain the true exterior orientation elements of the monocular camera.

[0091] In detail, the relative positions and attitude relationships of the UAV's GNSS, attitude sensor, and monocular camera:

[0092] A monocular camera is mounted on a drone, which is then used to acquire images of a calibrated area. Simultaneously, the position and attitude data of the images, as well as the capture time, are recorded. This data can be represented as: .

[0093] In the above formula, each parameter represents the sequence number, image acquisition time, image x-coordinate, image y-coordinate, image z-coordinate, image roll angle, image pitch angle, and image yaw angle, respectively.

[0094] Since the camera is not calibrated, the real-time recorded position and orientation deviate from the camera's actual exterior orientation elements. Calibration involves determining these deviations, which are denoted here as: .

[0095] Calculate the correct exterior orientation elements of the corresponding photograph using the calibration field:

[0096] The coordinates of the ground control points are known to be

[0097] The exterior orientation elements for each photograph can be obtained by solving the following formula:

[0098] (a);

[0099] The solution obtained can be denoted as

[0100] The above data represents accurate exterior orientation elements.

[0101] For a photograph taken at the same moment, the difference between the value recorded by the attitude sensor and the true value (obtained through calculation) is as follows:

[0102] .

[0103] The correction is obtained by matching the calculated result with the actual observation result using timestamps and then calculating the difference. When there is more than one photograph, multiple corrections can be calculated. The average of these corrections is then taken to obtain the correction for the deviation value, denoted as . .

[0104] Applying the above corrections to the GNSS and attitude data recorded by the UAV, the true exterior orientation elements of the images acquired by the monocular camera can be obtained, which can be expressed as:

[0105] ;

[0106] After completing the above two steps, any image acquired by the UAV through the monocular camera will have accurate interior and exterior orientation elements, which can meet the needs of related image processing work.

[0107] S02. Create a three-dimensional model in a local coordinate system with the acoustic center of the sonar probe on the towed body as the origin, and accurately measure the three-dimensional coordinates of multiple coded markers arranged on the upper surface of the towed body in the local coordinate system.

[0108] The aforementioned coding mark is a QR code; while accurate measurement is achieved by acquiring fine color point cloud data of the towed body using a total station or high-resolution laser scanner, and establishing a local coordinate system based on the fine color point cloud data and measuring the three-dimensional coordinates of each QR code corner point.

[0109] In detail, QR codes are sprayed onto the upper surface of the small tow vehicle. Each QR code needs to be sprayed onto a flat surface. After spraying, the tow vehicle is then subjected to the following operations:

[0110] First, use a total station or similar equipment to accurately determine the size, dimensions, and relative position of each QR code;

[0111] Then, a high-resolution laser scanner is used to scan the towed body to obtain detailed color point cloud data of the towed body;

[0112] Model the point cloud data, establish the local coordinate system of the towed body, and obtain the three-dimensional coordinates of each QR code and depth sounding transducer based on it. The coordinate system of the towed body is defined as follows: the acoustic center of the transducer is the origin, the bow direction of the towed body is the positive X-axis, the Z-axis is positive upwards, and the Y-axis is positive to the starboard side of the ship. Construct a left-handed coordinate system.

[0113] Use a monocular camera to capture QR code photos of small towed objects at different distances.

[0114] Establish the relationship between QR code size and distance to the monocular camera:

[0115] For any QR code, its physical side length is When the distance to the monocular camera is The number of pixels in At this point, the relationship between the focal length and the focal length has the following coefficient relationship:

[0116] ;

[0117] In the above formulas, f is the focal length, which has already been obtained using the interior orientation elements. It can be obtained through measurement using a total station or similar instruments. It can be read from the image. Given a known value, k is obtained by taking pictures of the QR code at different distances with a monocular camera. Multiple equations can be established, calculated, and then the average value is taken.

[0118] S03. The drone tows a towed body for detection, and simultaneously acquires the sequence of images containing the towed body captured by the monocular camera, the depth measurement data acquired by the sonar probe on the towed body, and the exterior orientation elements of the monocular camera when capturing each frame of the image, which are obtained by the GNSS and attitude measurement system of the drone and corrected in step S01.

[0119] Specifically, drones are used to tow a vehicle via cables for exploration. The exploration mainly records water depth measurement data, timestamp data, and images of the towed vehicle acquired simultaneously.

[0120] The connected equipment was deployed into the water for measurement: the GNSS and attitude sensors on the UAV were used to record the position and attitude of the images acquired by the monocular camera, the sonar equipment on the towed vehicle was used to record the water depth, and the 1pps system of the GNSS equipment was used to synchronize the time of image acquisition by the monocular camera and the time of water depth acquisition by the sonar equipment, resulting in the following sequence data:

[0121] .

[0122] In the above formula, t represents the time of image acquisition, photo represents the acquired image of the towed object, and x, y, and z are the spatial coordinates of the camera. These represent the roll angle, pitch angle, and yaw angle of the image.

[0123] S04. Process the sequence images obtained in step S03, identify and extract the image coordinates of each coded marker, and combine them with the parameters calibrated in step S01 to calculate the three-dimensional position and three-dimensional attitude of the towed body in the global coordinate system at each moment through spatial forward intersection.

[0124] Specifically, in combination Figure 2 As shown, the above image coordinate image processing for extracting sequence image recognition and extracting each coded marker includes:

[0125] S41. Using the distortion parameters in step S01, the original image of each frame in the sequence image is distorted to obtain the distorted image.

[0126] S42. Using the FLANN-based fast nearest neighbor search algorithm, the corrected image is matched with the pre-established towed body template image to locate the towed body region in the image.

[0127] S43. The Canny operator based on edge detection segments the local image of the towed body from the region where the towed body is located in the image;

[0128] S44. In the segmented local image of the towed body, a CNN-based QR code detector is used to identify and decode each QR code, and the image pixel coordinates of the center point of each QR code are extracted and defined as the image coordinates of the encoded marker.

[0129] In step S42, the FLANN algorithm is used, and the index parameters of the FLANN algorithm are configured to prioritize the search of the k-means tree algorithm, and to filter and exclude incorrect matching points through k-nearest neighbor matching and Lowe's ratio test.

[0130] The Canny operator edge detection used in step S43 performs at least the following processing on the image in the region where the drag body is located in the image: Gaussian filtering to remove noise, calculating gradient magnitude and direction, performing non-maximum suppression on the gradient magnitude, and detecting and connecting edges using a double thresholding method.

[0131] The above process applies the following steps to each frame of the image:

[0132] (1) First, the distortion parameters obtained in advance are used to correct the image:

[0133] ;

[0134] The above, and This represents the image data before and after the correction.

[0135] (2) Then, the range of the towed body image is determined by image matching technology. Here, the FLANN priority search k-means tree algorithm is used.

[0136] In the preliminary work, photos of the towed objects were acquired. Photos containing all the QR codes were used as the reference photos for matching, and then matching was performed on each set of observation data. Furthermore, based on prior conditions, it is known that an image can contain exactly one image of a towed object.

[0137] Considering that the pose of the towed object may change, the FLANN algorithm is used for image matching to find the towed object from the corrected monocular camera.

[0138] Using FLANN matching requires two dictionaries as arguments. These determine the algorithm to use and other relevant parameters. The first is `IndexParams`, which uses a k-means tree search algorithm in this case; the second is `SearchParams`, which specifies the number of recursive traversals. Higher values ​​result in more accurate results, but also consume more time.

[0139] Some of the detected matches may be incorrect positive examples. This is because the k-value used in the kNN matching is 2 (finding two points in the training set). The first match is the nearest neighbor, and the second match is the second nearest neighbor. Intuitively, a correct match will be closer to the first neighbor. In other words, an incorrect match results in two neighbors with similar distances. Therefore, the quality of the match can be judged by the difference in their distances. Poor matches are discarded, and good matches are retained.

[0140] FLANN is short for Fast Library for Approximate Nearest Neighbors. It is a collection of optimized algorithms for performing nearest neighbor searches on large datasets and high-dimensional features.

[0141] (3) Next, the local image of the dragging body is segmented from the image using image segmentation technology;

[0142] After matching, rectangular image data containing the towed body was obtained. Considering the significant difference between the color of the towed body and the water color, an edge detection-based segmentation method was used to separate the local image of the towed body from the rectangular image data. Specifically, the Canny operator was used, and the steps are as follows:

[0143] First, use a kill filter to remove noise from the image:

[0144] The two-dimensional Gaussian formula is:

[0145] ;

[0146] Then the gradient image and angle image are calculated.

[0147] Based on this, non-maximum suppression is performed on the gradient image, and interference from edge coarseness and weak edges is proposed.

[0148] Finally, edge concatenation is performed using double thresholds to obtain the edges.

[0149] (4) Extract the feature information of each QR code through image enhancement technology;

[0150] After completing the image matching and segmentation tasks, the main task is to filter out each QR code from the image and read the corresponding feature information.

[0151] Use a CNN-based QR code detector to detect QR codes:

[0152] Based on the SSD framework, a compact and efficient QR code detector was constructed, and targeted optimizations were made using techniques such as residual concat, depthwise convolution, dilated convolution, and convolution projection.

[0153] After QR code detection is complete, each QR code is decoded, and its coordinates are recorded. The information for each QR code is then saved and output in the following format: The position and attitude data of the towed body are calculated using image data.

[0154] Furthermore, by solving the three-dimensional position and three-dimensional attitude of the towed body in the global coordinate system at each moment through spatial forward intersection, the image coordinates of at least three non-collinear coded markers extracted in step S44 and their three-dimensional coordinates in the local coordinate system determined in step S02 are substituted into the collinearity equation, and the coordinates of these marker points in the global coordinate system are solved by spatial forward intersection.

[0155] Based on the coordinate sets of these marker points in the global coordinate system and the local coordinate system of the towed body, the optimal spatial transformation parameters between the two coordinate systems are calculated by the least squares method. The transformation parameters include a three-dimensional translation vector representing the position of the towed body and a rotation matrix representing the attitude of the towed body.

[0156] In detail, based on the interior and exterior azimuth parameters of each photo, the position and attitude information of the dragged object in each frame of the image are restored.

[0157] According to formula (a), by substituting the information of the QR codes into the equation, the geographical coordinates of each QR code can be obtained. .

[0158] Based on the QR code encoding, each location can be precisely matched with the QR code. Then, by distributing multiple QR codes, a local ship coordinate system is established and compared with the standard ship coordinate system to calculate the center position and rotation angle of the towed body.

[0159] ;

[0160] .

[0161] S05. Based on the calculated three-dimensional position and attitude of the towed body, and combined with the coordinates of the sonar probe in the local coordinate system of the towed body established in step S02, the depth measurement data obtained in step S03 is reduced to the three-dimensional coordinates of the underwater terrain points in the global coordinate system.

[0162] The above-mentioned three-dimensional coordinates of underwater topographic points It can be calculated using the following formula:

[0163] ;

[0164] in, The calculated coordinates of the dragged body center in the global coordinate system. The rotation matrix from the local coordinate system to the global coordinate system of the towed body, calculated in step S04. The fixed coordinates of the sonar probe in the local coordinate system of the towed body.

[0165] In detail, the water depth data is converted to the user-specified coordinate system using the ship's coordinate system.

[0166] The specific coordinates of the depth point are:

[0167] .

[0168] The above formulas are used to project water depth with attitude onto the correct geographical location.

[0169] This embodiment employs a drone-towed sonar system, eliminating the need for a survey vessel to enter the water. This fundamentally overcomes the limitations of traditional methods, which struggle with deployment in complex waters such as tidal flats, shoals, and dangerous rapids, or have extremely short operational windows. Drones are less affected by wind and waves, allowing for safer and more flexible operations over longer periods, significantly improving surveying efficiency and feasibility in challenging environments. Secondly, innovative monocular visual positioning technology allows for high-precision reconstruction of the towed body's 3D position and attitude using only a single camera on the drone to photograph coded markings. This avoids integrating expensive GNSS and inertial navigation units onto the towed body, significantly reducing hardware costs, system weight, and power consumption, thus enhancing the drone's endurance. Furthermore, compared to ships, drones fly faster and have more flexible path planning, enabling rapid coverage of target waters. By combining the drone's high maneuverability with the reliable depth sounding capabilities of sonar, and through visual positioning and post-processing, accurate 3D coordinates of underwater topographic points are ultimately output.

[0170] Example 2

[0171] This invention provides a non-transitory computer-readable storage medium storing at least one instruction or at least one program segment, which is loaded and executed by a processor to implement the following steps:

[0172] Determine the interior orientation elements and distortion parameters of the monocular camera of the UAV, and calibrate the relative position and attitude relationship between the UAV's GNSS and attitude measurement system and the monocular camera to obtain the exterior orientation element deviation correction.

[0173] A three-dimensional model is created using a local coordinate system with the acoustic center of the sonar probe on the towed body as the origin, and the three-dimensional coordinates of multiple coded markers arranged on the upper surface of the towed body in the local coordinate system are accurately measured.

[0174] The detection is carried out by a drone towing a towed body, and the sequence of images including the towed body taken by a monocular camera, the depth measurement data obtained by the sonar probe on the towed body, and the exterior orientation elements of the monocular camera when taking each frame of the image obtained by the GNSS and attitude measurement system of the drone and corrected.

[0175] The acquired sequence of images is processed, the image coordinates of each coded marker are identified and extracted, and combined with the calibrated parameters, the three-dimensional position and three-dimensional attitude of the towed body in the global coordinate system at each moment are calculated by spatial forward intersection.

[0176] Based on the calculated three-dimensional position and attitude of the towed body, combined with the coordinates of the sonar probe in the local coordinate system of the towed body, the acquired depth sounding data is converted to the three-dimensional coordinates of underwater topographic points in the global coordinate system.

[0177] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0178] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0179] Example 3

[0180] This invention provides an electronic device, including a processor and a memory, wherein the memory stores at least one instruction or at least one program segment, and the at least one instruction or the at least one program segment is loaded and executed by the processor to implement the following steps:

[0181] Determine the interior orientation elements and distortion parameters of the monocular camera of the UAV, and calibrate the relative position and attitude relationship between the UAV's GNSS and attitude measurement system and the monocular camera to obtain the exterior orientation element deviation correction.

[0182] A three-dimensional model is created using a local coordinate system with the acoustic center of the sonar probe on the towed body as the origin, and the three-dimensional coordinates of multiple coded markers arranged on the upper surface of the towed body in the local coordinate system are accurately measured.

[0183] The detection is carried out by a drone towing a towed body, and the sequence of images including the towed body taken by a monocular camera, the depth measurement data obtained by the sonar probe on the towed body, and the exterior orientation elements of the monocular camera when taking each frame of the image obtained by the GNSS and attitude measurement system of the drone and corrected.

[0184] The acquired sequence of images is processed, the image coordinates of each coded marker are identified and extracted, and combined with the calibrated parameters, the three-dimensional position and three-dimensional attitude of the towed body in the global coordinate system at each moment are calculated by spatial forward intersection.

[0185] Based on the calculated three-dimensional position and attitude of the towed body, combined with the coordinates of the sonar probe in the local coordinate system of the towed body, the acquired depth sounding data is converted to the three-dimensional coordinates of underwater topographic points in the global coordinate system.

[0186] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A method for UAV depth measurement based on monocular vision positioning, characterized in that, The method includes the following steps: S01. Determine the interior orientation elements and distortion parameters of the monocular camera of the UAV, and calibrate the relative position and attitude relationship between the UAV's GNSS and attitude measurement system and the monocular camera to obtain the exterior orientation element deviation correction number. S02. Create a three-dimensional model in a local coordinate system with the acoustic center of the sonar probe on the towed body as the origin, and accurately measure the three-dimensional coordinates of multiple coded marks arranged on the upper surface of the towed body in the local coordinate system. S03. The UAV tows the towed body to conduct detection, and simultaneously acquires the sequence of images containing the towed body captured by the monocular camera, the depth measurement data acquired by the sonar probe on the towed body, and the exterior orientation elements of the monocular camera when capturing each frame of the image, which are obtained by the UAV's GNSS and attitude measurement system and corrected by step S01. S04. Process the sequence of images obtained in step S03, identify and extract the image coordinates of each coded marker, and combine them with the parameters calibrated in step S01 to calculate the three-dimensional position and three-dimensional attitude of the towed body in the global coordinate system at each moment through spatial forward intersection. S05. Based on the calculated three-dimensional position and three-dimensional attitude of the towed body, and combined with the coordinates of the sonar probe in the local coordinate system of the towed body established in step S02, the depth measurement data obtained in step S03 is reduced to the three-dimensional coordinates of the underwater topographic points in the global coordinate system.

2. The UAV depth measurement method based on monocular vision positioning according to claim 1, characterized in that, In step S01, the determination of the intrinsic orientation elements and distortion parameters of the monocular camera adopts the Zhang Zhengyou calibration method. By taking pictures of a chessboard calibration board of known size from multiple perspectives, the corner pixel coordinates are extracted, and the intrinsic parameter matrix and distortion coefficient model of the camera are solved directly using direct linear transformation and least squares method.

3. The UAV depth measurement method based on monocular vision positioning according to claim 1, characterized in that, The calibration of the relative position and attitude relationship between the GNSS and attitude measurement system and the monocular camera in step S01 includes: The drone flew in a calibration field with known ground control points and acquired multiple synchronized images and corresponding UAV GNSS position and attitude angle data. The precise exterior orientation elements of each image are calculated by spatial back intersection; The calculated precise exterior orientation elements are subtracted from the corresponding position and attitude angle data recorded by the UAV to obtain the position deviation. and attitude angle deviation ; The average of multiple sets of deviations is taken as the system deviation correction value to correct the UAV GNSS position and attitude angle data acquired in real time in step S03, so as to obtain the true exterior orientation elements of the monocular camera.

4. The UAV depth measurement method based on monocular vision positioning according to claim 1, characterized in that, The encoding mark in step S02 is a QR code; The precise measurement involves acquiring detailed color point cloud data of the towed body using a total station or high-resolution laser scanner, establishing a local coordinate system based on the detailed color point cloud data, and determining the three-dimensional coordinates of each QR code corner point.

5. The UAV depth measurement method based on monocular vision positioning according to claim 4, characterized in that, Step S02 further includes establishing a proportional relationship model between the physical size of the QR code and the pixel size of the camera image. Specifically, this is done by capturing the QR code with a monocular camera at different distances, based on the formula: ； in, Focal length d can be obtained by measuring with a total station. 像素 It can be read from the image. Given a known value, k is obtained by taking pictures of the QR code at different distances using a monocular camera. Multiple equations can be established, calculated, and then the average value can be taken.

6. The UAV depth measurement method based on monocular vision positioning according to claim 1, characterized in that, The image coordinate image processing for acquiring the sequence image recognition and extracting each coded marker in step S04 includes: S41. Using the distortion parameters described in step S01, the original image of each frame in the sequence image is distorted to obtain the distorted image. S42. Using the FLANN-based fast nearest neighbor search algorithm, the corrected image is matched with the pre-established drag body template image to locate the region of the drag body in the image. S43. The Canny operator based on edge detection segments a local image of the towed body from the region where the towed body is located in the image; S44. In the segmented local image of the towed body, a CNN-based QR code detector is used to identify and decode each QR code, and the image pixel coordinates of the center point of each QR code are extracted and defined as the image coordinates of the encoded marker. In step S42, the FLANN algorithm is used, and the index parameters of the FLANN algorithm are configured to prioritize the search of the k-means tree algorithm, and to filter and exclude incorrect matching points through k-nearest neighbor matching and Lowe's ratio test. The Canny operator edge detection used in step S43 performs at least the following processing on the image in the region where the dragging body is located in the image: Gaussian filtering to remove noise, calculating gradient magnitude and direction, performing non-maximum suppression on the gradient magnitude, and detecting and connecting edges using a double threshold method.

7. The UAV depth measurement method based on monocular vision positioning according to claim 6, characterized in that, The step S04, which involves calculating the three-dimensional position and three-dimensional attitude of the towed body in the global coordinate system at each moment by spatial forward intersection, includes substituting the image coordinates of at least three non-collinear coded markers extracted in step S44 and their three-dimensional coordinates in the local coordinate system determined in step S02 into the collinearity equation, and calculating the coordinates of these marker points in the global coordinate system by spatial forward intersection. Based on the coordinate sets of these marker points in the global coordinate system and the local coordinate system of the towed body, the optimal spatial transformation parameters between the two coordinate systems are calculated by the least squares method. The transformation parameters include a three-dimensional translation vector representing the position of the towed body and a rotation matrix representing the attitude of the towed body.

8. The UAV depth measurement method based on monocular vision positioning according to claim 1, characterized in that, The three-dimensional coordinates of the underwater topographic points in step S05 It can be calculated using the following formula: ； in, The calculated coordinates of the dragged body center in the global coordinate system. The rotation matrix from the local coordinate system to the global coordinate system of the towed body, calculated in step S04. The fixed coordinates of the sonar probe in the local coordinate system of the towed body.

9. A non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores at least one instruction or at least one program segment, characterized in that, The at least one instruction or the at least one program segment is loaded and executed by the processor to implement the UAV depth measurement method based on monocular vision positioning as described in any one of claims 1-8.

10. An electronic device, characterized in that, It includes a processor and a memory, wherein the memory stores at least one instruction or at least one program, the at least one instruction or the at least one program being loaded and executed by the processor to implement the UAV depth measurement method based on monocular vision positioning as described in any one of claims 1-8.

Images