Monocular camera-based unmanned aerial vehicle sea surface target distance measurement method, device and equipment, medium and product

By performing internal and external parameter calibration and YOLO v8 target detection on the monocular camera, combined with the Earth's curvature projection, the curve distance between the UAV and the sea surface target is calculated, and the traditional monocular distance measurement algorithm ignores the distance measurement error caused by the Earth's curvature is improved. The distance measurement accuracy is achieved.

CN119022877APending Publication Date: 2024-11-26SHANGHAI UNIV
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202411133975.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-19
Publication Date
2024-11-26

AI Technical Summary

Technical Problem

The traditional monocular camera ranging algorithm ignores the influence of the earth's curvature, resulting in a deviation in measurement results when measuring sea surface targets, especially at long distances, which significantly affects the ranging accuracy.

Method used

By calibrating the internal and external parameters of the monocular camera, using the YOLO v8 target detection network to detect the sea surface target, calculate the depth coordinates of the target under the camera coordinate system, and calculate the drone to the actual sea surface target through rigid body three-dimensional transformation and earth curvature projection, the drone to the actual sea surface target Curve distance.

Benefits of technology

It significantly improves the accuracy of monocular ranging, ensures that the ranging results are closer to the actual distance, and is suitable for drones in areas such as ocean monitoring and navigation.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119022877A_ABST
    Figure CN119022877A_ABST
Patent Text Reader

Abstract

The invention discloses an unmanned aerial vehicle sea surface target ranging method, device and equipment based on a monocular camera, a medium and a product, and relates to the field of sea surface target ranging, and the method comprises the steps: calibrating internal and external parameters of the monocular camera; detecting a target in a sea surface scene by using a YOLO v8 target detection network based on a sea surface scene image collected by a monocular camera, and determining a target observation point; based on internal and external parameters of the monocular camera, calculating depth coordinates of the target observation point in a camera coordinate system according to a perspective projection geometrical relationship, and calculating three-dimensional coordinates of the target observation point in an unmanned aerial vehicle appendage coordinate system according to rigid body three-dimensional transformation; projecting a target observation point in the plane world coordinate system to the earth surface, and determining an actual sea surface observation point; and calculating a curve distance from a projection point of the unmanned aerial vehicle on the sea surface to an actual sea surface observation point based on the three-dimensional coordinates. According to the invention, the influence of the earth curvature is added into the distance measurement algorithm, so that the precision of monocular distance measurement can be improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of sea surface target ranging, and in particular to a method, device, equipment, medium and product for sea surface target ranging by an unmanned aerial vehicle based on a monocular camera. Background Art

[0002] In the task of observing and monitoring targets at sea, the image information obtained by drones is the most direct and effective source. Compared with active information acquisition methods such as radar, the passive information acquisition method using optical imaging equipment combined with visual image processing does not require the emission of electromagnetic pulse signals, so it has good concealment and strong anti-electromagnetic interference capabilities. These characteristics make it have important application value in the civil or military fields of ocean monitoring. With the continuous deepening of research in the field of computer vision, artificial intelligence technology has gradually replaced traditional manual analysis and become the main means of extracting key information from images. Based on this progress, restoring the three-dimensional information of the target through coordinate transformation to achieve visual ranging has become an effective and practical ranging method. This method not only improves the accuracy and efficiency of information acquisition, but also expands the application prospects of drones in various marine monitoring tasks.

[0003] Visual ranging technology can be divided into two categories: multi-view ranging and monocular ranging. Multi-view ranging obtains disparity information through feature extraction and stereo matching, thereby obtaining the distance information of all objects in the view. In contrast, monocular ranging has lower registration and installation requirements, is easier to operate, and has a wider range of applications. Although monocular ranging cannot directly obtain the disparity information of all objects, it can still accurately estimate the distance of the target through certain algorithms and models. This makes monocular ranging have obvious advantages in practical applications, especially in terms of equipment simplicity and ease of operation.

[0004] Traditional monocular camera ranging algorithms usually ignore the influence of the earth's curvature, which can lead to significant errors when using drones to measure long-distance targets on the sea surface. The reason why the earth's curvature affects the ranging accuracy is that when a monocular camera measures a long-distance target, the line of sight actually extends along the curve of the earth's surface rather than a straight line. This curvature causes a difference between the actual distance between the camera and the target and the straight-line distance calculated geometrically, which in turn affects the geometric calculation based on camera imaging. At the same time, the curvature of the earth causes distant objects to gradually approach the horizon or even be partially blocked, resulting in a shorter distance estimate when the curvature is not taken into account. On the sea surface, since the target is farther away and the influence of the earth's curvature is more significant, ignoring this factor will lead to deviations in the measurement results. Summary of the invention

[0005] The purpose of this application is to provide a method, device, equipment, medium and product for UAV sea surface target ranging based on a monocular camera, which can improve the accuracy of monocular ranging.

[0006] To achieve the above objectives, this application provides the following solutions:

[0007] In a first aspect, the present application provides a method for measuring sea surface targets by a UAV based on a monocular camera, comprising:

[0008] Calibrate the internal and external parameters of the monocular camera to obtain the internal and external parameters of the monocular camera;

[0009] Based on the sea scene images collected by the monocular camera, the YOLO v8 target detection network is used to detect the targets in the sea scene and determine the target observation points;

[0010] Based on the internal and external parameters of the monocular camera, the depth coordinates of the target observation point in the camera coordinate system are calculated according to the perspective projection geometric relationship;

[0011] Based on the depth coordinates, the three-dimensional coordinates of the target observation point in the coordinate system of the drone are calculated according to the rigid body three-dimensional transformation;

[0012] Project the target observation point in the plane world coordinate system onto the earth's surface to determine the actual sea surface observation point;

[0013] The curve distance between the projection point of the UAV on the sea surface and the actual sea surface observation point is calculated based on the three-dimensional coordinates to complete the UAV sea surface target ranging.

[0014] In a second aspect, the present application provides a UAV sea surface target ranging device based on a monocular camera, comprising:

[0015] The calibration module is used to calibrate the internal and external parameters of the monocular camera to obtain the internal and external parameters of the monocular camera;

[0016] The target detection module is used to detect targets in the sea scene based on the sea scene image collected by the monocular camera using the YOLO v8 target detection network and determine the target observation point;

[0017] A depth coordinate calculation module, used to calculate the depth coordinate of the target observation point in the camera coordinate system according to the perspective projection geometric relationship based on the internal and external parameters of the monocular camera;

[0018] A three-dimensional coordinate calculation module, used to calculate the three-dimensional coordinates of the target observation point in the coordinate system of the drone body based on the depth coordinates and according to the three-dimensional transformation of the rigid body;

[0019] The actual sea surface observation point determination module is used to project the target observation point in the plane world coordinate system onto the earth surface to determine the actual sea surface observation point;

[0020] The curve distance calculation module is used to calculate the curve distance between the projection point of the UAV on the sea surface and the actual sea surface observation point based on the three-dimensional coordinates, so as to complete the UAV sea surface target ranging.

[0021] In a third aspect, the present application provides a computer device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of any one of the above-mentioned methods for measuring sea surface targets by a UAV based on a monocular camera.

[0022] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the above-mentioned methods for measuring sea surface targets by an unmanned aerial vehicle based on a monocular camera.

[0023] In a fifth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of any of the above-mentioned methods for measuring sea surface targets by an unmanned aerial vehicle based on a monocular camera.

[0024] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0025] The present application provides a method, device, equipment, medium and product for measuring sea surface targets using a drone based on a monocular camera. The drone can directly and effectively perform three-dimensional transformation in the acquired image information through the monocular camera carried by the drone to obtain the three-dimensional coordinates of the target observation point in the drone's attached coordinate system, and consider the influence of the earth's curvature to project the target observation point in the plane world coordinate system onto the earth's surface to obtain the actual sea surface observation point, and calculate the curvilinear distance between the drone's projection point on the sea surface and the actual sea surface observation point. The present application considers the earth's curvature to make the ranging result closer to the actual distance, thereby significantly improving the ranging accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0027] Figure 1 A schematic diagram of a flow chart of a method for measuring sea surface targets using a UAV based on a monocular camera according to an embodiment of the present application;

[0028] Figure 2 This is the principle diagram of monocular vision ranging;

[0029] Figure 3 It is a schematic diagram of the projection of the plane world coordinate system onto the surface of the earth;

[0030] Figure 4 A schematic diagram of the functional modules of a monocular camera-based UAV sea surface target ranging device provided in one embodiment of the present application;

[0031] Figure 5 A schematic diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0032] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0033] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0034] In an exemplary embodiment, Figure 1 As shown, a method for measuring sea surface targets of a UAV based on a monocular camera is provided. The method is executed by a computer device, and can be executed by a computer device such as a terminal or a server alone, or by a terminal and a server together. In the embodiment of the present application, the method is applied to a server as an example for explanation, and includes the following steps S1 to S6. Among them:

[0035] S1: Calibrate the internal and external parameters of the monocular camera to obtain the internal and external parameters of the monocular camera. The internal and external parameters of the monocular camera include the monocular camera internal parameter matrix and the monocular camera external parameter matrix.

[0036] S2: Based on the sea scene image captured by the monocular camera, the YOLO v8 target detection network is used to detect the targets in the sea scene and determine the target observation points.

[0037] S3: Based on the internal and external parameters of the monocular camera, the depth coordinates of the target observation point in the camera coordinate system are calculated according to the perspective projection geometric relationship.

[0038] S4: Based on the depth coordinates, the three-dimensional coordinates of the target observation point in the drone attached coordinate system are calculated according to the rigid body three-dimensional transformation.

[0039] S5: Project the target observation point in the plane world coordinate system onto the earth's surface to determine the actual sea surface observation point.

[0040] S6: Based on the three-dimensional coordinates, the curve distance between the projection point of the drone on the sea surface and the actual sea surface observation point is calculated to complete the drone sea surface target ranging. This application has important practical application value and can be widely used in the fields of drone navigation, target tracking, etc., providing strong support for technological innovation and progress in related industries.

[0041] By implementing the above steps S1 to S6, the present application adds the influence of the earth's curvature into the ranging algorithm, thereby improving the accuracy of monocular ranging.

[0042] In another exemplary embodiment of the present application, step S1 specifically includes:

[0043] S11: Use the calibration checkerboard to calibrate the intrinsic parameters of the monocular camera using the Zhang Zhengyou calibration method to obtain the intrinsic parameter matrix.

[0044] The camera intrinsic parameters are the inherent properties of the camera, which usually do not change. They are calibrated using the calibration checkerboard through the Zhang Zhengyou calibration method. First, the calibrated checkerboard is placed at different positions and angles in the field of view of the monocular camera, and 15-20 images containing the calibration checkerboard are taken. Next, a preliminary estimate of the intrinsic parameters is made, and the intrinsic parameters of the camera are estimated using a linear method. After that, the intrinsic parameters of the monocular camera are further optimized by minimizing the reprojection error through nonlinear optimization methods (such as the Levenberg-Marquardt algorithm). Through this process, the intrinsic parameter matrix K of the monocular camera can be obtained, including the focal length f x , f y (normalized focal lengths in the x and y directions, respectively), and optical center coordinates u0 and v0 (the horizontal and vertical coordinates of the image center, respectively), provide a data basis for the subsequent distance measurement of the target observation point.

[0045] S12: Using the IMU data output by the drone, calibrate the extrinsic parameters of the monocular camera using the Rodriguez formula to obtain an extrinsic parameter matrix; the extrinsic parameter matrix includes a rotation matrix and a translation matrix.

[0046] The camera external parameters will change with the camera's position and posture. The IMU data output by the drone is used to calibrate the external parameters through the Rodriguez method to obtain the rotation matrix and translation matrix of the monocular camera. First, the IMU output data of the drone includes the height H of the drone from the sea surface, the yaw angle τ, pitch angle θ and roll angle of the monocular camera. In the marine monocular ranging algorithm model, the position of the monocular camera on the horizontal plane (X0, Y0) defaults to (0m, 0m), and the coordinate in the Z direction is H, which can be calibrated by the data output by the IMU to obtain the translation matrix T. Among the angles of the monocular camera, the yaw angle τ, pitch angle θ and roll angle of the monocular camera are The data can also be directly output by the IMU, and the camera's yaw angle τ, pitch angle θ and roll angle are calculated using the Rodriguez formula. Calculate to obtain the rotation matrix R.

[0047] In another exemplary embodiment of the present application, step S2 specifically includes:

[0048] S21: Input the sea scene image captured by the monocular camera into the YOLO v8 target detection network to obtain the anchor frame for target detection.

[0049] S22: Calculate the mean coordinates of the pixels in the anchor box.

[0050] S23: Using the coordinate mean as the coordinate of the target observation point to determine the target observation point.

[0051] The YOLO v8 network is used to detect and identify targets such as ships and warships in the sea scene images collected by the monocular camera. The YOLO v8 target detection network is used to output the anchor frame of the target detection, and the mean of the vertical and horizontal coordinates of the pixels in the area where the anchor frame is located is calculated and extracted. The combined horizontal and vertical coordinates are used as the pixel coordinates of the target observation point.

[0052] In another exemplary embodiment of the present application, step S3 specifically includes:

[0053] Reference Figure 2 As shown, the sea surface is first optimized to a standard plane. Figure 2 where xOy is the image coordinate system, Z c represents the Z axis and optical axis of the camera coordinate system, XO2Y represents the Z=0 plane in the coordinate system of the drone, i.e., the sea surface, O1 represents the location of the camera lens, the two dotted lines a and b represent the visual field range, θ represents the camera pitch angle, λ is the angle between the light at point P0 and the optical axis of the monocular camera, and H is the height of the drone from the sea surface. The imaging point of observation point P1 on the image plane is P, the projection point on the optical axis is P2, the projection point on the X-axis is P3, and the imaging point of point P3 on the image plane is P0. The depth information of the target observation point in the camera coordinate system is obtained based on the internal and external parameters of the monocular camera and the perspective projection geometric relationship.

[0054] In the pixel coordinate system, λ can be expressed as

[0055]

[0056] Where v is the ordinate of point P in the pixel coordinate system, which is output by the YOLO v8 target detection network. v0 is the ordinate of the center of the image, and f y is the normalized focal length in the y direction, which is obtained by the intrinsic calibration of the monocular camera.

[0057] The distance O1P3 from the point O1 where the monocular camera lens is located to the projection point P3 of the target observation point on the X-axis can be obtained according to the geometric relationship:

[0058]

[0059] Where H is the height of the UAV from the sea surface, and θ is the camera pitch angle, both of which are output by the UAV IMU.

[0060] According to the geometric relationship, the depth coordinate z of the target observation point in the camera coordinate system c , that is, the distance O1P2 from the point O1 where the monocular camera lens is located to the projection point P2 of the target observation point P1 on the optical axis is:

[0061] z c =O1P2=O1P3×cosλ.

[0062] In another exemplary embodiment of the present application, step S4 specifically includes:

[0063] The three-dimensional coordinates of the target observation point in the coordinate system of the drone are calculated based on the rigid body three-dimensional transformation (i.e., the pinhole camera model), and then the distance between the target observation point and the point projected onto the sea surface by the drone body is calculated. The pinhole camera model is used to calculate the coordinates p(u, v) of the point P in the pixel coordinate system and project them to the three-dimensional coordinates P in the coordinate system of the drone body. s (x s ,y s , z s ), point P is the imaging point of the target observation point P1 on the sea scene image plane, and the solution process is:

[0064]

[0065] Where u0 is the horizontal coordinate of the center of the image, f x is the normalized focal length in the x direction, which is obtained by the camera internal calibration. R is the rotation matrix from the drone body coordinate system to the camera coordinate system, and T is the translation matrix from the drone body coordinate system to the camera coordinate system, which is obtained by the camera external calibration. The remaining parameters are known, and the three-dimensional coordinate P of the observation point projected to the drone body coordinate system is solved linearly using the matrix s (x s ,y s , z s ).

[0066] In another exemplary embodiment of the present application, step S5 specifically includes:

[0067] During the three-dimensional transformation process, in order to simplify the projection relationship, the sea surface where the monocular camera is located is optimized to a standard plane. However, in the real world, the sea surface is closer to a sphere due to the influence of the earth's curvature. Simplifying the sea surface into a plane will produce a large ranging error, and this error will become larger and larger as the distance between the target and the camera increases.

[0068] Reference Figure 3 The figure shows the projection of the plane world coordinate system onto the earth's surface, where G is the center of the earth. The projection point of the observation point P4 on the actual sea surface (spherical surface) on the plane world coordinate system is Figure 2 The target observation point P1 in the mid-plane world coordinate system. The influence of the earth's curvature is taken into account in the above monocular vision ranging algorithm.

[0069] In another exemplary embodiment of the present application, step S6 specifically includes:

[0070] according to Figure 3 According to the geometric relationship in , the distance O2P1 from the projection point O2 of the UAV on the sea surface to the target observation point P1 is:

[0071]

[0072] ∠GO1P4 can be obtained:

[0073]

[0074] According to the triangle sine theorem and triangle cosine theorem, ∠O1GP4 can be obtained:

[0075]

[0076] Among them, O1O2 is the distance from the point O1 where the monocular camera lens is located to the projection point O2 of the drone on the sea surface, GP4 is the distance from the center of the earth G to the actual sea surface observation point P4, O1P4 is the distance from the point O1 where the monocular camera lens is located to the actual sea surface observation point P4, O1G is the distance from the point O1 where the monocular camera lens is located to the center of the earth G, GP4 = r, O1G = O1O2 + O2G = r + H. In the formula, H is the height of the drone from the sea surface, and the radius of the earth r is taken as 6366707m based on experience.

[0077] From the properties of the interior angles of a triangle, we can find ∠O1GP4:

[0078] ∠O1GP4=π-∠GO1P4-∠O1P4G

[0079] Furthermore, from the corresponding relationship between the central angle and the circumference arc, we can get O2P4:

[0080]

[0081] Therefore, the curve distance between the projection point O2 of the drone on the sea surface and the actual sea surface observation point P4 is O2P4.

[0082] The present application also provides an application scenario, which applies the above-mentioned UAV sea surface target ranging method based on a monocular camera. Specifically: The UAV sea surface target ranging method based on a monocular camera provided in this embodiment can be applied in a marine target tracking scenario. The UAV captures the image of the sea scene through the monocular camera carried by the drone, and uses the ranging method provided in the present application to determine the distance of the tracking target, thereby achieving target tracking.

[0083] Based on the same inventive concept, the embodiment of the present application also provides a UAV sea surface target ranging device based on a monocular camera for implementing the UAV sea surface target ranging method based on a monocular camera. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in the embodiments of one or more UAV sea surface target ranging devices based on a monocular camera provided below can refer to the limitations of the UAV sea surface target ranging method based on a monocular camera above, and will not be repeated here.

[0084] In an exemplary embodiment, Figure 4 As shown, a UAV sea surface target ranging device based on a monocular camera is provided, comprising:

[0085] The calibration module 1 is used to calibrate the internal and external parameters of the monocular camera to obtain the internal and external parameters of the monocular camera.

[0086] The target detection module 2 is used to detect the target in the sea scene based on the sea scene image collected by the monocular camera and determine the target observation point using the YOLO v8 target detection network.

[0087] The depth coordinate calculation module 3 is used to calculate the depth coordinate of the target observation point in the camera coordinate system based on the internal and external parameters of the monocular camera and the perspective projection geometric relationship.

[0088] The three-dimensional coordinate calculation module 4 is used to calculate the three-dimensional coordinates of the target observation point in the coordinate system of the drone body based on the depth coordinates and according to the rigid body three-dimensional transformation.

[0089] The actual sea surface observation point determination module 5 is used to project the target observation point in the plane world coordinate system onto the earth's surface to determine the actual sea surface observation point.

[0090] The curve distance calculation module 6 is used to calculate the curve distance between the projection point of the UAV on the sea surface and the actual sea surface observation point based on the three-dimensional coordinates, so as to complete the UAV sea surface target ranging.

[0091] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program. The computer device may be a server or a terminal, and its internal structure diagram may be as shown in FIG. Figure 5 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used for data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for measuring sea surface targets of an unmanned aerial vehicle based on a monocular camera is implemented.

[0092] Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components. In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0093] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0094] In an exemplary embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0095] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0096] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0097] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but is not limited thereto.

[0098] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0099] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A method for measuring sea surface targets by a UAV based on a monocular camera, characterized in that: include: Calibrate the internal and external parameters of the monocular camera to obtain the internal and external parameters of the monocular camera; Based on the sea scene images collected by the monocular camera, the YOLO v8 target detection network is used to detect the targets in the sea scene and determine the target observation points; Based on the internal and external parameters of the monocular camera, the depth coordinates of the target observation point in the camera coordinate system are calculated according to the perspective projection geometric relationship; Based on the depth coordinates, the three-dimensional coordinates of the target observation point in the coordinate system of the drone are calculated according to the rigid body three-dimensional transformation; Project the target observation point in the plane world coordinate system onto the earth's surface to determine the actual sea surface observation point; The curve distance between the projection point of the UAV on the sea surface and the actual sea surface observation point is calculated based on the three-dimensional coordinates to complete the UAV sea surface target ranging.

2. The method for measuring sea surface targets by using a monocular camera on an unmanned aerial vehicle according to claim 1, characterized in that: The internal and external parameters of the monocular camera include a monocular camera internal parameter matrix and a monocular camera external parameter matrix; Calibrate the internal and external parameters of the monocular camera to obtain the internal and external parameters of the monocular camera, including: Using the calibration chessboard, the monocular camera intrinsic parameters are calibrated by Zhang Zhengyou calibration method to obtain the intrinsic parameter matrix; Using the IMU data output by the drone, the monocular camera extrinsic parameters are calibrated using the Rodriguez formula to obtain an extrinsic parameter matrix; the extrinsic parameter matrix includes a rotation matrix and a translation matrix.

3. The method for measuring sea surface targets by using a UAV based on a monocular camera according to claim 1, characterized in that: Based on the sea scene image collected by the monocular camera, the YOLO v8 target detection network is used to detect the target in the sea scene and determine the target observation point, including: The sea scene image captured by the monocular camera is input into the YOLO v8 target detection network to obtain the anchor frame for target detection; Calculate the mean coordinates of the pixels in the anchor box; The coordinate mean is used as the coordinate of the target observation point to determine the target observation point.

4. The method for measuring sea surface targets by using a monocular camera on an unmanned aerial vehicle according to claim 1, wherein: The calculation formula of the depth coordinate of the target observation point in the camera coordinate system is: from c =O1P2=O1P3×cosλ Among them, z c is the depth coordinate of the target observation point in the camera coordinate system, O1P2 is the distance from the point O1 where the monocular camera lens is located to the projection point P2 of the target observation point P1 on the optical axis, O1P3 is the distance from the point O1 where the monocular camera lens is located to the projection point P3 of the target observation point on the X-axis, λ is the angle between the light at point P0 and the optical axis of the monocular camera, point P0 is the imaging point of point P3 on the image plane, H is the height of the drone from the sea surface, θ is the pitch angle of the monocular camera, v is the ordinate of point P in the pixel coordinate system, point P is the imaging point of the target observation point P1 on the image plane of the sea scene, v0 is the ordinate of the image center, and f y is the normalized focal length in the y direction.

5. The method for measuring sea surface targets by using a monocular camera on an unmanned aerial vehicle according to claim 4, characterized in that: The calculation formula of the three-dimensional coordinates of the target observation point in the coordinate system of the drone is: Among them, (u, v) is the coordinate of point P in the pixel coordinate system, u0 is the horizontal coordinate of the center of the image, and f x is the normalized focal length in the x direction, R is the rotation matrix from the drone's body coordinate system to the camera coordinate system, and T is the translation matrix from the drone's body coordinate system to the camera coordinate system. s ,y s , z s ) is the three-dimensional coordinate of the target observation point in the coordinate system of the drone.

6. The method for measuring sea surface targets by using a monocular camera on an unmanned aerial vehicle according to claim 5, characterized in that: The calculation formula for the curvilinear distance between the projection point of the UAV on the sea surface and the actual sea surface observation point is: ∠O1GP4=π-∠GO1P4-∠O1P4G Among them, O2P4 is the curve distance between the projection point O2 of the UAV on the sea surface and the actual sea surface observation point P4, G is the center of the earth, O2P1 is the distance between the projection point O2 of the UAV on the sea surface and the target observation point P1, O1O2 is the distance between the point O1 where the monocular camera lens is located and the projection point O2 of the UAV on the sea surface, x s is the horizontal coordinate of the target observation point in the coordinate system of the drone, y s is the ordinate of the target observation point in the coordinate system of the drone, GP4 is the distance from the center of the earth G to the actual sea surface observation point p4, O1G is the distance from the point O1 where the monocular camera lens is located to the center of the earth G, GP4=r, O1G=O1O2+O2G=r+H, r is the radius of the earth, H is the height of the drone from the sea surface, O1p4 is the distance from the point O1 where the monocular camera lens is located to the actual sea surface observation point P4.

7. A UAV sea surface target ranging device based on a monocular camera, characterized in that: include: The calibration module is used to calibrate the internal and external parameters of the monocular camera to obtain the internal and external parameters of the monocular camera; The target detection module is used to detect targets in the sea scene based on the sea scene image collected by the monocular camera using the YOLO v8 target detection network and determine the target observation point; A depth coordinate calculation module, used to calculate the depth coordinate of the target observation point in the camera coordinate system according to the perspective projection geometric relationship based on the internal and external parameters of the monocular camera; A three-dimensional coordinate calculation module, used to calculate the three-dimensional coordinates of the target observation point in the coordinate system of the drone body based on the depth coordinates and according to the three-dimensional transformation of the rigid body; The actual sea surface observation point determination module is used to project the target observation point in the plane world coordinate system onto the earth surface to determine the actual sea surface observation point; The curve distance calculation module is used to calculate the curve distance between the projection point of the UAV on the sea surface and the actual sea surface observation point based on the three-dimensional coordinates, so as to complete the UAV sea surface target ranging.

8. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the monocular camera-based UAV sea surface target ranging method described in any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for measuring sea surface targets on an unmanned aerial vehicle based on a monocular camera according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method for measuring sea surface targets on an unmanned aerial vehicle based on a monocular camera according to any one of claims 1 to 6 are implemented.

Citation Information

Cited By

  • Unmanned aerial vehicle landing guidance method based on monocular depth estimation network

    CN120333445A

  • Monocular human body distance measurement method and system based on AI depth and segmentation model

    CN120339296A