Method, device, drone and medium for determining target distance based on airborne camera
By using the target recognition model and camera parameters to calculate the distance between the UAV camera and the target, the problem of inaccurate UAV positioning in the existing technology is solved, and rapid and accurate positioning of the target is achieved.
Patent Information
- Application Number
- CN202411587373.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-08
- Publication Date
- 2025-01-21
- Estimated Expiration
- 2044-11-08
AI Technical Summary
The existing technology of positioning target objects based on images is not mature and accurate, making it difficult for drones to position target objects.
By obtaining the target image captured by the camera, the target recognition model is used to detect the target and obtain the detection frame, and the physical distance between the target and the camera is calculated by combining the camera parameters and transformation matrix.
It realizes the rapid positioning of target objects by drones and improves the accuracy and efficiency of positioning.
Smart Images

Figure CN119124090B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of unmanned aerial vehicles, and in particular to a method, a device, a unmanned aerial vehicle and a medium for determining a target distance based on an airborne camera. Background Art
[0002] Drones are highly maneuverable and flexible to operate. They can effectively patrol in a variety of environments to obtain various information. They are often used in tasks such as reconnaissance, surveillance, and target positioning.
[0003] Generally, target detection on a drone is achieved by carrying a gimbal camera. This part will find the target to be followed in the image obtained by the camera and mark it with a rectangular frame (or other form).
[0004] However, the existing image-based target positioning technology is not mature and accurate, making it difficult for drones to locate targets. Summary of the invention
[0005] In view of the above problems, the present invention is proposed to provide a method and system for determining target distance based on an airborne camera, which overcomes the above problems or at least partially solves the above problems.
[0006] According to one aspect of the present invention, a method for determining target distance based on an airborne camera is provided, the method comprising:
[0007] Acquire a target image captured by a camera, detect the target image using a target recognition model, and obtain a detection frame of the target;
[0008] Based on the pixel coordinates of the four vertices of the detection frame, combined with the camera parameters and the transformation matrix, determine the angles of the four vertices relative to the camera;
[0009] A standard height of a target is set, and a physical distance between the target and the camera is calculated based on the vertex and the angle.
[0010] In some implementations, obtaining a target image captured by a camera, detecting the target image using a target recognition model, and obtaining a detection frame of the target include:
[0011] A vertical coordinate system is established with the optical center of the camera as the origin, the straight line parallel to the horizontal plane as the X-axis, and the straight line perpendicular to the horizontal plane as the Y-axis.
[0012] In some implementations, the target recognition model is a YOLO series recognition model, and the target includes a human body. Then, obtaining a target image captured by a camera, detecting the target image using the target recognition model, and obtaining a detection frame of the target includes:
[0013] The trained YOLO series recognition model is used to identify the human body in the target image and obtain a detection frame that just frames the entire human body.
[0014] In some embodiments, the camera is mounted on a drone, and based on the pixel coordinates of the four vertices of the detection frame, combined with the camera parameters and the transformation matrix, determining the angles of the four vertices relative to the camera includes:
[0015] According to the pixel coordinates of the four vertices of the detection frame, the elevation angles of the lines between each vertex and the optical center of the camera are calculated respectively through camera parameters and matrix conversion, and the maximum elevation angle of the four elevation angles is used as the top elevation angle of the target.
[0016] In some implementations, setting a standard height of a target and calculating a physical distance between the target and the camera based on the vertex and the angle comprises:
[0017] Obtain the pixel coordinates of the center point of the detection frame, and determine the elevation angle of the line between the center point and the optical center of the camera as the center elevation angle after matrix conversion;
[0018] The horizontal distance between the target and the camera is determined based on the center elevation angle, the standard height of the target, and the top elevation angle.
[0019] In some embodiments, determining the horizontal distance between the target and the camera based on the center elevation angle, the standard height of the target, and the top elevation angle includes:
[0020] Assuming that the horizontal distance is x, half of the standard height of the target is L, the top elevation angle is α, the center elevation angle is β, a is the tangent value of the top elevation angle, and b is the tangent value of the center elevation angle, then:
[0021] x = L / (a -b).
[0022] In some implementations, setting a standard height of a target, and calculating a physical distance between the target and the camera based on the vertex and the angle further comprises:
[0023] The physical distance between the camera and the target can be expressed as the distance D between the optical center of the camera and the center point of the target, then:
[0024] D= .
[0025] According to another aspect of the present invention, a device for determining target distance based on an airborne camera is provided, the device comprising:
[0026] The target detection module is adapted to obtain a target image captured by a camera, detect the target image using a target recognition model, and obtain a detection frame of the target;
[0027] An angle determination module, adapted to determine the angles of the four vertices relative to the camera based on the pixel coordinates of the four vertices of the detection frame in combination with the camera parameters and the transformation matrix;
[0028] The distance calculation module is adapted to set a standard height of a target and calculate a physical distance between the target and the camera based on the vertex and the angle.
[0029] According to another aspect of the present invention, there is provided an unmanned aerial vehicle, comprising: a processor and a memory arranged to store computer executable instructions, wherein when the executable instructions are executed, the processor executes the target distance determination method based on an airborne camera according to any one of the above-mentioned embodiments.
[0030] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores one or more programs, and when the one or more programs are executed by a processor, the method for determining the target distance based on an airborne camera according to any of the above items is implemented.
[0031] From the above, it can be seen that according to the target distance determination scheme based on the airborne camera disclosed in the present invention, the distance between the camera and the target can be obtained only through the target image taken by the camera, and then the position of the target object can be obtained through the position of the intelligent body such as the drone, thereby realizing the rapid positioning of the target object.
[0032] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS
[0033] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:
[0034] Figure 1 A schematic diagram of the process of a method for determining target distance based on an airborne camera according to an embodiment of the present invention is shown;
[0035] Figure 2 A schematic diagram showing the position relationship between a target and a camera according to an embodiment of the present invention;
[0036] Figure 3A schematic diagram showing the structure of a target distance determination device based on an airborne camera according to an embodiment of the present invention
[0037] Figure 4 A schematic structural diagram of a drone according to an embodiment of the present invention is shown. DETAILED DESCRIPTION
[0038] The exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present invention and to enable the scope of the present invention to be fully communicated to those skilled in the art.
[0039] The overall concept of the embodiment of the present invention is to achieve visual positioning of the target object. First, assuming that the height of a person is known, the position of the target object relative to the drone is calculated by detecting the size of the detection frame of the recognition model.
[0040] Figure 1 The following is a flow chart of a method for determining target distance based on an airborne camera according to an embodiment of the present invention. Optionally, the method can be implemented by a program in a control system of an intelligent body such as a drone. The method specifically includes:
[0041] Step S110, obtaining a target image captured by a camera, detecting the target image using a target recognition model, and obtaining a detection frame of the target;
[0042] Step S120, based on the pixel coordinates of the four vertices of the detection frame, combined with the camera parameters and the transformation matrix, specifically, the camera's internal parameters, external parameters and the posture transformation matrix of the intelligent body such as the drone can be combined to determine the angles of the four vertices relative to the camera; wherein, the pixel coordinates of the vertex can be converted to obtain the position of the target object in the vertical coordinate system by using the conversion formula from the pixel coordinate system or the image coordinate system to the vertical coordinate system, and then the angle between the above vertex and the camera optical center can be obtained; wherein the vertical coordinate system is a coordinate system established with the camera optical center as the origin, the straight line parallel to the horizontal plane as the X-axis, and the straight line perpendicular to the horizontal plane as the Y-axis;
[0043] Step S130, setting a standard height of the target, and calculating the physical distance between the target and the camera based on the vertex and the angle; wherein the standard height of the target can be set in a variety of ways according to the type of the target, and can also be set in a graded manner for each type of target, for example, a person's height can be divided into the height of men, women, and children, wherein the height of a man can be calibrated to 1.75m, and the height of a woman can be calibrated to 1.6m, and no specific restrictions are made here; and after knowing the standard height of the target, the distance between the camera and the target can be obtained based on the standard height, the relative position of the vertex and the angle, etc.
[0044] According to an embodiment of the present invention, the distance between the camera and the target can be obtained only through the target image taken by the camera, and then the position of the target object can be obtained through the position of an intelligent body such as a drone, thereby achieving rapid positioning of the target object.
[0045] In some embodiments, obtaining a target image captured by a camera, detecting the target image using a target recognition model, and obtaining a detection frame of the target include:
[0046] A vertical coordinate system is established with the optical center of the camera as the origin, the straight line parallel to the horizontal plane as the X-axis, and the straight line perpendicular to the horizontal plane as the Y-axis.
[0047] according to Figure 2 As shown, the bottom horizontal line is the ground, point O is the location of the camera, and a vertical coordinate system is established with point O as the center. Among them, L is half of the vertical target height (set standard height). The end point of the first vector at the top is the top of the target, with coordinates [x, ax], and the end point of the second vector is the center point of the target with coordinates [x, bx], where a and b are the tangent values of the top elevation angle and the center elevation angle, respectively. The angle between the first vector and the x-axis is called the top elevation angle α, and the angle between the second vector and the x-axis is called the center elevation angle β. Both elevation angles are negative values, so the top elevation angle is greater than the center elevation angle.
[0048] In some embodiments, the target recognition model is a YOLO series recognition model, and the target includes a human body. Then, a target image captured by a camera is acquired, and the target image is detected using the target recognition model. Obtaining a detection frame of the target includes:
[0049] The trained YOLO series recognition model is used to identify the human body in the target image and obtain a detection frame that just frames the entire human body.
[0050] The large model YOLO (You Only Look Once) is an advanced deep learning target detection algorithm first proposed by Joseph Redmon et al. in 2015. YOLO is mainly used in the field of target detection (a subfield of machine vision). It can identify and locate objects in an image with a single look. It has the advantages of fast speed, high accuracy, strong interpretability and wide applicability. It is one of the most important representatives in the current field of target detection.
[0051] It should be noted that the main detection object of the embodiment of the present invention is a target in a vertical state, such as a person. The camera shoots the target from different angles, and the image size of the target is different. The image position and size are related to the shape of the target, the relative position of the camera and the target, and the relative posture. Knowing the relative posture and the image position and size, the relative position of the target and the camera can be obtained.
[0052] If the target is placed horizontally, the roll angle and pitch angle in the relative attitude can be obtained by the flight control, but the relative yaw angle cannot be obtained.
[0053] Therefore, in order to decouple the imaging size from the relative yaw angle, it is assumed that the target is a vertical line segment of known length. The imaging length is only related to the actual length of the line segment, the shooting distance, the shooting pitch angle, and the shooting roll angle. Since the camera's shooting pitch angle and roll angle can be obtained through the attitude angle of the flight control, the distance between the camera and the vertical column can be calculated based on the mathematical relationship between the projection length of the vertical line segment and the actual length of the vertical column.
[0054] A standing human body is like a pillar that is approximately perpendicular to the ground. Assuming that the height of the person is known, the distance between the camera and the person can be calculated.
[0055] It should be emphasized that the target of the present invention is not limited to the human body, and vertically elongated objects similar to the human body are all within the scope of protection.
[0056] In some embodiments, the camera is mounted on a drone, and based on the pixel coordinates of the four vertices of the detection frame, combined with the camera parameters and the transformation matrix, determining the angles of the four vertices relative to the camera includes:
[0057] According to the pixel coordinates of the four vertices of the detection frame, the elevation angles of the lines between each vertex and the optical center of the camera are calculated respectively through camera parameters and matrix conversion, and the maximum elevation angle of the four elevation angles is used as the top elevation angle of the target.
[0058] In some embodiments, setting a standard height of a target and calculating a physical distance between the target and the camera based on the vertex and the angle comprises:
[0059] Obtain the pixel coordinates of the center point of the detection frame, and determine the elevation angle of the line between the center point and the optical center of the camera as the center elevation angle after matrix conversion;
[0060] The horizontal distance between the target and the camera is determined based on the center elevation angle, the standard height of the target, and the top elevation angle.
[0061] In some embodiments, determining the horizontal distance between the target and the camera based on the center elevation angle, the standard height of the target, and the top elevation angle comprises:
[0062] Assuming that the horizontal distance is x, half of the standard height of the target is L, the top elevation angle is α, the center elevation angle is β, a is the tangent value of the top elevation angle, and b is the tangent value of the center elevation angle, then:
[0063] x = L / (a -b).
[0064] In some embodiments, setting a standard height of a target, and calculating a physical distance between the target and the camera based on the vertex and the angle further comprises:
[0065] The physical distance between the camera and the target can be expressed as the distance D between the optical center of the camera and the center point of the target, then:
[0066] D= .
[0067] According to the above embodiment, combined with Figure 2 As shown in the figure, the four vertices of the Bounding box detection frame identified by the YOLO model have four pixel coordinates. The pixel coordinates refer to the coordinates in the pixel coordinate system of the target image. The pixel coordinates are combined with the camera intrinsic parameters, the camera installation angle, the transformation matrix, etc. to calculate its coordinates in the vertical coordinate system. Then, according to the positional relationship between them and the camera origin, the four sight elevation angles in the vertical coordinate system are calculated. The four sight elevation angles are sorted, and the maximum elevation angle obtained is the top elevation angle α.
[0068] The elevation angle obtained by combining the pixel at the center of the bounding box detection box with the internal posture is the center elevation angle β.
[0069] a is the tangent of the top elevation angle, a = tan(α);
[0070] b is the tangent of the central elevation angle, b = tan(β);
[0071] From the figure, we can see that ax-bx = L;
[0072] So, x = L / (a -b);
[0073] So the second vector (from Figure 2 The length of the second vector from top to bottom is: ;
[0074] The target distance is: D = .
[0075] According to another aspect of the present invention, in combination Figure 3 As shown, a target distance determination device 300 based on an airborne camera is provided, and the device 300 includes:
[0076] The target detection module 310 is adapted to obtain a target image captured by a camera, detect the target image using a target recognition model, and obtain a detection frame of the target;
[0077] An angle determination module 320, adapted to determine the angles of the four vertices relative to the camera based on the pixel coordinates of the four vertices of the detection frame in combination with the camera parameters and the transformation matrix;
[0078] The distance calculation module 330 is adapted to set a standard height of the target and calculate the physical distance between the target and the camera based on the vertex and the angle.
[0079] According to an embodiment of the present invention, the distance between the camera and the target can be obtained only through the target image taken by the camera, and then the position of the target object can be obtained through the position of an intelligent body such as a drone, thereby achieving rapid positioning of the target object.
[0080] In some embodiments, the target detection module 310 obtains a target image captured by a camera, detects the target image using a target recognition model, and obtains a detection frame of the target, including:
[0081] A vertical coordinate system is established with the optical center of the camera as the origin, the straight line parallel to the horizontal plane as the X-axis, and the straight line perpendicular to the horizontal plane as the Y-axis.
[0082] In some embodiments, the target recognition model is a YOLO series recognition model, and the target includes a human body. Then, the target detection module 310 obtains a target image captured by a camera, and uses the target recognition model to detect the target image. Obtaining a detection frame of the target includes:
[0083] The trained YOLO series recognition model is used to identify the human body in the target image and obtain a detection frame that just frames the entire human body.
[0084] It should be noted that the main detection object of the embodiment of the present invention is a target in a vertical state, such as a person. The camera shoots the target from different angles, and the image size of the target is different. The image position and size are related to the shape of the target, the relative position of the camera and the target, and the relative posture. Knowing the relative posture and the image position and size, the relative position of the target and the camera can be obtained.
[0085] If the target is placed horizontally, the roll angle and pitch angle in the relative attitude can be obtained by the flight control, but the relative yaw angle cannot be obtained.
[0086] Therefore, in order to decouple the imaging size from the relative yaw angle, it is assumed that the target is a vertical line segment of known length. The imaging length is only related to the actual length of the line segment, the shooting distance, the shooting pitch angle, and the shooting roll angle. Since the camera's shooting pitch angle and roll angle can be obtained through the attitude angle of the flight control, the distance between the camera and the vertical column can be calculated based on the mathematical relationship between the projection length of the vertical line segment and the actual length of the vertical column.
[0087] A standing human body is like a pillar that is approximately perpendicular to the ground. Assuming that the height of the person is known, the distance between the camera and the person can be calculated.
[0088] In some embodiments, the camera is mounted on a drone, and the height of the camera is greater than the maximum height of the target. Then, the angle determination module 320 determines the angles of the four vertices relative to the camera based on the pixel coordinates of the four vertices of the detection frame, in combination with the camera parameters and the transformation matrix, including:
[0089] According to the pixel coordinates of the four vertices of the detection frame, the elevation angles of the lines between each vertex and the optical center of the camera are calculated respectively through camera parameters and matrix conversion, and the maximum elevation angle of the four elevation angles is used as the top elevation angle of the target.
[0090] In some embodiments, setting a standard height of the target in the distance calculation module 330 and calculating the physical distance between the target and the camera based on the vertex and the angle includes:
[0091] Obtain the pixel coordinates of the center point of the detection frame, and determine the elevation angle of the line between the center point and the optical center of the camera as the center elevation angle after matrix conversion;
[0092] The horizontal distance between the target and the camera is determined based on the center elevation angle, the standard height of the target, and the top elevation angle.
[0093] In some embodiments, determining the horizontal distance between the target and the camera based on the center elevation angle, the standard height of the target, and the top elevation angle comprises:
[0094] Assuming that the horizontal distance is x, half of the standard height of the target is L, the top elevation angle is α, the center elevation angle is β, a is the tangent value of the top elevation angle, and b is the tangent value of the center elevation angle, then:
[0095] x = L / (a -b).
[0096] In some embodiments, the distance calculation module 330 sets a standard height of the target, and calculates the physical distance between the target and the camera based on the vertex and the angle further includes:
[0097] The physical distance between the camera and the target can be expressed as the distance D between the optical center of the camera and the center point of the target, then:
[0098] D= .
[0099] In summary, the embodiments of the present invention can achieve the following technical effects:
[0100] By setting the standard height of the target object and combining the conversion relationship between the image coordinate system and the vertical coordinate system, the vertical coordinates of the camera are obtained according to the pixel coordinates of the target object, thereby determining the angle between the target object and the camera, and determining the position of the target relative to the drone according to the standard height.
[0101] The solution of the embodiment of the present invention can obtain the distance between the camera and the target only through the target image taken by the camera, and then the position of the target object can be obtained through the position of an intelligent body such as a drone, thereby realizing rapid positioning of the target object.
[0102] It should be noted that:
[0103] The algorithm and display provided herein are not inherently related to any particular computer, virtual device or other equipment. Various general-purpose devices can also be used together with the teachings based on this. According to the above description, it is obvious to construct the structure required for this type of device. In addition, the present invention is not directed to any specific programming language either. It should be understood that various programming languages can be utilized to realize the content of the present invention described herein, and the description of the above specific language is for the purpose of disclosing the best mode of the present invention.
[0104] In the description provided herein, a large number of specific details are described. However, it is understood that embodiments of the present invention can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.
[0105] Similarly, it should be understood that in order to streamline the present invention and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of the present invention, various features of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof. However, this method of disclosure should not be interpreted as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim.
[0106] Those skilled in the art will appreciate that the modules in the devices in the embodiments may be adaptively changed and arranged in one or more devices different from the embodiments. The modules or units or components in the embodiments may be combined into one module or unit or component, and further may be divided into a plurality of submodules or subunits or subcomponents. All features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device so disclosed may be combined in any combination, except that at least some of such features and / or processes or units are mutually exclusive. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) may be replaced by an alternative feature that provides the same, equivalent or similar purpose.
[0107] Furthermore, those skilled in the art will appreciate that although some embodiments described herein include certain features included in other embodiments but not other features, the combination of features from different embodiments is meant to be within the scope of the present invention and to form different embodiments.
[0108] The various component embodiments of the present invention may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. It should be understood by those skilled in the art that a microprocessor or a digital signal processor (DSP) may be used in practice to implement some or all of the functions of some or all of the components in the target distance determination device based on an airborne camera according to an embodiment of the present invention. The present invention may also be implemented as a device or device program (e.g., a computer program and a computer program product) for executing part or all of the methods described herein. Such a program for implementing the present invention may be stored on a computer-readable medium, or may be in the form of one or more signals. Such a signal may be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.
[0109] An embodiment of the present invention provides a non-volatile computer storage medium, wherein the computer storage medium stores at least one executable instruction, and the computer executable instruction can execute the above-mentioned target distance determination method based on an airborne camera in any of the above-mentioned method embodiments.
[0110] Figure 4 A schematic diagram of the structure of an embodiment of a UAV of the present invention is shown. The specific embodiment of the present invention does not limit the specific structure of the tethered UAV.
[0111] like Figure 4 As shown, the drone includes a flight control system, which further includes: a processor (processor) 402, a communication interface (Communications Interface) 404, a memory (memory) 406, and a communication bus 408.
[0112] The processor 402, the communication interface 404, and the memory 406 communicate with each other via the communication bus 408. The communication interface 404 is used to communicate with other devices such as a client or other server network elements. The processor 402 is used to execute the program 410, which can specifically execute the relevant steps in the above-mentioned embodiment of the target distance determination method based on the airborne camera for tethering the drone.
[0113] Specifically, the program 410 may include program codes, which include computer operation instructions.
[0114] The processor 402 may be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention. The one or more processors included in the tethered drone may be processors of the same type, such as one or more CPUs; or processors of different types, such as one or more CPUs and one or more ASICs.
[0115] The memory 406 is used to store the program 410. The memory 406 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.
[0116] The program 410 may be specifically used to enable the processor 402 to perform operations corresponding to the above-mentioned target distance determination method embodiment based on the airborne camera.
[0117] It should be noted that the above embodiments illustrate the present invention rather than limit it, and that those skilled in the art may devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference symbol between brackets shall not be construed as a limitation on the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "one" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present invention may be implemented by means of hardware comprising a number of different elements and by means of a suitably programmed computer. In a unit claim enumerating a number of devices, several of these devices may be embodied by the same hardware item. The use of the words first, second, and third, etc., does not indicate any order. These words may be interpreted as names.
Claims
1. A method for determining target distance based on an airborne camera, characterized in that: The method comprises: Acquire a target image captured by a camera, detect the target image using a target recognition model, and obtain a detection frame of the target; Based on the pixel coordinates of the four vertices of the detection frame, combined with the camera parameters and the transformation matrix, determine the angles of the four vertices relative to the camera; Setting a standard height of a target, and calculating a physical distance between the target and the camera based on the vertex and the angle; The process of acquiring a target image captured by a camera and detecting the target image using a target recognition model to obtain a detection frame of the target includes: A vertical coordinate system is established with the optical center of the camera as the origin, the straight line parallel to the horizontal plane as the X-axis, and the straight line perpendicular to the horizontal plane as the Y-axis; The camera is mounted on the drone, and based on the pixel coordinates of the four vertices of the detection frame, combined with the camera parameters and the transformation matrix, the angles of the four vertices relative to the camera are determined, including: According to the pixel coordinates of the four vertices of the detection frame, the elevation angles of the lines between each vertex and the optical center of the camera are calculated respectively through camera parameters and matrix conversion, and the maximum elevation angle of the four elevation angles is used as the top elevation angle of the target; Setting a standard height of a target and calculating a physical distance between the target and the camera based on the vertex and the angle comprises: Obtain the pixel coordinates of the center point of the detection frame, and determine the elevation angle of the line between the center point and the optical center of the camera as the center elevation angle after matrix conversion; determining a horizontal distance between the target and the camera based on the center elevation angle, the standard height of the target, and the top elevation angle; Determining the horizontal distance between the target and the camera according to the center elevation angle, the standard height of the target and the top elevation angle comprises: Assuming that the horizontal distance is x, half of the standard height of the target is L, the top elevation angle is α, the center elevation angle is β, a is the tangent value of the top elevation angle, and b is the tangent value of the center elevation angle, then: x = L / (a -b); Setting a standard height of the target and calculating the physical distance between the target and the camera based on the vertex and the angle further includes: The physical distance between the camera and the target can be expressed as the distance D between the optical center of the camera and the center point of the target, then: 。 2. The method according to claim 1, characterized in that The target recognition model is a YOLO series recognition model, and the target includes a human body. Then, a target image captured by a camera is acquired, and the target image is detected using the target recognition model. The target detection frame is obtained by: The trained YOLO series recognition model is used to identify the human body in the target image and obtain a detection frame that just frames the entire human body.
3. A target distance determination device based on an airborne camera, characterized in that: The device comprises: The target detection module is adapted to obtain a target image captured by a camera, detect the target image using a target recognition model, and obtain a detection frame of the target; An angle determination module, adapted to determine the angles of the four vertices relative to the camera based on the pixel coordinates of the four vertices of the detection frame in combination with the camera parameters and the transformation matrix; a distance calculation module, adapted to set a standard height of a target and calculate a physical distance between the target and the camera based on the vertex and the angle; The target detection module obtains the target image captured by the camera, detects the target image using the target recognition model, and obtains the target detection frame, including: A vertical coordinate system is established with the optical center of the camera as the origin, the straight line parallel to the horizontal plane as the X-axis, and the straight line perpendicular to the horizontal plane as the Y-axis; The camera is mounted on a drone, and the height of the camera is greater than the maximum height of the target. The angle determination module determines the angles of the four vertices relative to the camera based on the pixel coordinates of the four vertices of the detection frame, combined with the camera parameters and the transformation matrix, including: According to the pixel coordinates of the four vertices of the detection frame, the elevation angles of the lines between each vertex and the optical center of the camera are calculated respectively through camera parameters and matrix conversion, and the maximum elevation angle of the four elevation angles is used as the top elevation angle of the target; Setting a standard height of the target in the distance calculation module and calculating the physical distance between the target and the camera based on the vertex and the angle includes: Obtain the pixel coordinates of the center point of the detection frame, and determine the elevation angle of the line between the center point and the optical center of the camera as the center elevation angle after matrix conversion; determining a horizontal distance between the target and the camera based on the center elevation angle, the standard height of the target, and the top elevation angle; Determining the horizontal distance between the target and the camera according to the center elevation angle, the standard height of the target and the top elevation angle comprises: Assuming that the horizontal distance is x, half of the standard height of the target is L, the top elevation angle is α, the center elevation angle is β, a is the tangent value of the top elevation angle, and b is the tangent value of the center elevation angle, then: x = L / (a -b); The distance calculation module sets a standard height of the target, and calculates the physical distance between the target and the camera based on the vertex and the angle, further comprising: The physical distance between the camera and the target can be expressed as the distance D between the optical center of the camera and the center point of the target, then: 。 4. A drone, characterized in that: The invention comprises a processor and a memory arranged to store computer executable instructions, wherein when the executable instructions are executed, the processor executes the target distance determination method based on an airborne camera according to any one of claims 1 to 2.
5. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores one or more programs, and when the one or more programs are executed by the processor, the method for determining the target distance based on the airborne camera according to any one of claims 1 to 2 is implemented.
Citation Information
Patent Citations
Monocular visual 3D feature extraction method based on four-rotor unmanned aerial vehicle (UAV)
CN106570820A
Object identification and positioning method based on laser-monocular vision fusion
CN111191625A