Target point positioning method and device, electronic equipment and storage medium

Abstract

The application relates to the computer technical field and discloses a target point positioning method and device, electronic equipment and a storage medium. A current position parameter and an attitude parameter of a UAV are acquired; a first pixel coordinate of a first target point in a current image frame is determined, and the first pixel coordinate is converted into a first direction vector in a camera coordinate system; the first direction vector is converted into a second direction vector in a navigation coordinate system according to the attitude parameter; and the intersection point coordinates of the second direction vector and a horizontal plane where the first target point is located are determined according to the position parameter, and the intersection point coordinates are taken as the position coordinates of the first target point. According to the application, the position of the first target point is acquired, the first direction vector is generated, the second direction vector is determined through the attitude parameter, the attitude angle of the UAV is added to the calculation process, and the positioning error is reduced. For any first target point, the position coordinates of the first target point can be calculated, the manual intervention factor is reduced, and automatic real-time calculation is realized.

Description

Technical Field

[0001] This application relates to the field of computer technology, specifically to methods, devices, electronic devices, and storage media for locating target points. Background Technology

[0002] Drone video surveillance and target localization technology has broad application prospects in security patrol, emergency rescue, and geographic surveying. Among related technologies, one method for obtaining the latitude and longitude of ground target points using drones involves operators manually clicking on the target in the drone's live feed. Ground station software then calculates the target coordinates (e.g., the latitude and longitude of the target point) based on the position and attitude information from the drone's on-screen display (OSD) data, combined with a simple perspective projection model. However, this technology relies on manual intervention and cannot achieve automated real-time calculation. Furthermore, some solutions use pinhole camera models, assuming the camera's optical axis is vertically downward or ignoring the influence of drone attitude angles (especially roll and pitch angles), which is only suitable for near-vertical shooting scenarios. When the drone operates at large tilt angles (such as oblique shooting or flying around), the positioning error increases dramatically because the influence of yaw, pitch, and roll attitude angles on the line of sight is not fully considered. Summary of the Invention

[0003] This application provides a method, apparatus, electronic device, and storage medium for locating a target point, in order to solve the problems of inability to achieve automated real-time calculation and large positioning errors in related technologies.

[0004] Firstly, this application provides a method for locating a target point, including: Obtain the current position and attitude parameters of the drone; Determine the first pixel coordinates of the first target point in the current image frame, and transform the first pixel coordinates into a first direction vector in the camera coordinate system; the first direction vector is the vector from the camera optical center to the corresponding position of the first pixel coordinates on the imaging plane; Based on the attitude parameters, the first direction vector is converted into a second direction vector in the navigation coordinate system; The coordinates of the intersection point between the second direction vector and the horizontal plane where the first target point is located are determined based on the position parameters, and the coordinates of the intersection point are used as the position coordinates of the first target point.

[0005] The target point localization method provided in this embodiment determines the position of the first target point in the current image frame by obtaining its pixel coordinates. It then generates a first direction vector in the camera coordinate system, ensuring that this vector, originating from the optical center, points to the first target point in the current image frame. By processing the first direction vector using the UAV's attitude parameters and converting it into a second direction vector, the method transforms the camera-based first direction vector into a second direction vector pointing from the UAV to the target point. This incorporates the influence of the UAV's yaw, pitch, and roll attitude angles on the line-of-sight direction into the calculation process, reducing localization errors for the first target point. Furthermore, this method can calculate the position coordinates of any first target point in the current image frame, reducing human intervention and achieving automated real-time calculation.

[0006] In some optional implementations, the first direction vector is converted into a second direction vector in the navigation coordinate system based on attitude parameters, including: Based on the attitude parameters, construct the rotation matrix of the navigation coordinate system; The first direction vector is transformed using the navigation coordinate system rotation matrix to determine the second direction vector in the navigation coordinate system.

[0007] In some alternative implementations, after converting the first direction vector into a second direction vector in the navigation coordinate system, the target point localization method further includes: Determine the vertical component corresponding to the second direction vector; When the vertical component is oriented towards the ground, the subsequent step of determining the coordinates of the intersection point of the second direction vector and the horizontal plane where the first target point is located is performed based on the position parameters.

[0008] In some optional implementations, the first pixel coordinates are transformed into a first direction vector in the camera coordinate system, including: Obtain the current physical focal length, digital zoom magnification, and original pixel size of the camera on the drone; Determine the equivalent pixel size based on the digital zoom ratio and the original pixel size; Determine the pixel difference between the coordinates of the first pixel and the coordinates of the center pixel in the current image frame, and convert the pixel difference into offset coordinates in the image coordinate system according to the equivalent pixel size; Based on the offset coordinates and the current physical focal length, determine the first direction vector in the camera coordinate system; the component of the optical axis in the first direction vector is the current physical focal length.

[0009] In some optional implementations, determining the coordinates of the intersection point between the second direction vector and the horizontal plane containing the first target point based on position parameters includes: Based on the height parameter in the position parameters, determine the height difference between the horizontal plane where the first target point is located; Determine the vertical component corresponding to the second direction vector, and use the ratio of the height difference to the vertical component as the propagation parameter; The product of the first horizontal component corresponding to the second direction vector and the propagation parameter is used as the first horizontal offset of the first target point relative to the UAV, and the product of the second horizontal component corresponding to the second direction vector and the propagation parameter is used as the second horizontal offset of the first target point relative to the UAV. Add the first horizontal parameter and the first horizontal offset from the position parameters, and add the second horizontal parameter and the second horizontal offset from the position parameters to determine the coordinates of the intersection point between the second direction vector and the horizontal plane where the first target point is located.

[0010] In some optional implementations, the target point localization method further includes: The horizontal distance between the UAV and the first target point is determined based on the first horizontal offset and the second horizontal offset. If the horizontal distance is greater than a preset threshold, the height difference is corrected according to the Earth's radius, and the coordinates of the intersection point of the second direction vector and the horizontal plane where the first target point is located are recalculated based on the corrected height difference.

[0011] In some optional implementations, the target point localization method further includes: Obtain the position coordinates of the second target point; If the second target point is within the current image frame, a third direction vector from the UAV to the second target point in the navigation coordinate system is determined based on the position parameters and attitude parameters, and the position coordinates of the second target point. The third direction vector is converted into a fourth direction vector in the camera coordinate system. The second pixel coordinates of the second target point in the current image frame are determined based on the fourth direction vector, and the second pixel coordinates of the second target point in the current image frame are marked.

[0012] The target point localization method provided in this embodiment determines the position of the first target point in the current image frame by obtaining its pixel coordinates and generating a first direction vector in the camera coordinate system. This first direction vector, originating from the optical center, points to the first target point in the current image frame. By processing the first direction vector using the UAV's attitude parameters and converting it into a second direction vector, the method transforms the first direction vector based on the camera coordinate system into a second direction vector pointing from the UAV to the target point. This incorporates the influence of the UAV's yaw, pitch, and roll attitude angles on the line-of-sight direction into the calculation process, reducing localization errors for the first target point. Furthermore, this method can calculate the position coordinates of any first target point in the current image frame, reducing human intervention and enabling automated real-time calculation. By calculating the equivalent pixel size and constructing the first direction vector based on it, the method can handle digital zoom scenes when the camera has digital zoom capabilities, reducing localization errors during zoom. For distant target points, by correcting for altitude differences, the method fully considers the influence of the Earth's curvature on target point localization, enhancing the accuracy of target point localization.

[0013] Secondly, this application provides a target point positioning device, comprising: The parameter acquisition module is used to acquire the current position and attitude parameters of the UAV.

[0014] The first vector module is used to determine the first pixel coordinates of the first target point in the current image frame and convert the first pixel coordinates into a first direction vector in the camera coordinate system. The first direction vector is a vector from the camera optical center to the corresponding position of the first pixel coordinates on the imaging plane.

[0015] The second vector module is used to convert the first direction vector into a second direction vector in the navigation coordinate system based on the attitude parameters.

[0016] The coordinate determination module is used to determine the coordinates of the intersection point between the second direction vector and the horizontal plane where the first target point is located, based on the position parameters, and uses the intersection point coordinates as the position coordinates of the first target point.

[0017] Thirdly, this application provides an electronic device, including: a memory and a processor, which are communicatively connected to each other. The memory stores computer instructions, and the processor executes the computer instructions to perform the target point positioning method described in the first aspect or any corresponding embodiment.

[0018] Fourthly, this application provides a computer-readable storage medium storing computer instructions for causing a computer to perform the target point positioning method described in the first aspect or any corresponding embodiment.

[0019] Fifthly, this application provides a computer program product, including computer instructions for causing a computer to execute the target point positioning method described in the first aspect or any corresponding embodiment. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the specific embodiments of this application or the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0021] Figure 1 This is a schematic diagram illustrating an application scenario according to an embodiment of this application; Figure 2 This is a schematic flowchart of a first method for locating a target point according to an embodiment of this application; Figure 3 This is a schematic diagram of camera imaging according to an embodiment of this application; Figure 4 This is a schematic diagram of the third method for locating target points according to an embodiment of this application; Figure 5 This is a structural block diagram of a target point positioning device according to an embodiment of this application; Figure 6 This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0023] It is understood that before using the technical solutions disclosed in the various embodiments of this application, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this application in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0024] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0025] As one optional application scenario in the embodiments of this application, such as Figure 1 As shown, application 101 is installed in terminal device 110, and user 130 can interact with application 101 through terminal device 110 and / or access device of terminal device 110.

[0026] For example, application 101 can be arbitrary and can display the current image frame of the drone. Figure 1 In the application scenario shown, if application 101 is active, the terminal device 110 can display the interface 102 of application 101. The interface 102 may include various pages that application 101 can provide.

[0027] In some embodiments, terminal device 110 is communicatively connected to server 120 to provide services to application 101. Terminal device 110 may be a mobile terminal, fixed terminal, or portable terminal, etc., including but not limited to mobile phones, desktop computers, laptop computers, multimedia tablets, or any combination thereof, including accessories and peripherals of these devices or any combination thereof. In some embodiments, terminal device 110 may also support any type of interface, and server 120 may be various types of computing systems or servers capable of providing computing power, including but not limited to mainframes, edge computing nodes, computing devices in cloud environments, etc.

[0028] It should be noted that, Figure 1 This is merely an example of an application scenario and does not limit the scope of protection of this application.

[0029] The embodiments of this application will be described below with reference to the accompanying drawings. It should be understood that the pages shown in the drawings are merely examples, and various page designs are possible in practice. The various graphic elements on the page may have different arrangements and different visual representations, one or more elements may be omitted or replaced, and one or more other elements may also be present; no limitations are imposed on the embodiments of this application. Furthermore, the embodiments are primarily described below with reference to terminal device 110. It should be understood that the actions described relative to terminal device 110 can be performed by application 101 on terminal device 110, or can be performed by application 101 in conjunction with its server (e.g., server 120).

[0030] This application provides a target point localization method. By obtaining the pixel coordinates of the first target point in the current image frame, the position of the first target point in the current image frame is determined, and a first direction vector in the camera coordinate system is generated. The first direction vector is then processed and converted into a second direction vector by the attitude parameters of the UAV. This method can transform the first direction vector based on the camera coordinate system into a second direction vector pointing from the UAV to the target point. The influence of the UAV's yaw, pitch, and roll attitude angles on the line of sight direction is incorporated into the calculation process, reducing the localization error of the first target point.

[0031] According to an embodiment of this application, a method for locating a target point is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0032] This embodiment provides a target point localization method, which can be used in the aforementioned terminal devices, such as desktop computers and laptops, or can be executed by drones. Figure 2 This is a flowchart of a target point localization method according to an embodiment of this application, such as... Figure 2 As shown, the process includes the following steps: Step S201: Obtain the current position and attitude parameters of the UAV.

[0033] The drone is equipped with a camera for capturing images. When acquiring the current image frame, it can simultaneously acquire the drone's position and attitude parameters at the time the image frame was captured. The position parameters can include a first horizontal parameter, a second horizontal parameter, and an altitude parameter. For example, the first horizontal parameter could be the drone's current longitude, the second horizontal parameter could be the drone's current latitude, and the altitude parameter could be the drone's current altitude. The attitude parameters can include the drone's yaw angle, pitch angle, and roll angle.

[0034] Step S202: Determine the first pixel coordinates of the first target point in the current image frame, and convert the first pixel coordinates into a first direction vector in the camera coordinate system. The first direction vector is a vector from the camera optical center to the corresponding position of the first pixel coordinates on the imaging plane.

[0035] In determining the first pixel coordinates of the first target point, the position of the first target point in the current video frame can be obtained first. The first target point can be a location point of interest to the user. The user can select the first target point through the current frame image. For example, the first target point can be selected by the user through the mouse or touch screen in the current frame image, or it can be automatically identified by the icon detection algorithm. No restrictions are imposed in this embodiment.

[0036] After obtaining the position of the first target point in the current video frame, the corresponding first pixel coordinates (u, v) in the current image frame can be determined. In this embodiment, the origin of the pixel coordinate system in the current image frame can be at the upper left corner of the current image frame, where the u-axis can be horizontal to the right and the v-axis can be vertically downward. After determining the first pixel coordinates, a first direction vector in the camera coordinate system can be constructed based on the first pixel coordinates, where the first direction vector is the vector from the camera optical center to the corresponding position of the first pixel coordinates on the imaging plane.

[0037] Figure 3 This is a diagram illustrating camera imaging, such as... Figure 3 As shown, there is an optical center O in the camera, and the distance OO' between the camera and the imaging plane is the focal length of the camera. For the first target point P, there is a pixel P' in the frame image that corresponds to the target point P. At this time, OP' is the first direction vector.

[0038] Step S203: Based on the attitude parameters, convert the first direction vector into a second direction vector in the navigation coordinate system.

[0039] Since the drone's attitude is not always parallel to the ground and is constantly changing, after obtaining the first direction vector, it is necessary to transform the first direction vector by referring to the drone's attitude parameters. In this embodiment, after determining the first direction vector, it can be processed according to the drone's attitude parameters to transform the first direction vector from the camera coordinate system into a second direction vector in the navigation coordinate system. The second direction vector is the vector from the drone toward the target point. In this embodiment, the navigation coordinate system can be a North-East-Down (NED) coordinate system, which includes a north axis pointing north, an east axis pointing east, and a ground axis pointing to the ground. It can also be an East-North-Up (ENU) coordinate system, which will not be elaborated here.

[0040] By processing the first direction vector using the drone's attitude parameters and converting it into a second direction vector, the first direction vector based on the camera coordinate system can be transformed into a second direction vector pointing from the drone to the target point. This incorporates the influence of the drone's yaw, pitch, and roll attitude angles on the line-of-sight direction into the calculation process, thereby reducing positioning errors.

[0041] Step S204: Determine the coordinates of the intersection point between the second direction vector and the horizontal plane where the first target point is located based on the position parameters, and use the intersection point coordinates as the position coordinates of the first target point.

[0042] After determining the second direction vector, the intersection point of the second direction vector (originating from the UAV) and the horizontal plane where the first target point is located can be determined based on the UAV's position parameters. The coordinates of this intersection point are then determined, which are the position coordinates of the first target point. In this embodiment, the height of the horizontal plane where the first target point is located can be taken as the altitude of the UAV's takeoff point, or it can be obtained through a Digital Elevation Model (DEM).

[0043] The target point localization method provided in this embodiment determines the position of the first target point in the current image frame by obtaining its pixel coordinates. It then generates a first direction vector in the camera coordinate system, ensuring that this vector, originating from the optical center, points to the first target point in the current image frame. By processing the first direction vector using the UAV's attitude parameters and converting it into a second direction vector, the method transforms the camera-based first direction vector into a second direction vector pointing from the UAV to the target point. This incorporates the influence of the UAV's yaw, pitch, and roll attitude angles on the line-of-sight direction into the calculation process, reducing localization errors for the first target point. Furthermore, this method can calculate the position coordinates of any first target point in the current image frame, reducing human intervention and achieving automated real-time calculation.

[0044] This embodiment provides a method for locating a target point, which can be used in the aforementioned terminal devices, such as desktop computers and laptop computers. Figure 4 This is a flowchart of a target point localization method according to an embodiment of this application, such as... Figure 4 As shown, the process includes the following steps: Step S401: Obtain the current position and attitude parameters of the UAV.

[0045] Please see details Figure 2 Step S201 of the illustrated embodiment will not be described again here.

[0046] Step S402: Determine the first pixel coordinates of the first target point in the current image frame, and convert the first pixel coordinates into a first direction vector in the camera coordinate system. The first direction vector is a vector from the camera optical center to the corresponding position of the first pixel coordinates on the imaging plane.

[0047] Please see details Figure 2 Step S202 of the illustrated embodiment will not be described again here.

[0048] In some optional implementations, step S202, "converting the first pixel coordinates into a first direction vector in the camera coordinate system", includes steps a1 to a4.

[0049] Step a1: Obtain the current physical focal length, digital zoom magnification, and original pixel size of the camera on the drone.

[0050] Step a2: Determine the equivalent pixel size based on the digital zoom magnification and the original pixel size.

[0051] Step a3: Determine the pixel difference between the coordinates of the first pixel and the coordinates of the center pixel in the current image frame, and convert the pixel difference into offset coordinates in the image coordinate system according to the equivalent pixel size.

[0052] Step a4: Determine the first direction vector in the camera coordinate system based on the offset coordinates and the current physical focal length. The component of the optical axis in the first direction vector is the current physical focal length.

[0053] When converting the first pixel coordinates into the first direction vector in the camera coordinate system, we can first obtain the current physical focal length, digital zoom factor, and original pixel size of the camera mounted on the drone when capturing the current image frame. The current physical focal length is the optical focal length value of the camera lens, that is, the actual focal length under the current optical zoom state. It should be noted that the current physical focal length in this step does not change with digital zoom. The digital zoom factor represents the digital magnification factor of the current image relative to the original sensor image. When the digital zoom factor is 1, it means there is no digital zoom; when the digital zoom factor is greater than 1, it means the image has been digitally cropped and enlarged. The original pixel size is the physical size of a single pixel in the horizontal and vertical directions of the camera sensor. Under normal circumstances, the physical size of a single pixel in the horizontal and vertical directions is the same.

[0054] Since digital zoom involves cropping the original sensor image and enlarging it to its original resolution, this process is equivalent to increasing the sensor area corresponding to a single pixel. Therefore, the equivalent pixel size in the current frame image can be determined using digital zoom and the original pixel size. The equivalent pixel size in the horizontal direction is also relevant. Equivalent cell size in the vertical direction ;in, The original cell size in the horizontal direction. is the original pixel size in the vertical direction, and z is the digital zoom factor.

[0055] After determining the equivalent pixel size, the pixel difference between the coordinates of the first pixel and the coordinates of the center pixel in the current image frame can be determined, and the pixel difference can be converted into offset coordinates in the image coordinate system by the equivalent pixel size. The offset coordinates in the image coordinate system include the horizontal coordinate and the vertical coordinate.

[0056] ; ; in, The x-coordinate is the offset in the image coordinate system. y is the offset ordinate in the image coordinate system; u is the x-coordinate of the first pixel, v is the ordinate of the first pixel; W is the resolution of the current frame image in the horizontal direction, W / 2 is the x-coordinate of the center pixel, H is the resolution of the current frame image in the vertical direction, H / 2 is the ordinate of the center pixel. This represents the pixel difference between the coordinates of the first pixel and the coordinates of the center pixel along the horizontal axis. This represents the pixel difference between the coordinates of the first pixel and the coordinates of the center pixel along the vertical axis.

[0057] After determining the offset coordinates in the image coordinate system, the first direction vector in the camera coordinate system can be constructed. Specifically, the horizontal coordinate in the offset coordinates can be used as the x-axis component of the first direction vector, the vertical coordinate as the y-axis component, and the current physical focal length as the z-axis component. The x-axis points to the right of the camera, the y-axis points downwards from the camera, and the z-axis points forward along the optical axis. That is, for the first direction vector... ,have , where f is the current physical focal length. In some optional implementations, the first direction vector can be normalized to a unit vector, which will not be elaborated here.

[0058] In some alternative implementations, since the camera coordinate system and the body coordinate system are not always unified—for example, in the body coordinate system, the X-axis points forward of the camera head, the Y-axis points to the right side of the body, and the Z-axis points downward of the body—while the camera is typically mounted with its Z-axis (optical axis) aligned with the X-axis (head direction), its X-axis (right) aligned with the Y-axis (right), and its Y-axis (downward) aligned with the Z-axis (downward)—a transformation can be performed on the first direction vector to make it consistent with the corresponding first direction vector in the body coordinate system. For example, a body coordinate rotation matrix can be used. Transform the first direction vector: ; By calculating the equivalent pixel size and constructing a first direction vector based on the equivalent pixel size, it is possible to process digital zoom scenes when the camera has digital zoom capabilities, thereby reducing positioning errors in zoom mode.

[0059] Step S403: Based on the attitude parameters, convert the first direction vector into a second direction vector in the navigation coordinate system.

[0060] Specifically, step S403, "converting the first direction vector into a second direction vector in the navigation coordinate system according to the attitude parameters," includes steps S4031 and S4032.

[0061] Step S4031: Construct the navigation coordinate system rotation matrix based on the attitude parameters.

[0062] Step S4032: Transform the first direction vector using the navigation coordinate system rotation matrix to determine the second direction vector in the navigation coordinate system.

[0063] When transforming the first direction vector, a navigation coordinate system rotation matrix can be constructed. This navigation coordinate system rotation matrix is ​​the rotation matrix from the camera coordinate system to the navigation coordinate system. If the camera coordinate system and the body coordinate system are not the same, the rotation matrix from the camera coordinate system to the body coordinate system can be determined, and then the rotation matrix from the body coordinate system to the navigation coordinate system can be determined. Combining the two rotation matrices, the final rotation matrix from the camera coordinate system to the navigation coordinate system can be obtained.

[0064] In this embodiment, the rotation matrix from the body coordinate system to the navigation coordinate system It can be: ; in, The yaw angle is a parameter in the attitude parameters of the UAV. The pitch angle is one of the attitude parameters of the UAV. The roll angle is a parameter in the attitude parameters of the UAV.

[0065] After determining the rotation matrix of the navigation coordinate system, the second direction vector in the navigation coordinate system can be determined. ,in ,and ; in, This is the first direction vector in the camera coordinate system; The first horizontal component is the first component corresponding to the second direction vector. When the navigation coordinate system is the northeast-northeast coordinate system, the first horizontal component can be the northward component. The second horizontal component corresponds to the second direction vector. When the navigation coordinate system is the northeast-northeast coordinate system, the second horizontal component can be the eastward component. The vertical component is the component corresponding to the second direction vector. When the navigation coordinate system is the northeast-northeast coordinate system, the vertical component can be the ground component.

[0066] In some optional implementations, after step S403, the target point localization method further includes steps b1 and b2.

[0067] Step b1: Determine the vertical component corresponding to the second direction vector.

[0068] Step b2, with the vertical component pointing towards the ground, proceed to the next step of determining the coordinates of the intersection point between the second direction vector and the horizontal plane where the first target point is located, based on the position parameters.

[0069] After determining the second direction vector in the navigation coordinate system, the orientation of the vertical component of the second direction vector can be determined to see if it intersects with the horizontal plane where the first target point is located. If the vertical component is positive (e.g., in the case of a northeast-northeast coordinate system, the vertical component is the ground direction component), the sign of the vertical component can be used for judgment. If the vertical component is greater than 0, it points towards the ground, indicating that the second direction vector intersects with the horizontal plane where the first target point is located. In this case, step S404, which determines the coordinates of the intersection point between the second direction vector and the horizontal plane of the first target point based on the position parameters, can be continued. If the vertical component is not greater than 0, it does not point towards the ground, meaning the direction of the second direction vector is parallel to the ground or outwards from the Earth. In this case, the second direction vector does not intersect with the horizontal plane where the first target point is located, and the current positioning of the first target point can be stopped.

[0070] Step S404: Determine the coordinates of the intersection point between the second direction vector and the horizontal plane where the first target point is located based on the position parameters, and use the intersection point coordinates as the position coordinates of the first target point.

[0071] Please see details Figure 2 Step S204 of the illustrated embodiment will not be described again here.

[0072] In some optional implementations, step S404, "determining the coordinates of the intersection point of the second direction vector and the horizontal plane where the first target point is located based on the position parameters," includes steps c1 to c4.

[0073] Step c1: Determine the height difference between the location and the horizontal plane where the first target point is located, based on the height parameter in the position parameters.

[0074] Step c2: Determine the vertical component corresponding to the second direction vector, and use the ratio of the height difference to the vertical component as the propagation parameter.

[0075] Step c3: The product of the first horizontal component corresponding to the second direction vector and the propagation parameter is taken as the first horizontal offset of the first target point relative to the UAV, and the product of the second horizontal component corresponding to the second direction vector and the propagation parameter is taken as the second horizontal offset of the first target point relative to the UAV.

[0076] Step c4: Add the first horizontal parameter and the first horizontal offset in the position parameters, and add the second horizontal parameter and the second horizontal offset in the position parameters to determine the coordinates of the intersection point of the second direction vector and the horizontal plane where the first target point is located.

[0077] By using the drone's position parameters, the current altitude of the drone can be determined through the altitude parameter within the position parameters. The difference between this altitude and the altitude at the horizontal plane where the first target point is located can be used to determine the altitude difference.

[0078] After determining the height difference, the ratio of the height difference to the vertical component can be used as the propagation parameter. The propagation parameter t can then be calculated as follows: ; in, For height difference, This is the vertical component corresponding to the second direction vector.

[0079] After determining the propagation parameter t, the product of the first horizontal component corresponding to the second direction vector and the propagation parameter can be used as the first horizontal offset of the first target point relative to the UAV. The product of the second horizontal component corresponding to the second direction vector and the propagation parameter is taken as the second horizontal offset of the first target point relative to the UAV. ,Right now: ; ; in, The first horizontal component corresponding to the second direction vector. This is the second horizontal component corresponding to the second direction vector.

[0080] After determining the first horizontal offset and the second horizontal offset, the first horizontal parameter in the position parameters can be added to the first horizontal offset, and the second horizontal parameter in the position parameters can be added to the second horizontal offset, based on the current position parameters of the UAV, to determine the coordinates of the intersection point between the second direction vector and the horizontal plane where the first target point is located.

[0081] When the intersection coordinates are represented as latitude and longitude, the first and second horizontal offsets can be converted into longitude and latitude offsets, respectively. Then, a longitude offset is added to the UAV's current longitude, and a latitude offset is added to the UAV's current latitude to determine the intersection's latitude and longitude. For example, for the latitude offset... and longitude offset It can be calculated in the following ways: ; ; in, The first horizontal offset, when the intersection coordinates are expressed as latitude and longitude, is the northward offset. The second horizontal offset, when the intersection coordinates are expressed in latitude and longitude, is the eastward offset. This is the first horizontal parameter of the drone, i.e., the longitude parameter of the drone; The radius is the Earth's radius.

[0082] In determining latitude offset and longitude offset Then, the latitude and longitude of the first target point can be determined: ; ; in, The longitude of the first target point The latitude of the first target point; This is the second horizontal parameter of the drone, namely the latitude parameter of the drone.

[0083] In some optional implementations, the target point localization method further includes steps d1 and d2.

[0084] Step d1: Determine the horizontal distance between the UAV and the first target point based on the first horizontal offset and the second horizontal offset.

[0085] Step d2: If the horizontal distance is greater than a preset threshold, the height difference is corrected according to the Earth's radius, and the coordinates of the intersection point of the second direction vector and the horizontal plane where the first target point is located are recalculated based on the corrected height difference.

[0086] Because of the Earth's curvature, errors may occur in determining the position of a distant target point. In this case, the horizontal distance between the drone and the target point can be calculated based on the first and second horizontal offsets. If the horizontal distance is greater than a preset threshold (e.g., 1 km), the altitude difference can be corrected based on the Earth's radius. ; in, This is the corrected height difference. This is the original height difference. Let be the Earth's radius, and d be the horizontal distance between the UAV and the first target point. .

[0087] After determining the corrected height difference, the coordinates of the intersection point between the second direction vector and the horizontal plane where the first target point is located can be recalculated using the corrected height difference.

[0088] For distant target points, the accuracy is enhanced by correcting for the height difference, taking into full account the impact of the Earth's curvature on target point positioning.

[0089] In some optional implementations, the target point localization method further includes steps e1 to e3.

[0090] Step e1: Obtain the position coordinates of the second target point.

[0091] Step e2: If the second target point is within the current image frame, determine the third direction vector from the UAV to the second target point in the navigation coordinate system based on the position parameters and attitude parameters and the position coordinates of the second target point.

[0092] Step e3: Convert the third direction vector into the fourth direction vector in the camera coordinate system, determine the second pixel coordinates of the second target point in the current image frame based on the fourth direction vector, and mark the second pixel coordinates in the current image frame.

[0093] This application also supports reverse lookup, meaning that for any actual target point (i.e., the second target point) on the map or terrain, if the second target point is within the current image frame, it can be marked in the current image frame. First, the position coordinates of the second target point can be obtained, such as latitude and longitude. Using the position coordinates of the second target point, along with the position and attitude parameters of the UAV, a third direction vector is constructed. This third direction vector is the direction vector pointing from the UAV to the second target point in the navigation coordinate system. After determining the third direction vector, it can be converted into a fourth direction vector in the camera coordinate system. This fourth direction vector is the vector from the camera's optical center to the pixel coordinates. Based on the fourth direction vector, the coordinates of the second pixel corresponding to the second target point in the current image frame can be determined. At this point, the second pixel in the current image frame can be marked, for example, by highlighting or selecting the corresponding second pixel.

[0094] The target point localization method provided in this embodiment determines the position of the first target point in the current image frame by obtaining its pixel coordinates and generating a first direction vector in the camera coordinate system. This first direction vector, originating from the optical center, points to the first target point in the current image frame. By processing the first direction vector using the UAV's attitude parameters and converting it into a second direction vector, the method transforms the first direction vector based on the camera coordinate system into a second direction vector pointing from the UAV to the target point. This incorporates the influence of the UAV's yaw, pitch, and roll attitude angles on the line-of-sight direction into the calculation process, reducing localization errors for the first target point. Furthermore, this method can calculate the position coordinates of any first target point in the current image frame, reducing human intervention and enabling automated real-time calculation. By calculating the equivalent pixel size and constructing the first direction vector based on it, the method can handle digital zoom scenes when the camera has digital zoom capabilities, reducing localization errors during zoom. For distant target points, by correcting for altitude differences, the method fully considers the influence of the Earth's curvature on target point localization, enhancing the accuracy of target point localization.

[0095] This embodiment also provides a target point positioning device, which is used to implement the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0096] This embodiment provides a target point positioning device, such as... Figure 5 As shown, it includes: The parameter acquisition module 501 is used to acquire the current position and attitude parameters of the UAV.

[0097] The first vector module 502 is used to determine the first pixel coordinates of the first target point in the current image frame, and convert the first pixel coordinates into a first direction vector in the camera coordinate system. The first direction vector is a vector from the camera optical center to the corresponding position of the first pixel coordinates on the imaging plane.

[0098] The second vector module 503 is used to convert the first direction vector into a second direction vector in the navigation coordinate system based on the attitude parameters.

[0099] The coordinate determination module 504 is used to determine the coordinates of the intersection point of the second direction vector and the horizontal plane where the first target point is located based on the position parameters, and to use the intersection point coordinates as the position coordinates of the first target point.

[0100] In some alternative implementations, the second vector module 503 includes: The navigation matrix submodule is used to construct the rotation matrix of the navigation coordinate system based on the attitude parameters.

[0101] The matrix rotation submodule is used to transform the first direction vector using a rotation matrix in the navigation coordinate system to determine the second direction vector in the navigation coordinate system.

[0102] In some optional implementations, the target point positioning device further includes: The vertical component module is used to determine the vertical component corresponding to the second direction vector.

[0103] The judgment module is used to perform the subsequent step of determining the coordinates of the intersection point of the second direction vector and the horizontal plane where the first target point is located, based on the position parameters, when the vertical component is oriented towards the ground.

[0104] In some alternative implementations, the first vector module 502 includes: The drone data submodule is used to obtain the current physical focal length, digital zoom magnification, and original pixel size of the camera on the drone.

[0105] The equivalent pixel submodule is used to determine the equivalent pixel size based on the digital zoom magnification and the original pixel size.

[0106] The offset coordinate submodule is used to determine the pixel difference between the coordinates of the first pixel and the coordinates of the center pixel in the current image frame, and convert the pixel difference into offset coordinates in the image coordinate system according to the equivalent pixel size.

[0107] The first vector submodule is used to determine the first direction vector in the camera coordinate system based on the offset coordinates and the current physical focal length. The component of the optical axis in the first direction vector is the current physical focal length.

[0108] In some alternative implementations, the coordinate determination module 504 includes: The height difference submodule is used to determine the height difference between the location and the horizontal plane where the first target point is located, based on the height parameter in the location parameters.

[0109] The propagation parameter submodule is used to determine the vertical component corresponding to the second direction vector, and uses the ratio of the height difference to the vertical component as the propagation parameter.

[0110] The horizontal offset submodule is used to take the product of the first horizontal component corresponding to the second direction vector and the propagation parameter as the first horizontal offset of the first target point relative to the UAV, and to take the product of the second horizontal component corresponding to the second direction vector and the propagation parameter as the second horizontal offset of the first target point relative to the UAV.

[0111] The coordinate determination submodule is used to add the first horizontal parameter and the first horizontal offset in the position parameters, and add the second horizontal parameter and the second horizontal offset in the position parameters to determine the coordinates of the intersection point of the second direction vector and the horizontal plane where the first target point is located.

[0112] In some optional implementations, the target point positioning device further includes: The horizontal distance module is used to determine the horizontal distance between the UAV and the first target point based on the first horizontal offset and the second horizontal offset.

[0113] The height difference correction module is used to correct the height difference based on the Earth's radius when the horizontal distance is greater than a preset threshold, and to recalculate the coordinates of the intersection point of the second direction vector and the horizontal plane where the first target point is located based on the corrected height difference.

[0114] In some optional implementations, the target point positioning device further includes: The second coordinate module is used to obtain the position coordinates of the second target point.

[0115] The third vector module is used to determine, based on the position parameters and attitude parameters, the third direction vector pointing from the UAV to the second target point in the navigation coordinate system, according to the position coordinates of the second target point, when the second target point is within the current image frame.

[0116] The marking module is used to convert the third direction vector into a fourth direction vector in the camera coordinate system, determine the second pixel coordinates of the second target point in the current image frame based on the fourth direction vector, and mark the second pixel coordinates in the current image frame.

[0117] The target point positioning device provided in this application can execute the target point positioning method provided in any embodiment of this application, and has the corresponding functional modules and beneficial effects for executing the method. Further functional descriptions of the above modules and units are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0118] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0119] The following is a detailed reference. Figure 6This diagram illustrates a suitable structural schematic for implementing the electronic device described in the embodiments of this application. The electronic device may include a processor (e.g., a central processing unit, graphics processor, etc.) 601, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 602 or a program loaded from memory 608 into random access memory (RAM) 603. The RAM 603 also stores various programs and data required for the operation of the electronic device. The processor 601, ROM 602, and RAM 603 are interconnected via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0120] Typically, the following devices can be connected to I / O interface 605: input devices 606 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 607 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; memory devices 608 including, for example, magnetic tapes, hard disks, etc.; and communication devices 609. Communication device 609 allows electronic devices to communicate wirelessly or wiredly with other devices to exchange data. Although... Figure 6 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown, and more or fewer devices may be implemented or have instead.

[0121] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 609, or installed from a memory 608, or installed from a ROM 602. When the computer program is executed by the processor 601, it performs the functions defined in the target point localization method of embodiments of this application.

[0122] Figure 6 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0123] This application also provides a computer-readable storage medium. The methods described in this application can be implemented in hardware or firmware, or implemented as recordable on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc. Further, the storage medium may also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code. When the software or computer code is accessed and executed by the computer, processor, or hardware, the target point localization method shown in the above embodiments is implemented.

[0124] A portion of this application can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to this application through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions, or the computer compiling the instructions and then executing the corresponding compiled program, or the computer reading and executing the instructions, or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0125] Although embodiments of this application have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of this application, and all such modifications and variations fall within the scope defined by the appended claims.

Claims

1. A method for locating a target point, characterized in that, The method includes: Obtain the current position and attitude parameters of the drone; Determine the first pixel coordinates of the first target point in the current image frame, and convert the first pixel coordinates into a first direction vector in the camera coordinate system; the first direction vector is a vector from the camera optical center to the corresponding position of the first pixel coordinates on the imaging plane; Based on the attitude parameters, the first direction vector is converted into a second direction vector in the navigation coordinate system; The coordinates of the intersection point between the second direction vector and the horizontal plane where the first target point is located are determined based on the position parameters, and the coordinates of the intersection point are used as the position coordinates of the first target point.

2. The method according to claim 1, characterized in that, The step of converting the first direction vector into a second direction vector in the navigation coordinate system based on the attitude parameters includes: Based on the attitude parameters, construct the navigation coordinate system rotation matrix; The first direction vector is transformed using the navigation coordinate system rotation matrix to determine the second direction vector in the navigation coordinate system.

3. The method according to claim 1, characterized in that, After converting the first direction vector into a second direction vector in the navigation coordinate system, the method further includes: Determine the vertical component corresponding to the second direction vector; If the vertical component is oriented towards the ground, the subsequent step of determining the coordinates of the intersection point of the second direction vector and the horizontal plane where the first target point is located is performed based on the position parameters.

4. The method according to claim 1, characterized in that, The step of converting the first pixel coordinates into a first direction vector in the camera coordinate system includes: Obtain the current physical focal length, digital zoom magnification, and original pixel size of the camera on the UAV; The equivalent pixel size is determined based on the digital zoom ratio and the original pixel size; Determine the pixel difference between the first pixel coordinates and the center pixel coordinates in the current image frame, and convert the pixel difference into offset coordinates in the image coordinate system according to the equivalent pixel size; The first direction vector in the camera coordinate system is determined based on the offset coordinates and the current physical focal length; the component of the optical axis direction in the first direction vector is the current physical focal length.

5. The method according to claim 1, characterized in that, Determining the coordinates of the intersection point between the second direction vector and the horizontal plane where the first target point is located based on the position parameters includes: Based on the height parameter in the position parameters, determine the height difference between the position and the horizontal plane where the first target point is located; Determine the vertical component corresponding to the second direction vector, and use the ratio of the height difference to the vertical component as the propagation parameter; The product of the first horizontal component corresponding to the second direction vector and the propagation parameter is used as the first horizontal offset of the first target point relative to the UAV, and the product of the second horizontal component corresponding to the second direction vector and the propagation parameter is used as the second horizontal offset of the first target point relative to the UAV. Add the first horizontal parameter in the position parameters to the first horizontal offset, and add the second horizontal parameter in the position parameters to the second horizontal offset to determine the coordinates of the intersection point between the second direction vector and the horizontal plane where the first target point is located.

6. The method according to claim 5, characterized in that, The method further includes: The horizontal distance between the UAV and the first target point is determined based on the first horizontal offset and the second horizontal offset; If the horizontal distance is greater than a preset threshold, the height difference is corrected according to the Earth's radius, and the coordinates of the intersection point of the second direction vector and the horizontal plane where the first target point is located are recalculated based on the corrected height difference.

7. The method according to claim 1, characterized in that, The method further includes: Obtain the position coordinates of the second target point; When the second target point is within the current image frame, a third direction vector from the UAV to the second target point in the navigation coordinate system is determined based on the position parameters and the attitude parameters, according to the position coordinates of the second target point. The third direction vector is converted into a fourth direction vector in the camera coordinate system. The second pixel coordinates of the second target point in the current image frame are determined based on the fourth direction vector, and the second pixel coordinates of the second target point in the current image frame are marked.

8. A target point positioning device, characterized in that, The device includes: The parameter acquisition module is used to acquire the current position and attitude parameters of the UAV. The first vector module is used to determine the first pixel coordinates of the first target point in the current image frame and convert the first pixel coordinates into a first direction vector in the camera coordinate system; the first direction vector is a vector from the camera optical center to the corresponding position of the first pixel coordinates on the imaging plane. The second vector module is used to convert the first direction vector into a second direction vector in the navigation coordinate system according to the attitude parameters. The coordinate determination module is used to determine the coordinates of the intersection point between the second direction vector and the horizontal plane where the first target point is located, based on the position parameters, and to use the intersection point coordinates as the position coordinates of the first target point.

9. An electronic device, characterized in that, include: A memory and a processor are communicatively connected, the memory stores computer instructions, and the processor executes the computer instructions to perform the target point localization method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to perform the target point localization method according to any one of claims 1 to 7.

Images