Target positioning method, apparatus and electronic device

Abstract

The application relates to the technical field of geographic positioning, and provides a target positioning method, device and electronic equipment. The method comprises the following steps: acquiring a UAV aerial image and flight attitude parameters; determining an offset angle of a target pixel point in an imaging plane according to the aerial image, and determining a unit direction vector of the target pixel point in a world coordinate system according to the offset angle and the flight attitude parameters; determining northward displacement and eastward displacement of the target relative to the UAV in a north-east coordinate system according to the vertical relative height and slant range of the UAV and the target, the unit direction vector, the flight attitude parameters and the offset angle; and positioning the target according to the latitude and longitude of the UAV, the northward displacement and the eastward displacement. The application can improve the accuracy of target positioning when the UAV performs large-angle maneuvering, so as to meet actual positioning requirements.

Description

Technical Field

[0001] This application belongs to the field of geographic positioning technology, and in particular relates to a target positioning method, device and electronic equipment. Background Technology

[0002] In the field of drones, real-time video feeds transmitted by drones are the core source of information. However, simple real-time video feeds are merely pixel stacks without spatial attributes. If a valid target is detected in the feed, its geographic coordinates cannot be directly provided, making it difficult to support subsequent judgment, target guidance, and other operational needs. In related technologies, the localization of valid targets is only highly accurate under ideal conditions where the camera is shooting vertically downwards. When the drone performs large-angle maneuvers, the obtained target position deviates significantly, failing to meet the requirements for high-precision positioning. Summary of the Invention

[0003] In view of the shortcomings of the prior art, this application provides a target positioning method, device and electronic device to solve the problem that the target position obtained by the related technology is greatly deviated when the UAV is performing large-angle maneuvers, which cannot meet the requirements of high-precision positioning.

[0004] Firstly, this application provides a target localization method, including: Acquire aerial images and flight attitude parameters of the UAV; based on the aerial images, determine the offset angle of the corresponding pixel in the imaging plane, and based on the offset angle and flight attitude parameters, determine the unit direction vector of the pixel in the world coordinate system; based on the vertical relative altitude and slant distance between the UAV and the target, the unit direction vector, the flight attitude parameters, and the offset angle, determine the northward and eastward displacement of the target relative to the UAV in the northeast coordinate system; based on the latitude and longitude of the UAV, the northward displacement, and the eastward displacement, locate the target.

[0005] In one embodiment of this application, the flight attitude parameters include pitch angle and roll angle. Based on the vertical relative altitude and slant range between the UAV and the target, the unit direction vector, the flight attitude parameters, and the offset angle, the northward and eastward displacements of the target relative to the UAV in the northeast coordinate system are determined, including: Based on the vertical relative altitude and slant range between the UAV and the target, the unit direction vector, the pitch angle, and the offset angle, the longitudinal and lateral displacements are determined. The longitudinal displacement is the increment of the projection of the target's corresponding pixel on the ground relative to the projection point of the camera's optical axis center in the longitudinal direction, and the lateral displacement is the increment of the projection of the target's corresponding pixel on the ground relative to the projection point of the camera's optical axis center in the lateral direction. Based on the slant range, unit direction vector, roll angle, longitudinal displacement, and lateral displacement, the northward and eastward displacements of the target relative to the UAV in the northeast coordinate system are determined.

[0006] In one embodiment of this application, the unit direction vector includes a first direction vector and a second direction vector, and the offset angle includes a vertical offset angle and a horizontal offset angle; based on the vertical relative altitude and slant range between the UAV and the target, the unit direction vector, the pitch angle, and the offset angle, the longitudinal displacement and lateral displacement are determined, including: Based on the slant distance between the UAV and the target, the first direction vector, and the second direction vector, determine the horizontal distance between the projection of the target's corresponding pixel on the ground and the projection point directly below the UAV; based on the vertical relative height between the UAV and the target, the horizontal distance, the pitch angle, and the vertical offset angle, determine the longitudinal displacement; based on the vertical relative height between the UAV and the target, the pitch angle, the vertical offset angle, and the horizontal offset angle, determine the lateral displacement.

[0007] In one embodiment of this application, the northward and eastward displacements of the target relative to the UAV in the NE coordinate system are determined based on slant range, unit direction vector, roll angle, longitudinal displacement, and lateral displacement, including: Based on the slant range, roll angle, first direction vector, and second direction vector, determine the first reference component and the second reference component; based on the slant range, first direction vector, horizontal distance, first reference component, second reference component, longitudinal displacement, and lateral displacement, determine the northward displacement; based on the slant range, second direction vector, horizontal distance, first reference component, second reference component, longitudinal displacement, and lateral displacement, determine the eastward displacement.

[0008] In one embodiment of this application, the unit direction vector includes a third direction vector; before determining the northward and eastward displacements of the target relative to the UAV in the northeast coordinate system based on the vertical relative altitude and slant distance between the UAV and the target, the unit direction vector, flight attitude parameters, and offset angle, the method further includes: The real-time status of the rangefinder is detected, including both working and non-working states. When the rangefinder is working, the slant distance is obtained using the rangefinder. When the rangefinder is non-working, the slant distance is determined based on the vertical relative height and a third-party vector.

[0009] In one embodiment of this application, determining the offset angle of the pixel corresponding to the target in the imaging plane based on the aerial image includes: Based on the aerial image, determine the pixel coordinates of the pixel corresponding to the target; normalize the pixel coordinates using the resolution of the aerial image to obtain normalized coordinates; determine the offset angle of the pixel corresponding to the target in the imaging plane based on the UAV's field of view and the normalized coordinates.

[0010] In one embodiment of this application, determining the unit direction vector of a pixel in the world coordinate system based on the offset angle and flight attitude parameters includes: Based on the flight attitude parameters, determine the direction cosine matrix of the UAV; based on the offset angle, determine the initial target vector of the pixel in the camera coordinate system; based on the initial target vector and the direction cosine matrix, determine the unit direction vector of the pixel in the world coordinate system.

[0011] In one embodiment of this application, the target is located based on the latitude and longitude, northward displacement, and eastward displacement of the UAV, including: Based on the northward displacement and the target conversion constant, determine the latitude increment; based on the latitude of the UAV and the latitude increment, determine the latitude of the target; based on the eastward displacement, the target conversion constant, and the latitude of the UAV, determine the longitude increment; based on the longitude of the UAV and the longitude increment, determine the longitude of the target, and locate the target based on the latitude and longitude of the target.

[0012] Secondly, this application also provides a target positioning device, comprising: The parameter acquisition module is configured to acquire aerial images and flight attitude parameters of the UAV; the first determination module is configured to determine the offset angle of the corresponding pixel in the imaging plane based on the aerial image, and determine the unit direction vector of the pixel in the world coordinate system based on the offset angle and flight attitude parameters; the second determination module is configured to determine the northward and eastward displacement of the target relative to the UAV in the northeast coordinate system based on the vertical relative height and slant distance between the UAV and the target, the unit direction vector, the flight attitude parameters, and the offset angle; the target positioning module is configured to locate the target based on the latitude and longitude of the UAV, the northward displacement, and the eastward displacement.

[0013] Thirdly, this application also provides an electronic device, which includes: one or more processors; and a storage device for storing one or more programs, which, when executed by one or more processors, cause the electronic device to perform the steps of the target positioning method described above.

[0014] The beneficial effects of this technical solution are as follows: First, it acquires aerial images and flight attitude parameters of the UAV; based on the aerial images, it determines the offset angle of the target's corresponding pixel in the imaging plane; using the offset angle and flight attitude parameters, it determines the unit direction vector of the pixel in the world coordinate system, which can correct image rotation and perspective distortion caused by aircraft maneuvers, ensuring accurate ground projection of the target pixel; then, based on the vertical relative height and slant distance between the UAV and the target, the unit direction vector, flight attitude parameters, and offset angle, it determines the northward and eastward displacement of the target relative to the UAV in the northeast coordinate system; based on the UAV's latitude and longitude, northward displacement, and eastward displacement, it locates the target. The vertical relative height and slant distance between the UAV and the target, the unit direction vector, flight attitude parameters, and offset angle all characterize the spatial relationship between the UAV and the target during large-angle maneuvers. By integrating these parameters, even when the UAV is performing large-angle maneuvers, it can accurately calculate the target's position relative to the UAV in three-dimensional space using multi-source data, improving positioning accuracy and effectively addressing positioning needs in complex flight environments.

[0015] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0016] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort. In the drawings: Figure 1 This is a schematic flowchart illustrating a target localization method according to an exemplary embodiment of this application; Figure 2 This is a schematic diagram of the target positioning device shown in an exemplary embodiment of this application; Figure 3 A schematic diagram of the structure of a computer system suitable for implementing the electronic device of the present application is shown. Detailed Implementation

[0017] The embodiments of this application will be described below with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. This application can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. It should be understood that the preferred embodiments are only for illustrating this application and are not intended to limit the scope of protection of this application.

[0018] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of this application. Therefore, the drawings only show the components related to this application and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the shape, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0019] In the following description, numerous details are explored to provide a more thorough explanation of embodiments of the present application. However, it will be apparent to those skilled in the art that embodiments of the present application may be practiced without these specific details. In other embodiments, well-known structures and devices are shown in block diagram form rather than in detail to avoid obscuring embodiments of the present application.

[0020] Please see Figure 1 , Figure 1 This is a flowchart illustrating a target localization method as shown in an exemplary embodiment of this application. Figure 1 As shown, in an exemplary embodiment, the target localization method includes steps S110 to S140, and each step is described in detail below.

[0021] S110 acquires aerial images and flight attitude parameters of the drone; It is understandable that during flight, drones can capture aerial images in real time using cameras mounted on them, while simultaneously acquiring flight attitude parameters through built-in sensors. These parameters include the drone's roll, pitch, and yaw angles.

[0022] Furthermore, in practical application scenarios, the flight environment and mission requirements of drones may affect the quality of aerial images and flight attitude parameters. Therefore, the acquired aerial images and flight attitude parameters can be preprocessed to remove noise and improve their usability.

[0023] S120: Based on the aerial image, determine the offset angle of the pixel corresponding to the target in the imaging plane, and based on the offset angle and flight attitude parameters, determine the unit direction vector of the pixel in the world coordinate system. It is understandable that the offset angle represents the angle between the line connecting the actual imaging position of the target (i.e., the pixel corresponding to the target) and the center of the imaging plane, relative to the optical axis. The unit direction vector represents the actual geographic spatial orientation of the target from the drone's camera in the world coordinate system.

[0024] In some embodiments, determining the offset angle of the pixel corresponding to the target in the imaging plane based on the aerial image includes: Based on the aerial image, determine the pixel coordinates of the pixel corresponding to the target; normalize the pixel coordinates using the resolution of the aerial image to obtain normalized coordinates; determine the offset angle of the pixel corresponding to the target in the imaging plane based on the UAV's field of view and the normalized coordinates.

[0025] For example, if a suspicious target is detected in the upper right corner of the aerial image, intelligence personnel can click or select the suspicious vehicle on the display device, then capture the current frame of the aerial image and the UAV's full attitude data as raw data. This allows them to further determine the pixel coordinates of the target's corresponding pixel as (1440, 270) and the resolution of the aerial image as (1920, 1080). The pixel coordinates can then be normalized using the resolution of the aerial image according to the normalization formula shown below:

[0026]

[0027] The normalized coordinates (pixeIU, pixelIV) of the corresponding pixel point are obtained as (0.75, 0.25).

[0028] Where pixelIU is the horizontally normalized pixel coordinate (denoted as the first normalized coordinate), and pixelIV is the vertically normalized pixel coordinate (denoted as the second normalized coordinate). It is understandable that the drone's field of view includes a horizontal field of view and a vertical field of view, and the offset angle includes a horizontal offset angle and a vertical offset angle. Based on the drone's field of view and normalized coordinates, the offset angle of the target pixel in the imaging plane is determined, including: Determine the vertical field of view based on the horizontal field of view and the resolution of the aerial image; The horizontal offset angle is determined based on the horizontal field of view and the first normalized coordinates, and the vertical offset angle is determined based on the vertical field of view and the second normalized coordinates.

[0029] For example, based on the horizontal field of view (denoted as...) Based on the resolution of the aerial image, determine the vertical field of view (denoted as ). You can refer to the following method:

[0030] If the horizontal field of view is at this time If the angle is 60°, then the vertical field of view can be calculated to be 33.75°.

[0031] Then, using the horizontal offset angle calculation formula, the horizontal offset angle is determined based on the horizontal field of view and the first normalized coordinate pixelIU. The formula for calculating the horizontal offset angle is as follows:

[0032] in, The horizontal offset angle. For the first normalized coordinates, For horizontal field of view, The first normalization coefficient is used in this embodiment. It can be 1.

[0033] Next, using the vertical offset angle calculation formula, the vertical offset angle is determined based on the vertical field of view and the second normalized coordinate pixel IV. The formula for calculating the vertical offset angle is as follows:

[0034] in, The vertical offset angle. For the second normalized coordinates, For vertical field of view, The second normalization coefficient is used in this embodiment. It can be 1.

[0035] Continuing with the previous example, if the target's normalized coordinates (pixeIU, pixelIV) are (0.75, 0.25), the horizontal field of view angle... Given a vertical field of view of 60° and a vertical field of view of 33.75°, substituting these values ​​into the formula above, the horizontal offset angle can be calculated. The vertical offset angle is 16.1°. The value is 8.6°, indicating that the target is located 16.1° to the right and 8.6° above the camera's optical axis.

[0036] In this way, the two-dimensional pixels (1440, 720) of the target in the aerial image can be transformed into the initial target vector (16.1° to the right, 8.6° up) in the camera coordinate system through algebraic calculations with extremely low computational cost (in microseconds). This improves the calculation speed and provides more accurate data for subsequent steps.

[0037] In some embodiments, determining the unit direction vector of a pixel in the world coordinate system based on the offset angle and flight attitude parameters includes: Based on the flight attitude parameters, determine the direction cosine matrix of the UAV; based on the offset angle, determine the initial target vector of the pixel in the camera coordinate system; based on the initial target vector and the direction cosine matrix, determine the unit direction vector of the pixel in the world coordinate system.

[0038] Continuing the previous example, the flight attitude parameters include the UAV's roll angle, pitch angle, and yaw angle during flight. A 3×3 direction cosine matrix can be constructed based on these parameters. This matrix is ​​then used to compensate the initial target vector in the camera coordinate system to the world coordinate system, correcting image rotation caused by aircraft maneuvers, and obtaining the unit direction vector of the UAV's camera pointing at the target in the world coordinate system. The method for constructing the 3×3 direction cosine matrix based on the roll, pitch, and yaw angles is shown below:

[0039] in, Here is the direction cosine matrix. The matrix is ​​determined based on the roll angle. The matrix is ​​determined based on the pitch angle. This is the matrix determined based on the heading angle.

[0040] In some examples, if the roll angle is 15°, the pitch angle is -45°, and the yaw angle is 17.92°, then:

[0041]

[0042]

[0043] Continuing from the previous example, we obtain the horizontal offset angle. The vertical offset angle is 16.1°. After setting the depth to 8.6°, an initial target vector in the camera coordinate system can be constructed. It is understood that, to simplify the model, this embodiment sets the depth Z=1, at which point the initial target vector is:

[0044] in, This is the initial target vector.

[0045] Then, using the direction cosine matrix, the initial target vector is compensated to the world coordinate system, resulting in the unit direction vector of the UAV camera pointing at the target:

[0046] in, It is a unit direction vector. Let be the first direction vector. The second direction vector, It is a third-direction vector.

[0047] In this way, by using the direction cosine matrix to compensate the initial target vector in the camera coordinate system to the world coordinate system, the image rotation caused by aircraft maneuvers (such as side flight or turning) can be corrected, ensuring that the ground projection point remains accurate.

[0048] S130, based on the vertical relative height and slant distance between the UAV and the target, the unit direction vector, flight attitude parameters and offset angle, determine the northward and eastward displacement of the target relative to the UAV in the northeast coordinate system; It is understandable that the vertical relative height and slant distance between the UAV and the target, the unit direction vector, flight attitude parameters, and offset angle can all characterize the spatial relationship between the UAV and the target when the UAV is performing large-angle maneuvers (such as roll caused by rapid turning or pitch caused by diving). By combining these parameters, the position of the target relative to the UAV in three-dimensional space can be accurately calculated. Even when the UAV is performing large-angle maneuvers, it can maintain positioning accuracy by taking advantage of multi-source computing data and effectively cope with the positioning needs in complex flight environments.

[0049] In some embodiments, the flight attitude parameters include pitch angle and roll angle. Based on the vertical relative altitude and slant range between the UAV and the target, the unit direction vector, the flight attitude parameters, and the offset angle, the northward and eastward displacements of the target relative to the UAV in the northeast coordinate system are determined, including: Based on the vertical relative altitude and slant distance between the UAV and the target, the unit direction vector, the pitch angle, and the offset angle, the longitudinal and lateral displacements are determined. Based on the slant distance, the unit direction vector, the roll angle, the longitudinal displacement, and the lateral displacement, the northward and eastward displacements of the target relative to the UAV in the northeast coordinate system are determined.

[0050] The vertical displacement is the increment of the projection of the target pixel on the ground relative to the projection point of the camera's optical axis center in the vertical direction, and the horizontal displacement is the increment of the projection of the target pixel on the ground relative to the projection point of the camera's optical axis center in the horizontal direction.

[0051] The north distance represents the distance the target is offset from the drone on a horizontal surface along the geographic north direction, while the east distance represents the distance the target is offset from the drone on a horizontal surface along the geographic east direction.

[0052] It is understandable that the longitudinal and lateral displacements are offsets obtained based on the UAV camera coordinate system. They reflect the distance of the target's corresponding pixel's projection on the ground relative to the center projection point of the camera's optical axis in terms of front-back and left-right distances. These are intermediate results from the visual calculation, and their directions change with the UAV's heading and attitude, so they cannot directly correspond to geographical location. They need to be further converted into northward and eastward displacements relative to the UAV in the northeast coordinate system.

[0053] The northward and eastward displacements can represent the north-south and east-west components in the northeast coordinate system after the longitudinal and lateral displacements have been transformed. They are not affected by the orientation of the UAV itself and can truly reflect the relative position of the target in geographic space.

[0054] In some embodiments, the unit direction vector includes a first direction vector and a second direction vector, and the offset angle includes a vertical offset angle and a horizontal offset angle; Based on the vertical relative altitude and slant range between the UAV and the target, the unit direction vector, the pitch angle, and the offset angle, determine the longitudinal and lateral displacements, including: Based on the slant distance between the UAV and the target, the first direction vector, and the second direction vector, determine the horizontal distance between the projection of the target's corresponding pixel on the ground and the projection point directly below the UAV; based on the vertical relative height between the UAV and the target, the horizontal distance, the pitch angle, and the vertical offset angle, determine the longitudinal displacement; based on the vertical relative height between the UAV and the target, the pitch angle, the vertical offset angle, and the horizontal offset angle, determine the lateral displacement.

[0055] In some examples, the horizontal distance calculation formula can be used to determine the horizontal distance between the projection of the target's corresponding pixel on the ground and the projection point directly below the drone, based on the slant distance between the drone and the target, the first direction vector, and the second direction vector. The horizontal distance calculation formula is as follows:

[0056]

[0057]

[0058] in, Horizontal distance Let be the first direction vector. R is the second direction vector, and R is the slant distance. To calculate the first target component, This is the calculated second target component.

[0059] Continuing from the previous example, if the slant distance R is 220 meters, the first direction vector... The value is 0.673, and the second direction vector is... Substituting 0.217 into the above formula, the horizontal distance can be calculated. It is approximately 155.56 meters.

[0060] Then, the longitudinal displacement can be determined using the longitudinal displacement calculation formula, based on the vertical relative height between the UAV and the target, the horizontal distance, the pitch angle, and the vertical offset angle; the longitudinal displacement calculation formula is shown below:

[0061] in, Here, H represents the longitudinal displacement, H represents the vertical relative height, and pitch represents the drone's pitch angle. The vertical offset angle. is the horizontal distance. It represents the angle of the camera's principal optical axis relative to the vertical direction.

[0062] Continuing the previous example, if the vertical relative height H is 155.56 meters, the drone's pitch angle is -45°, and the vertical offset angle is... It is 8.6°, horizontal distance Approximately 155.56 meters. Substituting this into the above formula, the longitudinal displacement is calculated. It is approximately 55.63 meters.

[0063] Additionally, in some examples, the lateral displacement can be determined using a formula based on the vertical relative height between the UAV and the target, the pitch angle, the vertical offset angle, and the horizontal offset angle. The formula for calculating lateral displacement is shown below:

[0064] in, This is a lateral displacement. This is the horizontal offset angle.

[0065] Continuing the previous example, if the vertical relative height H is 155.56 meters, the pitch angle is -45°, and the vertical offset angle is... The horizontal offset angle is 8.6°. It is 16.1°, horizontal distance Approximately 155.56 meters. Substituting this into the above formula, the lateral displacement is calculated. It is approximately 75.72 meters.

[0066] In some embodiments, determining the target's northward and eastward displacements relative to the UAV in the northeast coordinate system based on slant range, unit direction vector, roll angle, longitudinal displacement, and lateral displacement includes: Based on the slant range, roll angle, first direction vector, and second direction vector, determine the first reference component and the second reference component; based on the slant range, first direction vector, horizontal distance, first reference component, second reference component, longitudinal displacement, and lateral displacement, determine the northward displacement; based on the slant range, second direction vector, horizontal distance, first reference component, second reference component, longitudinal displacement, and lateral displacement, determine the eastward displacement.

[0067] First, the camera center point is projected and transformed based on the slant range, roll angle, first direction vector, and second direction vector to obtain the first reference component and the second reference component.

[0068] In some examples, the first and second reference components can be calculated using the following formula:

[0069]

[0070] in, As the first reference component, As the second reference component, The roll angle is given by the formula above, which gives the first target component. The value is 148.02, the second target component. It is 47.87.

[0071] Continuing from the previous example, if the roll angle The angle is 15°, combined with the first target component calculated above. Second target component The first reference component can be calculated. The value is 130.58, the second reference component. It is 84.54.

[0072] Then, using the northward displacement calculation formula, the northward displacement is determined based on the slant distance, the first direction vector, the horizontal distance, the first reference component, the second reference component, the longitudinal displacement, and the lateral displacement; the northward displacement calculation formula is as follows:

[0073] Continuing from the previous example, the first reference component... The value is 130.58, the second reference component. The value is 84.54, the first target component. The longitudinal displacement is 148.02. The lateral displacement is 55.63 meters. It is 75.72 meters, horizontal distance The value is 155.56 meters. Substituting this into the above formula, the northward displacement is approximately 153.56 meters.

[0074] Finally, using the eastward displacement calculation formula, the eastward displacement is determined based on the slant distance, the second direction vector, the horizontal distance, the first reference component, the second reference component, the longitudinal displacement, and the lateral displacement. The eastward displacement calculation formula is shown below:

[0075] Continuing from the previous example, the first reference component... The value is 130.58, the second reference component. The value is 84.54, the second target component. The longitudinal displacement is 47.87. The lateral displacement is 55.63 meters. It is 75.72 meters, horizontal distance The value is 155.56 meters. Substituting this into the above formula, the eastward displacement is approximately 141.66 meters.

[0076] In this way, after converting the longitudinal and lateral displacements, we obtain the northward and eastward displacements in the northeast coordinate system. This ensures that the target position is not affected by the UAV's own orientation, truly reflects the target's relative position in geographic space, and thus achieves precise target positioning, ensuring the accuracy and consistency of the target's geographic coordinates.

[0077] In some embodiments, the unit direction vector includes a third direction vector; before determining the northward and eastward displacements of the target relative to the UAV in the northeast coordinate system based on the vertical relative altitude and slant range between the UAV and the target, the unit direction vector, flight attitude parameters, and offset angle, the method further includes: The real-time status of the rangefinder is detected, including both working and non-working states. When the rangefinder is working, the slant distance is obtained using the rangefinder. When the rangefinder is non-working, the slant distance is determined based on the vertical relative height and a third-party vector.

[0078] It is understandable that in real-world applications, such as drone operations with heavy smoke or dust, the rangefinder may be interfered with and fail to function properly due to the influence of smoke or dust.

[0079] Therefore, this embodiment detects the real-time status of the rangefinder, which includes a working state and a non-working state; when the rangefinder is in a working state, the slant distance between the UAV and the target is directly obtained through the rangefinder.

[0080] Even when the rangefinder is not operational, the slant distance can be determined using the slant distance calculation formula, based on the vertical relative height and a third-party vector. This ensures accurate slant distance between the UAV and the target, even when the rangefinder is unavailable. The slant distance calculation formula is as follows:

[0081] Where R is the slant distance and H is the vertical relative height. It is a third-direction vector.

[0082] Continuing with the previous example, the vertical relative height H is 163 meters, while the third-direction vector... If the value is approximately -0.707, then the slope distance R can be calculated to be approximately 230.5 meters.

[0083] In this way, when the rangefinder is operational, the slant range data it measures can be directly used to improve the efficiency and accuracy of target positioning. Even when the rangefinder is not operational, the slant range between the UAV and the target can still be obtained relatively accurately using the vertical relative height between them and the third-order vector, thereby improving the continuity and reliability of the target positioning process and ensuring the robustness of the system.

[0084] The S140 locates the target based on the UAV's latitude, longitude, northward displacement, and eastward displacement.

[0085] Understandably, once the latitude and longitude, northward displacement, and eastward displacement of the drone are determined, coordinate conversion can be performed to obtain the latitude and longitude of the target. The target's latitude and longitude can then be used to locate the target, thereby obtaining a more accurate positioning result for the target.

[0086] In some embodiments, the target is located based on the latitude and longitude, northward displacement, and eastward displacement of the UAV, including: Based on the northward displacement and the target conversion constant, determine the latitude increment; based on the latitude of the UAV and the latitude increment, determine the latitude of the target; based on the eastward displacement, the target conversion constant, and the latitude of the UAV, determine the longitude increment; based on the longitude of the UAV and the longitude increment, determine the longitude of the target, and locate the target based on the latitude and longitude of the target.

[0087] In some examples, the latitude increment formula can be used to determine the latitude increment based on the northward displacement and the target transformation constant, as shown below:

[0088] in, x represents the latitude increment, and x represents the northward displacement. The target transformation constant (approximately 8.983 × 10⁻⁶) -6 (degrees / meter).

[0089] Secondly, based on the latitude of the drone and the latitude increment, the latitude of the target is determined. Continuing the previous example, the northward distance x is approximately 153.56 meters. Substituting this into the formula, the target's latitude can be calculated. The latitude is approximately 0.00137946°. If the latitude of the drone's current location is determined to be 30.6738528°, then the latitude of the target is equal to 30.6738528° + 0.00137946° = 30.6752322°.

[0090] It is understandable that in practical applications, the distance between meridians will continuously shrink as latitude increases. The same angular increment will result in a smaller ground distance at higher latitudes. Therefore, this embodiment uses a cosine compensation formula to ensure that the longitude increment is proportional to the actual ground distance, thereby improving the accuracy of target positioning.

[0091] Specifically, the longitude increment can be determined using the cosine compensation formula, based on the eastward displacement, the target conversion constant, and the latitude of the UAV; the cosine compensation formula is shown below:

[0092] in, y represents the longitude increment, and y represents the eastward displacement. The latitude of the drone. is the target transformation constant.

[0093] Next, based on the longitude of the UAV and the longitude increment, determine the longitude of the target; continuing the previous example, the eastward distance y is approximately 141.66 meters, and the latitude of the UAV... The longitude increment is 30.6738528°. Substituting this into the above formula, we can calculate the longitude increment. It is approximately 0.001479517°. If the longitude of the drone's current location is determined to be 104.5310144°, then...

[0094] The longitude of the target is equal to 104.5310144° + 0.001479517° = 104.5324939°.

[0095] Finally, the target is located based on its latitude and longitude to obtain a more accurate target position, effectively reducing positioning errors caused by high-latitude regions and improving the accuracy of target positioning.

[0096] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the process of the embodiments of this application.

[0097] All of the above-mentioned optional technical solutions can be combined in any way to form the optional embodiments of this application, and will not be described in detail here.

[0098] The following are embodiments of the apparatus described in this application, which can be used to execute the embodiments of the method described in this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the method described in this application.

[0099] Figure 2 This is a schematic diagram illustrating the structure of a target positioning device according to an exemplary embodiment of this application. Figure 2 As shown, the exemplary target positioning device includes: The parameter acquisition module 210 is configured to acquire aerial images and flight attitude parameters of the UAV. The first determining module 220 is configured to determine the offset angle of the pixel corresponding to the target in the imaging plane based on the aerial image, and to determine the unit direction vector of the pixel in the world coordinate system based on the offset angle and flight attitude parameters. The second determining module 230 is configured to determine the northward and eastward displacements of the target relative to the UAV in the northeast coordinate system based on the vertical relative height and slant distance between the UAV and the target, the unit direction vector, the flight attitude parameters, and the offset angle. The target positioning module 240 is configured to locate the target based on the latitude and longitude, northward displacement, and eastward displacement of the UAV.

[0100] In some embodiments, the flight attitude parameters include pitch angle and roll angle. The second determining module 230 is further configured to determine longitudinal displacement and lateral displacement based on the vertical relative height and slant distance between the UAV and the target, the unit direction vector, the pitch angle, and the offset angle. The longitudinal displacement is the increment in the longitudinal direction of the projection of the target's corresponding pixel on the ground relative to the projection point of the camera's optical axis center. The lateral displacement is the increment in the lateral direction of the projection of the target's corresponding pixel on the ground relative to the projection point of the camera's optical axis center. Based on the slant distance, the unit direction vector, the roll angle, the longitudinal displacement, and the lateral displacement, the northward displacement and eastward displacement of the target relative to the UAV in the northeast coordinate system are determined.

[0101] In some embodiments, the unit direction vector includes a first direction vector and a second direction vector, and the offset angle includes a vertical offset angle and a horizontal offset angle. The second determining module 230 is further configured to determine, based on the slant distance between the UAV and the target, the first direction vector, and the second direction vector, the horizontal distance between the projection of the target's corresponding pixel on the ground and the projection point directly below the UAV; determine the longitudinal displacement based on the vertical relative height between the UAV and the target, the horizontal distance, the pitch angle, and the vertical offset angle; and determine the lateral displacement based on the vertical relative height between the UAV and the target, the pitch angle, the vertical offset angle, and the horizontal offset angle.

[0102] In some embodiments, the second determining module 230 is further configured to determine a first reference component and a second reference component based on slant range, roll angle, a first direction vector, and a second direction vector; determine a northward displacement based on slant range, a first direction vector, horizontal distance, a first reference component, a second reference component, longitudinal displacement, and lateral displacement; and determine an eastward displacement based on slant range, a second direction vector, horizontal distance, a first reference component, a second reference component, longitudinal displacement, and lateral displacement.

[0103] In some embodiments, the unit direction vector includes a third direction vector, and the second determining module 230 is further configured to detect the real-time state of the rangefinder, wherein the real-time state includes an operational state and an inoperable state; when the rangefinder is in an operational state, the slant distance is obtained through the rangefinder; when the rangefinder is in an inoperable state, the slant distance is determined based on the vertical relative height and the third direction vector.

[0104] In some embodiments, the first determining module 220 is further configured to determine the pixel coordinates of the pixel corresponding to the target based on the aerial image; normalize the pixel coordinates using the resolution of the aerial image to obtain normalized coordinates; and determine the offset angle of the pixel corresponding to the target in the imaging plane based on the field of view of the UAV and the normalized coordinates.

[0105] In some embodiments, the first determining module 220 is further configured to determine the direction cosine matrix of the UAV based on flight attitude parameters; determine the initial target vector of the pixel in the camera coordinate system based on the offset angle; and determine the unit direction vector of the pixel in the world coordinate system based on the initial target vector and the direction cosine matrix.

[0106] In some embodiments, the target positioning module 240 is further configured to determine the latitude increment based on the northward displacement and the target conversion constant; determine the latitude of the target based on the latitude of the UAV and the latitude increment; determine the longitude increment based on the eastward displacement, the target conversion constant, and the latitude of the UAV; determine the longitude of the target based on the longitude of the UAV and the longitude increment; and locate the target based on the latitude and longitude of the target.

[0107] Embodiments of this application also provide an electronic device, including: one or more processors; and a storage device for storing one or more programs, which, when executed by one or more processors, cause the electronic device to implement the methods provided in the above embodiments.

[0108] Figure 3 A schematic diagram of a computer system suitable for implementing the embodiments of this application is shown. It should be noted that... Figure 3 The computer system 300 of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0109] like Figure 3As shown, the computer system 300 includes a CPU 301, which can perform various appropriate actions and processes according to a program stored in ROM 302 or a program loaded from storage portion 308 into RAM 303, such as executing the methods in the above embodiments. The CPU 301 is a Central Processing Unit, ROM 302 is a Read-Only Memory, and RAM 303 is a Random Access Memory.

[0110] RAM 303 also stores various programs and data required for system operation. CPU 301, ROM 302, and RAM 303 are interconnected via bus 304. I / O interface 305 is also connected to bus 304. I / O interface 305 is an input / output interface.

[0111] The following components are connected to I / O interface 305: an input section 306 including a keyboard, mouse, etc.; an output section 307 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 308 including a hard disk, etc.; and a communication section 309 including a network interface card such as a LAN (Local Area Network) card, modem, etc. The communication section 309 performs communication processing via a network such as the Internet. A drive 310 is also connected to I / O interface 305 as needed. Removable media 311, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 310 as needed so that computer programs read from them can be installed into storage section 308 as needed.

[0112] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program including a computer program for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 309, and / or installed from removable medium 311. When the computer program is executed by CPU 301, it performs various functions defined in the system of this application. It should be noted that the computer-readable medium shown in embodiments of this application can be a computer-readable signal medium or a computer-readable storage medium or any combination thereof. The computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying a computer-readable computer program. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. A computer program contained on a computer-readable medium may be transmitted using any suitable medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0113] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation that may be implemented in systems, methods, and computer program products according to various embodiments of this application. Each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0114] The units described in the embodiments of this application can be implemented in software or hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the specific unit itself.

[0115] Another aspect of this application provides a computer-readable storage medium storing a computer program thereon, which, when executed by a computer's processor, causes the computer to perform the method as described above. This computer-readable storage medium may be included in the electronic device described in the above embodiments, or it may exist independently and not assembled into the electronic device.

[0116] Another aspect of this application provides a computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods described in the various embodiments above.

[0117] The above embodiments are merely illustrative of the principles and effects of this application and are not intended to limit this application. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of this application. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in this application should still be covered by the steps of this application.

Claims

1. A target localization method, characterized in that, include: Acquire aerial images and flight attitude parameters of the drone; Based on the aerial image, determine the offset angle of the pixel corresponding to the target in the imaging plane, and based on the offset angle and flight attitude parameters, determine the unit direction vector of the pixel in the world coordinate system. Based on the vertical relative altitude and slant distance between the UAV and the target, the unit direction vector, the flight attitude parameters, and the offset angle, the northward and eastward displacements of the target relative to the UAV in the northeast coordinate system are determined. The target is located based on the latitude and longitude, northward displacement, and eastward displacement of the UAV.

2. The method according to claim 1, characterized in that, The flight attitude parameters include pitch angle and roll angle. Determining the northward and eastward displacement of the target relative to the UAV in the northeast coordinate system based on the vertical relative altitude and slant distance between the UAV and the target, the unit direction vector, the flight attitude parameters, and the offset angle includes: Based on the vertical relative height and slant distance between the UAV and the target, the unit direction vector, the pitch angle, and the offset angle, the longitudinal displacement and the lateral displacement are determined. The longitudinal displacement is the increment of the projection of the target's corresponding pixel on the ground relative to the projection point of the camera's optical axis center in the longitudinal direction, and the lateral displacement is the increment of the projection of the target's corresponding pixel on the ground relative to the projection point of the camera's optical axis center in the lateral direction. Based on the slant range, unit direction vector, roll angle, longitudinal displacement, and lateral displacement, the northward and eastward displacements of the target relative to the UAV in the northeast coordinate system are determined.

3. The method according to claim 2, characterized in that, The unit direction vector includes a first direction vector and a second direction vector, and the offset angle includes a vertical offset angle and a horizontal offset angle; The determination of longitudinal and lateral displacements based on the vertical relative height and slant distance between the UAV and the target, the unit direction vector, the pitch angle, and the offset angle includes: Based on the slant distance between the UAV and the target, the first direction vector, and the second direction vector, the horizontal distance between the projection of the target's corresponding pixel on the ground and the projection point directly below the UAV is determined. The longitudinal displacement is determined based on the vertical relative height, horizontal distance, pitch angle, and vertical offset angle between the UAV and the target. The lateral displacement is determined based on the vertical relative height, pitch angle, vertical offset angle, and horizontal offset angle between the UAV and the target.

4. The method according to claim 3, characterized in that, The step of determining the northward and eastward displacements of the target relative to the UAV in the northeast coordinate system based on the slant range, unit direction vector, roll angle, longitudinal displacement, and lateral displacement includes: The first reference component and the second reference component are determined based on the slant distance, roll angle, first direction vector and second direction vector; The northward displacement is determined based on the slant distance, the first direction vector, the horizontal distance, the first reference component, the second reference component, the longitudinal displacement, and the lateral displacement. The eastward displacement is determined based on the slant distance, the second direction vector, the horizontal distance, the first reference component, the second reference component, the longitudinal displacement, and the lateral displacement.

5. The method according to any one of claims 1-4, characterized in that, The unit direction vector includes a third direction vector; before determining the northward and eastward displacement of the target relative to the UAV in the northeast coordinate system based on the vertical relative altitude and slant distance between the UAV and the target, the unit direction vector, the flight attitude parameters, and the offset angle, the method further includes: The real-time status of the rangefinder is detected, wherein the real-time status includes a working status and a non-working status; When the rangefinder is in a working state, the slope distance is obtained through the rangefinder; when the rangefinder is not in a working state, the slope distance is determined based on the vertical relative height and the third directional vector.

6. The method according to claim 1, characterized in that, The step of determining the offset angle of the pixel corresponding to the target in the imaging plane based on the aerial image includes: Based on the aerial image, determine the pixel coordinates of the pixel corresponding to the target; The pixel coordinates are normalized using the resolution of the aerial image to obtain normalized coordinates; Based on the UAV's field of view and the normalized coordinates, the offset angle of the pixel corresponding to the target in the imaging plane is determined.

7. The method according to claim 1, characterized in that, The step of determining the unit direction vector of the pixel in the world coordinate system based on the offset angle and flight attitude parameters includes: Based on the flight attitude parameters, determine the direction cosine matrix of the UAV; Based on the offset angle, determine the initial target vector of the pixel in the camera coordinate system; Based on the initial target vector and the direction cosine matrix, the unit direction vector of the pixel in the world coordinate system is determined.

8. The method according to claim 1, characterized in that, The method of locating the target based on the latitude, longitude, northward displacement, and eastward displacement of the UAV includes: The latitude increment is determined based on the northward displacement and the target conversion constant. The latitude of the target is determined based on the latitude of the UAV and the latitude increment; Based on the eastward displacement, target conversion constant, and UAV latitude, determine the longitude increment; The longitude of the target is determined based on the longitude and longitude increment of the UAV, and the target is located based on the latitude and longitude of the target.

9. A target positioning device, characterized in that, include: The parameter acquisition module is configured to acquire aerial images and flight attitude parameters of the UAV. The first determining module is configured to determine the offset angle of the pixel corresponding to the target in the imaging plane based on the aerial image, and to determine the unit direction vector of the pixel in the world coordinate system based on the offset angle and flight attitude parameters. The second determining module is configured to determine the northward and eastward displacements of the target relative to the UAV in the northeast coordinate system based on the vertical relative height and slant distance between the UAV and the target, the unit direction vector, the flight attitude parameters, and the offset angle. The target positioning module is configured to locate the target based on the latitude, longitude, northward displacement, and eastward displacement of the UAV.

10. An electronic device, characterized in that, include: One or more processors and a memory, the memory storing a computer program that, when executed by the one or more processors, causes the device to perform the steps of the target localization method as described in any one of claims 1 to 8.

Images