A method and apparatus for acquiring target spatial position information of a target object

Abstract

The application aims to provide a method and device for obtaining target space position information of a target object, comprising: obtaining a UAV image of the target object captured by a camera of a UAV device and camera pose information of the camera when the UAV image is captured, wherein the camera pose information comprises camera position information and camera attitude information of the camera; obtaining image position information of the corresponding target object in the UAV image; obtaining space position information of a plurality of map points; and determining target space position information of the target object in a three-dimensional rectangular coordinate system according to the space position information of the plurality of map points, the image position information, the camera position information and the camera attitude information. The application helps monitoring personnel to quickly and accurately correspond the real geographical position of the UAV image and the target object, and provides a good UAV device patrol and aerial photography experience.

Description

Technical Field

[0001] This application relates to the field of communications, and more particularly to a technique for acquiring target spatial location information of a target object. Background Technology

[0002] With the advancement and development of technology, drones are widely used in various fields, such as urban monitoring and patrol. Currently, most drone monitoring and patrols are conducted using traditional video surveillance. By transmitting the drone's video stream data to the ground control terminal and monitoring system, the real-time status of target points such as roads and buildings within the drone's images, as well as the drone's location information, can be viewed. Summary of the Invention

[0003] One object of this application is to provide a method, apparatus, medium, and program product for obtaining target spatial location information of a target object.

[0004] According to one aspect of this application, a method for obtaining target spatial location information of a target object is provided, wherein the method includes:

[0005] The method acquires drone images of a target object captured by the camera device of the drone equipment, as well as camera pose information of the camera device when the drone image is captured, wherein the camera pose information includes camera position information and camera posture information of the camera device;

[0006] Obtain the image location information of the corresponding target object in the UAV image;

[0007] Acquire spatial location information of multiple map points, wherein the spatial location information includes the spatial coordinate information of the corresponding map points in a three-dimensional Cartesian coordinate system;

[0008] The target spatial position information of the target object in the three-dimensional Cartesian coordinate system is determined based on the spatial position information of the multiple map points, the image position information, the camera position information, and the camera posture information.

[0009] According to another aspect of this application, a device for acquiring target spatial location information of a target object is provided, wherein the device includes:

[0010] A module is used to acquire drone images of a target object captured by the camera device of the drone equipment and the camera pose information of the camera device when the drone image is captured, wherein the camera pose information includes the camera position information and camera posture information of the camera device;

[0011] The first and second modules are used to obtain the image location information of the corresponding target object in the UAV image;

[0012] The first and third modules are used to obtain the spatial location information of multiple map points, wherein the spatial location information includes the spatial coordinate information of the corresponding map points in a three-dimensional rectangular coordinate system;

[0013] The first and fourth modules are used to determine the target spatial position information of the target object in the three-dimensional rectangular coordinate system based on the spatial position information of the multiple map points, the image position information, the camera position information, and the camera posture information.

[0014] According to one aspect of this application, a computer device is provided, wherein the device includes:

[0015] Processor; and

[0016] A memory configured to store computer-executable instructions, which, when executed, cause the processor to perform the steps of any of the methods described above.

[0017] According to one aspect of this application, a computer-readable storage medium is provided that stores a computer program / instructions thereon, characterized in that, when executed, the computer program / instructions cause a system to perform the steps of any of the methods described above.

[0018] According to one aspect of this application, a computer program product is provided, comprising a computer program / instructions, characterized in that, when executed by a processor, the computer program / instructions implement the steps of any of the methods described above.

[0019] Compared with existing technologies, this application can determine the target spatial location information of the target object in real time based on the image location information of the target object in the UAV image, thereby establishing a correlation between the UAV image and relevant information in space, providing a reference benchmark for UAV image monitoring personnel, helping monitoring personnel to quickly and accurately match the UAV image with the real geographical location related to the target object, and providing a good inspection and aerial photography experience for UAV equipment. Attached Figure Description

[0020] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0021] Figure 1 This diagram illustrates a method for obtaining target spatial location information of a target object according to an embodiment of this application.

[0022] Figure 2 This diagram illustrates a device structure of a computer apparatus according to another embodiment of the present application;

[0023] Figure 3 Exemplary systems that can be used to implement the various embodiments described in this application are shown.

[0024] The same or similar reference numerals in the accompanying drawings represent the same or similar parts. Detailed Implementation

[0025] The present application will now be described in further detail with reference to the accompanying drawings.

[0026] In a typical configuration of this application, the terminal, the device of the service network, and the trusted party all include one or more processors (e.g., a central processing unit (CPU)), input / output interfaces, network interfaces, and memory.

[0027] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash memory. Memory is an example of computer-readable media.

[0028] Computer-readable media, including both permanent and non-permanent, removable and non-removable media, can store information using any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PCM), programmable random access memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0029] The devices referred to in this application include, but are not limited to, user equipment, network equipment, or devices composed of user equipment and network equipment integrated through a network. The user equipment includes, but is not limited to, any mobile electronic product capable of human-computer interaction (e.g., via a touchpad), such as smartphones, tablets, and drones. These mobile electronic products can use any operating system, such as Android or iOS. The network equipment includes an electronic device capable of automatically performing numerical calculations and information processing according to pre-set or stored instructions. Its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), and embedded devices. The network equipment includes, but is not limited to, computers, network hosts, single network servers, multiple network server clusters, or a cloud composed of multiple servers. Here, the cloud consists of a large number of computers or network servers based on cloud computing, where cloud computing is a type of distributed computing, consisting of a virtual supercomputer composed of a group of loosely coupled computer clusters. The network includes, but is not limited to, the Internet, wide area network, metropolitan area network, local area network, VPN network, wireless ad hoc network, etc. Preferably, the device can also be a program running on the user equipment, network device, or a device formed by integrating user equipment and network device, network device, touch terminal, or network device and touch terminal through a network.

[0030] Of course, those skilled in the art should understand that the above-described devices are merely examples, and other existing or future devices that are applicable to this application should also be included within the scope of protection of this application, and are hereby incorporated by reference.

[0031] In the description of this application, "multiple" means two or more, unless otherwise expressly and specifically defined.

[0032] Figure 1A method for obtaining target spatial location information of a target object according to one aspect of this application is illustrated. This method is applied to a computer device and specifically includes steps S101, S102, S103, and S104. In step S101, a drone image of the target object captured by the camera device of the drone equipment and the camera pose information of the camera device when the drone image was captured are obtained, wherein the camera pose information includes the camera position information and camera posture information of the camera device; in step S102, the image position information of the corresponding target object in the drone image is obtained; in step S103, the spatial location information of multiple map points is obtained, wherein the spatial location information includes the spatial coordinate information of the corresponding map points in a three-dimensional Cartesian coordinate system; in step S104, the target spatial location information of the target object in the three-dimensional Cartesian coordinate system is determined based on the spatial location information of the multiple map points, the image position information, the camera position information, and the camera posture information. For example, the computer equipment includes, but is not limited to, user equipment, network equipment, or a combination of user equipment and network equipment; wherein, the user equipment includes, but is not limited to, any mobile electronic product capable of human-computer interaction (e.g., human-computer interaction via touchpad), such as smartphones, tablets, drones, smart glasses, smart helmets, etc.; the network equipment includes, but is not limited to, computers, network hosts, single network servers, sets of multiple network servers, or clouds composed of multiple servers, such as ground control center servers, etc. Steps S101, S102, and S103 are not sequential; steps S101 and S102 can be executed first, followed by step S103, or vice versa.

[0033] Specifically, in step S101, the drone image of the target object captured by the camera device of the drone equipment and the camera pose information of the camera device when the drone image was captured are acquired. The camera pose information includes the camera position information and camera posture information of the camera device. For example, the drone equipment includes an unmanned flying device controlled by a radio remote control device and a self-contained program control device. The drone equipment includes a corresponding communication device for establishing corresponding communication connections and transmitting information with other devices (e.g., user equipment, ground control center, etc.); the drone equipment also includes a camera device for acquiring drone images of the target object. The drone equipment flies based on the drone control user's operation or based on a preset flight route. During the flight of the drone equipment, the camera device of the drone equipment acquires ground targets based on user acquisition commands, in real time, or based on preset camera intervals to determine the corresponding drone image. The target object is used to indicate objects in the drone image that have a distinctive function or that the user is particularly interested in, for example, objects determined based on specified operations by the user in the drone image. For example, objects identified by users through clicking, selecting, or other operations in drone images, or objects identified in drone images based on relevant template features in a database, such as ground-based buildings or target landmarks; specifically, target objects can be geographical locations, iconic streets, buildings, cars, or pedestrians, etc. Those skilled in the art should understand that the above-mentioned target objects are merely examples, and other existing or future target objects that are applicable to this application should also be included within the scope of protection of this application, and are hereby incorporated by reference.

[0034] The drone device departs from the drone's takeoff point for this flight, flies to the corresponding location (for example, it could be a preset target location or the current location at the current moment, etc.), and then takes ground-related images of the corresponding location through the corresponding camera device. The ground-related images can be identified as drone images of the target object based on user operation, or the real-time captured ground-related images can be identified as drone images of the target object, etc. The drone device can acquire camera pose information corresponding to the camera device when capturing images of the drone. This camera pose information includes the camera position information and camera attitude information of the corresponding camera device. The camera position information includes, but is not limited to, the relative or absolute position information of the camera device (e.g., the relative or absolute position information of the origin of the camera coordinate system corresponding to the camera device). The relative position information includes, but is not limited to, the three-dimensional coordinates of the origin of the camera coordinate system corresponding to the camera device (e.g., the center of the camera device) in a three-dimensional coordinate system (e.g., the drone coordinate system, the geocentric coordinate system, or other three-dimensional Cartesian coordinate systems). The absolute position information includes the latitude and longitude information of the origin of the camera coordinate system corresponding to the camera device (e.g., the center of the camera device). Furthermore, in addition to latitude and longitude information, the absolute position information also includes altitude information. The corresponding camera position information is determined by the real-time position information of the drone image captured by the drone device.

[0035] In some cases, ignoring the translational relationship between the drone and the camera device, the geographical location information of the drone can be directly determined as the camera position information of the camera device. In other cases, the real-time position information of the drone when capturing drone images is first determined, and the camera position information of the camera device is determined based on this real-time position information and the translational relationship between the camera device and the drone. The translational relationship includes the translation of the origin of the camera coordinate system corresponding to the camera device relative to the origin of the drone coordinate system corresponding to the drone. The camera attitude information of the camera device includes the camera angle information of the camera device when capturing drone images. In some cases, the camera device adjusts the corresponding camera angle information through a corresponding support device (e.g., a gimbal), using the angle information of the corresponding gimbal to indicate the camera attitude information of the camera device. Specifically, the angle information of the drone gimbal is provided by the gimbal, including yaw / pitch / roll three-axis angle information, where yaw is the yaw angle, pitch is the pitch angle, and roll is the roll angle. For example, the angle information (α, β, γ) of the corresponding drone can be obtained based on the angle information of the drone's carrier device (e.g., gimbal). This drone angle information can be directly determined as the camera posture information of the camera device, or a coordinate axis transformation can be performed based on the drone angle information to convert each angle of the drone angle information to the camera coordinate system, thereby determining the camera posture information corresponding to the camera device. The camera coordinate system corresponding to the camera device is usually considered a special "object" coordinate system, defined within the visible area of ​​the camera screen. For example, in the camera coordinate system, with the camera's optical center as the origin, the x-axis is to the right, the z-axis is forward (towards the outside of the screen or the camera direction), and the y-axis is downward (not below the world but below the camera itself). The pixel coordinate system corresponding to the drone image includes a rectangular coordinate system uv established with the upper left corner of the drone image as the origin, using pixels as units. For example, the horizontal coordinate u and vertical coordinate v of a pixel are the column number and row number in its image array, respectively.

[0036] For example, a computer device (such as a drone) reads in map information of a scene, where one of the geographic coordinates includes longitude and latitude. Preferably, the computer device reads in an elevation map of the scene, where the elevation information... Figure 1 Generally, it is described based on the WGS84 coordinate system, where one of the geographic coordinate information includes longitude, latitude, and altitude, represented as (Longitude, latitude, altitude). The coordinate information of the UAV takeoff point is recorded as S0(φ0, λ0, g), and the elevation information h in the elevation information map is queried based on the latitude and longitude information of S0. map Obtain the WGS84 coordinates S of the drone's takeoff point. wgs84(φ0,λ0,h0), where h0=h+h map h is the altitude relative to the takeoff point (measured by the UAV system). map This refers to the altitude of the takeoff point. Based on coordinate transformation, the coordinates of the UAV's takeoff point in a three-dimensional Cartesian coordinate system (such as the ENU coordinate system) can be determined. The corresponding computer equipment acquires the pose data (φ, λ, h, α, β, γ) of the camera mounted on the UAV in real time. The UAV outputs its position information φ, λ, h in the WGS84 coordinate system, with α, β, and γ representing the yaw, pitch, and roll angles of the camera mounted on the gimbal, respectively, based on the UAV's takeoff point. φ and λ can be obtained through GPS sensors, RTK, or calculated using a nine-axis inertial measurement unit and the initial pose S0 (φ0, λ0, h0). The position (φ, λ, h) transformed into a three-dimensional Cartesian coordinate system (such as the ENU coordinate system) is the pose. cam (x,y,z).

[0037] In step S102, the image location information of the corresponding target object in the drone image is obtained. For example, the image location information can be determined by user clicks, selections, or other operations on the target object in the drone image, or by identifying the target object in the drone image based on relevant template features in the database. Specifically, the target object can be a geographical location, a landmark street, building, car, or pedestrian. Alternatively, the image location information can also be pixel coordinate information determined by other users' operations / image recognition on other devices, and sent to the computer device through a communication connection between the other device and the computer device. The computer device receives the image location information and determines it as the image location information of the target object.

[0038] In step S103, spatial location information of multiple map points is acquired, wherein the spatial location information includes the spatial coordinate information of the corresponding map points in a three-dimensional rectangular coordinate system. For example, a computer device acquires the spatial location information of multiple elevation information map points, which includes coordinate information in a corresponding three-dimensional rectangular coordinate system. This can be achieved by transforming all elevation information map points from a geographic coordinate system to a three-dimensional direct coordinate system, including but not limited to a station-centered coordinate system, a navigation coordinate system, and any three-dimensional rectangular coordinate system. The spatial location information of these multiple map points is stored in a corresponding database, which can be stored on a computer device or other device. Alternatively, the spatial location information of these multiple map points can be obtained by real-time transformation of corresponding geographic coordinate information. The corresponding geographic coordinate information is easy to store and can meet the transformation requirements between three-dimensional rectangular coordinate systems of different UAV flight trajectories.

[0039] In some embodiments, in step S103, the geographical location information of multiple map points is acquired, wherein the geographical location information includes the latitude and longitude information of the corresponding map points in the geographic coordinate system; first coordinate transformation information for converting the geographic coordinate system to the corresponding three-dimensional rectangular coordinate system is acquired; and based on the geographical location information of the multiple map points and the first coordinate transformation information, the spatial position information of the multiple map points in the three-dimensional rectangular coordinate system is determined. In some embodiments, the three-dimensional rectangular coordinate system includes, but is not limited to: a station-centered coordinate system; a navigation coordinate system; or any three-dimensional rectangular coordinate system. For example, the corresponding three-dimensional rectangular coordinate system includes a three-dimensional coordinate system with a certain point as the origin, two mutually perpendicular directions among the six positive directions (north, south, east, west, north, up, and down) as the corresponding x and y axes, and a positive direction perpendicular to the plane formed by the two perpendicular directions as the z axis. For example, the commonly used station-centered coordinate system or navigation coordinate system; of course, any three-dimensional rectangular coordinate system is acceptable. For example, the station-centered coordinate system includes a three-dimensional rectangular coordinate system with a station center point as the origin, the X-axis pointing east, the Y-axis pointing north, and the Z-axis pointing upwards, mainly used for geographical purposes. The corresponding navigation coordinate system includes a three-dimensional rectangular coordinate system with a coordinate point as the origin, the X-axis pointing north, the Y-axis pointing east, and the Z-axis perpendicular to the Earth's surface and pointing downwards, mainly used for navigation purposes. Any three-dimensional rectangular coordinate system can be obtained by transforming the aforementioned station-centered or navigation coordinate systems through simple coordinate axis directions, such as the NWU coordinate system (X-axis north, Y-axis west, Z-axis upwards) and the WND coordinate system (X-axis west, Y-axis north, Z-axis downwards), etc., which are merely examples and not intended to be limiting. When the three-dimensional rectangular coordinate system is known, the first coordinate transformation corresponding to the conversion of the geographic location information from the geographic coordinate system to the three-dimensional rectangular coordinate system is also known. Based on this known first coordinate transformation information, we can transform the map point in the geographic coordinate system to the three-dimensional rectangular coordinate system, thereby determining the corresponding spatial location information based on the geographic coordinate information of the map point. In some cases, for example, the elevation information of the map point is ignored, and the geographic location information includes the latitude and longitude information of the map point; or, in cases where the current geographic location is relatively flat, the geographic location information includes, but is not limited to, the latitude and longitude information of the map point, by determining the corresponding elevation information as a preset elevation plane or reference plane, such as an urban road plane or plain terrain. In some embodiments, the geographic location information also includes the elevation information of the corresponding map point. For example, in terrain with significant undulations, we also need to consider the influence of elevation information, by recording the elevation information of the map point and calculating the corresponding spatial location information, thereby ensuring the accuracy and precision of the calculation results. In some cases, the geographic location information of the map point further includes the geographic label information of the corresponding point, such as the identification information or descriptive information corresponding to the map point.In some cases, for ease of calculation, the origin of the three-dimensional Cartesian coordinate system is set at the takeoff point of the UAV's current flight. Of course, the origin of the three-dimensional Cartesian coordinate system can also be set at any other fixed map point, such as a fixed point on the map, or the origin of the Earth-Centered, Earth-Fixed (ECEF) coordinate system.

[0040] In some implementations, obtaining the first coordinate transformation information from the geographic coordinate system to the corresponding three-dimensional rectangular coordinate system includes: obtaining the second coordinate transformation information from the geographic coordinate system to the corresponding geocentric coordinate system; obtaining the third coordinate transformation information from the geocentric coordinate system to the corresponding three-dimensional rectangular coordinate system; and determining the first coordinate transformation information from the geographic coordinate system to the corresponding three-dimensional rectangular coordinate system based on the second and third coordinate transformation information. For example, the geographic coordinate system defines the Earth's surface location using a three-dimensional sphere, enabling the reference of Earth's surface point locations using latitude and longitude information. This is also called a geodetic coordinate system. In this system, the Earth is first abstracted into a regular ellipsoid that approximates the original natural Earth surface, called a reference ellipsoid. Then, a series of meridians and parallels are defined on the reference ellipsoid to form a latitude and longitude grid, thereby achieving the purpose of describing the location of points on the Earth's surface using latitude and longitude. This latitude and longitude geographic coordinate system is not a planar coordinate system because degree is not a standard unit of length and cannot be used to directly measure area and length. Therefore, we need to transform this geographic coordinate system, converting the latitude and longitude information into a measurable rectangular coordinate system. For example, we can first transform the geographic coordinate system to the geocentric coordinate system, and then transform it from the geocentric coordinate system to the corresponding three-dimensional rectangular coordinate system. The corresponding first coordinate transformation information is determined by the second coordinate transformation information from the geographic coordinate system to the corresponding geocentric coordinate system and the third coordinate transformation information from the geocentric coordinate system to the three-dimensional rectangular coordinate system corresponding to the UAV equipment. For example, the Earth-Centered, Earth-Fixed (ECEF) coordinate system is a Cartesian rectangular coordinate system with the Earth's center of mass as the origin. The origin O(0,0,0) is the Earth's center of mass. The z-axis is parallel to the Earth's axis and points towards the North Pole. The x-axis points towards the intersection of the Prime Meridian and the equator. The y-axis is perpendicular to the xOz plane (i.e., the intersection of 90 degrees east longitude and the equator), forming a right-handed coordinate system. The geocentric coordinate system is also a special type of three-dimensional rectangular coordinate system. Transforming from the geocentric coordinate system to the corresponding three-dimensional rectangular coordinate system only requires calculations such as translation of the origin and rotation along the three axes. The prefixes "first," "second," "third," and subsequent similar terms are only used to transform coordinate information between different coordinate systems and do not involve any order, sequence, or hierarchical ranking.

[0041] Here, the transformation relationship from the geographic coordinate system to the three-dimensional rectangular coordinate system is shown below:

[0042] 1) Transform the geographic coordinates (φ, λ, h) corresponding to the geographic coordinate system to the geocentric coordinate system to obtain the three-dimensional coordinates (x, y, h) in the geocentric coordinate system. in ,y in ,z in ):

[0043] x in =(N(φ)+h)cosφcosλ

[0044] y in =(N(φ)+h)cosφsinλ

[0045] z in =(N(φ)(1-e^2)+h)sinφ (1)

[0046] Among them, corresponding

[0047] The values ​​of a and e represent the semi-major axis and the first numerical eccentricity of the ellipse, respectively, where a = 6378137 and e = 1 / 2. 2 =6.69437999014*10 -3 In this context, a point (e.g., the UAV takeoff point) (φ0, λ0, h0) in a geographic coordinate system (such as the WGS84 coordinate system) is taken as the origin of a three-dimensional rectangular coordinate system (such as the ENU coordinate system). The coordinates of this point in the geocentric coordinate system are (X... r ,Y r Z r ), three-dimensional coordinates in the geocentric coordinate system (x in ,y in ,z in Transform to a three-dimensional rectangular coordinate system (such as the ENU coordinate system) to obtain the three-dimensional coordinates (x, y, x) in the three-dimensional rectangular coordinate system. out ,y out ,z out ):

[0048]

[0049] Here, the above embodiments use a three-dimensional rectangular coordinate system, including the ENU coordinate system, as an example to illustrate the coordinate transformation information from a geographic coordinate system to a three-dimensional rectangular coordinate system. Those skilled in the art should understand that the above embodiments are also applicable to scenarios with other three-dimensional rectangular coordinate systems. For example, the transformation relationship between the ENU coordinate system and the NED coordinate system is as follows:

[0050]

[0051] Based on the above formula, coordinate transformations can be achieved between the ENU and NED coordinate systems, thus realizing coordinate transformations from the geographic coordinate system to the NED coordinate system. Furthermore, the transformation relationships between the ENU / NED coordinate system and other three-dimensional rectangular coordinate systems can be utilized to achieve coordinate transformations from the geographic coordinate system to other three-dimensional rectangular coordinate systems.

[0052] In step S104, the target spatial position information of the target object in the three-dimensional Cartesian coordinate system is determined based on the spatial position information of the multiple map points, the image position information, the camera position information, and the camera posture information. For example, after the computer device obtains the known spatial position information of multiple map points, the image position information, the camera position information, and the corresponding camera posture information, since the intrinsic parameters of the camera device are known, we can construct a spatial ray from the camera's optical center through the image position information corresponding to the target object based on the camera imaging model. Based on this spatial ray, the spatial position information of the multiple map points, and the camera position information, the target spatial position information of the target object is determined. For example, we can assume that the image position information is perpendicular to the plane where the camera film is located (e.g., the optical axis corresponding to the center of the UAV image is perpendicular to the plane where the camera film is located, etc.), and then determine the corresponding spatial ray information based on the normal vector of the plane where the film is located and the image position information. Then, based on the spatial ray information and the ground information composed of multiple map points, the corresponding intersection point is determined, and the spatial coordinate information of the intersection point is used as the target spatial position information of the target object, etc. Of course, if the pixel corresponding to the image position information is not at the center of the image, there will be an error between the normal vector determined based on the film and the actual ray vector. In this case, we need to determine the vector information of the spatial ray corresponding to the image position information through the camera's imaging model, image position information, and camera posture information. The spatial ray is described by the coordinates of its optical center and its vector information. After the computer determines the vector information of the corresponding spatial ray, it can calculate the intersection point of the ray with respect to the ground based on the ray's vector information, camera position information, and the spatial position information of multiple map points. The spatial coordinate information of this intersection point can then be used as the target spatial position information of the target object, etc.

[0053] In some embodiments, step S104 includes sub-steps S1041 (not shown) and S1042 (not shown). In step S1041, the corresponding target ray vector information is determined based on the imaging model of the camera device, the image position information, and the camera posture information. In step S1042, the target spatial position information of the target object in the three-dimensional Cartesian coordinate system is determined based on the spatial position information of the plurality of map points, the target ray vector information, and the camera position information. For example, a computer device acquires the image position information I(u) of the target object in the drone image corresponding to the camera device carried by the drone device.I ,v I Based on the imaging model, a spatial ray L is constructed with the optical center of the camera device passing through point I. Ray L is composed of the coordinates of the optical center and the vector information of the ray. The description states that the imaging model is determined based on the intrinsic parameters of the camera device. The coordinates of the optical center are given by pose. cam The optical center coordinates (x, y, z) are consistent with the camera position information of the camera device in the three-dimensional Cartesian coordinate system when determining the image position information of the target object in the UAV image, i.e., the optical center coordinates pose. cam (x, y, z) changes according to the position of the UAV corresponding to the image position information of the target object. Specifically, the pose data of the UAV-loaded camera device is acquired in real time. When the image position information of the target object in the UAV image is determined (such as when the user clicks or selects the target object in the UAV image), the pose data of the UAV-loaded camera device is pose(φ, λ, h, α, β, γ). The UAV outputs the position information φ, λ, h in the WGS84 coordinate system with the take-off point as the reference. α, β, γ are the yaw, pitch, and roll angles of the gimbal-loaded camera device, respectively. The position pose of the load camera device in the three-dimensional rectangular coordinate system (such as the ENU coordinate system) is calculated according to formulas (1)-(2) based on (φ, λ, h). cam (x,y,z) are determined as the coordinates of the optical center.

[0054] Vector information of rays The calculation process is as follows:

[0055] Calculate the distortion-free pixel I based on the distortion parameters. undistort (u ud ,v ud This step is optional.

[0056] Calculated based on camera intrinsic parameters:

[0057] α′=arctan((cx-u ud ) / fx)

[0058] β′=arctan((cy-v ud ) / fy)

[0059] γ′=0 (4)

[0060] Based on the camera intrinsic parameters, the corresponding α′, β′, γ′ are determined, and R is calculated using the following formula. L :

[0061]

[0062]

[0063] Among them, R cam To determine the rotation matrix corresponding to the camera attitude information of the camera device when determining the image position information of the target object in the UAV image, the rotation matrix R of the load camera device is obtained by inputting the gimbal angle (α,β,γ) according to formula (5). cam f x and f y c represents the focal length of the gimbal camera. x and c y f represents the coordinates of the principal point. x f y c x c y This can be obtained through camera calibration. Specifically, the rotation matrix R of the load-bearing camera device is calculated. cam If the coordinate system of the UAV's rotation angle is different from the established three-dimensional rectangular coordinate system (different UAVs use different coordinate systems, such as the UAV's rotation angle coordinate system being the NED coordinate system, while the three-dimensional rectangular coordinate system is the ENU coordinate system), it is necessary to transform the UAV's rotation matrix to the three-dimensional rectangular coordinate system. Then, the rotation matrix R of the load camera device is calculated using formula (5). cam It is also necessary to transform the coordinate system using the transformation relationship between the two coordinate systems; if the coordinate system of the UAV rotation angle is the same as the established three-dimensional rectangular coordinate system, then the rotation matrix R of the load camera device is calculated using formula (5). cam No further conversion is needed.

[0064] After the computer device determines the vector information of the corresponding ray based on the aforementioned formula, it can determine the target spatial location information of the target object based on the ray vector information, camera position information, and spatial location information of multiple known map points.

[0065] In some implementations, in step S1042, the spatial position information of the optical center of the camera device in the three-dimensional Cartesian coordinate system is obtained based on the camera position information; the target spatial position information of the target object in the three-dimensional Cartesian coordinate system is determined according to the spatial position information of the multiple map points, the target ray vector information, and the spatial position information of the optical center. For example, to save computing resources and simplify the calculation process, we set the UAV takeoff point as the origin of the three-dimensional Cartesian coordinate system, and then acquire the pose data of the UAV-loaded camera device in real time. When the image position information of the target object in the UAV image is determined (such as when the user clicks or selects the target object in the UAV image), the pose data of the UAV-loaded camera device is pose(φ,λ,h,α,β,γ). The UAV outputs the position information φ,λ,h in the WGS84 coordinate system based on the takeoff point, where α,β,γ are the yaw, pitch, and roll angles of the gimbal-loaded camera device, respectively. The position (φ,λ,h) of the load camera device in a three-dimensional rectangular coordinate system (such as the ENU coordinate system) is calculated according to formulas (1)-(2). cam (x, y, z) are determined as the spatial coordinates of the optical center. After the computer equipment determines the vector information of the corresponding ray, it can determine the actual position of the ray in a three-dimensional rectangular coordinate system based on the spatial coordinate information of the optical center. Thus, it can calculate the intersection point of the ray with the ground plane. The ground plane is determined by the spatial coordinate information of at least three known map points, or by multiple known map points through plane fitting, etc.

[0066] In some implementations, determining the target spatial position information of the target object in the three-dimensional Cartesian coordinate system based on the spatial position information of the plurality of map points, the target ray vector information, and the optical center spatial position information includes: determining the target map point closest to the target ray from the plurality of map points based on the target ray vector information, the spatial position information of the plurality of map points, and the optical center spatial position information; sub-step e takes two map points from the plurality of map points other than the target map point, forms a corresponding spatial triangle with the target map point, and determines the corresponding spatial intersection point based on the target ray and the corresponding spatial triangle; and uses the spatial coordinate information of the spatial intersection point as the target spatial position information of the target object. The target ray is described by the optical center coordinates and the ray's vector information. For example, after a computer device acquires the spatial position information of multiple map points, it determines the target map point closest to the target ray based on this spatial position information, performs plane fitting between the target map point and other map points in the plurality of map points to determine the corresponding ground plane, and then determines the intersection point of the ray and the ground plane based on the target ray. For example, to calculate the target map point Q that is closest to the target ray L, let the coordinates of the map point in the map in a three-dimensional rectangular coordinate system (such as the ENU coordinate system) be P. mappoint (x,y,z), then the target map point Q is calculated as follows:

[0067]

[0068] Among them, pose cam This refers to the spatial coordinates of the optical center.

[0069] After determining the target map point, the computer device searches for surrounding map points R, J, K, and H of Q on the map. It then sequentially selects two surrounding map points and Q to construct a spatial triangle, such as △RJQ, △JKQ, △KHQ, or △HRQ. The number of surrounding map points is unlimited and can exceed four. The selected surrounding points must cover the area around Q, and the surrounding points and Q must form a triangle on the terrain surface. For example, a distance threshold can be set, and map points whose distance difference from Q is within that threshold can be used as surrounding map points; alternatively, the points closest to Q can be selected as surrounding map points. Based on the target map point and at least two surrounding map points, the computer device determines the corresponding spatial triangle, calculates the spatial intersection of the spatial triangle with the target ray, and uses the spatial coordinates of the intersection as the target spatial location information of the target object. For example, suppose the intersection point of the triangle and the target ray L is P′. Tag The target ray is known to pass through point pose. cam (x, y, z), the vector information of the ray is Assume the three vertices of triangle R, J, and Q. Then the plane normal vector of triangle RJQ is:

[0070]

[0071] but,

[0072]

[0073] Wherein, R is the spatial coordinate information of vertex R in a three-dimensional rectangular coordinate system.

[0074] In some implementations, selecting two map points from the plurality of map points other than the target map point includes: selecting a predetermined number of map points closest to the target map point as surrounding map points, and then selecting two map points from the surrounding map points. For example, after determining the target map point, a predetermined number of map points closest to the target map point are selected from the other map points, such as a predetermined number of four or more, and these predetermined number of map points are selected as surrounding map points. Two map points are then selected from the surrounding map points to form a spatial triangle with the target map point. The selected surrounding points also satisfy the requirement that they can cover the area around the target map point, and that the surrounding points and the target map point can form a triangle on the terrain surface.

[0075] In some implementations, selecting two map points from the plurality of map points other than the target map point includes: selecting a predetermined number of map points as surrounding map points with the target map point as the center and different radii based on the latitude and longitude of the other map points; and selecting two map points from the surrounding map points. For example, using the horizontal direction as the precision and the vertical direction as the latitude, only the latitude and longitude of the map points are used; a predetermined number of map points are selected from the other map points with the target map point as the center and different radii R, such as a predetermined number of 4 or more, and these predetermined number of map points are selected as surrounding map points. Two map points are then selected from the surrounding map points to form a spatial triangle with the target map point. The selected surrounding points also satisfy the requirement that they can cover the area around the target map point, and that the surrounding points and the target map point can form a triangle on the terrain surface.

[0076] In some implementations, using the spatial coordinates of the spatial intersection point as the target spatial location information of the target object includes: if the spatial intersection point is inside the spatial triangle, then using the spatial coordinates of the spatial intersection point as the target spatial location information of the target object. For example, after calculating the corresponding intersection point, the computer device further determines whether the spatial intersection point is inside the spatial triangle, for example, by calculating whether the following expressions hold true:

[0077]

[0078]

[0079]

[0080] If all the inequalities in equation (10) are false, then P′ Tag Within the current spatial triangle, the intersection point of the current triangle and the target ray L is successfully calculated, and P′ is then set. Tag The spatial coordinate information is determined as the spatial location information of the target object.

[0081] In some implementations, using the spatial coordinates of the spatial intersection point as the target spatial location information of the target object further includes: if the spatial intersection point is not within the spatial triangle, polling other map points and repeating step e until a new spatial intersection point is obtained within the corresponding new spatial triangle, then using the spatial coordinates of the new spatial intersection point as the target spatial location information of the target object. For example, if the spatial intersection point is not within the spatial triangle, discarding the current spatial triangle, reconstructing a triangle using the aforementioned formulas (8)-(10) combined with other surrounding map points, and calculating the corresponding intersection point until it is determined that the first output new spatial intersection point is within the newly constructed spatial triangle, thereby outputting the spatial coordinates of the corresponding spatial intersection point in the three-dimensional rectangular coordinate system, and using the spatial coordinates as the target spatial location information of the target object.

[0082] In some embodiments, the method further includes step S105 (not shown), in which the geographic coordinates of the target object in a geographic coordinate system are determined and stored based on the target spatial location information of the target object. For example, after the computer device determines the target spatial location information of the target object, it can store P... Tag The coordinate information of a point (in a 3D Cartesian coordinate system) is converted from the 3D spatial coordinate system to a geographic coordinate system (e.g., the WGS84 coordinate system) for storage. Later, when other drone devices or the drone device flies from different takeoff points, this geographic location information can be converted back into spatial location information (from the geodetic coordinate system to the 3D Cartesian coordinate system) based on the target object's call (e.g., presentation). This way, the drone can display the target object's coordinate information or tag information when taking off from different points. This is because if the 3D Cartesian coordinate system is established with the takeoff point as the origin, the corresponding 3D Cartesian coordinate system will differ depending on the takeoff point, while the geodetic coordinate system remains unchanged, facilitating subsequent calculations.

[0083] In some embodiments, the method further includes step S106 (not shown), in which target tag information of the target object is generated based on the target spatial location information and the object description information of the target object. For example, a computer device can acquire the target spatial location information of the target object and generate the target tag information of the target object based on the target spatial location information and the object description information of the target object. The object description information includes, but is not limited to, target object identification information, target object description information, target object geographic coordinate information, or other augmented reality information. The augmented reality information includes, but is not limited to, the rendering, marking, or 3D projection information of the target object. For example, by transforming the target spatial location information to the pixel coordinate system corresponding to the UAV image based on the corresponding coordinate transformation, the target tag information of the target object can be superimposed on the UAV image in real time during the flight of the current UAV. Alternatively, by determining the corresponding geographic coordinate information based on the target spatial location information of the target object, the geographic coordinate information can be transformed to the pixel coordinate system corresponding to the UAV image based on the corresponding coordinate transformation when other UAV devices are flying or when the UAV device takes off subsequently, and the target tag information of the target object can be superimposed on the UAV image.

[0084] In some embodiments, the method further includes step S107 (not shown), in which real-time drone images captured by the camera device and real-time pose information of the camera device are acquired; the real-time image position information of the target object in the real-time drone image is determined based on the target spatial position information and the real-time pose information; and the target label information is superimposed on the real-time drone image based on the real-time image position information. For example, using the above method, a computer device can calculate the real-time image position information corresponding to each real-time captured real-time drone image in real time, wherein the corresponding real-time image position information includes the real-time pixel coordinate information of the target object in the real-time pixel coordinate system of the real-time drone image, etc. For example, P Tag The point (e.g., in a three-dimensional rectangular coordinate system) is transformed into the camera coordinate system to obtain P. ca Let the pose data of the camera device on the current UAV be pose(φ1,λ1,h1,α1,β1,γ1). The camera coordinates in the current three-dimensional Cartesian coordinate system are calculated from (φ1,λ1,h1) using formulas (1)-(2), and are given as pose′. cam (x′1,y′1,z′1), according to formula (5), input the gimbal angles α1,β1,γ1 to obtain the rotation matrix R′ of the current load camera device. cam Then the coordinates in the camera coordinate system are P. cam (x c ,y c ,zc ):

[0085] P cam =R′ cam *P Tag -R′ cam *pose′ cam (11)

[0086] Among them, P cam (x c ,y c ,z c P is obtained by converting it to the pixel coordinate system of the camera device. i (u,v),

[0087]

[0088] Among them, f x f y c x c y This serves as an internal reference for the camera device. Subsequently, for (μ) undistort ,v undistort Distortion processing can be added to obtain P. i (u,v), of course, this distortion processing is optional, and (μ) can be directly applied. undistort ,v undistort ) is determined to be P i (u,v) etc.

[0089] Then, the computer device overlays target label information of the target object based on the corresponding real-time image location information. This target label information includes, but is not limited to, rendering, marking, or 3D projection information of the target object. Similarly, the real-time image location information of the target object in the real-time UAV image is determined based on the geographic coordinate information corresponding to the target object and the real-time pose information; the target label information is then overlaid and presented in the real-time UAV image based on the real-time image location information. For example, the geographic coordinate information can be transformed to the pixel coordinate system corresponding to the UAV image based on the coordinate transformation corresponding to the geographic coordinate system to the pixel coordinate system, and the target label information of the target object can be overlaid and presented in the real-time UAV image.

[0090] The foregoing section primarily describes various embodiments of this application for obtaining target spatial location information of a target object. Furthermore, this application also provides specific devices capable of implementing the above embodiments. Below, we will combine... Figure 2 Let me introduce it.

[0091] Figure 2A computer device 100 for acquiring target spatial location information of a target object according to one aspect of this application is shown, specifically including a first module 101, a second module 102, a third module 103, and a fourth module 104. The first module 101 is used to acquire a drone image of the target object captured by the camera device of the drone equipment, and the camera pose information of the camera device when the drone image was captured, wherein the camera pose information includes the camera position information and camera posture information of the camera device; the second module 102 is used to acquire the image position information of the corresponding target object in the drone image; the third module 103 is used to acquire the spatial location information of multiple map points, wherein the spatial location information includes the spatial coordinate information of the corresponding map points in a three-dimensional Cartesian coordinate system; the fourth module 104 is used to determine the target spatial location information of the target object in the three-dimensional Cartesian coordinate system based on the spatial location information of the multiple map points, the image position information, the camera position information, and the camera posture information.

[0092] In some embodiments, module 103 is used to acquire geographic location information of multiple map points, wherein the geographic location information includes the latitude and longitude information of the corresponding map points in a geographic coordinate system; acquire first coordinate transformation information for converting the geographic coordinate system to a corresponding three-dimensional rectangular coordinate system; and determine the spatial location information of the multiple map points in the three-dimensional rectangular coordinate system based on the geographic location information of the multiple map points and the first coordinate transformation information. In some embodiments, the three-dimensional rectangular coordinate system includes, but is not limited to: a station-centered coordinate system; a navigation coordinate system; or any three-dimensional rectangular coordinate system. In some embodiments, the geographic location information also includes the elevation information of the corresponding map points.

[0093] In some embodiments, obtaining the first coordinate transformation information of the geographic coordinate system to the corresponding three-dimensional rectangular coordinate system includes: obtaining the second coordinate transformation information of the geographic coordinate system to the corresponding geocentric coordinate system; obtaining the third coordinate transformation information of the geocentric coordinate system to the corresponding three-dimensional rectangular coordinate system; and determining the first coordinate transformation information of the geographic coordinate system to the corresponding three-dimensional rectangular coordinate system based on the second coordinate transformation information and the third coordinate transformation information.

[0094] In some embodiments, the four-module 104 includes a four-one unit (not shown) and a four-two unit (not shown). The four-one unit is used to determine the corresponding target ray vector information based on the imaging model of the camera device, the image position information, and the camera posture information. The four-two unit is used to determine the target spatial position information of the target object in the three-dimensional rectangular coordinate system based on the spatial position information of the plurality of map points, the target ray vector information, and the camera position information.

[0095] In some embodiments, unit 1042 is used to obtain the spatial position information of the optical center of the camera device in the three-dimensional rectangular coordinate system based on the camera position information; and to determine the target spatial position information of the target object in the three-dimensional rectangular coordinate system based on the spatial position information of the plurality of map points, the target ray vector information, and the spatial position information of the optical center.

[0096] In some embodiments, determining the target spatial position information of the target object in the three-dimensional Cartesian coordinate system based on the spatial position information of the plurality of map points, the target ray vector information, and the optical center spatial position information includes: determining the target map point closest to the target ray vector information from the plurality of map points based on the target ray vector information, the spatial position information of the plurality of map points, and the optical center spatial position information; sub-step e involves selecting two map points from the plurality of map points other than the target map point, forming a corresponding spatial triangle with the target map point, and determining the corresponding spatial intersection point based on the target ray vector information and the corresponding spatial triangle; and using the spatial coordinate information of the spatial intersection point as the target spatial position information of the target object. In some embodiments, selecting two map points from the plurality of map points other than the target map point includes: selecting a preset number of map points closest to the target map point from the other map points as surrounding map points, and selecting two map points from the surrounding map points. In other embodiments, the step of selecting two map points from the plurality of map points other than the target map point includes: selecting a preset number of map points as surrounding map points with different radii, centered on the target map point, based on the latitude and longitude of the other map points, and selecting two map points from the surrounding map points.

[0097] In some implementations, using the spatial coordinates of the spatial intersection as the target spatial location information of the target object includes: if the spatial intersection is located inside the spatial triangle, then using the spatial coordinates of the spatial intersection as the target spatial location information of the target object.

[0098] In some implementations, using the spatial coordinates of the spatial intersection point as the target spatial location information of the target object further includes: if the spatial intersection point is not within the spatial triangle, polling the other map points and repeating step e until a new spatial intersection point is obtained within the corresponding new spatial triangle, then using the spatial coordinates of the new spatial intersection point as the target spatial location information of the target object.

[0099] Here, the Figure 2The specific implementation methods corresponding to module 101, module 102, module 103, and module 104 shown are the same as those described above. Figure 1 The embodiments of steps S101, S102, S103 and S104 shown are the same or similar, and therefore will not be described again, but are incorporated herein by reference.

[0100] In some embodiments, the device further includes a five-module (not shown) for determining and storing the geographic coordinate information of the target object in a geographic coordinate system based on the target spatial location information of the target object.

[0101] In some embodiments, the device further includes a six-module (not shown) for generating target tag information of the target object based on the target spatial location information and the object description information of the target object.

[0102] In some embodiments, the device further includes a module (not shown) for acquiring real-time drone images captured by the camera device and real-time pose information of the camera device; determining the real-time image position information of the target object in the real-time drone image based on the target spatial position information and the real-time pose information; and overlaying the target label information onto the real-time drone image based on the real-time image position information.

[0103] Here, the specific implementation methods corresponding to modules 15 to 17 are the same as or similar to the embodiments of steps S105 to S107 mentioned above, and therefore will not be repeated here, but are included by reference.

[0104] In addition to the methods and devices described in the above embodiments, this application also provides a computer-readable storage medium storing computer code that, when executed, performs the method described in any of the preceding embodiments.

[0105] This application also provides a computer program product that, when executed by a computer device, performs the method described in any of the preceding claims.

[0106] This application also provides a computer device, the computer device comprising:

[0107] One or more processors;

[0108] Memory, used to store one or more computer programs;

[0109] When the one or more computer programs are executed by the one or more processors, the one or more processors cause the one or more processors to perform the method as described in any of the preceding methods.

[0110] Figure 3 Exemplary systems that can be used to implement the various embodiments described in this application are shown;

[0111] like Figure 3 As shown in some embodiments, system 300 can function as any of the aforementioned devices in each of the described embodiments. In some embodiments, system 300 may include one or more computer-readable media having instructions (e.g., system memory or NVM / storage device 320) and one or more processors (e.g., one or more processors 305) coupled to the one or more computer-readable media and configured to execute the instructions to implement the module and thus perform the actions described in this application.

[0112] In one embodiment, the system control module 310 may include any suitable interface controller to provide any suitable interface to at least one of the processors 305 and / or any suitable device or component communicating with the system control module 310.

[0113] The system control module 310 may include a memory controller module 330 to provide an interface to the system memory 315. The memory controller module 330 may be a hardware module, a software module, and / or a firmware module.

[0114] System memory 315 can be used, for example, to load and store data and / or instructions for system 300. In one embodiment, system memory 315 may include any suitable volatile memory, such as suitable DRAM. In some embodiments, system memory 315 may include double data rate type quad synchronous dynamic random access memory (DDR4 SDRAM).

[0115] In one embodiment, the system control module 310 may include one or more input / output (I / O) controllers to provide interfaces to the NVM / storage device 320 and (one or more) communication interfaces 325.

[0116] For example, NVM / storage device 320 can be used to store data and / or instructions. NVM / storage device 320 may include any suitable non-volatile memory (e.g., flash memory) and / or may include any suitable (one or more) non-volatile storage devices (e.g., one or more hard disk drives (HDDs), one or more optical disc drives (CDs), and / or one or more digital universal optical disc (DVD) drives).

[0117] NVM / storage device 320 may include storage resources that are physically part of a device on which system 300 is mounted, or that can be accessed by the device without necessarily being part of it. For example, NVM / storage device 320 may be accessed via a network through one or more communication interfaces 325.

[0118] One or more communication interfaces 325 may provide the system 300 with an interface to communicate over one or more networks and / or with any other suitable device. The system 300 may wirelessly communicate with one or more components of a wireless network in accordance with any of one or more wireless network standards and / or protocols.

[0119] In one embodiment, at least one of the processors 305 may be logically packaged with one or more controllers of the system control module 310 (e.g., memory controller module 330). In one embodiment, at least one of the processors 305 may be logically packaged with one or more controllers of the system control module 310 to form a system-in-package (SiP). In one embodiment, at least one of the processors 305 may be integrated with the logic of one or more controllers of the system control module 310 on the same die. In one embodiment, at least one of the processors 305 may be integrated with the logic of one or more controllers of the system control module 310 on the same die to form a system-on-a-chip (SoC).

[0120] In various embodiments, system 300 may be, but is not limited to, a server, workstation, desktop computing device, or mobile computing device (e.g., laptop computing device, handheld computing device, tablet computer, netbook, etc.). In various embodiments, system 300 may have more or fewer components and / or different architectures. For example, in some embodiments, system 300 includes one or more cameras, a keyboard, a liquid crystal display (LCD) screen (including a touchscreen display), a non-volatile memory port, multiple antennas, a graphics chip, an application-specific integrated circuit (ASIC), and a speaker.

[0121] It should be noted that this application can be implemented in software and / or a combination of software and hardware, for example, using an application-specific integrated circuit (ASIC), a general-purpose computer, or any other similar hardware device. In one embodiment, the software program of this application can be executed by a processor to implement the steps or functions described above. Similarly, the software program of this application (including related data structures) can be stored in a computer-readable recording medium, such as RAM memory, magnetic or optical drives, floppy disks, and similar devices. Furthermore, some steps or functions of this application can be implemented in hardware, for example, as circuitry that cooperates with a processor to perform the various steps or functions.

[0122] Furthermore, a portion of this application can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to this application through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions, or the computer compiling the instructions and then executing the corresponding compiled program, or the computer reading and executing the instructions, or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0123] Communication media include media through which communication signals containing, for example, computer-readable instructions, data structures, program modules, or other data are transmitted from one system to another. Communication media can include guided transmission media (such as cables and wires (e.g., optical fibers, coaxial cables, etc.)) and wireless (unguided transmission) media capable of propagating energy waves, such as sound, electromagnetic, RF, microwave, and infrared. Computer-readable instructions, data structures, program modules, or other data can be embodied as modulated data signals in, for example, wireless media (such as carrier waves or similar mechanisms embodied as part of spread spectrum technology). The term "modulated data signal" refers to a signal whose one or more characteristics are altered or set in a manner that encodes information in the signal. Modulation can be analog, digital, or a hybrid modulation technique.

[0124] By way of example and not limitation, computer-readable storage media may include volatile and non-volatile, removable and non-removable media implemented by any method or technique for storing information such as computer-readable instructions, data structures, program modules or other data. For example, computer-readable storage media include, but are not limited to, volatile memories such as random access memory (RAM, DRAM, SRAM); and non-volatile memories such as flash memory, various read-only memories (ROM, PROM, EPROM, EEPROM), magnetic and ferromagnetic / ferroelectric memories (MRAM, FeRAM); and magnetic and optical storage devices (hard disks, magnetic tapes, CDs, DVDs); or other media now known or hereafter developed capable of storing computer-readable information / data for use by a computer system.

[0125] Herein, one embodiment of this application includes an apparatus comprising a memory for storing computer program instructions and a processor for executing the program instructions, wherein when the computer program instructions are executed by the processor, the apparatus is triggered to run a method and / or technical solution based on the foregoing embodiments of this application.

[0126] It will be apparent to those skilled in the art that this application is not limited to the details of the exemplary embodiments described above, and that this application can be implemented in other specific forms without departing from the spirit or essential characteristics of this application. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of this application is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be embraced within this application. No reference numerals in the claims should be construed as limiting the scope of the claims. Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices recited in the apparatus claims may also be implemented by a single unit or device in software or hardware. The terms "first," "second," etc., are used to indicate names and do not indicate any particular order.

Claims

1. A method for obtaining target spatial location information of a target object, wherein, The method includes: The method involves acquiring drone images of a target object captured by the camera device of a drone equipment, as well as the camera pose information of the camera device when the drone image was captured, wherein the camera pose information includes the camera position information and camera posture information of the camera device; Obtain the image location information of the corresponding target object in the UAV image; Acquire spatial location information of multiple map points, wherein the spatial location information includes the spatial coordinate information of the corresponding map points in a three-dimensional Cartesian coordinate system; Based on the imaging model of the camera device, the image position information, and the camera posture information, the corresponding target ray vector information is determined; Based on the camera position information, obtain the spatial position information of the optical center of the camera device in the three-dimensional rectangular coordinate system; Based on the target ray vector information, the spatial location information of the plurality of map points, and the spatial location information of the optical center, determine the target map point closest to the target ray from the plurality of map points; Step e involves selecting two map points from the plurality of map points other than the target map point, forming a corresponding spatial triangle with the target map point, and determining the corresponding spatial intersection point based on the target ray and the corresponding spatial triangle. The spatial coordinates of the spatial intersection points are used as the target spatial location information of the target object.

2. The method according to claim 1, wherein, The acquisition of spatial location information of multiple map points includes: Obtain the geographic location information of multiple map points, wherein the geographic location information includes the latitude and longitude information of the corresponding map points in the geographic coordinate system; Obtain the first coordinate transformation information of the geographic coordinate system to the corresponding three-dimensional rectangular coordinate system; Based on the geographical location information of the multiple map points and the first coordinate transformation information, the spatial location information of the multiple map points in the three-dimensional rectangular coordinate system is determined.

3. The method according to claim 2, wherein, The geographic location information also includes the elevation information of the corresponding map points.

4. The method according to claim 3, wherein, The step of obtaining the first coordinate transformation information from the geographic coordinate system to the corresponding three-dimensional rectangular coordinate system includes: Obtain the second coordinate transformation information from the geographic coordinate system to the corresponding geocentric coordinate system; Obtain the third coordinate transformation information from the geocentric coordinate system to the corresponding three-dimensional rectangular coordinate system; Based on the second coordinate transformation information and the third coordinate transformation information, the first coordinate transformation information for transforming the geographic coordinate system to the corresponding three-dimensional rectangular coordinate system is determined.

5. The method according to any one of claims 1 to 4, wherein, The three-dimensional rectangular coordinate system includes any one of the following: Station-centered coordinate system; Navigation coordinate system; Any three-dimensional rectangular coordinate system.

6. The method according to claim 1, wherein, The step of selecting two map points from the plurality of map points other than the target map point includes: From the other map points, select a preset number of map points that are closest to the target map point as surrounding map points, and then select two map points from the surrounding map points.

7. The method according to claim 1, wherein, The step of selecting two map points from the plurality of map points other than the target map point includes: Based on the latitude and longitude of the other map points, a preset number of map points are selected from the other map points with different radii, centered on the target map point, as surrounding map points, and two map points are selected from the surrounding map points.

8. The method according to any one of claims 1, 6, and 7, wherein, The step of using the spatial coordinates of the intersection point as the target spatial location information of the target object includes: If the spatial intersection point is located inside the spatial triangle, then the spatial coordinate information of the spatial intersection point is used as the target spatial location information of the target object.

9. The method according to claim 8, wherein, The step of using the spatial coordinates of the intersection point as the target spatial location information of the target object further includes: If the spatial intersection point is not within the spatial triangle, then poll the other map points and repeat step e until a new spatial intersection point is obtained that is within the corresponding new spatial triangle. Then, use the spatial coordinate information of the new spatial intersection point as the target spatial location information of the target object.

10. The method according to claim 1, wherein, The method further includes: Based on the target spatial location information of the target object, determine and store the geographic coordinate information of the target object in the geographic coordinate system.

11. The method according to claim 1 or 10, wherein, The method further includes: The target label information of the target object is generated based on the target spatial location information and the object description information of the target object.

12. The method according to claim 11, wherein, The method further includes: Acquire real-time drone images captured by the camera device and real-time pose information of the camera device; The real-time image position information of the target object in the real-time UAV image is determined based on the target spatial position information and the real-time pose information. The target label information is overlaid on the real-time drone image based on the real-time image location information.

13. A device for acquiring target spatial location information of a target object, wherein, The device includes: The module is used to acquire drone images of a target object captured by the camera device of a drone equipment and the camera pose information of the camera device when the drone image is captured, wherein the camera pose information includes the camera position information and camera posture information of the camera device; The first and second modules are used to obtain the image location information of the corresponding target object in the UAV image; The first and third modules are used to obtain the spatial location information of multiple map points, wherein the spatial location information includes the spatial coordinate information of the corresponding map points in a three-dimensional rectangular coordinate system; The first four modules are used to determine the corresponding target ray vector information based on the imaging model of the camera device, the image position information, and the camera posture information; obtain the optical center spatial position information of the camera device in the three-dimensional rectangular coordinate system based on the camera position information; determine the target map point closest to the target ray from the multiple map points based on the target ray vector information, the spatial position information of the multiple map points, and the optical center spatial position information; select two map points from the multiple map points other than the target map point, form a corresponding spatial triangle with the target map point, and determine the corresponding spatial intersection point based on the target ray and the corresponding spatial triangle; and use the spatial coordinate information of the spatial intersection point as the target spatial position information of the target object.

14. A computer device, wherein, The device includes: Processor; and A memory configured to store computer-executable instructions, which, when executed, cause the processor to perform the steps of the method as described in any one of claims 1 to 12.

15. A computer-readable storage medium having a computer program / instructions stored thereon, characterized in that, When executed, the computer program / instruction causes the system to perform the steps of the method as described in any one of claims 1 to 12.

16. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method described in any one of claims 1 to 12.

Images