A method and apparatus for acquiring geographic location information of a target object

Abstract

The application aims to provide a method and device for obtaining geographical position information of a target object, comprising: obtaining coordinate transformation information corresponding to a current image currently presented by a display device, wherein the coordinate transformation information comprises first coordinate transformation information corresponding to transformation from a screen coordinate system to a virtual world three-dimensional rectangular coordinate system, and second coordinate transformation information corresponding to transformation from the virtual world three-dimensional rectangular coordinate system to a geographical coordinate system; obtaining screen position information of a target object to be measured in the current image in the screen coordinate system; obtaining corresponding reference data information; and determining geographical position information of the target object according to the screen position information, the first coordinate transformation information, the second coordinate transformation information and the reference data information. The application provides intuitive reference of screen position and actual position for a user, and helps the user to quickly and accurately correspond real geographical position related to the current image and the target object.

Description

Technical Field

[0001] This application relates to the field of communications, and more particularly to a technique for obtaining geographic location information of a target object. Background Technology

[0002] With the advancement and development of technology, new surveillance equipment is widely used in various fields, such as using drones and AR glasses for urban monitoring and patrol. Currently, most surveillance and patrol operations rely on traditional video surveillance. By transmitting the video stream data from the surveillance equipment to the ground control terminal and monitoring system, the real-time status of target objects such as equipment, roads, buildings, and locations within the surveillance footage can be viewed. However, this approach lacks sufficient situational awareness. Viewing information such as video and police force distribution using two-dimensional electronic maps is not intuitive enough, and operational personnel lack a global understanding of the situation, relying solely on personal experience for judgment and unable to determine the actual geographical location of target objects based on their positions within the footage. Summary of the Invention

[0003] One objective of this application is to provide a method and apparatus for obtaining the geographic location information of a target object.

[0004] According to one aspect of this application, a method for obtaining the geographic location information of a target object is provided, wherein the method includes:

[0005] Obtain coordinate transformation information corresponding to the current image currently displayed on the display device, wherein the coordinate transformation information includes first coordinate transformation information of the screen coordinate system to the virtual world three-dimensional rectangular coordinate system, and second coordinate transformation information of the virtual world three-dimensional rectangular coordinate system to the geographic coordinate system;

[0006] Obtain the screen position information of the target object to be measured in the current image in the screen coordinate system;

[0007] Obtain corresponding reference data information, wherein the reference data information includes reference location information or reference distance information;

[0008] The geographical location information of the target object in the geographic coordinate system is determined based on the screen position information, the first coordinate transformation information, the second coordinate transformation information, and the reference data information.

[0009] According to another aspect of this application, a device for acquiring the geographic location information of a target object is provided, wherein the device includes:

[0010] The module is used to obtain coordinate transformation information corresponding to the current image currently displayed on the display device. The coordinate transformation information includes first coordinate transformation information of the screen coordinate system to the virtual world three-dimensional rectangular coordinate system, and second coordinate transformation information of the virtual world three-dimensional rectangular coordinate system to the geographic coordinate system.

[0011] The first and second modules are used to obtain the screen position information of the target object to be measured in the current image in the screen coordinate system;

[0012] The first and third modules are used to obtain the screen position information of the target object to be measured in the current image in the screen coordinate system;

[0013] The first and fourth modules are used to determine the geographical location information of the target object in the geographic coordinate system based on the screen position information, the first coordinate transformation information, the second coordinate transformation information, and the reference data information.

[0014] According to one aspect of this application, a computer device is provided, wherein the device includes:

[0015] Processor; and

[0016] A memory configured to store computer-executable instructions, which, when executed, cause the processor to perform the steps of any of the methods described above.

[0017] According to one aspect of this application, a computer-readable storage medium is provided that stores a computer program / instructions thereon, characterized in that, when executed, the computer program / instructions cause a system to perform the steps of any of the methods described above.

[0018] According to one aspect of this application, a computer program product is provided, comprising a computer program / instructions, characterized in that, when executed by a processor, the computer program / instructions implement the steps of any of the methods described above.

[0019] Compared with the prior art, this application can determine the geographical location information of the target object based on the screen position information of the target object in the current image, thereby obtaining the actual position of the target object while displaying the target object on the display device, providing users with an intuitive reference between the screen position and the actual position, and helping users to quickly and accurately match the current image with the real geographical location related to the target object. Attached Figure Description

[0020] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0021] Figure 1A flowchart of a method for obtaining the geographic location information of a target object according to an embodiment of this application;

[0022] Figure 2 This diagram illustrates a device structure of a computer apparatus according to another embodiment of the present application;

[0023] Figure 3 Exemplary systems that can be used to implement the various embodiments described in this application are shown.

[0024] The same or similar reference numerals in the accompanying drawings represent the same or similar parts. Detailed Implementation

[0025] The present application will now be described in further detail with reference to the accompanying drawings.

[0026] In a typical configuration of this application, the terminal, the device of the service network, and the trusted party all include one or more processors (e.g., a central processing unit (CPU)), input / output interfaces, network interfaces, and memory.

[0027] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash memory. Memory is an example of computer-readable media.

[0028] Computer-readable media, including both permanent and non-permanent, removable and non-removable media, can store information using any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PCM), programmable random access memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0029] The devices referred to in this application include, but are not limited to, user equipment, network equipment, or devices composed of user equipment and network equipment integrated through a network. The user equipment includes, but is not limited to, any mobile electronic product capable of human-computer interaction (e.g., via a touchpad), such as drones, smartphones, and tablets. These mobile electronic products can use any operating system, such as Android or iOS. The network equipment includes an electronic device capable of automatically performing numerical calculations and information processing according to pre-set or stored instructions. Its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), and embedded devices. The network equipment includes, but is not limited to, computers, network hosts, single network servers, multiple network server sets, or clouds composed of multiple servers. Here, a cloud consists of a large number of computers or network servers based on cloud computing, where cloud computing is a type of distributed computing, consisting of a virtual supercomputer composed of a group of loosely coupled computer sets. The network includes, but is not limited to, the Internet, wide area network, metropolitan area network, local area network, VPN network, wireless ad hoc network, etc. Preferably, the device can also be a program running on the user equipment, network device, or a device formed by integrating user equipment and network device, network device, touch terminal, or network device and touch terminal through a network.

[0030] Of course, those skilled in the art should understand that the above-described devices are merely examples, and other existing or future devices that are applicable to this application should also be included within the scope of protection of this application, and are hereby incorporated by reference.

[0031] In the description of this application, "multiple" means two or more, unless otherwise expressly and specifically defined.

[0032] Figure 1A method for obtaining the geographic location information of a target object according to one aspect of this application is illustrated. This method is applied to a computer device and specifically includes steps S101, S102, S103, and S104. In step S101, coordinate transformation information corresponding to the current image currently displayed on the display device is obtained. The coordinate transformation information includes a first coordinate transformation from the screen coordinate system to a virtual world three-dimensional rectangular coordinate system, and a second coordinate transformation from the virtual world three-dimensional rectangular coordinate system to a geographic coordinate system. In step S102, the screen position information of the target object to be measured in the current image in the screen coordinate system is obtained. In step S103, corresponding reference data information is obtained, including reference position information or reference distance information. In step S104, the geographic location information of the target object in the geographic coordinate system is determined based on the screen position information, the first coordinate transformation information, the second coordinate transformation information, and the reference data information. For example, the computer equipment includes, but is not limited to, user equipment, network equipment, or a combination of user equipment and network equipment; wherein, the user equipment includes, but is not limited to, any mobile electronic product that can interact with the user (e.g., through a touchpad), such as smartphones, tablets, drones, smart glasses, smart helmets, etc.; the network equipment includes, but is not limited to, computers, network hosts, single network servers, sets of multiple network servers, or a cloud consisting of multiple servers, such as a ground control center server, etc.; wherein, the drone equipment includes unmanned flying equipment operated using radio remote control equipment and its own program control device.

[0033] Specifically, in step S101, coordinate transformation information corresponding to the current image displayed on the display device is obtained. This coordinate transformation information includes a first coordinate transformation from the screen coordinate system to the virtual world's three-dimensional Cartesian coordinate system, and a second coordinate transformation from the virtual world's three-dimensional Cartesian coordinate system to the geographic coordinate system. For example, the computer device includes a camera device for acquiring image information of the current scene, such as a drone camera, a smart glasses camera, a PTZ surveillance camera, or a mobile phone camera. In some cases, the computer device also includes a corresponding communication device for establishing communication connections with other devices (e.g., user equipment, ground control center, etc.) and transmitting information. The computer device determines the current image by acquiring image information of the current scene based on user operations or preset instructions. For example, a drone flies along a preset flight route, acquiring the ground scene through a camera device during flight to determine the current image; a user wearing smart glasses captures the scene in front of them through a camera device to determine the current image; or a user holding a mobile phone captures the scene around them through the phone's front / rear camera to determine the current image.

[0034] In some cases, when the intrinsic parameters of the camera device are known, the computer can transform from the screen coordinate system corresponding to the display device to the camera coordinate system, thereby determining the spatial coordinates in the corresponding camera coordinate system based on the screen position information. The camera coordinate system is usually considered a special "object" coordinate system defined within the visible area of ​​the camera screen. For example, in the camera coordinate system, the origin is the camera's optical center, the x-axis is to the right, the z-axis is backward (towards the screen), and the y-axis is upward (not above the world but above the camera itself). Furthermore, we can set up a virtual three-dimensional Cartesian coordinate system as an intermediate coordinate system. First, the spatial position in the camera coordinate system is transformed from the corresponding camera coordinate system to the corresponding virtual three-dimensional Cartesian coordinate system, and then from the virtual three-dimensional Cartesian coordinate system to the geographic coordinate system, thereby determining the corresponding geographic location information. The coordinates in the corresponding virtual three-dimensional Cartesian coordinate system (…) ),on the one hand, This refers to altitude, where (Xa, Za) are the coordinate values ​​after projecting latitude and longitude using a projection algorithm. On the other hand, ( The coordinate transformation information refers to the coordinate values ​​transformed based on the screen space coordinate system and the inverse view projection matrix. This transformation includes a first coordinate transformation from the screen coordinate system to the virtual world's three-dimensional rectangular coordinate system, and a second coordinate transformation from the virtual world's three-dimensional rectangular coordinate system to the geographic coordinate system. In some cases, the first coordinate transformation is determined by a third coordinate transformation from the screen coordinate system to the camera coordinate system and a fourth coordinate transformation from the camera coordinate system to the virtual world's three-dimensional rectangular coordinate system. The prefixes "first," "second," "third," "fourth," and subsequent similar terms are used only to describe the transformation information between different coordinate systems and do not involve any order, sequence, or hierarchy. Specifically, the geographic coordinate system includes coordinate systems established based on the Global Positioning System (GPS), represented by latitude and longitude coordinates and altitude information, such as the World Geodetic System-1984 Coordinate System (WGS84), the National Geodetic Coordinate System, and the Xi'an Coordinate System. For example, the corresponding geographic location information is represented by geographic coordinates (…). ) indicates that, where L represents longitude information, This represents latitude information, and H represents altitude (such as elevation). Latitude and longitude information can be obtained through GPS or point location on an electronic map, while altitude information can be obtained through a Digital Elevation Model (DEM) or other methods. Here, geographic coordinates (… The method of obtaining ) is only an example and is not limited here.

[0035] In some embodiments, step S101 includes sub-steps S1011 (not shown) and S1012 (not shown); in step S1011, camera position information, camera posture information, camera parameter information, and corresponding screen parameter information are obtained when the corresponding camera device captures the current image; in step S1012, corresponding coordinate transformation information is determined based on the camera position information, camera posture information, camera parameter information, and screen parameter information, wherein the coordinate transformation information includes first coordinate transformation information of transforming the corresponding screen coordinate system to a virtual world three-dimensional rectangular coordinate system, and second coordinate transformation information of transforming the virtual world three-dimensional rectangular coordinate system to a geographic coordinate system. For example, the computer device also includes a display device for presenting the current image, such as a display screen. When acquiring the current image, the computer device can also acquire the camera pose information corresponding to the camera device. This camera pose information includes the camera position information and camera posture information of the corresponding camera device. The camera position information includes, but is not limited to, the relative or absolute position information of the camera device (e.g., the relative or absolute position information of the origin of the camera coordinate system corresponding to the camera device). The relative position information includes, but is not limited to, the three-dimensional coordinate information of the origin of the camera coordinate system corresponding to the camera device (e.g., the center of the camera device) in the three-dimensional coordinate system. The absolute position information includes the latitude, longitude, and altitude information of the origin of the camera coordinate system corresponding to the camera device (e.g., the center of the camera device). Correspondingly, the computer device can also acquire the screen parameter information corresponding to the display screen of the display device. This screen parameter information indicates the pixel width W and pixel height H corresponding to the display area of ​​the screen. In some embodiments, the pixel width W and pixel height H are the same as the image resolution (e.g., resolution W*H).

[0036] In some cases, ignoring the translation relationship between the corresponding user device (e.g., drone or smart glasses) and the camera device, the real-time geographical location information of the user device can be directly determined as the real-time camera position information of the camera device. In other cases, the real-time geographical location information of the user device when the current image is captured is first determined, and the real-time camera position information of the camera device is determined based on the real-time geographical location information and the translation relationship between the camera device and the user device. The translation relationship includes the translation relationship between the origin of the camera coordinate system corresponding to the camera device and the origin of the coordinate system corresponding to the user device. The camera attitude information of the camera device includes the camera angle information when the camera device is capturing the current image. In some cases, the camera device adjusts the corresponding camera angle information. Specifically, the angle includes yaw / pitch / roll three-axis angle information, where yaw is the yaw angle, pitch is the pitch angle, and roll is the roll angle, etc. When the user device is a drone, the camera angle is provided by the gimbal. When the user device is smart glasses or a mobile phone, the camera angle is obtained through the corresponding attitude sensor (such as a three-axis gyroscope, etc.).

[0037] Here, the camera position information and camera posture information are used to determine the fourth coordinate transformation information from the camera coordinate system to the corresponding virtual world three-dimensional Cartesian coordinate system. The camera parameter information and screen parameter information are used to determine the third coordinate transformation information from the corresponding screen coordinate system to the camera coordinate system. Based on this third and fourth coordinate transformation information, the first coordinate transformation information from the screen coordinate system to the virtual world three-dimensional Cartesian coordinate system is then determined. Specifically, in some embodiments, in step S1012, the third coordinate transformation information from the corresponding screen coordinate system to the camera coordinate system is determined based on the camera parameter information and the screen parameter information. The fourth coordinate transformation information from the camera coordinate system to the corresponding virtual world three-dimensional Cartesian coordinate system is determined based on the camera position information and the camera posture information, thereby determining the first coordinate transformation information from the screen coordinate system to the virtual world three-dimensional Cartesian coordinate system; and the second coordinate transformation information from the virtual world three-dimensional Cartesian coordinate system to the corresponding geographic coordinate system is obtained. For example, we calculate perspective relationships by calibrating the intrinsic parameters of the camera device. The camera device is calibrated using commonly used computer vision calibration algorithms to obtain its camera parameter information when it is not zoomed. This camera parameter information includes, but is not limited to, long focal length, short focal length, long axis optical center offset, and short axis optical center offset. We also obtain the initial zoom size of the camera device, the current image width W, and height H through the application programming interface. The pixel width and height of the current image are the same as the display range of the screen parameter information. In some embodiments, determining the third coordinate transformation information from the corresponding screen coordinate system to the camera coordinate system based on the camera parameter information and the screen parameter information includes: determining the projection transformation information from the screen space coordinate system of the camera device to the corresponding camera coordinate system based on the camera parameter information; determining the fifth coordinate transformation information from the screen coordinate system to the screen space coordinate system based on the screen parameter information; and determining the third coordinate transformation information from the screen coordinate system to the camera coordinate system based on the projection transformation information and the fifth coordinate transformation information. For example, let's assume a far plane... and near plane Assuming the camera supports a zoom-enabled perspective projection matrix, let the relative zoom factor be... Then the perspective projection matrix from the camera coordinate system to the screen space coordinate system can be determined:

[0038] (1)

[0039] When calibrating intrinsic parameters, if the zoom is not 1x, the relative zoom ratio is determined based on the zoom ratio during calibration. For example, if the camera's zoom ratio is 4x during intrinsic parameter calibration and the real-time zoom ratio is 8x, then the relative zoom ratio is... It is 8 / 4 = 2 times. and Represents the focal length of the gimbal camera. and Represents the coordinates of the principal point. , , , This can be obtained through camera calibration. The perspective projection matrix information describes the transformation relationship from the camera coordinate system to the screen space coordinate system (e.g., clipping coordinate system, normalized device coordinate system, etc.). The essence of the perspective projection matrix transformation is to transform the view frustum into clipping space. The view frustum has six faces: near clipping plane, far clipping plane, left clipping plane, right clipping plane, top clipping plane, and bottom clipping plane. All surfaces outside the view frustum are discarded, i.e., clipped. Our subsequent operations are all calculations performed inside the view frustum. When the acquisition device does not support zoom / no zoom / 1x zoom, Substituting into the above equation, we obtain the perspective projection matrix from the camera coordinate system to the screen space coordinate system:

[0040] (2)

[0041] Based on the aforementioned perspective projection matrix, an inverse transformation can be performed to determine the projection transformation information from the screen space coordinate system to the camera coordinate system. Further, the screen space coordinate system is a small segment of space coordinates with x, y, and z values ​​ranging from -1.0 to 1.0. Any coordinates falling outside this range will be discarded / cropped and will not be displayed on the current screen. Based on the aforementioned screen space coordinate system, we can obtain the fifth coordinate transformation information from the screen coordinate system to the corresponding screen space coordinate system. For example, we can obtain the pixel range information (e.g., pixel width and pixel height) of the current image captured by the camera device, determine the coordinate transformation information from the screen coordinate system to the corresponding screen space coordinate system based on the pixel range information, and then determine the third coordinate transformation information from the screen coordinate system to the camera coordinate system based on the fifth coordinate transformation information and the projection transformation information. For example, in screen coordinates... Click to add a tag. At this point, the screen width and height are W and H, and the pixel depth is... Determine the positions of the far plane and near plane in the screen space coordinate system, respectively. Two sets of point positions were obtained. Where the near plane D is 0.0 and the far plane D is 1.0, the specific transformation is as follows:

[0042] 1) Convert screen coordinates to spatial coordinates:

[0043] (3)

[0044] (4)

[0045] (5)

[0046] 2) Map this spatial coordinate system to the screen space coordinate system:

[0047] (6)

[0048] (7)

[0049] (8)

[0050] Based on the above process, the screen coordinates are transformed to the screen space coordinate system to determine the two spatial points on the corresponding far plane and near plane. Subsequent coordinate transformations only require transforming these two spatial points.

[0051] In some implementations, determining the fourth coordinate transformation information of the virtual world three-dimensional coordinate system corresponding to the camera coordinate system transformation based on the camera position information and the camera posture information includes: determining the coordinate offset information of the camera device's camera coordinate system transformation to the virtual world three-dimensional Cartesian coordinate system based on the camera position information, and determining the coordinate rotation information of the camera coordinate system transformation to the virtual world three-dimensional Cartesian coordinate system based on the camera posture information, thereby determining the fourth coordinate transformation information of the virtual world three-dimensional Cartesian coordinate system corresponding to the camera coordinate system transformation. For example, a computer device can obtain the angle information (ϕ, θ, λ) corresponding to a user device, where ϕ, θ, and λ correspond to pitch, yaw, and roll, respectively. Here, we can directly determine this angle information as the camera posture information of the camera device, or perform coordinate axis transformation based on the user device angle information to transform each angle of the angle information to the camera coordinate system to determine the corresponding camera posture information, etc. In some cases, the translation matrix information for transforming the virtual world's three-dimensional Cartesian coordinate system to the camera coordinate system of the camera device is determined based on the camera position information. For example, after transforming the geographic coordinates to the virtual world's three-dimensional Cartesian coordinate system using the inverse transformation of the aforementioned second coordinate transformation information to determine the corresponding coordinates (X, Y, Z), the corresponding translation matrix is ​​as follows:

[0052] (9)

[0053] In other cases, after the computer device determines the camera posture information corresponding to the camera coordinate system, it can determine the rotation matrix information for transforming the virtual world's three-dimensional Cartesian coordinate system to the camera device's coordinate system based on the angle information. For example...

[0054] (10)

[0055] (11)

[0056] (12)

[0057] Based on the aforementioned equations (10)-(12), we can determine the rotation matrix corresponding to the angle information:

[0058] (13)

[0059] The rotation order can vary, and different rotation orders correspond to different rotation matrices; this is not limited here. In some cases, if the gimbal is in a left-handed coordinate system, then the angle information ( After converting the information into angle information in a right-handed coordinate system, the rotation matrix is ​​calculated.

[0060] at this time,

[0061] (14)

[0062] Furthermore, we can determine the transformation matrix information from the virtual world's three-dimensional Cartesian coordinate system to the camera coordinate system of the camera device:

[0063] (15)

[0064] Here, I record This can be understood as the transformation matrix from the virtual world's three-dimensional rectangular coordinate system to the screen space coordinate system. Based on this transformation matrix, the inverse transformation can be performed to determine the transformation matrix from the screen space coordinate system to the virtual world's three-dimensional rectangular coordinate system. Combined with the aforementioned fifth coordinate transformation information from the screen coordinate system to the screen space coordinate system, the first coordinate transformation information from the screen coordinate system to the virtual world's three-dimensional rectangular coordinate system can be determined.

[0065] In some embodiments, obtaining the second coordinate transformation information of the virtual world's three-dimensional Cartesian coordinate system to the corresponding geographic coordinate system includes: determining the second coordinate transformation information of the virtual world's three-dimensional Cartesian coordinate system to the corresponding geographic coordinate system through a projection algorithm. In some embodiments, the second coordinate transformation information of the virtual world's three-dimensional Cartesian coordinate system to the corresponding geographic coordinate system can be determined by the inverse algorithm of a preset projection algorithm, wherein the projection algorithm includes, but is not limited to, Mercator projection, web Mercator projection, universal Mercator projection, Gaussian projection, Miller projection, Lambert projection, and geocentric coordinate system with the geocenter as the origin without projection. The following embodiments are illustrated using Mercator projection as an example, and those skilled in the art should understand that these embodiments are also applicable to other projection algorithms. Specifically, the coordinates in the virtual world's three-dimensional Cartesian coordinate system ( Convert to geographic coordinates ):

[0066] (16)

[0068] (17)

[0069] (18)

[0071] in,

[0072] (19)

[0074] In the aforementioned equations (16)-(19), a is the semi-major axis of the ellipsoid, and the WGS84 ellipsoid parameter is 6378137;

[0075] b is the minor semi-axis of the ellipsoid, and the WGS84 ellipsoid parameter is taken as 6356752.3142;

[0076] f is the flattening, i.e. (ab) / a;

[0077] e is the first eccentricity, i.e. ;

[0078] The second eccentricity, i.e. ;

[0079] N is the radius of curvature of the zonal loop, i.e. ;

[0080] R is the radius of curvature of the meridian, i.e. ;

[0081] Latitude Longitude;

[0082] Using rectangular coordinates, The vertical coordinate is a rectangular coordinate. For height coordinates;

[0083] And the Mercator projection inverse solution formula: ( )→( , ), origin latitude B0, origin longitude L0.

[0084] In step S102, the screen position information of the target object to be measured in the current image within the screen coordinate system is obtained. The target object can be any object of user interest. For example, the target object could be an object determined based on a specified user operation (such as clicking or selecting) in the current image, such as a building or street on the ground; or it could be any spatial point, line segment, or region, which could be a geographical location, street location, building location, or any other arbitrary spatial point, line segment, or region. Alternatively, the target object could be an object identified in the current image based on relevant template features identified in a database, such as a landmark or target marker on the ground. In some cases, the screen position information of the target object in the screen coordinate system can be determined by the pixel range of the corresponding display screen in the display device, and based on the template information or selected position of the target object in the current image. In some cases, the marker point or spatial point may carry corresponding indication information (e.g., location indication, name indication, or other operation indication). The screen coordinate system refers to a two-dimensional planar coordinate system defined by the entire screen of the display device. For example, a two-dimensional rectangular coordinate system with the upper left corner of the display screen as the origin, the X-axis as positive to the right, and the Y-axis as positive downwards. The X and Y coordinates in the corresponding screen coordinate information are the corresponding planar coordinate information.

[0085] In step S103, corresponding reference data information is obtained, wherein the reference data information includes reference location information or reference distance information. For example, since this solution can only obtain the screen location information of the target object, which is the planar coordinate information of the target object on the screen (e.g., the coordinates of a certain plane point or the coordinates of a set of points), in order to facilitate the calculation of the geographical location of the target object, in the process of transforming the target object from the screen coordinate system to the corresponding geographic coordinate system, we need to establish a corresponding reference benchmark, for example, through multiple known geographic location information of the positioning points, or the distance information of the target object or a certain known positioning point from the camera device, etc.

[0086] In step S104, the geographical location information of the target object in the geographic coordinate system is determined based on the screen position information, the first coordinate transformation information, the second coordinate transformation information, and the reference data information. For example, after the computer device obtains the corresponding screen position information, it can determine the ray information of the screen point corresponding to the screen position information in the virtual world's three-dimensional Cartesian coordinate system based on the aforementioned first coordinate transformation information. Then, based on this ray information and the reference data information, the target point of the screen point in the virtual world's three-dimensional Cartesian coordinate system is determined. Finally, based on the second coordinate transformation information, the spatial coordinates of the target point are transformed to the geographic coordinate system to determine the geographical location information of the target object.

[0087] In some cases, when the reference data information is the geographical location information of multiple elevation points, we can build a surface model through the elevation point data to calculate the intersection of the corresponding ray information and the surface, thereby determining the spatial coordinates of the target point and thus the geographical location information of the target object. For example, in some embodiments, in step S103, the geographical location information of multiple elevation points in the relevant area of ​​the current image is obtained, wherein the geographical location information corresponds to the geographic coordinate system; wherein, in step S104, the geographical location information of the target object in the geographic coordinate system is determined based on the screen position information, the first coordinate transformation information, the second coordinate transformation information, and the geographical location information of the multiple elevation points. For example, a computer device loads elevation data (e.g., 1000m or 5000m) of a certain range near the current image collected by the user device according to business needs. This can be achieved by loading the geographical location information of multiple elevation points corresponding to the Digital Elevation Model (DEM) or Digital Surface Model (DSM) of the nearby area. Taking the geographical location information of one elevation point as an example, the corresponding geographical location information is represented by (…). The computer device uses the inverse transformation of the second coordinate transformation information to convert the geographic coordinate system to a virtual three-dimensional Cartesian coordinate system, representing the geographic location information of multiple elevation points. Specifically, the geographic coordinates ( The coordinates are converted to a virtual three-dimensional Cartesian coordinate system, and the converted coordinates are (X,Y,Z).

[0088] Location information obtained from sensors such as GPS is in degrees, while the Mercator algorithm uses radians for latitude and longitude. First, convert the angles to radians before performing the Mercator algorithm: Radius (rad) = (degrees / 180)π

[0089] (20)

[0090] (twenty one)

[0091] (twenty two)

[0092] in,

[0093] a -- semi-major axis of the ellipsoid, b -- semi-minor axis of the ellipsoid, f -- flattening (ab) / a, e -- first eccentricity , -- Second eccentricity N -- Radius of curvature of the zonal loop R -- radius of curvature of the meridional circle ,

[0094] B -- latitude, L -- longitude, in radians (RAD)

[0095] Z -- Vertical rectangular coordinate, X -- Horizontal rectangular coordinate, Y -- Coordinate, unit: meters (m)

[0096] Mercator projection forward solution formula: (B,L)→(X,Z), origin latitude B0, origin longitude L0

[0097] The elevation data of the surrounding area can be determined according to preset rules, such as elevation points within a certain range centered on the camera device, or within a certain range centered on the image center of the current image, or by determining the elevation points of the area to which the current image acquisition area falls based on preset area divisions. The computer equipment can determine the corresponding surface model based on the geographical location information of multiple elevation points, thereby determining the intersection point between the ray information corresponding to the screen position information and the surface model, and converting the coordinates of the corresponding intersection point to a geographic coordinate system to determine the geographical location information of the target object.

[0098] In some embodiments, step S104 includes sub-steps S1041 (not shown), S1042 (not shown), S1043 (not shown), and S1044 (not shown); in step S1041, the ray information of the target object in the three-dimensional rectangular coordinate system of the virtual world is determined according to the screen position information and the first coordinate transformation information; in step S1042, the multiple triangular mesh information corresponding to the three-dimensional rectangular coordinate system of the virtual world is determined according to the inverse transformation of the second coordinate transformation information and the geographical location information of the multiple elevation points; in step S1043, the corresponding intersection point coordinate information is determined according to the ray information and the multiple triangular mesh information; in sub-step S1044, the intersection point coordinate information is transformed to the geographic coordinate system according to the second coordinate transformation information, thereby determining the geographical location information of the target object. For example, after a computer loads DEM data and converts it to determine the spatial coordinates of multiple elevation points within a certain range in the virtual world's three-dimensional Cartesian coordinate system, it can generate corresponding triangular mesh information (e.g., N triangles composed of multiple elevation points) based on these spatial coordinates. For instance, it can generate an irregular triangular network (TIN), etc. The specific method for generating this triangular mesh information is not limited. We can use this triangular mesh information as the corresponding surface model, and then calculate the corresponding intersection points based on the ray information and the triangular mesh information. The intersection point closest to the camera device is determined as the intersection point between the target object and the surface, thus determining the geographical location information of the target object based on the second coordinate transformation.

[0099] In some implementations, the first coordinate transformation information includes fifth coordinate transformation information (transforming the screen coordinate system to the corresponding screen space coordinate system), projection transformation information (transforming the screen space coordinate system to the corresponding camera coordinate system), and fourth coordinate transformation information (transforming the camera coordinate system to the corresponding virtual world three-dimensional Cartesian coordinate system). Specifically, in step S1041, the intersection points of the target object with the far plane and the near plane in the screen space coordinate system are determined based on the screen position information. The intersection points are then transformed to the virtual world three-dimensional Cartesian coordinate system based on the projection transformation information and the fourth coordinate transformation information, determining the corresponding far plane space points and near plane space points. Corresponding ray information is determined based on the point coordinates of the far plane space points and the near plane space points, wherein the origin of the ray is the near plane space point. For example, a computer device determines two sets of screen space coordinates corresponding to the far plane and the near plane based on the plane position corresponding to the screen position information, using the aforementioned transformation relationship from the screen coordinate system to the screen space coordinate system. Then, it is converted to three-dimensional Cartesian coordinates in the virtual world, thus obtaining two sets of spatial point positions. The specific process is as follows:

[0100] = (twenty three)

[0101] (twenty four)

[0102] (25)

[0103] (26)

[0104] Based on the above coordinate transformation, the spatial coordinates of the far-plane point can be determined. ), spatial coordinates of near-plane points ( ), and then determine the ray direction of the corresponding ray information based on the two spatial points ( The origin of this ray information is a near-plane point:

[0105] (27)

[0106] (28)

[0107] (29)

[0108] In some cases, for ease of calculation, computer equipment can also specify the direction of the ray ( Normalize the corresponding three-dimensional vector to determine the unit vector corresponding to the ray information.

[0109] In some implementations, in step S1043, one or more bounding box information corresponding to the plurality of triangular mesh information are determined; intersection detection is performed sequentially based on the ray information and the one or more bounding box information; if the ray information intersects with a certain bounding box information among the one or more bounding box information, then the certain bounding box information is determined as the target bounding box information; the corresponding intersection point coordinate information is determined based on the target bounding box information and the ray information. For example, since there are many irregular triangular meshes, the computational workload of sequentially determining whether a ray intersects with each triangular mesh is large. We can take one or more bounding box information corresponding to the triangular mesh and reduce the corresponding computation by determining the intersection between the bounding box and the ray, thereby improving the corresponding detection efficiency. For example, based on the multiple triangular mesh information, the bounding box information corresponding to the multiple triangular meshes is determined, such as a circumscribed cuboid, a sphere, etc. Specifically, the corresponding axis-aligned bounding box (AABB box) is determined based on the multiple triangular meshes. The purpose of dividing the terrain into AABB boxes is to divide the entire terrain into N*M blocks for simple detection. AABB box division methods include: 1) Max-min representation: using a top-right and bottom-left corner to uniquely define a bounding box; 2) Center-radial representation: using the center point to represent the midpoint, and a radial array storing the radius of the bounding box in the x, y, and z directions; 3) Min-Width representation: using the min point to define the bottom-left corner, and the width to store the length in the x, y, and z directions. Each bounding box can correspond one-to-one with a specific triangle, or multiple triangles can be divided into meshes using bounding boxes of the same or different sizes. The computer device sequentially determines whether the ray information intersects with at least one AABB box. If the corresponding AABB box does not intersect, the triangular mesh contained in that AABB box is rejected, meaning that the triangular mesh contained in that AABB box does not have any points intersecting with the ray information, and no further calculation of the triangular mesh contained in that AABB box and the ray information is required. If an AABB intersects with a ray, the intersecting AABB box is recorded as target bounding box information, and further calculation of the triangular mesh contained in that AABB box and the ray information is required subsequently. In some embodiments, the computer device first determines the target bounding box information intersecting with the ray based on the ray information and one or more corresponding bounding box information, and then determines at least one intersection point between the corresponding triangular mesh and the ray based on the triangular mesh in the target bounding box information, thereby determining the final intersection point from at least one intersection point, etc. In other embodiments, the computer device determines one or more first-level bounding box information corresponding to the multiple triangular meshes based on the multiple triangular mesh information, and then determines one or more second-level bounding box information corresponding to the first-level bounding box information.The computer device first determines the target second-level bounding box information intersecting with the ray based on the ray information and one or more corresponding second-level bounding box information. Then, based on the ray information and the target second-level bounding box information, it determines the target first-level bounding box information intersecting with the ray. Next, based on the triangular mesh in the target first-level bounding box information, it determines at least one intersection point between the corresponding triangular mesh and the ray, thereby determining the final intersection point from at least one intersection point. In some embodiments, determining the corresponding intersection point coordinate information based on the target bounding box information and the ray information includes: determining at least one intersection point based on the ray information and the target triangular mesh information corresponding to the target bounding box information; determining the intersection point closest to the camera device from the at least one intersection point; and determining the spatial coordinate information of the intersection point as the corresponding intersection point coordinate information. For example, after the computer device determines the AABB box intersecting with the ray, it reads the triangular mesh contained in the intersecting AABB box and determines the triangular mesh information contained in the intersecting AABB box as the target triangular mesh information. Furthermore, the computer device sequentially determines the intersections of the target triangular mesh information and the rays (for example, using Badouel's simple strategy to first determine the intersections of the rays and the plane containing the triangles), thereby determining one or more corresponding intersection points. If there is only one intersection point, the computer device can directly determine the spatial coordinates of that intersection point as the intersection point coordinate information. If there are multiple intersection points, the intersection point closest to the camera device is taken as the corresponding intersection point. For example, the computer device takes the spatial coordinates of the intersection point closest to the camera device as the spatial coordinates in the virtual world's three-dimensional Cartesian coordinate system corresponding to the screen coordinates. Subsequently, based on the second coordinate transformation information, the spatial coordinates in the three-dimensional Cartesian coordinate system of the virtual world are... The coordinates are transformed to a geographic coordinate system to determine the geographic location information of the target object. Specifically, in some embodiments, determining at least one intersection point based on the target triangle mesh information corresponding to the ray information and the target bounding box information includes: sequentially performing intersection detection between the ray information and one or more target triangle mesh information; if the ray information intersects the plane of a target triangle mesh information among the one or more target triangle mesh information, and the corresponding intersection point is inside the target triangle mesh information, then the intersection point is determined as the corresponding intersection point to obtain at least one corresponding intersection point. For example, after the computer device determines the corresponding target triangle mesh information, it judges the ray information with each target triangle mesh information, first determining whether the ray information intersects the plane of the target triangle mesh information; if it intersects, it calculates the intersection point of the ray information and the plane, and determines whether the intersection point is inside the target triangle mesh information (e.g., calculating whether the centroid coordinates of the intersection point are inside the target triangle); if so, the intersection point is determined as the intersection point; otherwise, the target triangle mesh information is discarded.

[0110] In some cases, the computer device further includes a laser rangefinder for measuring the distance information between the target object and the camera device, using the corresponding distance information as reference data. In some embodiments, the camera device and the corresponding laser rangefinder are arranged parallel to each other; wherein, in step S102, the center coordinate information of the current image at the center of the screen of the display device is determined as the screen position information of the target object to be measured in the screen coordinate system; wherein, in step S103, the distance information between the target object and the camera device measured by the rangefinder is obtained; wherein, in step S104, the geographical location information of the target object in the geographic coordinate system is determined based on the screen position information, the first coordinate transformation information, the second coordinate transformation information, and the distance information. For example, we can set the laser rangefinder and the camera device in parallel. In other words, the laser beam of the laser rangefinder is parallel to the optical axis of the camera device. To simplify the calculation process, we can directly determine the center coordinates of the current image at the center of the screen as the screen position information of the target object to be measured in the screen coordinate system. Then, based on the screen position information, we can determine the corresponding ray information, and based on the ray information, distance information, etc., we can determine the spatial coordinates of the corresponding target object in the three-dimensional rectangular coordinate system of the virtual world. Then, based on the second coordinate transformation information, we can transform it to the geographic coordinate system to determine the geographic location information of the target object. Specifically, in some embodiments, determining the geographic location information of the target object in the geographic coordinate system based on the screen position information, the first coordinate transformation information, the second coordinate transformation information, and the distance information includes: determining the ray information of the target object in the virtual world's three-dimensional Cartesian coordinate system based on the screen position information and the first coordinate transformation information, wherein the ray information includes ray direction and origin information; determining the target spatial position information of the target object in the virtual world's three-dimensional Cartesian coordinate system based on the ray information and the distance information; and determining the geographic location information of the target object in the geographic coordinate system based on the target spatial position information and the second coordinate transformation information. For example, based on the aforementioned process of determining ray information, the origin and direction of the ray in the virtual world's three-dimensional Cartesian coordinate system can be calculated using the current image center as a reference point, wherein the origin is a near-plane point, and the distance information successfully determined by the laser ranging device is recorded as... Then the spatial coordinates of the ranging target ( )calculate:

[0111] (30)

[0112] (31)

[0113] (32)

[0114] Computer equipment determines the spatial location of the intersection point between the center of the screen and the ground surface. After that, it can be transformed to a geographic coordinate system based on the second coordinate transformation information to determine the corresponding geographic location information. In some cases, computer devices can store the geographic location information of the target object in a geographic information database and generate corresponding geographic tag information, which may include the geographic location information and / or spatial location information of the target object.

[0115] The foregoing mainly described various embodiments of a method for obtaining the geographic location information of a target object, representing one aspect of this application. Furthermore, this application also provides specific devices capable of implementing the above embodiments. Below, we will combine... Figure 2 Let me introduce it.

[0116] Figure 2 A computer device 100 for acquiring geographic location information of a target object according to one aspect of this application is shown, specifically including a first module 101, a second module 102, a third module 103, and a fourth module 104. The first module 101 is used to acquire coordinate transformation information corresponding to the current image currently displayed on the display device, wherein the coordinate transformation information includes first coordinate transformation information corresponding to the transformation from the screen coordinate system to a virtual world three-dimensional rectangular coordinate system, and second coordinate transformation information corresponding to the transformation from the virtual world three-dimensional rectangular coordinate system to a geographic coordinate system; the second module 102 is used to acquire the screen position information of the target object to be measured in the current image in the screen coordinate system; the third module 103 is used to acquire corresponding reference data information, wherein the reference data information includes reference position information or reference distance information; the fourth module 104 is used to determine the geographic location information of the target object in the geographic coordinate system based on the screen position information, the first coordinate transformation information, the second coordinate transformation information, and the reference data information.

[0117] In some embodiments, the module 101 includes a first unit (not shown) and a second unit (not shown); the first unit is used to acquire camera position information, camera posture information, camera parameter information, and corresponding screen parameter information when the corresponding camera device captures the current image; the second unit is used to determine corresponding coordinate transformation information based on the camera position information, camera posture information, camera parameter information, and screen parameter information, wherein the coordinate transformation information includes first coordinate transformation information of transforming the corresponding screen coordinate system to the virtual world three-dimensional rectangular coordinate system, and second coordinate transformation information of transforming the virtual world three-dimensional rectangular coordinate system to the geographic coordinate system.

[0118] In some implementations, unit 112 is used to determine, based on the camera parameter information and the screen parameter information, the third coordinate transformation information corresponding to the transformation from the screen coordinate system to the camera coordinate system, and based on the camera position information and the camera posture information, the fourth coordinate transformation information corresponding to the transformation from the camera coordinate system to the virtual world three-dimensional rectangular coordinate system, so as to determine the first coordinate transformation information of the transformation from the screen coordinate system to the virtual world three-dimensional rectangular coordinate system; and to obtain the second coordinate transformation information of the transformation from the virtual world three-dimensional rectangular coordinate system to the corresponding geographic coordinate system.

[0119] In some embodiments, determining the third coordinate transformation information from the corresponding screen coordinate system to the camera coordinate system based on the camera parameter information and the screen parameter information includes: determining homogeneous perspective projection transformation information from the screen space coordinate system of the camera device to the corresponding camera coordinate system based on the camera parameter information; determining fifth coordinate transformation information from the screen coordinate system to the screen space coordinate system based on the screen parameter information; and determining the third coordinate transformation information from the screen coordinate system to the camera coordinate system based on the homogeneous perspective projection transformation information and the fifth coordinate transformation information.

[0120] In some embodiments, determining the fourth coordinate transformation information of the virtual world three-dimensional spatial coordinate system corresponding to the camera coordinate system transformation based on the camera position information and the camera posture information includes: determining the coordinate offset information of the camera device's camera coordinate system transformation to the virtual world three-dimensional rectangular coordinate system based on the camera position information, and determining the coordinate rotation information of the camera coordinate system transformation to the virtual world three-dimensional rectangular coordinate system based on the camera posture information, so as to determine the fourth coordinate transformation information of the virtual three-dimensional rectangular coordinate system corresponding to the camera coordinate system transformation.

[0121] In some implementations, obtaining the second coordinate transformation information from the virtual world's three-dimensional Cartesian coordinate system to the corresponding geographic coordinate system includes: determining the second coordinate transformation information from the virtual world's three-dimensional Cartesian coordinate system to the corresponding geographic coordinate system through a projection algorithm.

[0122] In some implementations, module 103 is used to acquire geographic location information of multiple elevation points in the relevant area of ​​the current image, wherein the geographic location information corresponds to the geographic coordinate system; and module 104 is used to determine the geographic location information of the target object in the geographic coordinate system based on the screen position information, the first coordinate transformation information, the second coordinate transformation information, and the geographic location information of the multiple elevation points.

[0123] In some embodiments, the four-module 104 includes a four-one unit (not shown), a four-two unit (not shown), a four-three unit (not shown), and a four-four unit (not shown); the four-one unit is used to determine the ray information of the target object in the three-dimensional rectangular coordinate system of the virtual world based on the screen position information and the first coordinate transformation information; the four-two unit is used to determine the multiple triangular mesh information corresponding to the three-dimensional rectangular coordinate system of the virtual world based on the inverse transformation of the second coordinate transformation information and the geographical location information of the multiple elevation points; the four-three unit is used to determine the corresponding intersection point coordinate information based on the ray information and the multiple triangular mesh information; the four-four unit is used to transform the intersection point coordinate information to the geographic coordinate system based on the second coordinate transformation information, thereby determining the geographical location information of the target object.

[0124] In some implementations, the first coordinate transformation information includes fifth coordinate transformation information (transforming the screen coordinate system to the corresponding screen space coordinate system), homogeneous perspective projection transformation information (transforming the screen space coordinate system to the corresponding camera coordinate system), and fourth coordinate transformation information (transforming the camera coordinate system to the corresponding virtual world three-dimensional rectangular coordinate system). The unit 141 is used to determine the intersection points of the target object with the far plane and the near plane in the screen space coordinate system based on the screen position information; to transform the intersection points of the far plane and the near plane to the virtual world three-dimensional rectangular coordinate system based on the homogeneous perspective projection transformation information and the fourth coordinate transformation information; to determine the corresponding far plane spatial point and the near plane spatial point; and to determine the corresponding ray information based on the point coordinates of the far plane spatial point and the near plane spatial point, wherein the origin of the ray is the near plane spatial point.

[0125] In some implementations, a 1-4-3 unit is used to determine one or more bounding box information corresponding to the plurality of triangular mesh information; to perform intersection detection sequentially between the ray information and the one or more bounding box information; if the ray information intersects with a certain bounding box information among the one or more bounding box information, then the certain bounding box information is determined as the target bounding box information; and to determine the corresponding intersection point coordinate information based on the target bounding box information and the ray information.

[0126] In some implementations, determining the corresponding intersection coordinate information based on the target bounding box information and the ray information includes: determining at least one intersection point based on the ray information and the target triangle mesh information corresponding to the target bounding box information; determining the intersection point closest to the camera device from the at least one intersection point; and determining the spatial coordinate information of the intersection point as the corresponding intersection coordinate information.

[0127] In some implementations, determining at least one intersection point based on the target triangle mesh information corresponding to the ray information and the target bounding box information includes: sequentially performing intersection detection between the ray information and one or more target triangle mesh information in the target triangle mesh information; if the ray information intersects with the plane containing a certain target triangle mesh information in the one or more target triangle mesh information, and the corresponding intersection point is inside the certain target triangle mesh information, then the intersection point is determined as the corresponding intersection point, so as to obtain at least one corresponding intersection point.

[0128] In some embodiments, the camera device and the corresponding laser rangefinder are arranged in parallel; wherein, module 102 is used to determine the center coordinate information of the current image at the center of the screen of the display device as the screen position information of the target object to be measured in the screen coordinate system; wherein, module 103 is used to obtain the distance information between the target object and the camera device measured by the rangefinder; wherein, module 104 is used to determine the geographical location information of the target object in the geographic coordinate system based on the screen position information, the first coordinate transformation information, the second coordinate transformation information, and the distance information.

[0129] In some embodiments, determining the geographic location information of the target object in the geographic coordinate system based on the screen position information, the first coordinate transformation information, the second coordinate transformation information, and the distance information includes: determining the ray information of the target object in the virtual world's three-dimensional Cartesian coordinate system based on the screen position information and the first coordinate transformation information, wherein the ray information includes ray direction and origin information; determining the target spatial position information of the target object in the virtual world's three-dimensional Cartesian coordinate system based on the ray information and the distance information; and determining the geographic location information of the target object in the geographic coordinate system based on the target spatial position information and the second coordinate transformation information.

[0130] Here, the specific implementation methods corresponding to the first module 101, the second module 102, the third module 103 and the fourth module 104 are the same as or similar to the embodiments of the aforementioned steps S101, S102, S103 and S104, and therefore will not be repeated here, but are included by reference.

[0131] In addition to the methods and devices described in the above embodiments, this application also provides a computer-readable storage medium storing computer code that, when executed, performs the method described in any of the preceding embodiments.

[0132] This application also provides a computer program product that, when executed by a computer device, performs the method described in any of the preceding claims.

[0133] This application also provides a computer device, the computer device comprising:

[0134] One or more processors;

[0135] Memory, used to store one or more computer programs;

[0136] When the one or more computer programs are executed by the one or more processors, the one or more processors cause the one or more processors to perform the method as described in any of the preceding methods.

[0137] Figure 3 Exemplary systems that can be used to implement the various embodiments described in this application are shown;

[0138] like Figure 3 As shown in some embodiments, system 300 can function as any of the aforementioned devices in each of the described embodiments. In some embodiments, system 300 may include one or more computer-readable media having instructions (e.g., system memory or NVM / storage device 320) and one or more processors (e.g., one or more processors 305) coupled to the one or more computer-readable media and configured to execute the instructions to implement the module and thus perform the actions described in this application.

[0139] In one embodiment, the system control module 310 may include any suitable interface controller to provide any suitable interface to at least one of the processors 305 and / or any suitable device or component communicating with the system control module 310.

[0140] The system control module 310 may include a memory controller module 330 to provide an interface to the system memory 315. The memory controller module 330 may be a hardware module, a software module, and / or a firmware module.

[0141] System memory 315 can be used, for example, to load and store data and / or instructions for system 300. In one embodiment, system memory 315 may include any suitable volatile memory, such as suitable DRAM. In some embodiments, system memory 315 may include double data rate type quad synchronous dynamic random access memory (DDR4 SDRAM).

[0142] In one embodiment, the system control module 310 may include one or more input / output (I / O) controllers to provide interfaces to the NVM / storage device 320 and (one or more) communication interfaces 325.

[0143] For example, NVM / storage device 320 can be used to store data and / or instructions. NVM / storage device 320 may include any suitable non-volatile memory (e.g., flash memory) and / or may include any suitable (one or more) non-volatile storage devices (e.g., one or more hard disk drives (HDDs), one or more optical disc drives (CDs), and / or one or more digital universal optical disc (DVD) drives).

[0144] NVM / storage device 320 may include storage resources that are physically part of a device on which system 300 is mounted, or that can be accessed by the device without necessarily being part of it. For example, NVM / storage device 320 may be accessed via a network through one or more communication interfaces 325.

[0145] One or more communication interfaces 325 may provide the system 300 with an interface to communicate over one or more networks and / or with any other suitable device. The system 300 may wirelessly communicate with one or more components of a wireless network in accordance with any of one or more wireless network standards and / or protocols.

[0146] In one embodiment, at least one of the processors 305 may be logically packaged with one or more controllers of the system control module 310 (e.g., memory controller module 330). In one embodiment, at least one of the processors 305 may be logically packaged with one or more controllers of the system control module 310 to form a system-in-package (SiP). In one embodiment, at least one of the processors 305 may be integrated with the logic of one or more controllers of the system control module 310 on the same die. In one embodiment, at least one of the processors 305 may be integrated with the logic of one or more controllers of the system control module 310 on the same die to form a system-on-a-chip (SoC).

[0147] In various embodiments, system 300 may be, but is not limited to, a server, workstation, desktop computing device, or mobile computing device (e.g., laptop computing device, handheld computing device, tablet computer, netbook, etc.). In various embodiments, system 300 may have more or fewer components and / or different architectures. For example, in some embodiments, system 300 includes one or more cameras, a keyboard, a liquid crystal display (LCD) screen (including a touchscreen display), a non-volatile memory port, multiple antennas, a graphics chip, an application-specific integrated circuit (ASIC), and a speaker.

[0148] It should be noted that this application can be implemented in software and / or a combination of software and hardware, for example, using an application-specific integrated circuit (ASIC), a general-purpose computer, or any other similar hardware device. In one embodiment, the software program of this application can be executed by a processor to implement the steps or functions described above. Similarly, the software program of this application (including related data structures) can be stored in a computer-readable recording medium, such as RAM memory, a magnetic or optical drive, a floppy disk, or similar devices. Furthermore, some steps or functions of this application can be implemented in hardware, for example, as circuitry that cooperates with a processor to perform the various steps or functions.

[0149] Furthermore, a portion of this application can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to this application through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions, or the computer compiling the instructions and then executing the corresponding compiled program, or the computer reading and executing the instructions, or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0150] Communication media include media through which communication signals containing, for example, computer-readable instructions, data structures, program modules, or other data are transmitted from one system to another. Communication media can include guided transmission media (such as cables and wires (e.g., optical fibers, coaxial cables, etc.)) and wireless (unguided transmission) media capable of propagating energy waves, such as sound, electromagnetic, RF, microwave, and infrared. Computer-readable instructions, data structures, program modules, or other data can be embodied as modulated data signals in, for example, wireless media (such as carrier waves or similar mechanisms embodied as part of spread spectrum technology). The term "modulated data signal" refers to a signal whose one or more characteristics are altered or set in a manner that encodes information in the signal. Modulation can be analog, digital, or a hybrid modulation technique.

[0151] By way of example and not limitation, computer-readable storage media may include volatile and non-volatile, removable and non-removable media implemented by any method or technique for storing information such as computer-readable instructions, data structures, program modules or other data. For example, computer-readable storage media include, but are not limited to, volatile memories such as random access memory (RAM, DRAM, SRAM); and non-volatile memories such as flash memory, various read-only memories (ROM, PROM, EPROM, EEPROM), magnetic and ferromagnetic / ferroelectric memories (MRAM, FeRAM); and magnetic and optical storage devices (hard disks, magnetic tapes, CDs, DVDs); or other media now known or hereafter developed capable of storing computer-readable information / data for use by a computer system.

[0152] Herein, one embodiment of this application includes an apparatus comprising a memory for storing computer program instructions and a processor for executing the program instructions, wherein when the computer program instructions are executed by the processor, the apparatus is triggered to run a method and / or technical solution based on the foregoing embodiments of this application.

[0153] It will be apparent to those skilled in the art that this application is not limited to the details of the exemplary embodiments described above, and that this application can be implemented in other specific forms without departing from the spirit or essential characteristics of this application. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of this application is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be embraced within this application. No reference numerals in the claims should be construed as limiting the scope of the claims. Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices recited in the apparatus claims may also be implemented by a single unit or device in software or hardware. The terms "first," "second," etc., are used to indicate names and do not indicate any particular order.

Claims

1. A method for obtaining the geographic location information of a target object, wherein, The method includes: Obtain coordinate transformation information corresponding to the current image currently displayed on the display device, wherein the coordinate transformation information includes first coordinate transformation information of the screen coordinate system to the virtual world three-dimensional rectangular coordinate system, and second coordinate transformation information of the virtual world three-dimensional rectangular coordinate system to the geographic coordinate system; Obtain the screen position information of the target object to be measured in the current image in the screen coordinate system; Obtain corresponding reference data information, wherein the reference data information includes reference location information or reference distance information, the reference location information includes the geographical location information of a known positioning point, and the reference distance information includes the distance information between the target object or the known positioning point and the camera device; The geographical location information of the target object in the geographic coordinate system is determined based on the screen position information, the first coordinate transformation information, the second coordinate transformation information, and the reference data information; The step of determining the geographic location information of the target object in the geographic coordinate system based on the screen location information, the first coordinate transformation information, the second coordinate transformation information, and the reference data information includes: Based on the first coordinate transformation information, the ray information of the screen point corresponding to the screen position information in the virtual world three-dimensional rectangular coordinate system is determined. Based on the ray information and the reference data information, the target point of the screen point in the virtual world three-dimensional rectangular coordinate system is determined. Based on the second coordinate transformation information, the spatial coordinates of the target point are transformed to the geographic coordinate system to determine the geographic location information of the target object.

2. The method according to claim 1, wherein, The step of obtaining the coordinate transformation information corresponding to the current image currently displayed on the display device includes: Obtain the camera position information, camera posture information, camera parameter information, and corresponding screen parameter information when the corresponding camera device captures the current image; Based on the camera position information, camera posture information, camera parameter information, and screen parameter information, corresponding coordinate transformation information is determined. The coordinate transformation information includes a first coordinate transformation information that transforms the corresponding screen coordinate system to a virtual world three-dimensional rectangular coordinate system, and a second coordinate transformation information that transforms the virtual world three-dimensional rectangular coordinate system to a geographic coordinate system.

3. The method according to claim 2, wherein, The step of determining the corresponding coordinate transformation information based on the camera position information, camera posture information, camera parameter information, and screen parameter information includes: Based on the camera parameter information and the screen parameter information, the third coordinate transformation information corresponding to the transformation from the screen coordinate system to the camera coordinate system is determined. Based on the camera position information and the camera posture information, the fourth coordinate transformation information corresponding to the transformation of the camera coordinate system to the virtual world three-dimensional rectangular coordinate system is determined. This is used to determine the first coordinate transformation information of the transformation from the screen coordinate system to the virtual world three-dimensional rectangular coordinate system. Obtain the second coordinate transformation information from the virtual world's three-dimensional rectangular coordinate system to the corresponding geographic coordinate system.

4. The method according to claim 3, wherein, The step of determining the third coordinate transformation information from the screen coordinate system to the camera coordinate system based on the camera parameter information and the screen parameter information includes: Based on the camera parameter information, determine the projection transformation information of the camera device from the screen space coordinate system to the corresponding camera coordinate system; The fifth coordinate transformation information, which transforms the screen coordinate system to the screen space coordinate system, is determined based on the screen parameter information. The third coordinate transformation information, which transforms the screen coordinate system to the camera coordinate system, is determined based on the projection transformation information and the fifth coordinate transformation information.

5. The method according to claim 3 or 4, wherein, The step of determining the fourth coordinate transformation information of the virtual world three-dimensional space coordinate system corresponding to the camera coordinate system transformation based on the camera position information and the camera posture information includes: Based on the camera position information, determine the coordinate offset information of the camera coordinate system of the camera device to the three-dimensional rectangular coordinate system of the virtual world. Based on the camera posture information, determine the coordinate rotation information of the camera coordinate system to the three-dimensional rectangular coordinate system of the virtual world, so as to determine the fourth coordinate transformation information of the camera coordinate system to the three-dimensional rectangular coordinate system of the virtual world.

6. The method according to claim 3, wherein, The step of obtaining the second coordinate transformation information from the virtual world's three-dimensional Cartesian coordinate system to the corresponding geographic coordinate system includes: The projection algorithm determines the second coordinate transformation information from the virtual world's three-dimensional rectangular coordinate system to the corresponding geographic coordinate system.

7. The method according to claim 1, wherein, The acquisition of the corresponding reference data information includes: Obtain the geographic location information of multiple elevation points in the relevant area of ​​the current image, wherein the geographic location information corresponds to the geographic coordinate system; The step of determining the geographic location information of the target object in the geographic coordinate system based on the screen location information, the first coordinate transformation information, the second coordinate transformation information, and the reference data information includes: The geographical location information of the target object in the geographic coordinate system is determined based on the screen position information, the first coordinate transformation information, the second coordinate transformation information, and the geographical location information of the multiple elevation points.

8. The method according to claim 7, wherein, Determining the geographic location information of the target object in the geographic coordinate system based on the screen location information, the first coordinate transformation information, the second coordinate transformation information, and the reference data information includes: Based on the screen position information and the first coordinate transformation information, determine the ray information of the target object in the three-dimensional Cartesian coordinate system of the virtual world; Based on the inverse transformation of the second coordinate transformation information and the geographical location information of the multiple elevation points, determine the multiple triangular mesh information corresponding to the three-dimensional rectangular coordinate system of the virtual world; The coordinates of the corresponding intersection points are determined based on the ray information and the multiple triangular mesh information. The intersection coordinates are transformed into a geographic coordinate system based on the second coordinate transformation information, thereby determining the geographic location information of the target object.

9. The method according to claim 8, wherein, The first coordinate transformation information includes fifth coordinate transformation information from the screen coordinate system to the corresponding screen space coordinate system, projection transformation information from the screen space coordinate system to the corresponding camera coordinate system, and fourth coordinate transformation information from the camera coordinate system to the corresponding virtual world three-dimensional Cartesian coordinate system; wherein, determining the ray information of the target object in the virtual world three-dimensional Cartesian coordinate system based on the screen position information and the first coordinate transformation information includes: Based on the screen position information, determine the intersection points of the target object with the far plane and the near plane in the screen space coordinate system; Based on the projection transformation information and the fourth coordinate transformation information, the far plane intersection point and the near plane intersection point are transformed into the three-dimensional rectangular coordinate system of the virtual world to determine the corresponding far plane spatial point and near plane spatial point. The corresponding ray information is determined based on the coordinates of the far-plane spatial point and the near-plane spatial point, wherein the origin of the ray is the near-plane spatial point.

10. The method according to claim 9, wherein, The step of determining the corresponding intersection point coordinates based on the ray information and the multiple triangular mesh information includes: Determine one or more bounding box information corresponding to the plurality of triangular mesh information; Intersection detection is performed sequentially between the ray information and one or more bounding box information. If the ray information intersects with a certain bounding box information among the one or more bounding box information, then the certain bounding box information is determined as the target bounding box information. The corresponding intersection coordinates are determined based on the target bounding box information and the ray information.

11. The method according to claim 10, wherein, The step of determining the corresponding intersection point coordinates based on the target bounding box information and the ray information includes: At least one intersection point is determined based on the target triangle mesh information corresponding to the ray information and the target bounding box information; From the at least one intersection point, determine the intersection point closest to the camera device, and determine the spatial coordinate information of the intersection point as the corresponding intersection point coordinate information.

12. The method according to claim 11, wherein, Determining at least one intersection point based on the ray information and the target triangle mesh information corresponding to the target bounding box information includes: Intersection detection is performed sequentially between the ray information and one or more target triangle mesh information in the target triangle mesh information. If the ray information intersects with the plane of a target triangle mesh information in the one or more target triangle mesh information, and the corresponding intersection point is inside the target triangle mesh information, then the intersection point is determined as the corresponding intersection point, so as to obtain at least one corresponding intersection point.

13. The method according to claim 1, wherein, The camera device and the corresponding laser rangefinder are arranged in parallel; wherein, acquiring the screen position information of the target object to be measured in the current image in the screen coordinate system includes: The center coordinate information of the current image at the center of the screen of the display device is determined as the screen position information of the target object to be measured in the screen coordinate system; The acquisition of corresponding reference data information includes: Obtain the distance information between the target object and the camera device as measured by the ranging device; The step of determining the geographic location information of the target object in the geographic coordinate system based on the screen location information, the first coordinate transformation information, the second coordinate transformation information, and the reference data information includes: The geographical location information of the target object in the geographic coordinate system is determined based on the screen position information, the first coordinate transformation information, the second coordinate transformation information, and the distance information.

14. The method according to claim 13, wherein, Determining the geographic location information of the target object in the geographic coordinate system based on the screen position information, the first coordinate transformation information, the second coordinate transformation information, and the distance information includes: The ray information of the target object in the three-dimensional Cartesian coordinate system of the virtual world is determined based on the screen position information and the first coordinate transformation information, wherein the ray information includes ray direction and origin information; Based on the ray information and the distance information, determine the target spatial position information of the target object in the three-dimensional Cartesian coordinate system of the virtual world; The geographic location information of the target object in the geographic coordinate system is determined based on the target spatial location information and the second coordinate transformation information.

15. A device for acquiring the geographic location information of a target object, wherein, The device includes: The module is used to obtain coordinate transformation information corresponding to the current image currently displayed on the display device. The coordinate transformation information includes first coordinate transformation information of the screen coordinate system to the virtual world three-dimensional rectangular coordinate system, and second coordinate transformation information of the virtual world three-dimensional rectangular coordinate system to the geographic coordinate system. The first and second modules are used to obtain the screen position information of the target object to be measured in the current image in the screen coordinate system; The first and third modules are used to acquire corresponding reference data information, wherein the reference data information includes reference location information or reference distance information, the reference location information includes the geographical location information of a known positioning point, and the reference distance information includes the distance information between the target object or the known positioning point and the camera device; The first and fourth modules are used to determine the geographical location information of the target object in the geographic coordinate system based on the screen position information, the first coordinate transformation information, the second coordinate transformation information, and the reference data information. The determination of the geographical location information of the target object in the geographic coordinate system based on the screen position information, the first coordinate transformation information, the second coordinate transformation information, and the reference data information includes: determining the ray information of the screen point corresponding to the screen position information in the virtual world's three-dimensional Cartesian coordinate system based on the first coordinate transformation information; determining the target point of the screen point in the virtual world's three-dimensional Cartesian coordinate system based on the ray information and the reference data information; and transforming the spatial coordinates of the target point to the geographic coordinate system based on the second coordinate transformation information to determine the geographical location information of the target object.

16. A computer device, wherein, The device includes: Processor; and A memory configured to store computer-executable instructions, which, when executed, cause the processor to perform the steps of the method as described in any one of claims 1 to 14.

17. A computer-readable storage medium having a computer program / instructions stored thereon, characterized in that, When executed, the computer program / instruction causes the system to perform the steps of the method as described in any one of claims 1 to 14.

18. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method described in any one of claims 1 to 14.

Images