Target object positioning method, device, medium, equipment and trajectory mapping method

Abstract

The present application discloses a method, apparatus, medium, device for target object positioning and a trajectory mapping method, including: for a target area, pre-shooting at each predetermined reference shooting position to collect a number of reference aerial images; the total shooting range of each reference aerial image fully covers the target area; obtaining in real time the current aerial image taken by the aircraft for the target area; based on the coverage range of the current aerial image, determining a number of target reference aerial images that match the current aerial image from each reference aerial image; based on the current aerial image, each target reference aerial image and the reference position coordinates of the reference objects in each target reference aerial image, determining the target position coordinates of the target object in the current aerial image. The present application can accurately and quickly determine the actual position coordinates of the target object in the current aerial image, realizes the automatic mapping of the pixel position of the target object to the actual space position, and improves the positioning accuracy and positioning speed.

Description

Technical Field

[0001] The present invention relates to the field of electronic information technology, and particularly to a method for locating a target object, a device, a medium, a device and a trajectory mapping method. Background Art

[0002] Unmanned aerial vehicle (UAV) remote sensing systems, with their significant advantages such as strong mobility, high image resolution, and low cost, are widely used in multiple fields, especially in scenarios where it is necessary to quickly obtain large-scale and high-resolution image data. The high-performance sensors carried by UAVs can collect image data in real time, which is usually presented in the form of a video stream for intuitive observation. However, such video images only provide visual information and lack positioning data associated with the geographical space, so it is difficult to provide accurate spatial references for decision-makers. Thus, methods for locating target objects in aerial images have emerged. Currently, the methods for UAVs to locate ground targets mainly include two types. One method is to perform target positioning through the telemetry data of the UAV, using the instantaneous distance and angle information relative to the target during UAV aerial photography, as well as the position information and attitude data read from the flight control system for calculation, so as to obtain the geographical coordinates of the target. Another method is based on image matching technology. First, the images taken by the UAV are matched with a reference map, and then the coordinate transformation relationship between the images is established, and finally the position information of the target is calculated.

[0003] However, the above-mentioned positioning method using UAV telemetry data relies heavily on the positioning accuracy of the UAV itself. When the UAV hovers in the air, affected by external factors, the positioning accuracy is often unstable. Although the positioning method based on image matching can provide relatively accurate trajectory positioning, due to the spatial resolution difference between the image and the map, it is difficult to match the image with the map, and the positioning accuracy is also difficult to guarantee in complex environments.

[0004] Therefore, there is an urgent need for a method for locating a target object to solve the problem in the prior art that it is impossible to accurately and quickly locate the target object in an aerial image. Summary of the Invention

[0005] In view of this, the present invention provides a method for locating a target object, a device, a medium, a device and a trajectory mapping method, mainly aiming to solve the problem that it is currently impossible to accurately and quickly locate the target object in an aerial image.

[0006] To solve the above problems, the present application provides a method for locating a target object, including:

[0007] For a target area, pre-shoot at each predetermined reference shooting position to collect a number of reference aerial images; wherein, the total shooting range of each of the reference aerial images fully covers the target area;

[0008] Obtain the current aerial image captured by the aircraft for the target area in real time;

[0009] Based on the coverage range of the current aerial image, determine several target reference aerial images that match the current aerial image from each of the reference aerial images;

[0010] Based on the current aerial image, each of the target reference aerial images, and the reference position coordinates of the reference objects within each of the target reference aerial images, determine the target position coordinates of the target object within the current aerial image.

[0011] Optionally, before taking pictures at each predetermined reference shooting position for the target area, the method further includes: predetermining each of the reference shooting positions, specifically including:

[0012] Based on the horizontal field of view angle of the shooting device, the vertical field of view angle of the shooting device, and the flight altitude of the aircraft, determine the shooting range of the reference aerial image;

[0013] Based on the shooting range and the total area range of the target area, determine each of the reference shooting positions.

[0014] Optionally, the shooting height of the current aerial image is lower than or equal to the shooting heights of each of the reference aerial images;

[0015] The determining, from each of the reference aerial images, several target reference aerial images that match the current aerial image based on the coverage range of the current aerial image specifically includes:

[0016] Based on the current shooting position of the current aerial image, determine the target position interval;

[0017] Based on the reference shooting positions of each of the reference aerial images, determine the reference aerial images whose reference shooting positions are within the target position interval as the target reference aerial images.

[0018] Optionally, the determining the target position interval based on the current shooting position of the current aerial image specifically includes:

[0019] Based on the shooting height of the current aerial image, the horizontal field of view angle of the shooting device, and the vertical field of view angle of the shooting device, determine the course distance and the cross-track distance;

[0020] Based on the current shooting position, the course distance, and the cross-track distance, determine the target position interval.

[0021] Optionally, determining the target position coordinates of the target object in the current aerial image based on the current aerial image, each of the target reference aerial images, and the reference position coordinates of the reference objects in each of the target reference aerial images specifically includes:

[0022] Based on the current aerial image, each of the target reference aerial images, and the first pixel coordinates of the reference objects in each of the target reference aerial images, determining the second pixel coordinates of each of the reference objects in the current aerial image; the number of each of the reference objects is greater than or equal to 4;

[0023] Based on the second pixel coordinates corresponding to each of the reference objects, the reference position coordinates of each of the reference objects, and the target pixel coordinates of the target object in the current aerial image, determining the target position coordinates of the target object.

[0024] Optionally, determining the second pixel coordinates of each of the reference objects in the current aerial image based on the current aerial image, each of the target reference aerial images, and the first pixel coordinates of the reference objects in each of the target reference aerial images specifically includes:

[0025] Based on the current aerial image and each of the target reference aerial images, determining the transformation matrix between the current aerial image and each of the target reference aerial images;

[0026] Based on the transformation matrix corresponding to each of the target reference aerial images and the first pixel coordinates of the reference objects in each of the target reference aerial images, determining the second pixel coordinates of each of the reference objects in the current aerial image.

[0027] To solve the above problems, the present application provides a trajectory mapping method, including: using the target object positioning method described in any one of the above, positioning the target object in each video frame of the aerial video in real time to obtain the target position coordinates of the target object;

[0028] Based on each of the target position coordinates, determining the target coordinate trajectory of the target object to obtain the trajectory mapping result.

[0029] To solve the above problems, the present application provides a positioning device for a target object, including:

[0030] An acquisition module, configured to perform shooting at each predetermined reference shooting position for a target area in advance to acquire a plurality of reference aerial images; wherein, the total shooting range of each of the reference aerial images fully covers the target area;

[0031] An acquisition module, configured to acquire the current aerial image taken by the aircraft for the target area in real time;

[0032] A matching module, configured to determine a plurality of target reference aerial images that match the current aerial image from each of the reference aerial images based on the coverage range of the current aerial image;

[0033] A determination module, configured to determine the target position coordinates of the target object in the current aerial image based on the current aerial image, each of the target reference aerial images, and the reference position coordinates of the reference objects in each of the target reference aerial images.

[0034] To solve the above problems, the present application provides a storage medium storing a computer program, and when the computer program is executed by a processor, the steps of the target object positioning method described in any one of the above are implemented.

[0035] To solve the above problems, the present application provides an electronic device including at least a memory and a processor, where a computer program is stored on the memory, and when the processor executes the computer program on the memory, the steps of the target object positioning method described in any one of the above are implemented.

[0036] In a target object positioning method, device, medium, device and trajectory mapping method of the present application, by pre-collecting reference aerial images covering the entire target area, the current aerial image can be subsequently matched with each reference aerial image, so as to quickly and accurately determine the target reference aerial images overlapping with the current aerial image. Furthermore, based on the pixel coordinates and reference position coordinates / actual spatial positions of each reference object in each target reference aerial image, the actual position coordinates of the target object in the current aerial image can be accurately and quickly determined, realizing the automatic mapping of the pixel position of the target object to the actual spatial position, and improving the positioning accuracy and positioning speed.

[0037] The above description is only an overview of the technical solution of the present invention. In order to be able to understand the technical means of the present invention more clearly, it can be implemented according to the content of the specification. And in order to make the above and other purposes, features and advantages of the present invention more obvious and understandable, the specific embodiments of the present invention are hereinafter specifically exemplified. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] By reading the following detailed description of the preferred embodiments, various other advantages and benefits will become clear to those of ordinary skill in the art. The drawings are only for the purpose of showing the preferred embodiments and are not considered to be a limitation of the present invention. And throughout the drawings, the same reference numerals are used to represent the same components. In the drawings:

[0039] Figure 1 is a flowchart of a target object positioning method according to an embodiment of the present application;

[0040] Figure 2Schematic diagram of the shooting range of the shooting device according to another embodiment of the present application;

[0041] FIG. 3(a) shows the positional relationship diagram of two adjacent reference aerial images along the heading / X UAV axis direction in the embodiment of the present application;

[0042] FIG. 3(b) shows the positional relationship diagram of two adjacent reference aerial images along the cross-track / Y UAV axis direction in the embodiment of the present application;

[0043] FIG. 4(a) shows the index table constructed based on the reference aerial images in another embodiment of the present application;

[0044] FIG. 4(b) shows the matching flowchart of the current aerial image and each reference aerial image in another embodiment of the present application;

[0045] FIG. 4(c) shows the matching result diagram for the reference aerial images in another embodiment of the present application;

[0046] Figure 5 Structural block diagram of a positioning device for a target object in another embodiment of the present application;

[0047] Figure 6 Structural block diagram of an electronic device in another embodiment of the present application. Detailed implementation manners

[0048] Reference is made herein to the accompanying drawings to describe various solutions and features of the present application.

[0049] It should be understood that various modifications can be made to the embodiments applied herein. Therefore, the above description should not be regarded as a limitation, but only as an example of the embodiments. Those skilled in the art will think of other modifications within the scope and spirit of the present application.

[0050] The accompanying drawings included in and constituting a part of the specification illustrate the embodiments of the present application and, together with the general description of the present application given above and the detailed description of the embodiments given below, are used to explain the principles of the present application.

[0051] These and other features of the present application will become apparent from the following description of the preferred forms of the embodiments given by way of non-limiting examples with reference to the accompanying drawings.

[0052] It should also be understood that although the present application has been described with reference to some specific examples, those skilled in the art can surely implement many other equivalent forms of the present application.

[0053] When combined with the accompanying drawings, the above and other aspects, features, and advantages of the present application will become more apparent in view of the following detailed description.

[0054] Specific embodiments of the present application will be described hereinafter with reference to the accompanying drawings; however, it should be understood that the claimed embodiments are merely examples of the present application and can be implemented in various ways. Well-known and / or repetitive functions and structures are not described in detail to avoid obscuring the present application with unnecessary or redundant details. Therefore, the specific structural and functional details claimed herein are not intended to be limiting, but are merely used as a basis and representative basis for the claims to teach those skilled in the art to use the present application in substantially any suitable detailed structure in a variety of ways.

[0055] This specification may use the phrases "in one embodiment", "in another embodiment", "in yet another embodiment", or "in other embodiments", which may each refer to one or more of the same or different embodiments according to the present application.

[0056] The embodiments of the present application provide a method for locating a target object, a device, a medium, a device, and a trajectory mapping method, which can be specifically applied to electronic devices such as terminals and servers, such as Figure 1 As shown, a method for locating a target object in this embodiment includes the following steps:

[0057] Step S101, for a target area, pre-shoot at each predetermined reference shooting position to collect a number of reference aerial images; wherein, the total shooting range of each of the reference aerial images fully covers the target area;

[0058] In the specific implementation process of this embodiment, the shooting range of each reference aerial image can be determined according to the shooting height / the flight height of the aircraft, the horizontal field of view angle of the shooting device, and the vertical field of view angle of the shooting device. Finally, according to the shooting ranges of the aerial images and the total area range of the target area, each reference shooting position can be determined. In this step, the shooting height / the flight height corresponding to each reference shooting position can be different or the same, as long as it is ensured that the combined total shooting range of all reference aerial images fully covers the target area. Among them, each reference aerial image contains a number of reference objects.

[0059] Step S102, obtain in real time the current aerial image taken by the aircraft for the target area;

[0060] In the specific implementation process of this step, after the aircraft / drone hovers, since the position change is small when the drone hovers and the lens is equipped with a gimbal, the change of the real-time video image is very small and can be ignored. Therefore, a video frame is extracted from the real-time video as the current aerial image.

[0061] Step S103, based on the coverage range of the current aerial image, determine a number of target reference aerial images that match the current aerial image from each of the reference aerial images;

[0062] In the specific implementation process of this step, based on the coverage range of the current aerial image and the coverage ranges of the reference aerial images, several target reference aerial images that overlap with the current aerial image can be determined from the reference aerial images, so as to realize the dynamic matching between the current aerial image and the reference aerial images.

[0063] Step S104: Based on the current aerial image, each of the target reference aerial images, and the reference position coordinates of the reference objects within each of the target reference aerial images, determine the target position coordinates of the target object within the current aerial image.

[0064] In the specific implementation process of this step, specifically, the reference objects within each target reference aerial image can be mapped to the current aerial image to obtain the second pixel coordinates of each reference object within the current aerial image. Then, combined with the reference position coordinates / actual spatial positions of each reference object and the target pixel coordinates of the target object within the current aerial image, the target position coordinates / actual spatial positions of the target object can be determined, so as to realize the spatial positioning of the target object.

[0065] In the method of this embodiment, by pre-collecting reference aerial images covering the entire target area, the current aerial image can be matched with each reference aerial image subsequently, so that the target reference aerial images overlapping with the current aerial image can be quickly and accurately determined. Furthermore, according to the pixel coordinates and reference position coordinates / actual spatial positions of each reference object within each target reference aerial image, the actual position coordinates of the target object within the current aerial image can be accurately and quickly determined, realizing the automatic mapping of the pixel position of the target object to the actual spatial position, and improving the positioning accuracy and positioning speed.

[0066] Another embodiment of this application provides a target object positioning method, which specifically includes the following steps:

[0067] Step S201: Predetermine several reference objects;

[0068] In the specific implementation process of this step, several reference objects and the reference position coordinates / actual spatial positions of each reference object can be predetermined for the target area. That is, a certain number of reference objects are arranged within the aerial photography area / target area, and it is ensured that they are distributed as evenly as possible. The positions of the reference objects need to be selected relatively fixed to ensure that the positions change little during different time periods. A unique identifier can be configured for each reference object, in the format: P c =<cid, X, Y>, for subsequent identification and matching use. Where cid represents the number of the reference object, and (X, Y) represents the position coordinates of the reference object.

[0069] Step S202, pre-determine each reference shooting position;

[0070] In this step, the shooting range of the reference aerial image can be determined based on the horizontal field of view angle of the shooting device, the vertical field of view angle of the shooting device, and the flight altitude of the aircraft / the flight altitude of the UAV; then, based on the shooting range and the total area range of the target area, each of the reference shooting positions is determined.

[0071] Specifically, in order to improve the accuracy of target positioning, aerial photography stations can be pre-set to facilitate subsequent shooting of reference aerial images. First, design the operation route of the UAV / aircraft according to the aerial photography area / target area, and calculate the coverage range of the video / image based on the field of view angle and flight altitude of the lens. When the UAV flies at a constant altitude and keeps the lens vertically downward, its video coverage range / shooting range can be simplified, as Figure 2 shown. Wherein, H, ψ, and θ respectively represent the flight altitude, the horizontal field of view angle, and the vertical field of view angle, point O represents the vertical projection of the UAV on the ground (i.e., the shooting position), and the rectangular area represents the coverage range of the camera / shooting device. At this time, L = 2×H×tanψ is the length of the rectangular area / shooting range, and W = 2×H×tanθ is the width of the rectangular area / shooting range.

[0072] After determining the shooting range value, the reference shooting positions of the reference aerial images can be determined / set according to the shooting range of the UAV camera, as shown in Figures 3(a) and 3(b). During the setting process, it is stipulated that the flight distance along the X UAV axis is the heading flight distance Lp, and the flight distance along the Y UAV axis is the cross-track flight distance Wq. To ensure that each reference aerial image contains reference points and has sufficient overlap, it is necessary to control the distance between the reference shooting positions to avoid the situation of non-overlap between adjacent reference images. According to the camera shooting range and the actual ground situation, select appropriate heading flight distance Lp and cross-track flight distance Wq such that they satisfy Lp < L or Wq < W, that is, the heading flight distance Lp = X i -X i-1 between two adjacent reference shooting positions, and the cross-track flight distance Wq = Y i -Y i-1 . Thus, a number of reference shooting positions (X i , Y i ) can be obtained.

[0073] Step S203, for the target area, pre-shoot at each predetermined reference shooting position to collect a number of reference aerial images; wherein, the total shooting range of each of the reference aerial images fully covers the target area.

[0074] In this step, after each reference shooting position is determined, shooting can be performed at each reference shooting position, thereby obtaining a reference aerial image corresponding to each reference shooting position.

[0075] In this step, after obtaining several reference aerial images, each reference aerial image can be expressed in the form of a quadruple: pic= <pid i , X i , Y i , pts>. Where pid is the number of the i-th reference aerial image; (X i , Y i ) represents the spatial coordinates of the reference aerial image taken by the drone, that is, the reference shooting position. pts is the set of reference objects in the image, represented by pts={ c1 , P c1 >, …, cn , P cn >}. Among them, P cn is the nth reference object identifier, P cn = <cid n , X n , Y n >I cn is the point in the image corresponding to the reference object, I c =<pcid,u,v> It consists of 3 tuples, which are number and pixel coordinates, so pts can also be expressed as: pts={ <pcid 1 ,u 1 ,v 1 ,cid 1 ,x 1 ,y 1 >,…, <pcid n ,u n ,v n ,cid n ,x n ,y n >}.

[0076] Step S203, acquiring in real time a current aerial image taken by the aircraft for the target area;

[0077] In this step, after the aircraft / drone is hovering, since the position of the drone changes little in the hovering state and the lens is equipped with a gimbal, the real-time video picture changes very little and can be ignored. Therefore, a video frame is extracted from the real-time video as the current aerial image.

[0078] Step S204, based on the coverage of the current aerial image, determining a plurality of target reference aerial images matching the current aerial image from the reference aerial images; ​​

[0079] In this step, the following method can be used to match and obtain several target reference aerial images that overlap with the current aerial image. Taking the case where the shooting height of the current aerial image is lower than or equal to the shooting heights of the reference aerial images, when determining the target reference aerial images, the course distance Lr and the cross-track distance Wr can be determined first based on the shooting height of the current aerial image, the horizontal field of view angle of the shooting device, and the vertical field of view angle of the shooting device; based on the current shooting position, the course distance Lr, and the cross-track distance Wr, the target position intervals [R X -Lr, R X +Lr] and [R Y -Wr, R Y +Wr] are determined. Based on the reference shooting positions of the reference aerial images, the reference aerial images whose reference shooting positions are within the target position intervals are determined as the target reference aerial images.

[0080] Specifically, the process of matching and obtaining each target reference aerial image is as follows:

[0081] Step 1. Build an index; in order to quickly and accurately find relevant target reference aerial images in the subsequent dynamic matching / image matching process, an index needs to be established for these images. According to the reference shooting positions (X, Y) of the reference aerial images in step S203, they are sorted according to their positions, and an index table is generated after arranging the X and Y coordinate values in ascending order. Among them, X represents the row, Y represents the column, and the combination of X and Y corresponds to the ID of each image, as shown in Fig. 4(a).

[0082] Step 2. Image query; when performing dynamic matching, query the reference aerial image set according to the X and Y of the real-time position of the current UAV.

[0083] The specific method is as follows: Denote the coordinates of the current shooting position of the UAV as R X and R Y , and the range of the required aerial reference image is P X and P Y . First is the query in the X direction. To ensure that the reference aerial image set around the current shooting position of the UAV can be found during retrieval, its retrieval range should not be less than ±Lr, that is, Lr = 2×Hr×tanψ, Wr = 2×Hr×tanθ, where Hr is the flight height of the real-time UAV, and ψ and θ represent the horizontal field of view angle and the vertical field of view angle respectively. Then the retrieval range of the aerial reference image in the X direction of the real-time position of the UAV is R X -Lr <= P X <= R X +Lr. Similarly, the above method is used to select the index range in the Y direction, that is, the retrieval range of the aerial reference image in the direction of the real-time position of the UAV is R Y-Wr <= P Y <= R Y +Wr. When retrieving / matching using the process shown in Figure 4(b), when the sets of aerial reference image IDs that meet the requirements for the X value and the Y value range can be retrieved respectively, take the intersection of the two to complete the retrieval, and the set of reference aerial image index IDs can be obtained. As shown in the intersecting part of the two dashed rectangles in Figure 4(c), the corresponding target reference aerial images can be found according to the ID.

[0084] Step S205, based on the current aerial image, each of the target reference aerial images, and the first pixel coordinates of the reference objects in each of the target reference aerial images, determine the second pixel coordinates of each of the reference objects within the current aerial image; the number of each of the reference objects is greater than or equal to 4;

[0085] In the specific implementation process of this step, the transformation matrix between the current aerial image and each target reference aerial image can be determined based on the current aerial image and each target reference aerial image; based on the transformation matrix corresponding to each target reference aerial image and the first pixel coordinates of the reference objects in each target reference aerial image, determine the second pixel coordinates of each of the reference objects within the current aerial image.

[0086] Specifically, this step can specifically adopt the following process to map each reference object in each target reference aerial image to the current aerial image. Specifically, it includes:

[0087] Step 1, solve the transformation matrix between the real-time video / current aerial image and each target reference aerial image.

[0088] In this step, since there are many real-time video frames captured by the drone, resulting in video frame redundancy and time-consuming in the matching process, a video frame F can be selected as the current aerial image to establish the relationship between the image and the ground instead of the real-time video. The specific approach is that after the drone hovers, since the position change is small in the hovering state of the drone and the camera is equipped with a gimbal, the change of the real-time video image is very small and can be ignored. Therefore, a frame F is extracted from the real-time video as the video frame / current aerial image for establishing the connection with the ground points. Specifically, the SURF (Speeded-Up Robust Features), FLANN (Fast Library for Approximate Nearest Neighbors), and RANSAC (Random Sample Consensus) algorithms can be combined to perform feature matching between the video frame F and the set of aerial reference images in sequence, and calculate the transformation matrix between the video frame F and each aerial reference image.

[0089] Step 2: Transform the pixel coordinates of the reference objects in the target reference aerial image set to the video frame F / current aerial image according to each transformation matrix.

[0090] Step S206: Determine the target position coordinates of the target object based on the second pixel coordinates corresponding to each of the reference objects, the reference position coordinates of each of the reference objects, and the target pixel coordinates of the target object in the current aerial image.

[0091] In the specific implementation process of this step, it is specifically to solve the mutual mapping relationship between the video frame F / current aerial image and the spatial information. That is, after transforming the pixel coordinates of the reference objects in the target reference aerial image set to the video frame F / current aerial image, the mapping relationship between the ground points and the pixel points can be established by using the homography transformation based on the matching reference objects and pixel coordinates in the video frame F / current aerial image, so as to realize the mutual mapping between the video frame F / current aerial image and the spatial information, so that each pixel coordinate in the video frame F / current aerial image corresponds to a geographical spatial position, and thus the target position coordinates of the target object in the current aerial image can be determined accurately and quickly.

[0092] In the specific implementation process of this embodiment, when the position of the UAV changes, the changed real-time video is sequentially executed through the above steps S204 to S206, so as to realize the dynamic fusion of the UAV video and the spatial information and the accurate positioning of the target object.

[0093] In the specific implementation process of this embodiment, the target object in the current aerial image can be recognized by using the target detection and tracking method, so as to determine the target pixel coordinates of the target object. Specifically, the YOLO target detection algorithm can be used to achieve this. This embodiment uses the YOLOv8 detection model to detect the target object in the current aerial image. YOLOv8 adopts a single-stage network structure, which can quickly detect the target object in the current aerial image and output the bounding box, confidence level and class label of the target. For target tracking, the multi-object tracking algorithm of deep learning (DeepSort algorithm) is used. DeepSort combines the appearance features and motion information of the target, predicts the target trajectory through the Kalman filter, and uses the Hungarian algorithm for target matching and ID assignment. This enables DeepSort to maintain the tracking ID of the target and update the target position in real time to ensure accurate tracking in a complex environment. Combining YOLOv8 and DeepSort can achieve efficient and stable target detection and tracking, so as to output the target pixel coordinates of the target object in the UAV video / current aerial image, which is convenient for subsequent determination of the actual spatial coordinates / target position coordinates of the target object based on the target pixel coordinates.

[0094] In the method of this embodiment, by selecting a reference object according to the target area and collecting reference aerial images with the reference object, and dynamically matching a set of reference aerial images with the reference object, the pixel coordinates of the reference object in the current video / current aerial image are obtained. Then, the homography transformation method is used to establish a mutual mapping relationship between the image and the geographical space. At the same time, dynamic targets in the real-time video / current aerial image are detected and tracked, and the pixel coordinates of the target object in the real-time video / current aerial image are mapped to the geographical space according to the mutual mapping relationship, so as to realize the fusion and positioning of dynamic targets and geographical space information in the UAV video, and improve the accuracy of the positioning result.

[0095] Another embodiment of this application provides a trajectory mapping method, including:

[0096] Step 1: Use a target object positioning method in any of the above embodiments to locate the target object in each video frame of the aerial video in real time, and obtain the target position coordinates of the target object;

[0097] Step 2: Based on each of the target position coordinates, determine the target coordinate trajectory of the target object to obtain a trajectory mapping result.

[0098] In the specific implementation process of this embodiment, YOLOv8 and DeepSort can be used to identify and track the target object in the UAV video, and obtain the pixel coordinates corresponding to each target ID. Then, use a target object positioning method in any of the above embodiments to map the pixel coordinates of the target object to the actual position coordinates, that is, convert the sequence pixel trajectories of these target objects into geographical space coordinates, so as to realize the mapping and fusion of the target trajectory and geographical space information in the UAV video.

[0099] The method in this embodiment can accurately and quickly determine the actual position coordinates of the target object, realize the automatic mapping of the pixel position of the target object to the actual space position, and improve the positioning accuracy and positioning speed.

[0100] Another embodiment of this application provides a positioning device for a target object, as Figure 5 shown, including:

[0101] The acquisition module 11 is used to pre-take pictures at each predetermined reference shooting position for the target area to acquire a number of reference aerial images; wherein, the total shooting range of each of the reference aerial images fully covers the target area;

[0102] The acquisition module 12 is used to acquire the current aerial image taken by the aircraft for the target area in real time;

[0103] A matching module 13, configured to determine a number of target reference aerial images that match the current aerial image from each of the reference aerial images based on the coverage range of the current aerial image;

[0104] A determination module 14, configured to determine the target position coordinates of the target object in the current aerial image based on the current aerial image, each of the target reference aerial images, and the reference position coordinates of the reference objects in each of the target reference aerial images.

[0105] In the specific implementation process of this embodiment, the device further includes a position determination module for pre-determining each of the reference shooting positions. The position determination module is specifically configured to: determine the shooting range of the reference aerial image based on the horizontal field of view angle of the shooting device, the vertical field of view angle of the shooting device, and the flight altitude of the aircraft; determine each of the reference shooting positions based on the shooting range and the total area range of the target area.

[0106] In the specific implementation process of this embodiment, the shooting height of the current aerial image is lower than or equal to the shooting heights of each of the reference aerial images; the matching module 13 is specifically configured to: determine a target position interval based on the current shooting position of the current aerial image; determine the reference aerial images whose reference shooting positions are within the target position interval as the target reference aerial images based on the reference shooting positions of each of the reference aerial images.

[0107] In the specific implementation process of this embodiment, the matching module 13 is specifically configured to: determine the heading distance and the cross-track distance based on the shooting height of the current aerial image, the horizontal field of view angle of the shooting device, and the vertical field of view angle of the shooting device; determine the target position interval based on the current shooting position, the heading distance, and the cross-track distance.

[0108] In the specific implementation process of this embodiment, the determination module 14 specifically includes a first determination unit and a second determination unit; the first determination unit is configured to: determine the second pixel coordinates of each of the reference objects in the current aerial image based on the current aerial image, each of the target reference aerial images, and the first pixel coordinates of the reference objects in each of the target reference aerial images; the number of each of the reference objects is greater than or equal to 4; the second determination unit is configured to: determine the target position coordinates of the target object based on the second pixel coordinates corresponding to each of the reference objects, the reference position coordinates of each of the reference objects, and the target pixel coordinates of the target object in the current aerial image.

[0109] In the specific implementation process of this embodiment, the first determination unit is configured to: determine a transformation matrix between the current aerial image and each target reference aerial image based on the current aerial image and each target reference aerial image; and determine second pixel coordinates of each reference object within the current aerial image based on the transformation matrix corresponding to each target reference aerial image and the first pixel coordinates of the reference object within each target reference aerial image.

[0110] The device in this embodiment can, by pre-collecting reference aerial images covering the entire target area, subsequently match the current aerial image with each reference aerial image, so as to quickly and accurately determine the target reference aerial images overlapping with the current aerial image. Furthermore, based on the pixel coordinates and reference position coordinates / actual spatial positions of each reference object within each target reference aerial image, the actual position coordinates of the target object within the current aerial image can be accurately and quickly determined, realizing the automatic mapping of the pixel position of the target object to the actual spatial position, and improving the positioning accuracy and speed.

[0111] Another embodiment of this application provides a storage medium storing a computer program, and when the computer program is executed by a processor, the following method steps are implemented:

[0112] Step 1: For the target area, perform shooting at each predetermined reference shooting position in advance to collect a number of reference aerial images; wherein, the total shooting range of each reference aerial image fully covers the target area;

[0113] Step 2: Obtain in real time the current aerial image captured by the aircraft for the target area;

[0114] Step 3: Based on the coverage range of the current aerial image, determine a number of target reference aerial images that match the current aerial image from each reference aerial image;

[0115] Step 4: Based on the current aerial image, each target reference aerial image, and the reference position coordinates of the reference object within each target reference aerial image, determine the target position coordinates of the target object within the current aerial image.

[0116] For the specific implementation process of the above method steps, reference can be made to the embodiments of any of the above target object positioning methods, and this embodiment will not be repeated here.

[0117] In the storage medium of the present application, by pre-collecting reference aerial images covering the entire target area, the current aerial image can then be matched with each reference aerial image, so as to quickly and accurately determine the target reference aerial image that overlaps with the current aerial image. Furthermore, based on the pixel coordinates and reference position coordinates / actual spatial positions of each reference object in each target reference aerial image, the actual position coordinates of the target object in the current aerial image can be accurately and quickly determined, realizing the automatic mapping of the pixel position of the target object to the actual spatial position, and improving the positioning accuracy and positioning speed.

[0118] Another embodiment of the present application provides an electronic device, as Figure 6 shown, which at least includes a memory 1 and a processor 2. A computer program is stored on the memory 1, and when the processor 2 executes the computer program on the memory 1, the following method steps are implemented:

[0119] Step 1: For the target area, pre-take pictures at each predetermined reference shooting position to collect a number of reference aerial images; wherein, the total shooting range of each of the reference aerial images fully covers the target area;

[0120] Step 2: Real-time obtain the current aerial image taken by the aircraft for the target area;

[0121] Step 3: Based on the coverage range of the current aerial image, determine a number of target reference aerial images that match the current aerial image from each of the reference aerial images;

[0122] Step 4: Based on the current aerial image, each of the target reference aerial images, and the reference position coordinates of the reference objects in each of the target reference aerial images, determine the target position coordinates of the target object in the current aerial image.

[0123] For the specific implementation process of the above method steps, reference can be made to the embodiments of any of the above target object positioning methods, and this embodiment will not be repeated here.

[0124] In the electronic device of the present application, by pre-collecting reference aerial images covering the entire target area, the current aerial image can then be matched with each reference aerial image, so as to quickly and accurately determine the target reference aerial image that overlaps with the current aerial image. Furthermore, based on the pixel coordinates and reference position coordinates / actual spatial positions of each reference object in each target reference aerial image, the actual position coordinates of the target object in the current aerial image can be accurately and quickly determined, realizing the automatic mapping of the pixel position of the target object to the actual spatial position, and improving the positioning accuracy and positioning speed.

[0125] The above embodiments are only exemplary embodiments of the present application and are not used to limit the present application. The protection scope of the present application is defined by the claims. Those skilled in the art can make various modifications or equivalent substitutions within the essence and protection scope of the present application, and such modifications or equivalent substitutions should also be regarded as falling within the protection scope of the present application.

Claims

1. A method for locating a target object, characterized in that: include: For the target area, photographing is performed in advance at each predetermined reference photographing position to collect a number of reference aerial images; wherein the total photographing range of each of the reference aerial images fully covers the target area; Acquire the current aerial image taken by the aircraft for the target area in real time; Based on the coverage of the current aerial image, determining a plurality of target reference aerial images matching the current aerial image from the reference aerial images; Determining target position coordinates of the target object in the current aerial image based on the current aerial image, each of the target reference aerial images, and reference position coordinates of the reference object in each of the target reference aerial images; The shooting height of the current aerial image is lower than or equal to the shooting height of each of the reference aerial images; The determining, based on the coverage of the current aerial image, a plurality of target reference aerial images matching the current aerial image from the reference aerial images specifically includes: Determine the heading distance and the lateral distance based on the shooting height of the current aerial image, the horizontal field of view angle of the shooting device, and the vertical field of view angle of the shooting device; Determine a target position interval based on the current shooting position, the heading distance, and the lateral distance; Based on the reference shooting position of each of the reference aerial images, a reference aerial image whose reference shooting position is within the target position interval is determined as the target reference aerial image.

2. The method according to claim 1, characterized in that Before photographing the target area at each predetermined reference photographing position in advance, the method further includes: predetermining each of the reference photographing positions; Wherein, the predetermining each of the reference shooting positions specifically includes: Determine a shooting range of the reference aerial image based on a horizontal field of view angle of the shooting device, a vertical field of view angle of the shooting device, and a flight altitude of the aircraft; Based on the shooting range and the total area range of the target area, each of the reference shooting positions is determined.

3. The method according to claim 1, characterized in that The determining the target position coordinates of the target object in the current aerial image based on the current aerial image, each of the target reference aerial images, and the reference position coordinates of the reference object in each of the target reference aerial images specifically includes: Determine, based on the current aerial image, each of the target reference aerial images, and first pixel coordinates of reference objects in each of the target reference aerial images, second pixel coordinates of each of the reference objects located in the current aerial image; the number of each of the reference objects is greater than or equal to 4; The target position coordinates of the target object are determined based on the second pixel coordinates corresponding to each of the reference objects, the reference position coordinates of each of the reference objects, and the target pixel coordinates of the target object in the current aerial image.

4. The method according to claim 3, characterized in that The determining, based on the current aerial image, each of the target reference aerial images, and the first pixel coordinates of the reference objects in each of the target reference aerial images, the second pixel coordinates of each of the reference objects in the current aerial image specifically includes: Based on the current aerial image and each of the target reference aerial images, determining a transformation matrix between the current aerial image and each of the target reference aerial images; Based on the transformation matrix corresponding to each of the target reference aerial images and the first pixel coordinates of the reference object in each of the target reference aerial images, the second pixel coordinates of each of the reference objects in the current aerial image are determined.

5. A trajectory mapping method, characterized in that: include: Using a target object positioning method as described in any one of claims 1 to 4, the target object in each video frame of the aerial video is positioned in real time to obtain each target position coordinate of the target object; The target coordinate trajectory of the target object is determined based on each of the target position coordinates to obtain a trajectory mapping result.

6. A device for locating a target object, characterized in that: include: A collection module is used to pre-shoot at each predetermined reference shooting position for the target area to collect a number of reference aerial images; wherein the total shooting range of each of the reference aerial images fully covers the target area; An acquisition module, used for acquiring in real time the current aerial image taken by the aircraft for the target area; A matching module, configured to determine, from each of the reference aerial images, a plurality of target reference aerial images that match the current aerial image based on the coverage of the current aerial image; a determination module, configured to determine the target position coordinates of the target object in the current aerial image based on the current aerial image, each of the target reference aerial images, and the reference position coordinates of the reference object in each of the target reference aerial images; The shooting height of the current aerial image is lower than or equal to the shooting height of each of the reference aerial images; The matching module is specifically used for: Determine the heading distance and the lateral distance based on the shooting height of the current aerial image, the horizontal field of view angle of the shooting device, and the vertical field of view angle of the shooting device; Determine a target position interval based on the current shooting position, the heading distance, and the lateral distance; Based on the reference shooting position of each of the reference aerial images, a reference aerial image whose reference shooting position is within the target position interval is determined as the target reference aerial image.

7. A storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the target object positioning method according to any one of claims 1 to 4 are implemented.

8. An electronic device, characterized in that: The method comprises at least a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the target object positioning method according to any one of claims 1 to 4 when executing the computer program on the memory.

Images