Target object positioning method, device, medium, equipment and trajectory mapping method

Abstract

The present application discloses a method, device, medium, equipment and trajectory mapping method for positioning a target object, including: shooting at predetermined reference shooting positions in advance for a target area to collect a number of reference aerial images; the total shooting range of each reference aerial image fully covers the target area; real-time acquisition of the current aerial image shot by the aircraft for the target area; based on the coverage range of the current aerial image, determining a number of target reference aerial images that match the current aerial image from each reference aerial image; based on the current aerial image, each target reference aerial image and the reference position coordinates of the reference object in each target reference aerial image, determining the target position coordinates of the target object in the current aerial image. The present application can accurately and quickly determine the actual position coordinates of the target object in the current aerial image, realizes the automatic mapping of the pixel position of the target object to the actual spatial position, and improves the positioning accuracy and positioning speed.

Description

Technical Field

[0001] The present invention relates to the field of electronic information technology, and in particular to a target object positioning method, device, medium, equipment and trajectory mapping method. Background Art

[0002] With its significant advantages such as strong mobility, high image resolution and low cost, UAV remote sensing systems are widely used in many fields, especially in scenarios where large-scale, high-resolution image data needs to be quickly obtained. The high-performance sensors carried by UAVs can collect image data in real time, which are usually displayed in the form of video streams for intuitive observation. However, such video images only provide visual information and lack positioning data associated with geographic space, making it difficult to provide decision makers with accurate spatial references. As a result, methods for positioning target objects in aerial images have emerged. At present, there are mainly two types of methods for positioning ground targets by UAVs. One method is to locate the target through the telemetry data of the UAV, using the instantaneous distance and angle information relative to the target during the UAV aerial photography process, as well as the position information and attitude data read in the flight control system, to solve and obtain the geographic coordinates of the target. The other method is based on image matching technology. First, the image taken by the UAV is matched with the reference map, and then the coordinate conversion relationship between the images is established, and finally the position information of the target is calculated.

[0003] However, the positioning method using UAV telemetry data is highly dependent on the positioning accuracy of the UAV itself. When the UAV is hovering in the air, it is disturbed by external factors and the positioning accuracy is often unstable. Although the positioning method based on image matching can provide more accurate trajectory positioning, the difference in spatial resolution between the image and the map makes it difficult to match the image with the map, and the positioning accuracy is difficult to guarantee in complex environments.

[0004] Therefore, there is an urgent need for a target object positioning method to solve the problem in the prior art that the target object in the aerial image cannot be accurately and quickly positioned. Summary of the invention

[0005] In view of this, the present invention provides a target object positioning method, device, medium, equipment and trajectory mapping method, the main purpose of which is to solve the current problem that it is impossible to accurately and quickly locate the target object in the aerial image.

[0006] To solve the above problems, the present application provides a target object positioning method, comprising: For the target area, photographing is performed in advance at each predetermined reference photographing position to collect a number of reference aerial images; wherein the total photographing range of each of the reference aerial images fully covers the target area; Acquire the current aerial image taken by the aircraft for the target area in real time; Based on the coverage of the current aerial image, determining a plurality of target reference aerial images matching the current aerial image from the reference aerial images; The target position coordinates of the target object in the current aerial image are determined based on the current aerial image, each of the target reference aerial images, and the reference position coordinates of the reference object in each of the target reference aerial images.

[0007] Optionally, before photographing the target area at each predetermined reference photographing position in advance, the method further includes: predetermining each of the reference photographing positions, specifically including: Determine a shooting range of the reference aerial image based on a horizontal field of view angle of the shooting device, a vertical field of view angle of the shooting device, and a flight altitude of the aircraft; Based on the shooting range and the total area range of the target area, each reference shooting position is determined.

[0008] Optionally, the shooting height of the current aerial image is lower than or equal to the shooting height of each of the reference aerial images; The determining, based on the coverage of the current aerial image, a plurality of target reference aerial images matching the current aerial image from the reference aerial images specifically includes: Determining a target position interval based on a current shooting position of the current aerial image; Based on the reference shooting position of each of the reference aerial images, a reference aerial image whose reference shooting position is within the target position interval is determined as the target reference aerial image.

[0009] Optionally, the determining the target position interval based on the current shooting position of the current aerial image specifically includes: Determine the heading distance and the lateral distance based on the shooting height of the current aerial image, the horizontal field of view angle of the shooting device, and the vertical field of view angle of the shooting device; The target position interval is determined based on the current shooting position, the heading distance, and the lateral distance.

[0010] Optionally, determining the target position coordinates of the target object in the current aerial image based on the current aerial image, each of the target reference aerial images, and the reference position coordinates of the reference object in each of the target reference aerial images specifically includes: Determine, based on the current aerial image, each of the target reference aerial images, and first pixel coordinates of reference objects in each of the target reference aerial images, second pixel coordinates of each of the reference objects located in the current aerial image; the number of each of the reference objects is greater than or equal to 4; The target position coordinates of the target object are determined based on the second pixel coordinates corresponding to each of the reference objects, the reference position coordinates of each of the reference objects, and the target pixel coordinates of the target object in the current aerial image.

[0011] Optionally, determining the second pixel coordinates of each reference object in the current aerial image based on the current aerial image, each of the target reference aerial images, and the first pixel coordinates of the reference object in each of the target reference aerial images specifically includes: Based on the current aerial image and each of the target reference aerial images, determining a transformation matrix between the current aerial image and each of the target reference aerial images; Based on the transformation matrix corresponding to each of the target reference aerial images and the first pixel coordinates of the reference object in each of the target reference aerial images, the second pixel coordinates of each of the reference objects in the current aerial image are determined.

[0012] To solve the above problems, the present application provides a trajectory mapping method, comprising: using any one of the target object positioning methods described above to locate the target object in each video frame of the aerial video in real time, and obtaining each target position coordinate of the target object; The target coordinate trajectory of the target object is determined based on each of the target position coordinates to obtain a trajectory mapping result.

[0013] To solve the above problems, the present application provides a target object positioning device, including: A collection module is used to pre-shoot at each predetermined reference shooting position for the target area to collect a number of reference aerial images; wherein the total shooting range of each of the reference aerial images fully covers the target area; An acquisition module, used for acquiring in real time the current aerial image taken by the aircraft for the target area; A matching module, configured to determine, from each of the reference aerial images, a plurality of target reference aerial images that match the current aerial image based on the coverage of the current aerial image; The determination module is used to determine the target position coordinates of the target object in the current aerial image based on the current aerial image, each of the target reference aerial images and the reference position coordinates of the reference object in each of the target reference aerial images.

[0014] To solve the above problems, the present application provides a storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the steps of any of the above target object positioning methods are implemented.

[0015] To solve the above problems, the present application provides an electronic device, comprising at least a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program in the memory, the steps of any of the above target object positioning methods are implemented.

[0016] A target object positioning method, device, medium, equipment and trajectory mapping method in the present application, by pre-collecting reference aerial images covering the entire target area, the current aerial image can be matched with each reference aerial image later, so that the target reference aerial image overlapping with the current aerial image can be quickly and accurately determined, and then the actual position coordinates of the target object in the current aerial image can be accurately and quickly determined according to the pixel coordinates of each reference object in each target reference aerial image and the reference position coordinates / actual spatial position, thereby realizing the automatic mapping of the pixel position of the target object to the actual spatial position, and improving the positioning accuracy and positioning speed.

[0017] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings: Figure 1 This is a flow chart of a method for locating a target object according to an embodiment of the present application; Figure 2 This is a schematic diagram of the shooting range of a shooting device according to another embodiment of the present application; Figure 3 (a) In the embodiment of the present application, the heading / X UAV Schematic diagram of the positional relationship between two adjacent reference aerial images in the axial direction; FIG. 3 (b) shows the embodiment of the present application along the side / Y UAV Schematic diagram of the positional relationship between two adjacent reference aerial images in the axial direction; FIG4 (a) is an index table constructed based on a reference aerial image in another embodiment of the present application; FIG4( b ) is a flowchart of matching the current aerial image with each reference aerial image in another embodiment of the present application; FIG4 (c) is a schematic diagram of matching results for a reference aerial image in another embodiment of the present application; Figure 5 This is a structural block diagram of a target object positioning device according to another embodiment of the present application; Figure 6 This is a structural block diagram of an electronic device according to another embodiment of the present application. DETAILED DESCRIPTION

[0019] Various aspects and features of the present application are described herein with reference to the accompanying drawings.

[0020] It should be understood that various modifications may be made to the embodiments of the present application. Therefore, the above description should not be considered as limiting, but only as an example of an embodiment. Other modifications within the scope and spirit of the present application will occur to those skilled in the art.

[0021] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the present application and, together with the general description of the present application given above and the detailed description of the embodiments given below, serve to explain the principles of the present application.

[0022] These and other characteristics of the present application will become apparent from the following description of a preferred form of embodiment given as a non-limiting example with reference to the accompanying drawings.

[0023] It should also be understood that although the present application has been described with reference to some specific examples, those skilled in the art will be able to readily implement many other equivalent forms of the present application.

[0024] The above and other aspects, features and advantages of the present application will become more apparent in view of the following detailed description when taken in conjunction with the accompanying drawings.

[0025] Specific embodiments of the present application are described hereinafter with reference to the accompanying drawings; however, it should be understood that the embodiments applied for are merely examples of the present application, which may be implemented in a variety of ways. Well-known and / or repeated functions and structures are not described in detail to avoid unnecessary or redundant details that obscure the present application. Therefore, the specific structural and functional details applied for herein are not intended to be limiting, but merely serve as a basis and representative basis for the claims to teach those skilled in the art to use the present application in a variety of ways with substantially any suitable detailed structure.

[0026] This specification may use the phrases "in one embodiment," "in another embodiment," "in yet another embodiment," or "in other embodiments," all of which may refer to one or more of the same or different embodiments according to the present application.

[0027] The embodiments of the present application provide a method, apparatus, medium, device and trajectory mapping method for locating a target object, which can be specifically applied to electronic devices such as terminals and servers, such as Figure 1 As shown, a target object positioning method in this embodiment includes the following steps: Step S101, photographing a target area at predetermined reference photographing positions in advance to collect a number of reference aerial images; wherein the total photographing range of each of the reference aerial images fully covers the target area; In the specific implementation process of this embodiment, the shooting range of each reference aerial image can be determined according to the shooting height / flight height of the aircraft, the horizontal field of view of the shooting device, and the vertical field of view of the shooting device. Finally, each reference shooting position is determined according to the shooting range of each aerial image and the total area range of the target area. In this step, the shooting height / flight height of the aircraft corresponding to each reference shooting position can be different or the same, as long as the total shooting range after merging all reference aerial images fully covers the target area. Among them, each reference aerial image contains a number of reference objects.

[0028] Step S102, acquiring in real time a current aerial image taken by the aircraft for the target area; In the specific implementation process of this step, after the aircraft / drone is hovering, the real-time video image changes very little and can be ignored because the position of the drone changes little in the hovering state and the lens is equipped with a gimbal. Therefore, a video frame is extracted from the real-time video as the current aerial image.

[0029] Step S103, based on the coverage of the current aerial image, determining a plurality of target reference aerial images matching the current aerial image from the reference aerial images; In the specific implementation process of this step, based on the coverage of the current aerial image and the coverage of each reference aerial image, several target reference aerial images with overlapping areas with the current aerial image can be determined from each reference aerial image, thereby achieving dynamic matching of the current aerial image and the reference aerial image.

[0030] Step S104 , determining the target position coordinates of the target object in the current aerial image based on the current aerial image, each of the target reference aerial images, and the reference position coordinates of the reference object in each of the target reference aerial images.

[0031] In the specific implementation process of this step, the reference objects in each target reference aerial image can be mapped to the current aerial image to obtain the second pixel coordinates of each reference object in the current aerial image. Then, the target position coordinates / actual spatial position of each reference object and the target pixel coordinates of the target object in the current aerial image are combined to determine the target position coordinates / actual spatial position of the target object, thereby realizing the spatial positioning of the target object.

[0032] The method in this embodiment, by pre-collecting reference aerial images covering the entire target area, can subsequently match the current aerial image with the reference aerial images, so as to quickly and accurately determine the target reference aerial image overlapping with the current aerial image, and then can accurately and quickly determine the actual position coordinates of the target object in the current aerial image based on the pixel coordinates of each reference object in each target reference aerial image and the reference position coordinates / actual spatial position, thereby realizing the automatic mapping of the pixel position of the target object to the actual spatial position, thereby improving the positioning accuracy and positioning speed.

[0033] Another embodiment of the present application provides a method for locating a target object, which specifically includes the following steps: Step S201, predetermine a number of reference objects; In the specific implementation process of this step, several reference objects and the reference position coordinates / actual spatial positions of each reference object can be pre-determined for the target area. That is, a certain number of reference objects are arranged in the aerial photography area / target area, and they are distributed as evenly as possible. The positions of the reference objects need to be selected to be relatively fixed to ensure that the position changes little in different time periods. A unique identifier can be configured for each reference object in the format of: P c =<cid,X,Y> , so as to facilitate subsequent identification and matching. Where cid represents the reference object ID, and (X, Y) represents the location coordinates of the reference object.

[0034] Step S202, predetermine each reference shooting position; In this step, the shooting range of the reference aerial image can be determined based on the horizontal field of view angle of the shooting device, the vertical field of view angle of the shooting device, and the flight altitude of the aircraft / drone; and then, based on the shooting range and the total area range of the target area, each reference shooting position can be determined.

[0035] Specifically, in order to improve the accuracy of target positioning, aerial photography sites can be set up in advance to facilitate the subsequent shooting of aerial reference images. First, design the operation route of the drone / aircraft according to the aerial photography area / target area, and calculate the coverage range of the video / image based on the field of view angle and flight altitude of the camera. When the drone flies at a constant altitude and keeps the camera pointing vertically downward, its video coverage range / shooting range can be simplified, as Figure 2 shown. Wherein, H, ψ, and θ respectively represent the flight altitude, horizontal field of view angle, and vertical field of view angle, point O represents the vertical projection of the drone on the ground (i.e., the shooting position), and the rectangular area represents the coverage range of the camera / shooting device. At this time, L = 2×H×tanψ is the length of the rectangular area / shooting range, and W = 2×H×tanθ is the width of the rectangular area / shooting range.

[0036] After determining the shooting range value, the reference shooting positions of the reference aerial images can be determined / set according to the shooting range of the drone camera, as shown in Figures 3(a) and 3(b). During the setting process, it is stipulated that the flight distance along the X UAV axis is the heading flight distance Lp, and the flight distance along the Y UAV axis is the cross-track flight distance Wq. To ensure that each reference aerial image contains reference points and has sufficient overlap, it is necessary to control the distance between the reference shooting positions to avoid non-overlap between adjacent reference images. According to the camera shooting range and the actual ground situation, select appropriate heading flight distance Lp and cross-track flight distance Wq such that they satisfy Lp < L or Wq < W, that is, the heading flight distance Lp = X i -X i-1 between two adjacent reference shooting positions, and the cross-track flight distance Wq = Y i -Y i-1 . Thus, a number of reference shooting positions (X i , Y i ) can be obtained.

[0037] Step S203, for the target area, pre-shoot at each predetermined reference shooting position to collect a number of reference aerial images; wherein, the total shooting range of each of the reference aerial images fully covers the target area.

[0038] In this step, after determining each reference shooting position, shooting can be carried out at each reference shooting position to obtain the reference aerial images corresponding to each reference shooting position.

[0039] In this step, after obtaining a number of reference aerial images, each reference aerial image can be expressed in the form of a quadruple: pic = <pid i , X i , Y i, pts>. Where pid is the number of the i-th reference aerial image; (X i , Y i ) represents the spatial coordinates of the reference aerial image taken by the drone, that is, the reference shooting position. pts is the set of reference objects in the image, represented by pts={ c1 , P c1 >, …, cn , P cn >}. Among them, P cn is the nth reference object identifier, P cn = <cid n , X n , Y n >I cn is the point in the image corresponding to the reference object, I c =<pcid,u,v> It consists of 3 tuples, which are number and pixel coordinates, so pts can also be expressed as: pts={<pcid1,u1,v1,cid1,x1,y1> ,…, <pcid n ,u n ,v n ,cid n ,x n ,y n >}.

[0040] Step S203, acquiring in real time a current aerial image taken by the aircraft for the target area; In this step, after the aircraft / drone is hovering, since the position of the drone changes little in the hovering state and the lens is equipped with a gimbal, the real-time video picture changes very little and can be ignored. Therefore, a video frame is extracted from the real-time video as the current aerial image.

[0041] Step S204, based on the coverage of the current aerial image, determining a plurality of target reference aerial images matching the current aerial image from the reference aerial images; In this step, the following method can be used to match and obtain several target reference aerial images that overlap with the current aerial image. Taking the shooting height of the current aerial image as an example, when determining each target reference aerial image, the heading distance Lr and the lateral distance Wr can be determined based on the shooting height of the current aerial image, the horizontal field of view angle of the shooting device, and the vertical field of view angle of the shooting device; based on the current shooting position, the heading distance Lr and the lateral distance Wr, the target position interval [R X -Lr, R X +Lr] and [R Y -Wr, R Y ​​+Wr]. Based on the reference shooting positions of the reference aerial images, a reference aerial image whose reference shooting position is within the target position interval is determined as the target reference aerial image.

[0042] Specifically, the process of matching and obtaining the reference aerial images of each target is as follows: Step 1: Build an index: In order to quickly and accurately find the relevant target reference aerial images in the subsequent dynamic matching / image matching process, it is necessary to build an index for these images. According to the reference shooting position (X, Y) of the reference aerial image in step S203, sort them according to their positions, and generate an index table after arranging the X and Y coordinate values ​​in ascending order. Among them, X represents the row, Y represents the column, and the combination of X and Y corresponds to the ID of each image, as shown in Figure 4 (a).

[0043] Step 2: Image query: When performing dynamic matching, query the reference aerial image set based on the X and Y of the current drone's real-time position.

[0044] The specific method is as follows: The coordinates of the current shooting position of the drone are recorded as R X and R Y , the range of the required aerial reference image is P X and P Y . First, the query in the X direction. In order to ensure that the reference aerial image set around the current shooting position of the drone can be found during the retrieval, the retrieval range should not be less than ±Lr, that is, Lr=2×Hr×tanψ, Wr=2×Hr×tanθ, where Hr is the real-time drone's flight altitude, ψ and θ represent the horizontal and vertical field of view angles, respectively. Then the retrieval range of the aerial reference image in the X direction of the drone's real-time position is R X -Lr<=P X <=R X +Lr. Similarly, the Y direction uses the above method to select the index range, that is, the retrieval range of the aerial reference image in the direction of the real-time position of the drone is R Y -Wr<=P Y <=R Y +Wr. When the process shown in Figure 4 (b) is used for retrieval / matching, the aerial reference image ID sets that meet the required X value and Y value range can be retrieved respectively, and the intersection of the two is taken to complete the retrieval, and the reference aerial image index ID set can be obtained. As shown in Figure 4 (c), the intersection of the two dotted rectangles can find the corresponding reference aerial images of each target according to the ID.

[0045] Step S205, determining the second pixel coordinates of each reference object in the current aerial image based on the current aerial image, each of the target reference aerial images, and the first pixel coordinates of the reference objects in each of the target reference aerial images; the number of each of the reference objects is greater than or equal to 4; During the specific implementation of this step, the transformation matrix between the current aerial image and each target reference aerial image can be determined based on the current aerial image and each target reference aerial image; the second pixel coordinates of each reference object in the current aerial image can be determined based on the transformation matrix corresponding to each target reference aerial image and the first pixel coordinates of the reference object in each target reference aerial image.

[0046] Specifically, this step may use the following process to map each reference object in each target reference aerial image to the current aerial image. Specifically, it includes: Step 1: Solve the transformation matrix between the real-time video / current aerial image and the reference aerial images of each target.

[0047] In this step, since there are many real-time video frames taken by the drone, the video frames are redundant and the matching process is time-consuming. Therefore, a video frame F can be selected as the current aerial image to replace the real-time video to establish the relationship between the image and the ground. The specific method is that after the drone hovers, the position change of the drone is small in the hovering state and the lens is equipped with a gimbal, and the real-time video picture changes very little and can be ignored. Therefore, a frame F is extracted from the real-time video as the video frame / current aerial image that establishes a connection with the ground point. Specifically, a method combining SURF (Speeded-Up Robust Features), FLANN (Fast Library for Approximate Nearest Neighbors) and RANSAC (Random Sample Consensus) algorithms can be used to perform feature matching between the video frame F and the aerial reference image set in turn, and calculate the transformation matrix between the video frame F and each aerial reference image.

[0048] Step 2: transform the pixel coordinates of the reference objects in the target reference aerial image set into the video frame F / current aerial image according to each transformation matrix.

[0049] Step S206 , determining the target position coordinates of the target object based on the second pixel coordinates corresponding to each of the reference objects, the reference position coordinates of each of the reference objects, and the target pixel coordinates of the target object in the current aerial image.

[0050] In the specific implementation process, this step specifically solves the mutual mapping relationship between the video frame F / current aerial image and the spatial information. That is, after the pixel coordinates of the reference object in the target reference aerial image set are transformed into the video frame F / current aerial image, the mapping relationship between the ground point and the pixel point can be established by using the homography transformation based on the matched reference object and the pixel coordinates in the video frame F / current aerial image, thereby realizing the mutual mapping between the video frame F / current aerial image and the spatial information, so that each pixel coordinate in the video frame F / current aerial image corresponds to a geographic spatial position, so that the target position coordinates of the target object in the current aerial image can be accurately and quickly determined.

[0051] In the specific implementation process of this embodiment, when the position of the drone changes, the above steps S204 to S206 are executed in sequence on the changed real-time video, thereby realizing the dynamic fusion of the drone video and spatial information and the precise positioning of the target object.

[0052] In the specific implementation process of this embodiment, the target object in the current aerial image can be identified by target detection and tracking, so as to determine the target pixel coordinates of the target object. Specifically, the YOLO target detection algorithm can be used to implement it. This embodiment adopts the YOLOv8 detection model to detect the target object in the current aerial image. YOLOv8 adopts a single-stage network structure, which can quickly detect the target object in the current aerial image and output the bounding box, confidence and category label of the target. For target tracking, a deep learning multi-target tracking algorithm (DeepSort algorithm) is used. DeepSort combines the appearance features and motion information of the target, predicts the target trajectory through the Kalman filter, and uses the Hungarian algorithm for target matching and ID assignment. This enables DeepSort to maintain the tracking ID of the target and update the target position in real time to ensure accurate tracking in complex environments. Combining YOLOv8 with DeepSort can achieve efficient and stable target detection and tracking, thereby outputting the target pixel coordinates of the target object in the drone video / current aerial image, which is convenient for the subsequent determination of the actual spatial coordinates / target position coordinates of the target object based on the target pixel coordinates.

[0053] The method in this embodiment selects a reference object according to the target area and collects a reference aerial image with the reference object, obtains the pixel coordinates of the reference object in the current video / current aerial image by dynamically matching the reference aerial image set with the reference object, and then uses the homography transformation method to establish a mutual mapping relationship between the image and the geographic space. At the same time, the dynamic target in the real-time video / current aerial image is detected and tracked, and the pixel coordinates of the target object in the real-time video / current aerial image are mapped to the geographic space according to the mutual mapping relationship, thereby realizing the fusion and positioning of the dynamic target in the drone video and the geographic space information, and improving the accuracy of the positioning result.

[0054] Another embodiment of the present application provides a trajectory mapping method, including: Step 1: Using a target object positioning method in any of the above embodiments, the target object in each video frame of the aerial video is positioned in real time to obtain the target position coordinates of each target object; Step 2: determining the target coordinate trajectory of the target object based on the target position coordinates to obtain a trajectory mapping result.

[0055] In the specific implementation process of this embodiment, the target object in the drone video can be identified and tracked by YOLOv8 and DeepSort to obtain the pixel coordinates corresponding to each target ID. Then, a target object positioning method in any of the above embodiments is used to map the pixel coordinates of the target object to the actual position coordinates, that is, to convert the sequence pixel trajectories of these target objects into geographic space coordinates, thereby realizing the mapping and fusion of the target trajectory in the drone video and the geographic space information.

[0056] The method in this embodiment can accurately and quickly determine the actual position coordinates of the target object, realize automatic mapping of the pixel position of the target object to the actual spatial position, and improve the positioning accuracy and positioning speed.

[0057] Another embodiment of the present application provides a positioning device for a target object, such as Figure 5 As shown, including: The acquisition module 11 is used to pre-shoot at each predetermined reference shooting position for the target area to acquire a number of reference aerial images; wherein the total shooting range of each of the reference aerial images fully covers the target area; An acquisition module 12 is used to acquire in real time a current aerial image taken by the aircraft for the target area; A matching module 13 is used to determine, from each of the reference aerial images, a number of target reference aerial images that match the current aerial image based on the coverage of the current aerial image; The determination module 14 is configured to determine the target position coordinates of the target object in the current aerial image based on the current aerial image, each of the target reference aerial images, and the reference position coordinates of the reference object in each of the target reference aerial images.

[0058] During the specific implementation of this embodiment, the device also includes a position determination module for pre-determining each of the reference shooting positions, and the position determination module is specifically used to: determine the shooting range of the reference aerial image based on the horizontal field of view angle of the shooting device, the vertical field of view angle of the shooting device, and the flight altitude of the aircraft; determine each of the reference shooting positions based on the shooting range and the total area range of the target area.

[0059] During the specific implementation of this embodiment, the shooting height of the current aerial image is lower than or equal to the shooting height of each of the reference aerial images; the matching module 13 is specifically used to: determine the target position interval based on the current shooting position of the current aerial image; based on the reference shooting position of each of the reference aerial images, determine that the reference aerial image whose reference shooting position is within the target position interval is the target reference aerial image.

[0060] In the specific implementation process of this embodiment, the matching module 13 is specifically used to: determine the heading distance and the lateral distance based on the shooting height of the current aerial image, the horizontal field of view angle of the shooting device, and the vertical field of view angle of the shooting device; determine the target position interval based on the current shooting position, the heading distance and the lateral distance.

[0061] In the specific implementation process of this embodiment, the determination module 14 specifically includes a first determination unit and a second determination unit; the first determination unit is used to: determine the second pixel coordinates of each reference object in the current aerial image based on the current aerial image, each target reference aerial image, and the first pixel coordinates of the reference object in each target reference aerial image; the number of each reference object is greater than or equal to 4; the second determination unit is used to: determine the target position coordinates of the target object based on the second pixel coordinates corresponding to each reference object, the reference position coordinates of each reference object, and the target pixel coordinates of the target object in the current aerial image.

[0062] In the specific implementation process of this embodiment, the first determination unit is used to: determine the transformation matrix between the current aerial image and each target reference aerial image based on the current aerial image and each target reference aerial image; determine the second pixel coordinates of each reference object in the current aerial image based on the transformation matrix corresponding to each target reference aerial image and the first pixel coordinates of the reference object in each target reference aerial image.

[0063] The device in this embodiment can pre-collect reference aerial images covering the entire target area, and then match the current aerial image with each reference aerial image, so as to quickly and accurately determine the target reference aerial image overlapping with the current aerial image, and then accurately and quickly determine the actual position coordinates of the target object in the current aerial image based on the pixel coordinates of each reference object in each target reference aerial image and the reference position coordinates / actual spatial position, thereby realizing the automatic mapping of the pixel position of the target object to the actual spatial position, and improving the positioning accuracy and positioning speed.

[0064] Another embodiment of the present application provides a storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the following method steps are implemented: Step 1: photographing the target area at predetermined reference shooting positions in advance to collect a number of reference aerial images; wherein the total shooting range of each of the reference aerial images fully covers the target area; Step 2: acquiring in real time the current aerial image taken by the aircraft for the target area; Step 3: Based on the coverage of the current aerial image, determine a number of target reference aerial images that match the current aerial image from the reference aerial images; Step 4: Determine the target position coordinates of the target object in the current aerial image based on the current aerial image, each of the target reference aerial images, and the reference position coordinates of the reference object in each of the target reference aerial images.

[0065] The specific implementation process of the above method steps can refer to any of the above embodiments of the target object positioning method, and this embodiment will not be repeated here.

[0066] The storage medium in the present application can pre-collect reference aerial images covering the entire target area, and then match the current aerial image with each reference aerial image, so as to quickly and accurately determine the target reference aerial image overlapping with the current aerial image, and then accurately and quickly determine the actual position coordinates of the target object in the current aerial image according to the pixel coordinates of each reference object in each target reference aerial image and the reference position coordinates / actual spatial position, thereby realizing the automatic mapping of the pixel position of the target object to the actual spatial position, and improving the positioning accuracy and positioning speed.

[0067] Another embodiment of the present application provides an electronic device, such as Figure 6 As shown, at least a memory 1 and a processor 2 are included, the memory 1 stores a computer program, and the processor 2 implements the following method steps when executing the computer program on the memory 1: Step 1: photographing the target area at predetermined reference shooting positions in advance to collect a number of reference aerial images; wherein the total shooting range of each of the reference aerial images fully covers the target area; Step 2: acquiring in real time the current aerial image taken by the aircraft for the target area; Step 3: Based on the coverage of the current aerial image, determine a number of target reference aerial images that match the current aerial image from the reference aerial images; Step 4: Determine the target position coordinates of the target object in the current aerial image based on the current aerial image, each of the target reference aerial images, and the reference position coordinates of the reference object in each of the target reference aerial images.

[0068] The specific implementation process of the above method steps can refer to any of the above embodiments of the target object positioning method, and this embodiment will not be repeated here.

[0069] The electronic device in the present application can pre-collect reference aerial images covering the entire target area, and then match the current aerial image with each reference aerial image, so as to quickly and accurately determine the target reference aerial image overlapping with the current aerial image, and then accurately and quickly determine the actual position coordinates of the target object in the current aerial image according to the pixel coordinates of each reference object in each target reference aerial image and the reference position coordinates / actual spatial position, thereby realizing the automatic mapping of the pixel position of the target object to the actual spatial position, and improving the positioning accuracy and positioning speed.

[0070] The above embodiments are only exemplary embodiments of the present application and are not intended to limit the present application. The protection scope of the present application is defined by the claims. Those skilled in the art may make various modifications or equivalent substitutions to the present application within the essence and protection scope of the present application, and such modifications or equivalent substitutions shall also be deemed to fall within the protection scope of the present application.

Claims

1. A method for locating a target object, characterized in that: include: For the target area, photographing is performed in advance at each predetermined reference photographing position to collect a number of reference aerial images; wherein the total photographing range of each of the reference aerial images fully covers the target area; Acquire the current aerial image taken by the aircraft for the target area in real time; Based on the coverage of the current aerial image, determining a plurality of target reference aerial images matching the current aerial image from the reference aerial images; The target position coordinates of the target object in the current aerial image are determined based on the current aerial image, each of the target reference aerial images, and the reference position coordinates of the reference object in each of the target reference aerial images.

2. The method according to claim 1, characterized in that Before photographing the target area at each predetermined reference photographing position in advance, the method further includes: predetermining each of the reference photographing positions; Wherein, the predetermining each of the reference shooting positions specifically includes: Determine a shooting range of the reference aerial image based on a horizontal field of view angle of the shooting device, a vertical field of view angle of the shooting device, and a flight altitude of the aircraft; Based on the shooting range and the total area range of the target area, each reference shooting position is determined.

3. The method according to claim 1, characterized in that The shooting height of the current aerial image is lower than or equal to the shooting height of each of the reference aerial images; The determining, based on the coverage of the current aerial image, a plurality of target reference aerial images matching the current aerial image from the reference aerial images specifically includes: Determining a target position interval based on a current shooting position of the current aerial image; Based on the reference shooting position of each of the reference aerial images, a reference aerial image whose reference shooting position is within the target position interval is determined as the target reference aerial image.

4. The method according to claim 3, characterized in that The determining of the target position interval based on the current shooting position of the current aerial image specifically includes: Determine the heading distance and the lateral distance based on the shooting height of the current aerial image, the horizontal field of view angle of the shooting device, and the vertical field of view angle of the shooting device; The target position interval is determined based on the current shooting position, the heading distance, and the lateral distance.

5. The method according to claim 1, characterized in that The determining the target position coordinates of the target object in the current aerial image based on the current aerial image, each of the target reference aerial images, and the reference position coordinates of the reference object in each of the target reference aerial images specifically includes: Determine, based on the current aerial image, each of the target reference aerial images, and first pixel coordinates of reference objects in each of the target reference aerial images, second pixel coordinates of each of the reference objects located in the current aerial image; the number of each of the reference objects is greater than or equal to 4; The target position coordinates of the target object are determined based on the second pixel coordinates corresponding to each of the reference objects, the reference position coordinates of each of the reference objects, and the target pixel coordinates of the target object in the current aerial image.

6. The method according to claim 5, characterized in that The determining, based on the current aerial image, each of the target reference aerial images, and the first pixel coordinates of the reference objects in each of the target reference aerial images, the second pixel coordinates of each of the reference objects in the current aerial image specifically includes: Based on the current aerial image and each of the target reference aerial images, determining a transformation matrix between the current aerial image and each of the target reference aerial images; Based on the transformation matrix corresponding to each of the target reference aerial images and the first pixel coordinates of the reference object in each of the target reference aerial images, the second pixel coordinates of each of the reference objects in the current aerial image are determined.

7. A trajectory mapping method, characterized in that: include: Using a target object positioning method as described in any one of claims 1 to 6, the target object in each video frame of the aerial video is positioned in real time to obtain each target position coordinate of the target object; The target coordinate trajectory of the target object is determined based on each of the target position coordinates to obtain a trajectory mapping result.

8. A device for locating a target object, characterized in that: include: A collection module is used to pre-shoot at each predetermined reference shooting position for the target area to collect a number of reference aerial images; wherein the total shooting range of each of the reference aerial images fully covers the target area; An acquisition module, used for acquiring in real time the current aerial image taken by the aircraft for the target area; A matching module, configured to determine, from each of the reference aerial images, a plurality of target reference aerial images that match the current aerial image based on the coverage of the current aerial image; The determination module is used to determine the target position coordinates of the target object in the current aerial image based on the current aerial image, each of the target reference aerial images and the reference position coordinates of the reference object in each of the target reference aerial images.

9. A storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the target object positioning method according to any one of claims 1 to 6 are implemented.

10. An electronic device, characterized in that: The method comprises at least a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the target object positioning method according to any one of claims 1 to 6 when executing the computer program on the memory.

Images