Field scene non-cooperative target measurement method

Abstract

The invention provides a field scene non-cooperative target measurement method, which comprises the following steps that: step 1, an unmanned aerial vehicle shoots multiple frames of field scene images through hovering, obtains high-precision self-positioning geographic coordinates of pixels corresponding to a cooperative target at corresponding moments in each frame of image, and establishes a control point set among the coordinates; step 2, fitting the control point set by using a quadric surface function, and obtaining a mapping relation of mutual mapping between coordinates of any pixel in the image based on fitting of each control point; and step 3, according to the pixel coordinates of the non-cooperative target in the aerial image, calculating to obtain the geographic coordinates of the non-cooperative target according to the mapping relation, and realizing the positioning measurement of the non-cooperative target in the field scene. According to the invention, the real-time position of the non-cooperative target such as the unmanned vehicle is used as the control point of the aerial image of the unmanned aerial vehicle in the field scene, and the positioning detection of the non-cooperative target on the flat ground in the image in the field scene is realized.

Description

Technical Field

[0001] This invention belongs to the field of aerial target measurement technology, specifically relating to a method for measuring non-cooperative targets in field scenes. Background Technology

[0002] Drone aerial photography is widely used in fields such as land surveillance, environmental protection, military reconnaissance, and target search. Target localization within drone aerial images is a hot topic in both drone aerial photography and target localization. Currently, target localization methods based on drone aerial images mainly include binocular vision localization, two-drone intersection measurement, coordinate transformation methods relying on drone attitude information, and scene matching-based target localization methods.

[0003] Among them, the binocular vision positioning method is suitable for close-range targets, but its camera intrinsic and extrinsic parameters (focal length, baseline distance, distortion) need to be calibrated with high precision. When used at long distances, the error is large. The two-machine intersection measurement method and the coordinate transformation method that relies on the attitude information of the UAV require accurate information such as the position, attitude and gimbal pointing of the UAV. They are only suitable for UAVs equipped with high-precision gimbals and have high cost requirements. The target positioning method based on scene matching is sensitive to environmental changes and is not suitable for repetitive textures and weak feature scenes, such as the Gobi Desert, desert and ocean. Summary of the Invention

[0004] The purpose of this invention is to address the shortcomings of calibration dependence and scene dependence in non-cooperative target localization and measurement based on aerial images, and to provide a method for non-cooperative target measurement in field scenes. In field scenes, the real-time position of cooperative targets such as unmanned vehicles is used as the control point of the UAV aerial image. Through calculation, the localization and target detection of non-cooperative targets on flat ground in the aerial image of the field scene are realized.

[0005] To achieve the above objectives, the technical solution provided by this invention is:

[0006] A method for measuring non-cooperative targets in a field setting, comprising:

[0007] Step 1: The UAV hovers and captures images of the field scene containing both cooperative and non-cooperative targets moving within the camera's field of view. Based on the pixel coordinates of the cooperative targets in multiple frames of the field scene images and the self-localized geographic coordinates of the cooperative targets in the corresponding images, a set of control points for the cooperative targets in the field scene images is established.

[0008] Step 2: Fit the control point set obtained in Step 1 using a quadratic surface function to obtain the mapping relationship between any pixel coordinates in the aerial image and geographic coordinates;

[0009] Step 3: Based on the pixel coordinates of the non-cooperative target in the aerial image and the mapping relationship obtained in Step 2, calculate the geographic coordinates of the non-cooperative target to achieve the positioning and measurement of the non-cooperative target in the field.

[0010] As a further limitation of the present invention, in step one, establishing a set of control points for the cooperative target in the field scene images based on the pixel coordinates of the cooperative target in multiple frames of field scene images and the geographic coordinates of the cooperative target in the corresponding frame images further includes:

[0011] When the drone hovers and takes pictures of an area containing both cooperative and non-cooperative targets, it acquires the pixel coordinates of the cooperative target in the maneuvering state in the aerial image frame, as well as the precise self-localization geographic coordinates of the cooperative target at the corresponding moment in the image, and uses them as a control point in the field scene image.

[0012] The pixel coordinates and high-precision geographic coordinates of the cooperative target at different locations in the images are continuously obtained from multiple frames of field scene images, and a set of control points between the pixel coordinates of the cooperative target in the field scene images and the geographic coordinates of the corresponding pixels are established.

[0013] As a further limitation of the present invention, in step one, establishing a set of control points for the cooperative target in the field scene image from pixel coordinates to geographic coordinates specifically includes:

[0014] While the drone is hovering, an aerial camera captures images of the field scene within the target area during continuous maneuvering of the cooperative target. This yields multiple frames of field scene images ordered by time, and the geographic coordinates of the cooperative target at the same moment are obtained from these multiple frames. The selection criteria for these multiple frames are as follows: In the continuously captured and time-ordered field scene images, if a cooperative target is detected in the current field scene image and there is pixel movement between the cooperative target and the field scene image at the previous control point, and the moved pixels are greater than a preset threshold, then the pixel coordinates of the cooperative target are extracted from the current field scene image, and the geographic coordinates of the cooperative target in that frame are simultaneously obtained, and it is used as a control point; otherwise, the detection of cooperative targets continues from the next frame.

[0015] By using multiple frames of the field scene images, a set of control points between the pixel coordinates and geographic coordinates of the cooperative target at multiple corresponding times is established, so that the control point set can be fitted using a quadratic surface function in step two.

[0016] As a further limitation of the present invention, step two also includes:

[0017] The control point set obtained in step one is fitted using a quadratic surface function, and the fitting parameters are solved using the least squares method in order to obtain the mapping relationship between the pixel coordinates of any pixel in the aerial image and the geographic coordinates.

[0018] As a further limitation of the present invention, in step two, constructing an accurate mapping relationship between arbitrary pixel coordinates and geographic coordinates in the hovered aerial photograph of the outdoor scene specifically includes:

[0019] (1) The fitting function for the set of control points between pixel coordinates and geographic coordinates is expressed as:

[0020]

[0021] In the formula, Represents the longitude value of geographic coordinates. Representing pixel coordinates To numerical values, Representing pixel coordinates To numerical values, Represents the latitude value of geographic coordinates. , , , , , , , , , These are the fitting coefficients to be solved;

[0022] (2) Both longitude and latitude in geographic coordinates require multiple sets of non-collinear feature points to solve for the optimal fitting parameters using the least squares method;

[0023] (3) Based on the control point set and the fitting function, the least squares method is used to solve for the fitting parameters; specifically: for longitude, an observation vector is established based on the control point set from pixel coordinates to geographic coordinates. sum matrix Solve for the parameter vector of the fitted function. Then, the fitting function is obtained; the longitude data corresponding to any pixel is calculated; assuming there are n observation points, the pixel coordinates corresponding to their longitude L are... The following relationship must be satisfied:

[0024]

[0025] In the formula, Represents observed data, parameter vector , The observation vector of the group of non-collinear feature points is Using the least squares method, the optimal fitting parameters are expressed as: ; where, matrix ;

[0026] For latitude, the observation vector is represented as Its optimal fitting parameters are expressed as .

[0027] (4) Based on (3), the fitting parameters of the fitting functions for longitude and latitude are obtained, and then the mapping relationship between the real-time frame pixel coordinates and the corresponding geographic coordinates in the aerial field scene image is obtained.

[0028] As a further limitation of the present invention, step three includes:

[0029] While the drone is hovering, it takes images of at least one non-cooperative target within the target area and obtains the real-time pixel coordinates of the non-cooperative targets within that target area.

[0030] Based on the mapping relationship in step two, the geographic coordinates corresponding to the non-cooperative target and the pixel coordinates in step three are obtained in real time.

[0031] The advantages of this invention are:

[0032] 1. In field scenarios, this invention utilizes the real-time location of the cooperative target (preferably an unmanned vehicle) as the image control point for aerial photography, eliminating the need for personnel to set up and measure control points in advance, thus reducing the workload of personnel;

[0033] 2. This invention adopts a mapping relationship between pixel points and geographic coordinates based on fitting multiple known points, which effectively solves the problem that the camera needs to be calibrated or that it relies on real-time information from high-precision UAVs and gimbals when locating targets in aerial images. It also reduces the impact of lens distortion when capturing images to a certain extent, thereby realizing the location of non-cooperative targets in aerial images.

[0034] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0035] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which:

[0036] Figure 1 The present invention provides a flowchart of a method for measuring non-cooperative targets in a field setting;

[0037] Figure 2 The present invention provides a schematic diagram of known pixels in an image, wherein there are multiple known pixels in the image, and their pixel coordinates (x, y) and geographic coordinates (L, B) are known.

[0038] Figure 3 The present invention provides a flowchart for establishing a set of pixel coordinates to geographic coordinates based on the collected pixel coordinates;

[0039] Figure 4 The present invention provides a flowchart of the mapping parameters for solving longitude from pixel coordinates to geographic coordinates. Detailed Implementation

[0040] The embodiments of the present invention are described in detail below. These embodiments are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0041] See Figure 1 This invention provides a method for measuring non-cooperative targets in a field setting, comprising the following steps:

[0042] Step 1: The UAV hovers and captures images of the field scene, including both cooperative and non-cooperative targets, with the cooperative targets maneuvering within the camera's field of view. The pixel coordinates of the cooperative targets in the field scene images are detected, and the self-localized geographic coordinates of the cooperative targets in the corresponding images are obtained as control points from pixel coordinates to geographic coordinates. The pixel coordinates of the cooperative targets are continuously obtained from multiple frames of images, and the high-precision self-localized geographic coordinates of the cooperative targets at the corresponding times are also obtained, establishing a set of control points for pixel coordinates and geographic coordinates.

[0043] In step one above, the present invention establishes a set of control points for the cooperative target based on the captured field scene images. The specific requirements are as follows: (1) The camera is relatively fixed while the UAV is hovering, and the cooperative target in motion is captured to obtain multiple aerial field scene images; (2) Multiple non-collinear feature points (such as...) are selected one by one from the same cooperative target in each frame of the field scene images. Figure 2 As shown, the pixel coordinates (x, y) and geographic coordinates (L, B) of each known feature point are known. The pixel coordinates of each feature point in the field scene image are detected in real time; (3) At the same time, the geographic coordinates of the cooperative target at the corresponding time of the current image are obtained.

[0044] In practical applications, aerial cameras are used to photograph target areas while the drone is hovering. The aerial camera, relatively fixed to the drone, precisely locates the unmanned vehicle's (UAV) movements within the camera's field of view, detecting the UAV's pixel coordinates in multiple frames in real time, and simultaneously acquiring the UAV's position information at any given moment. (See also...) Figure 3In the lower part of this embodiment, in a series of continuously captured aerial images of a field scene ordered by time, if a cooperative target is detected in the current field scene image and there is pixel movement between the cooperative target and the previous control point in the field scene image, and the moved pixels are greater than a preset threshold, then the pixel coordinates of the cooperative target are extracted from the current field scene image, and the geographic coordinates of the cooperative target in that frame of the field scene image are obtained, and it is used as a control point; otherwise, the detection of cooperative targets continues from the next frame of the image. This embodiment of the invention uses multiple frames of the field scene image to establish multiple sets of control points for the cooperative target at corresponding times, from pixel coordinates to geographic coordinates, so that the control point set can be fitted using a quadratic surface function in the following step two.

[0045] Step 2: Fit the control point set obtained in Step 1 using a quadratic surface function, and use the least squares method to solve for the fitting coefficients in order to obtain the mapping relationship between any pixel coordinates in the image taken by the UAV and the geographic coordinates based on the fitting of each control point.

[0046] Step two of this embodiment specifically includes: based on the information obtained in step one... Figure 2 The location information of each pixel is shown, and the geographic coordinates (L, B) of the corresponding cooperative target in the current frame aerial image are obtained in the geographic coordinate system; continue reading Figure 2 For each frame of the image, the pixel coordinates (x, y) of each feature point of each cooperative target are obtained one by one, and the geographic coordinates (L, B) of the target are formed by multiple feature points. Based on each frame of the image and each cooperative target at each time, a data set of the target between pixel coordinates and geographic coordinates is established.

[0047] Specifically, in step two of the embodiments of the present invention, constructing an accurate mapping relationship between arbitrary pixel coordinates and geographic coordinates in the hovered aerial photography of the field scene image specifically includes:

[0048] (1) The fitting function for the set of control points between pixel coordinates and geographic coordinates is expressed as:

[0049]

[0050]

[0051] In the formula, Represents the longitude value of geographic coordinates. Representing pixel coordinates To numerical values, Representing pixel coordinates To numerical values, Represents the latitude value of geographic coordinates. , , , , , , , , , These are the fitting coefficients to be solved.

[0052] (2) Both longitude and latitude in geographic coordinates require multiple sets of non-collinear feature points. The least squares method is used to solve for the optimal fitting parameters.

[0053] (3) Based on the control point set and the fitting function, the least squares method is used to solve for the fitting parameters; specifically: for longitude, an observation vector is established based on the control point set from pixel coordinates to geographic coordinates. sum matrix Solve for the parameter vector of the fitted function. Then, the fitting function is obtained; the longitude data corresponding to any pixel is calculated; assuming there are n observation points, the pixel coordinates corresponding to their longitude L are... The following relationship must be satisfied:

[0054]

[0055] In the formula, Represents observed data, parameter vector , The observation vector of the group of non-collinear feature points is Using the least squares method, the optimal fitting parameters are expressed as: ; where, matrix ;

[0056] For latitude, the observation vector is represented as Its optimal fitting parameters are expressed as .

[0057] (4) Based on (3), the fitting parameters of the fitting functions for longitude and latitude are obtained, and then the mapping relationship between the real-time frame pixel coordinates and the corresponding geographic coordinates in the aerial field scene image is obtained.

[0058] Step 3: Based on the pixel coordinates of non-cooperative targets in the aerial image, and using the mapping relationship obtained in Step 2, calculate the geographic coordinates of the non-cooperative targets, thereby realizing the positioning and measurement of non-cooperative targets in the field.

[0059] The third step in the above embodiment of the present invention specifically includes: (1) taking pictures of non-cooperative targets in the target area while the UAV is hovering, and obtaining the real-time pixel coordinates of the non-cooperative targets in the target area; (2) obtaining the geographic coordinates of the non-cooperative targets in the field scene and the pixel coordinates in the real-time according to the real-time pixel coordinates of the non-cooperative targets and the mapping relationship obtained in the second step.

[0060] Preferably, in this embodiment of the invention, multiple frames of high-precision localization images of an unmanned vehicle are captured by a hovering drone. The pixel coordinates of the unmanned vehicle are detected using a YOLO model, and the location information of the unmanned vehicle at the corresponding time is also obtained. This allows the establishment of a set PointCollection composed of multiple target pixels and their geographic coordinates: {Pt1, Pt2, …, Pti, …, Ptn}, where Pti is the i-th point, which can be represented as (x... i , y i ,L i B i ), x i , y i L represents pixel coordinates. i B i The coordinates are longitude and latitude. Then, a quadratic surface is used to fit the mapping relationship from pixel coordinates to geographic coordinates. Finally, the least squares method is used to solve for the fitting parameters, establishing the mapping relationship from aerial image pixel coordinates to geodetic coordinates (longitude and latitude), thereby obtaining the geographic location (longitude and latitude) corresponding to any pixel coordinate. Using the method of this embodiment, the positioning and measurement of any target in an image can be achieved without the attitude and positioning information of the drone and gimbal.

[0061] In practical applications, the optional specific measurement process includes the following steps: (1) According to Figure 3 The process shown establishes a PointCollection (control point set) consisting of multiple points, including their pixel coordinates and geographic coordinates; (2) according to Figure 4 The illustrated process preferably uses a quadratic surface to fit the mapping relationship between pixel coordinates and geographic coordinates, and then uses the least squares method to solve for the fitting coefficients. After establishing the mapping relationship between image pixel coordinates and geodetic coordinates (longitude and latitude), the geographic location (longitude and latitude) corresponding to any pixel coordinate is obtained. Furthermore, based on the above-disclosed scheme, this embodiment of the invention can also utilize PointCollection to construct observation vectors. sum matrix Finally, embodiments of the present invention also utilize formulas Solving for the coefficient vector Thus, the embodiments of the present invention can solve for the longitude L of any pixel in an outdoor scene image, and of course, the latitude B of any pixel can also be obtained using a similar method.

[0062] This invention can utilize high-precision unmanned vehicle positioning to automatically establish multiple control points for aerial images of outdoor scenes, forming a control point set. This reduces the workload of staff and uses the mapping relationship from pixel coordinates to geographic coordinates of each pixel in the multiple control point set for fitting, thus enabling the positioning and measurement of non-cooperative targets without relying on real-time information from high-precision drones and gimbals.

[0063] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the scope of the technology disclosed in the present invention, and such modifications or substitutions should all be covered within the scope of protection of the present invention.

Claims

1. A method for measuring non-cooperative targets in a field setting, characterized in that, include: Step 1: The UAV hovers and captures images of the field scene containing both cooperative and non-cooperative targets moving within the camera's field of view. Based on the pixel coordinates of the cooperative targets in multiple frames of the field scene images and the self-localized geographic coordinates of the cooperative targets in the corresponding images, a set of control points for the cooperative targets in the field scene images is established. Step 2: Fit the control point set obtained in Step 1 using a quadratic surface function to obtain the mapping relationship between any pixel coordinates in the aerial image and geographic coordinates; Step 3: Based on the pixel coordinates of the non-cooperative target in the aerial image, and using the mapping relationship obtained in Step 2, calculate the geographic coordinates of the non-cooperative target to achieve the positioning and measurement of the non-cooperative target in the field.

2. The method for measuring non-cooperative targets in a field scene according to claim 1, characterized in that, In step one, based on the pixel coordinates of the cooperative target in multiple frames of field scene images and the geographic coordinates of the cooperative target in the corresponding frame images, a set of control points for the cooperative target in the field scene images is established, including: When the drone hovers and takes pictures of an area containing both cooperative and non-cooperative targets, it acquires the pixel coordinates of the cooperative target in the maneuvering state in the aerial image frame, as well as the precise self-localization geographic coordinates of the cooperative target at the corresponding moment in the image, and uses them as a control point in the field scene image. The pixel coordinates and high-precision geographic coordinates of the cooperative target at different locations in the images are continuously obtained from multiple frames of field scene images, and a set of control points between the pixel coordinates of the cooperative target in the field scene images and the geographic coordinates of the corresponding pixels are established.

3. The method for measuring non-cooperative targets in a field scene according to claim 1, characterized in that, In step one, establishing a set of control points for the cooperative target in the field scene image, from pixel coordinates to geographic coordinates, specifically includes: While the drone is hovering, an aerial camera captures images of the field scene within the target area during continuous maneuvering of the cooperative target. This yields multiple frames of field scene images ordered by time, and the geographic coordinates of the cooperative target at the same moment are obtained from these multiple frames. The selection criteria for these multiple frames are as follows: In the continuously captured and time-ordered field scene images, if a cooperative target is detected in the current field scene image and there is pixel movement between the cooperative target and the field scene image at the previous control point, and the moved pixels are greater than a preset threshold, then the pixel coordinates of the cooperative target are extracted from the current field scene image, and the geographic coordinates of the cooperative target in this frame are obtained and used as a control point; otherwise, the detection of cooperative targets continues from the next frame. By using multiple frames of the field scene images, a set of control points between the pixel coordinates and geographic coordinates of the cooperative target at multiple corresponding times is established, so that the control point set can be fitted using a quadratic surface function in step two.

4. The method for measuring non-cooperative targets in a field scene according to claim 1, characterized in that, Step two also includes: The control point set obtained in step one is fitted using a quadratic surface function, and the fitting parameters are solved using the least squares method in order to obtain the mapping relationship between the pixel coordinates of any pixel in the aerial image and the geographic coordinates.

5. A method for measuring non-cooperative targets in a field scene according to claim 1 or 4, characterized in that, In step two, an accurate mapping relationship between arbitrary pixel coordinates and geographic coordinates is constructed in the hovering aerial photography of the field scene image, specifically including: (1) The fitting function for the set of control points between pixel coordinates and geographic coordinates is expressed as: In the formula, Represents the longitude value of geographic coordinates. Representing pixel coordinates To numerical values, Representing pixel coordinates To numerical values, Represents the latitude value of geographic coordinates. These are the fitting coefficients to be solved; (2) Both longitude and latitude in geographic coordinates require multiple sets of non-collinear feature points. The least squares method is used to solve for the optimal fitting parameters. (3) Based on the control point set and the fitting function, the least squares method is used to solve for the fitting parameters; specifically: for longitude, an observation vector is established based on the control point set from pixel coordinates to geographic coordinates. sum matrix Solve for the parameter vector of the fitted function. Then, the fitting function is obtained; the longitude data corresponding to any pixel is calculated; assuming there are n observation points, the pixel coordinates corresponding to their longitude L are... The following relationship must be satisfied: In the formula, Represents observed data, parameter vector , The observation vector of the group of non-collinear feature points is Using the least squares method, the optimal fitting parameters are expressed as: , where the matrix ; For latitude, the observation vector is represented as Its optimal fitting parameters are expressed as . (4) Based on (3), the fitting parameters of the fitting functions for longitude and latitude are obtained, and then the mapping relationship between the real-time frame pixel coordinates and the corresponding geographic coordinates in the aerial field scene image is obtained.

6. The method for measuring non-cooperative targets in a field scene according to claim 1, characterized in that, Step three includes: The drone takes images of non-cooperative targets within a target area while hovering, and obtains the real-time pixel coordinates of the non-cooperative targets within that target area. Based on the mapping relationship in step two, the geographic coordinates corresponding to the non-cooperative target and the pixel coordinates in step three are obtained in real time.

Images