High-speed target measurement method and system based on unmanned aerial vehicle pod and calibration target

Abstract

The invention discloses a high-speed target measurement method and system based on an unmanned aerial vehicle pod and a calibration target. The method comprises the following steps: calibrating a binocular camera to obtain camera internal parameters; a plurality of calibration targets are arranged in the measurement field, a high-speed target motion image and a calibration target image are shot respectively, and unmanned aerial vehicle pod information is obtained in real time and recorded in an image frame header; calculating camera external parameters used by the current image frame based on the pod information, and correcting the camera external parameters based on the calibration target image; and analyzing the motion parameters of the high-speed target based on the camera internal parameters, the corrected camera external parameters and the high-speed target motion image. According to the method, pod information is collected in real time and recorded in an image frame header, camera external parameters used by a current frame can be calculated, and pod external errors caused by movement of the unmanned aerial vehicle are eliminated; the target information is used to correct the external parameters of the camera, and the displacement change of the camera relative to the pod is converted to a coordinate level, thereby reducing the influence of the jitter of the shooting equipment on the measurement result, and achieving the purpose of improving the measurement precision.

Description

Technical Field

[0001] The present invention relates to the field of unmanned aerial vehicle optical measurement, and in particular to a high-speed target measurement method and system based on an unmanned aerial vehicle pod and a calibration target. Background Art

[0002] The tracking and measurement of high-speed moving targets is widely used in aerospace, traffic safety, military reconnaissance, range testing, aerial target interception, mechanics research and other fields.

[0003] One way to achieve this is to use multi-sensor active measurement, mount multiple sensors on a moving carrier, monitor the instantaneous running state of the target, and then calculate the target's motion posture. However, the sensor mounting method is not suitable for high-speed target tracking and measurement of non-cooperative targets.

[0004] Another implementation method is to use a drone to track and measure high-speed targets, integrate a high-speed camera into the drone pod, use the high-speed camera to capture and store images of the moving target to obtain the target's motion image, and use the captured images to analyze and evaluate the target's motion characteristics.

[0005] However, the high-speed camera is suspended at the bottom of the drone as a payload. During the measurement process, the drone will be affected by internal and external factors and will vibrate slightly, so the high-speed camera will also vibrate during the measurement process. The high-speed camera jitter mainly affects the external parameters of the binocular measurement system, resulting in deviations in the position recognition of high-speed targets in the constructed three-dimensional space, affecting the measurement accuracy.

[0006] Therefore, when measuring the posture of a high-speed moving target with a large field of view, in order to ensure the true restoration of the working conditions, the main problem to be solved at this stage is to capture its position, posture, and motion state with high precision without any processing on the high-speed target. Summary of the invention

[0007] In view of the problems existing in the above-mentioned prior art, the present invention provides a high-speed target measurement method and system based on a drone pod and a calibration target, wherein a high-speed camera mounted on a drone is used to achieve high-precision tracking measurement under the conditions of a large measurement field of view and high-speed motion of non-cooperative targets, wherein no sensor is mounted on the target and no marking is performed, and the influence of the jitter problem of the drone-mounted camera on the measurement result is effectively reduced.

[0008] To achieve the above object, the present invention provides the following technical solutions:

[0009] In a first aspect, the present invention provides a high-speed target measurement method based on a drone pod and a calibration target, which is applied to a drone, wherein the drone includes a pod, and the pod is composed of an azimuth component, a pitch component, and a load compartment component equipped with a binocular camera system. The method comprises:

[0010] Calibrate the binocular camera to obtain the camera's intrinsic parameters;

[0011] Set up multiple calibration targets in the measurement field of high-speed targets, use binocular cameras to capture high-speed target motion images and calibration target images respectively, and collect the pod information of the UAV in real time during the shooting process and record it in the image frame header;

[0012] Calculating camera extrinsics used in the current image frame based on the pod information, and correcting the camera extrinsics based on the calibration target image;

[0013] The motion parameters of the high-speed target are analyzed based on the camera intrinsic parameters, the corrected camera extrinsic parameters, and the high-speed target motion image.

[0014] Furthermore, the camera calibration adopts Zhang's calibration method, and the steps include:

[0015] Use a chessboard as a calibration plate, and place the camera and calibration plate arbitrarily to capture calibration images;

[0016] Extract the pixel coordinates of corner points on the calibration image based on the corner detection algorithm;

[0017] Based on the pixel coordinates of the corner points on the calibration image and the spatial coordinates of the corresponding corner points of the checkerboard on the calibration board, a mapping relationship between the pixel coordinate system of the camera image and the world coordinate system is established to obtain the camera intrinsic parameters.

[0018] Furthermore, the calibration target adopts a ring-shaped coding mark, and the setting of multiple calibration targets in the measurement field of the high-speed target is specifically:

[0019] At least 6 calibration targets are evenly covered within the activity range of the high-speed target, and it is ensured that each calibration target is not on the same plane in three-dimensional space and is separated by a certain distance.

[0020] Furthermore, the pod information of the drone includes positioning information and attitude information of the pod; the positioning information is used to characterize the three-dimensional spatial coordinates of the pod; the attitude information is used to characterize the motion attitude of the pod during the drone tracking measurement process, including the yaw angle and the pitch angle;

[0021] The camera extrinsic parameters used in calculating the current image frame based on the pod information include:

[0022] After the drone pod completes the northing calibration, for the left and right pods at each image frame, the azimuth component is used to obtain the current yaw angle of the pod, the pitch component is used to obtain the current pitch angle of the pod, and the payload pod component is used to obtain the current three-dimensional space coordinates of the pod;

[0023] The rotation matrix of the left camera is calculated based on the yaw angle and pitch angle of the left pod, and the translation vector of the left camera is calculated based on the three-dimensional space coordinates of the left pod to obtain the external parameters of the left camera;

[0024] Taking the left camera in the binocular camera system as the origin of the world coordinate system, calculate the yaw angle and pitch angle of the right camera relative to the left camera and record them as the right relative yaw angle and the right relative pitch angle;

[0025] The relative rotation matrix and relative translation vector of the right camera are calculated according to the right relative yaw angle and the right relative pitch angle to obtain the external parameters of the right camera.

[0026] Furthermore, the correcting the camera extrinsic parameters based on the calibration target image includes:

[0027] Acquire real-time calibration target images and perform image preprocessing operations;

[0028] Using an ellipse detection method based on edge detection to identify all ellipses in the calibration target image;

[0029] Based on the characteristic constraint of the center positioning circle, the identified ellipse is screened to obtain a ring-shaped coded mark image;

[0030] Performing perspective deformation correction on the annular coded mark image;

[0031] In the corrected annular coded mark image, obtaining pixel coordinates of the coded mark points;

[0032] Converting the annular coding band in the corrected annular coding mark image into a vertically arranged strip coding band, parsing the coding value of the strip coding band to obtain the spatial coordinates of the coding mark point;

[0033] An objective function for extrinsic parameter correction is established based on the pixel coordinates and spatial coordinates of the coded marker points, and the camera extrinsic parameters used in the current image frame calculated based on the pod information are substituted into the objective function as initial values ​​for iterative correction.

[0034] Furthermore, the converting of the annular coding band in the corrected annular coding mark image into a vertically arranged strip coding band, and parsing the coding value of the strip coding band to obtain the spatial coordinates of the coding mark point includes:

[0035] Performing polar coordinate transformation on the annular coding belt to convert it into a vertically arranged strip coding belt;

[0036] Obtain the number of pixels in each coding block of the strip coding band from top to bottom;

[0037] Using the pixel value as the binary number required to be encoded for each coding block, the number of binary numbers of each coding block is calculated according to the relative proportion of the number of pixels of each coding block to the total number of pixels in the vertical direction of the strip coding band;

[0038] Obtaining a binary sequence of each coding block according to the binary number and the number of the binary numbers;

[0039] The binary sequences are sorted cyclically, and the decimal number corresponding to the binary sequence with the smallest value is selected as the encoding value;

[0040] The spatial coordinates of the coded marker points are obtained based on the one-to-one correspondence between the coded value and the spatial coordinates.

[0041] Furthermore, the objective function of extrinsic parameter correction is established based on the pixel coordinates and spatial coordinates of the coded marker points, and the camera extrinsic parameters calculated based on the pod information are substituted as initial values ​​into the objective function for iterative correction, including:

[0042] The spatial conversion error is calculated based on the projection relationship between the camera coordinates and the pixel coordinates and the camera intrinsic parameters;

[0043] Constructing an objective function for extrinsic parameter correction based on minimizing the spatial conversion error;

[0044] Substitute the rotation matrix of the camera extrinsic parameters calculated based on the pod information into the objective function as the initial rotation matrix to calculate the current translation vector;

[0045] Substitute the current rotation matrix and translation vector into the objective function about the current rotation matrix to obtain the rotation matrix expression for the next iteration;

[0046] Solve the iterative rotation matrix expression using singular value decomposition;

[0047] Repeat the iterative process until the rotation matrix converges and obtain the corrected camera extrinsics.

[0048] Furthermore, when a binocular camera is used to capture images of high-speed target motion, a Kalman filter is used to detect and track the high-speed target.

[0049] Further, analyzing the motion parameters of the high-speed target based on the camera intrinsic parameters, the corrected camera extrinsic parameters, and the high-speed target motion image includes:

[0050] The spatial coordinates of the high-speed target are calculated based on the camera intrinsic parameters, the camera extrinsic parameters, and the pixel positions of the high-speed target in the motion images of the left and right cameras; the spatial coordinates obtained in all the high-speed target motion images constitute the spatial coordinate sequence of the high-speed target;

[0051] Decompose the discrete spatial coordinate sequence into real signal data and zero-mean white noise data;

[0052] Representing the real signal data by a linear combination of a family of orthogonal polynomials of a set of time functions;

[0053] According to the least squares principle, by minimizing the sum of squares of differences between the real signal data and the spatial coordinate sequence, the coefficients of the orthogonal polynomial family are estimated by least squares;

[0054] Substitute the estimated orthogonal polynomial family coefficients into the decomposition of the spatial coordinate sequence to calculate the optimal estimate of the real signal data;

[0055] The optimal estimate of the real signal data is subjected to multi-order derivatives with respect to time to obtain the motion parameters of the high-speed target; wherein the 0th order derivative represents the displacement of the high-speed target; the 1st order derivative represents the speed of the high-speed target; and the 2nd order derivative represents the acceleration of the high-speed target.

[0056] In a second aspect, the present invention provides a high-speed target measurement system based on an unmanned aerial vehicle pod and a calibration target, comprising:

[0057] The intrinsic parameter calibration module is used to calibrate the binocular camera to obtain the camera's intrinsic parameters;

[0058] The data acquisition module is used to collect high-speed target motion images and calibration target images in the measurement field, and obtain the drone's pod information in real time during the shooting process and record it in the image frame header;

[0059] An extrinsic parameter calibration module, used to calculate the camera extrinsic parameters used in the current image frame based on the pod information, and to correct the camera extrinsic parameters based on the calibration target image;

[0060] The measurement and solution module is used to analyze the motion parameters of the high-speed target based on the camera intrinsic parameters, the corrected camera extrinsic parameters, and the high-speed target motion image.

[0061] The present invention provides a high-speed target measurement method and system based on an unmanned aerial vehicle pod and a calibration target, which has the following features:

[0062] Beneficial effects:

[0063] 1. The present invention uses the pod-mounted camera of the UAV to track and measure high-speed targets. It does not require any equipment mounting or feature marking processing for the high-speed targets. The calibration method is simple, the calibration target structure is simple, the extraction is convenient, and it is easy to identify and has high positioning accuracy. This method can be applied in complex environments such as outdoor and large fields of view, and has high accuracy, robustness and flexibility.

[0064] 2. The present invention uses a high-speed camera that synchronizes timing and receives trigger signals, collects pod information in real time and records it in the image frame header, and can calculate the camera external parameter information used in the current frame to meet the user's real-time measurement of the spatial coordinates, displacement, speed, acceleration, attitude and other information of the moving object. The external parameters obtained based on the real-time pod information can effectively eliminate the pod angle offset error caused by the movement during the UAV tracking process. At the same time, the real-time coordinate information of the calibration target is used to correct the camera external parameters, taking into account the internal error of the camera displacement relative to the pod, and further reducing the impact of the jitter of the shooting equipment on the measurement results, thereby achieving the purpose of improving the measurement accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 is a schematic diagram of the configuration of a UAV platform and a ground station in one embodiment of the present invention;

[0066] Figure 2 It is a flow chart of a high-speed target measurement method based on a drone pod and a calibration target of the present invention;

[0067] Figure 3 is a schematic diagram of an annular coding belt used in one embodiment of the present invention;

[0068] Among them, (a) is the principle diagram of calibration target coding; (b) is the example diagram of calibration target coding;

[0069] Figure 4 It is a schematic diagram of parsing the code value of the bar code band to obtain the spatial coordinates of the code mark point in one embodiment of the present invention;

[0070] Figure 5 The present invention is a structural block diagram of a high-speed target measurement system based on an unmanned aerial vehicle pod and a calibration target. DETAILED DESCRIPTION

[0071] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0072] In order to better illustrate the technical solution of the present invention, the terminology involved in the embodiments of the present invention is first explained.

[0073] 1. The camera calibration of the present invention involves three coordinate systems:

[0074] (1) Spatial coordinate system: also known as the world coordinate system, which describes the three-dimensional spatial position of objects in the real environment;

[0075] (2) Camera coordinate system: The origin is the optical center of the camera, and the z-axis coincides with the optical axis, that is, the z-axis points to the front of the camera (that is, perpendicular to the imaging plane).

[0076] (3) Pixel coordinate system: The unit is pixel, and the origin of the coordinate system is the upper left corner of the image.

[0077] 2. (1) Intrinsic parameters: The transformation from the camera coordinate system to the pixel coordinate system is represented by an intrinsic parameter matrix K. It is a property of the camera itself and is usually fixed for a specific camera model and is independent of the scene.

[0078] (2) External parameters: The transformation from the spatial coordinate system to the camera coordinate system, including the rotation matrix R and the translation vector t, which may change at different camera positions or shooting times.

[0079] It should be noted that, unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application.

[0080] To facilitate understanding of this embodiment, a high-speed target measurement method based on a drone pod and a calibration target disclosed in an embodiment of the present invention is first introduced in detail.

[0081] The embodiment of the present invention provides a high-speed target measurement method based on a drone pod and a calibration target, which is applied to a drone. Figure 1 The configuration of the UAV platform in this embodiment is shown. When implementing the method of the present invention, two UAVs are respectively equipped with a high-speed camera to complete the tracking measurement, which are recorded as left and right cameras in this embodiment. The high-speed camera used supports real-time preview and real-time image monitoring to meet the needs of observing high-speed moving objects; it supports synchronous timing and second-time alignment to meet the timestamp synchronization requirements between multiple cameras, and can realize the external parameter correction requirements of the real-time single-frame image of the present invention. Each UAV includes a pod, which is composed of an azimuth component, a pitch component and a load compartment component equipped with a high-speed camera.

[0082] like Figure 2 As shown, the method comprises the following steps:

[0083] Step 1: Intrinsic calibration: Use Zhang’s calibration method to calibrate the binocular camera to obtain the camera’s intrinsic parameters, including:

[0084] Use a chessboard as a calibration plate, and place the camera and calibration plate arbitrarily to capture calibration images;

[0085] Extract the pixel coordinates of corner points on the calibration image based on the corner detection algorithm;

[0086] Based on the pixel coordinates of the corner points on the calibration image and the spatial coordinates of the corresponding corner points of the checkerboard on the calibration board, a mapping relationship between the pixel coordinate system of the camera image and the world coordinate system is established to obtain the camera intrinsic parameters.

[0087] The coordinates p of the corner point in the pixel coordinate system and the spatial coordinates P of the checkerboard corner point on the calibration plate have the following relationship:

[0088] p=K[R|T]P

[0089] The homography matrix H = K[R|T] = K[r1 r2 t]. By solving the homography matrix H, the internal and external parameters of the camera can be obtained. A calibration image can provide 2 constraints, and at least 3 calibration images are required to calculate the internal parameters. where f x and f y is the focal length of the camera, which is related to the size of the pixel; c x and c y It is the translation distance, which is related to the size of the camera imaging plane.

[0090] The external parameters are relative to each calibration image, and different images correspond to different external parameters. It should be understood that the external parameters required here are the camera external parameters in the calibration scene, which should be distinguished from the external parameters mentioned later. Since the external parameters change with the movement of the camera, more accurate external parameters need to be re-solved when the drone tracks the target.

[0091] When considering distortion, the distortion coefficient is solved by maximum likelihood estimation, and reference may be made to the relevant contents of the prior art, which will not be described in detail in this embodiment.

[0092] Step 2: Real-time data acquisition: multiple calibration targets are set up in the measurement field of the high-speed target, and the high-speed target motion image and calibration target image are captured by a binocular camera respectively. The pod information of the drone is collected in real time during the shooting process and recorded in the image frame header;

[0093] Wherein, the calibration target adopts a circular coding mark, such as Figure 3As shown, the circular positioning point in the middle and the surrounding annular coding belt are composed of the same center. The spatial coordinates of the calibration target correspond to the position of the circular positioning point in the middle of the target. The coding value of each calibration target must be unique and correspond one-to-one with the spatial coordinate mark. Using the binary coding principle, the annular coding belt is divided into n equal parts. Each equal part is a binary bit, which forms a binary value clockwise (or counterclockwise). There is no start bit and end bit, so the coding belt has up to n different values. In order to ensure uniqueness, the smallest binary value is selected and its corresponding decimal number is taken as the coding value. This coding mark has a simple structure, convenient extraction, easy identification, and high positioning accuracy.

[0094] The method of setting a plurality of calibration targets in the measurement field of the high-speed target is specifically as follows:

[0095] At least 6 calibration targets are evenly covered within the activity range of the high-speed target, and it is ensured that each calibration target is not on the same plane in three-dimensional space and is separated by a certain distance.

[0096] Among them, the pod information of the drone includes the pod's positioning information and attitude information; the positioning information is used to characterize the pod's three-dimensional spatial coordinates; the attitude information is used to characterize the pod's motion attitude during the drone's tracking and measurement process, including the yaw angle and pitch angle.

[0097] The yaw angle, pitch angle and roll angle together constitute three important parameters that describe the attitude of the pod. The yaw angle is the angle between the projection of the longitudinal axis of the body coordinate system on the horizontal plane and the horizontal axis of the ground coordinate system. The pitch angle is the angle between the longitudinal axis of the body coordinate system and the horizontal plane. The roll angle is the angle between the horizontal axis of the body coordinate system and the plumb plane passing through the longitudinal axis of the body coordinate system. The present invention mainly considers the shooting problem of the drone during the hovering process after changing its attitude. Therefore, when it is in a stable state, the roll angle is not considered.

[0098] During data collection, the left and right binocular cameras synchronize the time stamps and receive trigger information to enable the two cameras to start recording at the same time. During the camera shooting process, the pitch and yaw angles and positioning information of the drone pod are saved in the image frame header in real time.

[0099] Furthermore, when a binocular camera is used to capture images of high-speed target motion, a Kalman filter is used to detect and track the high-speed target.

[0100] Specifically include:

[0101] Define a template image of a high-speed target;

[0102] Based on the template image, a Kalman filter is used to predict the position of the high-speed target in the next frame of the motion image.

[0103] The possible range of high-speed targets is delineated with this position as the center;

[0104] Searching for high-speed targets within a defined range based on their shape and grayscale information;

[0105] The search result closest to the predicted position is used as the matching target point to track the high-speed target.

[0106] Step 3: extrinsic parameter calibration, calculating the camera extrinsic parameters used in the current image frame based on the pod information, and correcting the camera extrinsic parameters based on the calibration target image;

[0107] The camera extrinsic parameters used in calculating the current image frame based on the pod information include:

[0108] After the drone pod completes the northing calibration, for the left and right pods at each image frame, the azimuth component is used to obtain the current yaw angle of the pod, the pitch component is used to obtain the current pitch angle of the pod, and the payload pod component is used to obtain the current three-dimensional space coordinates of the pod;

[0109] Calculate the rotation matrix R of the left camera based on the yaw angle ryl and pitch angle rxl of the left pod l , calculate the translation vector t of the left camera based on the 3D space coordinates of the left pod l , get the external parameters of the left camera;

[0110] Take the left camera in the binocular camera system as the origin of the world coordinate system, calculate the yaw angle and pitch angle of the right camera relative to the left camera, and record them as the right relative yaw angle ryr and the right relative pitch angle rxr;

[0111] The relative rotation matrix R of the right camera is calculated according to the right relative yaw angle and the right relative pitch angle. r and the relative translation vector t r , and get the external parameters of the right camera.

[0112] The following takes the right camera as an example for explanation (the left camera does not need to perform coordinate transformation, which is simpler than the right one. It can be obtained according to the calculation process of the right camera, which will not be repeated here). The left camera in the binocular camera system is used as the origin of the world coordinate system, and the yaw angle and pitch angle of the right camera are unified into one coordinate system:

[0113] rx=rxr-rxl

[0114] ry=ryr-ryl

[0115] Then the relative rotation matrix R of the right camera is l for:

[0116]

[0117] R r =Rrx*Rry

[0118] There is also a relative translation vector t r (t x ,t y ,t z )=G r -G l .

[0119] The translation vector t of the left camera l (t x ,t y ,t z )=G l .

[0120] Furthermore, the correcting the camera extrinsic parameters based on the calibration target image includes:

[0121] Acquire real-time calibration target images and perform image preprocessing operations;

[0122] In this embodiment, illumination correction, Gaussian filtering and edge detection are performed on the image.

[0123] Using an ellipse detection method based on edge detection to identify all ellipses in the calibration target image;

[0124] In one embodiment, it may include:

[0125] (1) Edge classification: arc segments are classified according to the gradient direction of edge points.

[0126] (2) Remove arcs with only a small number of pixels and arcs with a large number of collinear points;

[0127] (3) Group the arcs belonging to the same ellipse;

[0128] (4) Ellipse clustering: clustering multiple detections of the same ellipse.

[0129] Based on the characteristic constraint of the center positioning circle, the identified ellipse is screened to obtain a ring-shaped coded mark image;

[0130] In this embodiment, the characteristic constraints of the center positioning circle include:

[0131] (1) Size criteria: There are maximum and minimum limits on area and perimeter.

[0132] (2) Area difference criterion: avoid the situation where the unit coding block is mistakenly identified as the center circle of the coding point.

[0133] (3) Code value verification: When the decoded code value is not in the encoding dictionary, the code point is removed.

[0134] Performing perspective deformation correction on the annular coded mark image;

[0135] In the corrected annular coded mark image, obtaining pixel coordinates of the coded mark points;

[0136] It should be noted that in most cases, the image of the circular coded mark will be deformed. Perspective deformation is caused by the image plane of the shooting device forming a certain angle with the plane where the mark is located, which causes the acquired image to be distorted, which is a great obstacle to the correct recognition of the coded mark. Therefore, perspective transformation is an important part of mark recognition. By correcting the perspective transformation to the front view, the decoding accuracy of the coded mark can be greatly improved.

[0137] Before performing perspective correction, it is necessary to use the roundness of the center circle to determine the degree of deformation to determine whether correction is needed.

[0138] Converting the annular coding band in the corrected annular coding mark image into a vertically arranged strip coding band, parsing the coding value of the strip coding band to obtain the spatial coordinates of the coding mark point;

[0139] An objective function for extrinsic parameter correction is established based on the pixel coordinates and spatial coordinates of the coded marker points, and the camera extrinsic parameters used in the current image frame calculated based on the pod information are substituted into the objective function as initial values ​​for iterative correction.

[0140] Further, refer to Figure 4 The step of converting the annular coding band in the corrected annular coding mark image into a vertically arranged strip coding band, and parsing the coding value of the strip coding band to obtain the spatial coordinates of the coding mark point includes:

[0141] Performing polar coordinate transformation on the annular coding belt to convert it into a vertically arranged strip coding belt;

[0142] Obtain the number of pixels in each coding block of the strip coding band from top to bottom;

[0143] Assume there are K blocks, and the number of black and white pixels in each segment is stored in an array P[K], and the black and white information of each segment corresponding to it is stored in Code[K], which is defined as follows:

[0144]

[0145] Since the flag is cyclically coded, when Code[0]=Code[K-1], the number of pixels at the beginning and the end need to be merged, that is, P[0]=P[0]+P[K-1].

[0146] Using the pixel value as the binary number required to be encoded for each coding block, the number of binary numbers of each coding block is calculated according to the relative proportion of the number of pixels of each coding block to the total number of pixels in the vertical direction of the strip coding band;

[0147] Specifically, let m i is the number of 0 / 1 in the i-th segment,

[0148]

[0149] Among them, H is the number of pixels in the vertical direction of the transformed image, and N is the number of bits of the binary code value.

[0150] Obtaining a binary sequence of each coding block according to the binary number and the number of the binary numbers;

[0151] Finally, the number of 0s and 1s in each block is calculated, the binary sequences are cyclically sorted, and the decimal number corresponding to the binary sequence with the smallest value is selected as the encoding value;

[0152] The spatial coordinates of the coded marker points are obtained based on the one-to-one correspondence between the coded value and the spatial coordinates.

[0153] Furthermore, the objective function of extrinsic parameter correction is established based on the pixel coordinates and spatial coordinates of the coded marker points, and the camera extrinsic parameters calculated based on the pod information are substituted as initial values ​​into the objective function for iterative correction, including:

[0154] Suppose the spatial coordinates of a set of non-collinear coded marker points are P i =(x i ,y i ,z i ) T , i=1,…,n,n≥3, the corresponding coordinate of Pi in the camera coordinate system is q i =(x′ i ,y′ i ,z′ i ) T , the normalized coordinate of Pi in the pixel coordinate system is v i =(u i ,v i ,1) T .

[0155] The spatial conversion error is calculated based on the projection relationship between the camera coordinates and the pixel coordinates and the camera intrinsic parameters;

[0156] In the case that the drone does not shake, the orthogonal projection of the camera coordinates of the coded marker point on the pixel coordinates is itself, that is, RP i +t=V i (RP i+t).

[0157] When there is jitter, the projection is offset and the spatial conversion error is:

[0158] e i =(IV i )(RP i +t)

[0159] In the formula, is the projection matrix along the line of sight, and I is the identity matrix.

[0160] The objective function of extrinsic parameter correction is constructed based on minimizing the spatial conversion error:

[0161]

[0162] in, Represents the norm.

[0163] Substitute the rotation matrix of the camera extrinsic parameters calculated based on the pod information into the objective function as the initial rotation matrix to calculate the current translation vector;

[0164]

[0165] Where R is the rotation matrix of the camera extrinsic parameters calculated based on the pod information.

[0166] Substitute the current rotation matrix and translation vector into the objective function of the current rotation matrix to obtain the rotation matrix expression R for the next iteration (k+1) for:

[0167]

[0168] Solve the iterative rotation matrix expression using singular value decomposition;

[0169] Repeat the iterative process until the rotation matrix converges and obtain the corrected camera extrinsics.

[0170] During the process of shooting high-speed target movement, the drone will have multiple posture adjustments, so there will be slight movement or jitter during the drone hovering process, and the external parameters may change, which needs to be corrected accordingly according to the actual situation. The present invention calculates the camera external parameters by using the drone pod information, taking into account the external errors caused by the overall angle deviation of the pod when the drone is hovering. Then, the calibration target in the captured image is automatically identified by the algorithm, and the code value of the target is parsed. The spatial coordinates of the target are obtained through the code value of the target. The external parameters of the camera are corrected according to the camera internal parameter data and the image coordinates and spatial coordinates of the calibration target. The coordinate changes of the calibration target show the influence of the jitter inside the pod, that is, the camera and the pod are not integrated, and the jitter of the pod will cause the camera position to move slightly. The present invention takes into account the internal error of the camera relative to the pod. The correction of internal and external dual errors can ensure that the binocular camera system maintains an accurate spatial relationship during movement, thereby improving the accuracy of measurement and perception.

[0171] Step 4: measurement and solution, analyzing the motion parameters of the high-speed target based on the camera intrinsic parameters, the corrected camera extrinsic parameters, and the high-speed target motion image.

[0172] Further, analyzing the motion parameters of the high-speed target based on the camera intrinsic parameters, the corrected camera extrinsic parameters, and the high-speed target motion image includes:

[0173] The spatial coordinates of the high-speed target are calculated based on the camera intrinsic parameters, the camera extrinsic parameters, and the pixel positions of the high-speed target in the motion images of the left and right cameras; the spatial coordinates obtained in all the high-speed target motion images constitute the spatial coordinate sequence of the high-speed target;

[0174] Decompose the discrete spatial coordinate sequence into real signal data and zero-mean white noise data;

[0175] Assume t i (i=1,2,…,N, N is the width of the filtering window in the time dimension) for the real signal data s(t i ) is:

[0176]

[0177] where x i is the spatial coordinate sequence, ε(t i ) is zero mean white noise, p j (t i ) is an orthogonal polynomial family, a j are the coefficients of the family of orthogonal polynomials.

[0178] Representing the real signal data by a linear combination of a family of orthogonal polynomials of a set of time functions;

[0179] Assume that the time interval between adjacent images is h, and the orthogonal polynomial family is:

[0180]

[0181] According to the least squares principle, by minimizing the sum of squares of differences between the real signal data and the spatial coordinate sequence, the coefficients of the orthogonal polynomial family are estimated by least squares;

[0182] Substitute the estimated orthogonal polynomial family coefficients into the decomposition of the spatial coordinate sequence to calculate the optimal estimate of the real signal data:

[0183]

[0184] in,

[0185] At the N+αth moment, the optimal estimate of the real signal data is subjected to multi-order derivatives with respect to time,

[0186]

[0187] Wherein, L represents the L-order derivative, which is a non-negative integer.

[0188] The motion parameters of the high-speed target are obtained; wherein, the 0th-order derivative represents the displacement of the high-speed target; the 1st-order derivative represents the speed of the high-speed target; and the 2nd-order derivative represents the acceleration of the high-speed target.

[0189] The present invention uses a chessboard to calibrate the camera's intrinsic parameters. This calibration board is simple, can be placed arbitrarily in the captured image, is easy to operate, and has high calibration accuracy. The external parameter information used in the current frame image is calculated through the orientation and attitude information of the drone pod recorded in the image, thereby meeting the requirements of real-time measurement. The external parameter information calculated from the pod attitude information is corrected using a calibration target, and the target coding mark used has a simple structure, is convenient to extract, is easy to identify, and has high positioning accuracy, thereby reducing the influence of camera shake on the measurement result.

[0190] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0191] Based on the same inventive concept, the embodiment of the present application also provides a high-speed target measurement system for implementing the high-speed target measurement method based on a drone pod and a calibration target involved above. The implementation solution provided by the system to solve the problem is similar to the implementation solution recorded in the above method, so the specific limitations in one or more high-speed target measurement system embodiments provided below can refer to the above limitations on the high-speed target measurement method based on a drone pod and a calibration target, and will not be repeated here.

[0192] like Figure 5 As shown, the present invention also provides a high-speed target measurement system based on a drone pod and a calibration target, comprising:

[0193] The intrinsic parameter calibration module is used to calibrate the binocular camera to obtain the camera's intrinsic parameters;

[0194] The data acquisition module is used to collect high-speed target motion images and calibration target images in the measurement field, and obtain the drone's pod information in real time during the shooting process and record it in the image frame header;

[0195] An extrinsic parameter calibration module, used to calculate the camera extrinsic parameters used in the current image frame based on the pod information, and to correct the camera extrinsic parameters based on the calibration target image;

[0196] The measurement and solution module is used to analyze the motion parameters of the high-speed target based on the camera intrinsic parameters, the corrected camera extrinsic parameters, and the high-speed target motion image.

[0197] It should be noted that: when the system provided in this embodiment performs high-speed target measurement, only the division of the above-mentioned functional modules is used as an example for explanation. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the system can be divided into different functional modules. Each functional module can be composed of a single execution unit, or two or more execution units can be integrated into one functional module to realize all the functions of the functional module.

[0198] Those skilled in the art will appreciate that each module in the above system may be implemented in whole or in part by software, hardware, or a combination thereof. Each module may be embedded in or independent of a processor in a computer device in the form of hardware, or may be stored in a memory in a computer device in the form of software, so that the processor may call and execute operations corresponding to each module.

[0199] The present invention is not limited to the above-mentioned specific implementation modes. Various changes made by ordinary technicians in this field based on the above-mentioned concepts without creative work are all within the protection scope of the present invention.

Claims

1. A high-speed target measurement method based on an unmanned aerial vehicle pod and a calibration target, characterized in that: Applied to a drone, the drone includes a pod, the pod is composed of an azimuth component, a pitch component and a load compartment component equipped with a binocular camera system, and the method includes the following steps: Calibrate the binocular camera to obtain the camera's intrinsic parameters; Set up multiple calibration targets in the measurement field of high-speed targets, use binocular cameras to capture high-speed target motion images and calibration target images respectively, and collect the pod information of the UAV in real time during the shooting process and record it in the image frame header; Calculating camera extrinsics used in the current image frame based on the pod information, and correcting the camera extrinsics based on the calibration target image; The motion parameters of the high-speed target are analyzed based on the camera intrinsic parameters, the corrected camera extrinsic parameters, and the high-speed target motion image.

2. The high-speed target measurement method based on an unmanned aerial vehicle pod and a calibration target according to claim 1 is characterized in that: The camera calibration adopts Zhang's calibration method, and the steps include: Use a chessboard as a calibration plate, and place the camera and calibration plate arbitrarily to capture calibration images; Extract the pixel coordinates of corner points on the calibration image based on the corner detection algorithm; Based on the pixel coordinates of the corner points on the calibration image and the spatial coordinates of the corresponding corner points of the checkerboard on the calibration board, a mapping relationship between the pixel coordinate system of the camera image and the world coordinate system is established to obtain the camera intrinsic parameters.

3. The high-speed target measurement method based on an unmanned aerial vehicle pod and a calibration target according to claim 1 is characterized in that: The calibration target adopts a ring-shaped coding mark, and the multiple calibration targets are set in the measurement field of the high-speed target specifically as follows: At least 6 calibration targets are evenly covered within the activity range of the high-speed target, and it is ensured that each calibration target is not on the same plane in three-dimensional space and is separated by a certain distance.

4. The high-speed target measurement method based on an unmanned aerial vehicle pod and a calibration target according to claim 1 is characterized in that: The pod information of the UAV includes the pod's positioning information and attitude information; the positioning information is used to characterize the pod's three-dimensional spatial coordinates; the attitude information is used to characterize the pod's motion attitude during the UAV tracking measurement process, including the yaw angle and the pitch angle; The camera extrinsic parameters used for calculating the current image frame based on the pod information include: After the drone pod completes the northing calibration, for the left and right pods at each image frame, the azimuth component is used to obtain the current yaw angle of the pod, the pitch component is used to obtain the current pitch angle of the pod, and the payload pod component is used to obtain the current three-dimensional space coordinates of the pod; The rotation matrix of the left camera is calculated based on the yaw angle and pitch angle of the left pod, and the translation vector of the left camera is calculated based on the three-dimensional space coordinates of the left pod to obtain the external parameters of the left camera; Taking the left camera in the binocular camera system as the origin of the world coordinate system, calculate the yaw angle and pitch angle of the right camera relative to the left camera and record them as the right relative yaw angle and the right relative pitch angle; The relative rotation matrix and relative translation vector of the right camera are calculated according to the right relative yaw angle and the right relative pitch angle to obtain the external parameters of the right camera.

5. The high-speed target measurement method based on the drone pod and the calibration target according to claim 3 is characterized in that: The step of correcting the camera extrinsic parameters based on the calibration target image comprises: Acquire real-time calibration target images and perform image preprocessing operations; Using an ellipse detection method based on edge detection to identify all ellipses in the calibration target image; Based on the characteristic constraint of the center positioning circle, the identified ellipse is screened to obtain a ring-shaped coded mark image; Performing perspective deformation correction on the annular coded mark image; In the corrected annular coded mark image, obtaining pixel coordinates of the coded mark points; Converting the annular coding band in the corrected annular coding mark image into a vertically arranged strip coding band, parsing the coding value of the strip coding band to obtain the spatial coordinates of the coding mark point; An objective function for extrinsic parameter correction is established based on the pixel coordinates and spatial coordinates of the coded marker points, and the camera extrinsic parameters used in the current image frame calculated based on the pod information are substituted into the objective function as initial values ​​for iterative correction.

6. The high-speed target measurement method based on the UAV pod and the calibration target according to claim 5 is characterized in that: The step of converting the annular coding band in the corrected annular coding mark image into a vertically arranged strip coding band, and parsing the coding value of the strip coding band to obtain the spatial coordinates of the coding mark point comprises: Performing polar coordinate transformation on the annular coding belt to convert it into a vertically arranged strip coding belt; Obtain the number of pixels in each coding block of the strip coding band from top to bottom; Using the pixel value as the binary number required to be encoded for each coding block, the number of binary numbers of each coding block is calculated according to the relative proportion of the number of pixels of each coding block to the total number of pixels in the vertical direction of the strip coding band; Obtaining a binary sequence of each coding block according to the binary number and the number of the binary numbers; The binary sequences are sorted cyclically, and the decimal number corresponding to the binary sequence with the smallest value is selected as the encoding value; The spatial coordinates of the coded marker points are obtained based on the one-to-one correspondence between the coded value and the spatial coordinates.

7. The high-speed target measurement method based on an unmanned aerial vehicle pod and a calibration target according to claim 5 is characterized in that: The objective function of extrinsic parameter correction is established based on the pixel coordinates and spatial coordinates of the coded marker points, and the camera extrinsic parameters calculated based on the pod information are substituted as initial values ​​into the objective function for iterative correction, including: The spatial conversion error is calculated based on the projection relationship between the camera coordinates and the pixel coordinates and the camera intrinsic parameters; Constructing an objective function for extrinsic parameter correction based on minimizing the spatial conversion error; Substitute the rotation matrix of the camera extrinsic parameters calculated based on the pod information into the objective function as the initial rotation matrix to calculate the current translation vector; Substitute the current rotation matrix and translation vector into the objective function about the current rotation matrix to obtain the rotation matrix expression for the next iteration; Solve the iterative rotation matrix expression using singular value decomposition; Repeat the iterative process until the rotation matrix converges and obtain the corrected camera extrinsics.

8. The high-speed target measurement method based on an unmanned aerial vehicle pod and a calibration target according to claim 1 is characterized in that: When using a binocular camera to capture images of high-speed target motion, a Kalman filter is used to detect and track high-speed targets.

9. The high-speed target measurement method based on the drone pod and the calibration target according to claim 8 is characterized in that: Analyzing the motion parameters of the high-speed target based on the camera intrinsic parameters, the modified camera extrinsic parameters, and the high-speed target motion image includes: The spatial coordinates of the high-speed target are calculated based on the camera intrinsic parameters, the camera extrinsic parameters, and the pixel positions of the high-speed target in the motion images of the left and right cameras; the spatial coordinates obtained in all the high-speed target motion images constitute the spatial coordinate sequence of the high-speed target; Decompose the discrete spatial coordinate sequence into real signal data and zero-mean white noise data; Representing the real signal data by a linear combination of a family of orthogonal polynomials of a set of time functions; According to the least squares principle, by minimizing the sum of squares of differences between the real signal data and the spatial coordinate sequence, the coefficients of the orthogonal polynomial family are estimated by least squares; Substitute the estimated orthogonal polynomial family coefficients into the decomposition of the spatial coordinate sequence to calculate the optimal estimate of the real signal data; The optimal estimate of the real signal data is subjected to multi-order derivatives with respect to time to obtain the motion parameters of the high-speed target; wherein the 0th order derivative represents the displacement of the high-speed target; the 1st order derivative represents the speed of the high-speed target; and the 2nd order derivative represents the acceleration of the high-speed target.

10. A high-speed target measurement system based on an unmanned aerial vehicle pod and a calibration target, characterized in that: include: The intrinsic parameter calibration module is used to calibrate the binocular camera to obtain the camera's intrinsic parameters; The data acquisition module is used to collect high-speed target motion images and calibration target images in the measurement field, and obtain the drone's pod information in real time during the shooting process and record it in the image frame header; An extrinsic parameter calibration module, used to calculate the camera extrinsic parameters used in the current image frame based on the pod information, and to correct the camera extrinsic parameters based on the calibration target image; The measurement and solution module is used to analyze the motion parameters of the high-speed target based on the camera intrinsic parameters, the corrected camera extrinsic parameters, and the high-speed target motion image.

Images