Method and system for detecting visual features and measuring pose of a boom for autonomous aerial refueling

Abstract

The application discloses a kind of autonomous aerial refueling cone visual feature detection and pose measurement method, comprising: defining and calibrating the relative relationship between required coordinate systems, coordinate system includes world coordinate system, camera coordinate system, image coordinate system and pixel coordinate system;Collect the motion video of cone, label cone picture key point and construct cone dataset;Build model infrastructure, train to form the key point detection model of cone for the subpixel level key point pixel detection result of key point feature on non-cooperative target cone;Based on the subpixel level key point detection coordinate result fitting cone inner and outer circle ellipse equation;Based on the relative relationship between coordinate system and inner and outer circle ellipse equation of cone, the initial value of cone pose is solved by double space conical pose measurement method;Based on the double space circle pose measurement optimization method of chamfer distance loss, the initial value of cone pose is optimized by key point feature pose re-projection. Corresponding system, electronic equipment and computer readable storage medium are also disclosed.

Description

Technical Field

[0001] The present invention relates to the field of computer vision measurement technology, and in particular to a method and system for detecting visual features and measuring the posture of an autonomous aerial refueling drogue. Background Art

[0002] Autonomous aerial refueling can significantly increase an aircraft's endurance and combat radius, enhance its long-range strike capability, and improve its air defense capabilities. High-precision visual position measurement of the drogue is the key to successful docking.

[0003] Currently, methods for drogue visual feature detection and pose measurement primarily target drogues with active LED or passive color threshold optical cooperative targets, providing auxiliary feature detection and visual measurement. However, the features of these drogues differ significantly from those of drogues without optical cooperative targets in actual aerial refueling scenarios. Consequently, existing methods face key challenges in aerial refueling, including difficulty in feature localization and low measurement accuracy.

[0004] Deep learning-based feature detection methods offer advantages such as strong adaptability, high robustness, and high accuracy, making them suitable for detecting visual features in autonomous aerial refueling drogues. However, these methods suffer from computational complexity and long inference times. Furthermore, autonomous aerial refueling often relies on onboard embedded devices or edge computing devices with limited computing resources. Therefore, achieving real-time detection and measurement of visual features onboard autonomous aerial refueling platforms remains a key challenge. Summary of the Invention

[0005] The purpose of the present invention is to provide a method and system for visual feature detection and posture measurement of an autonomous aerial refueling drogue, wherein the visual feature detection method can realize real-time detection of sub-pixel key point features of a non-cooperative target drogue, and at the same time the posture measurement method can realize visual posture measurement based on the key point features of the drogue, thereby providing robust and accurate measurement results for the autonomous aerial refueling system and improving the efficiency of autonomous aerial refueling, thereby generally solving the problems of difficult positioning of key point features of non-cooperative targets, long reasoning time, and low measurement accuracy in the prior art, so as to achieve the purpose of robust and accurate measurement of non-cooperative target drogues.

[0006] A first aspect of the present invention is to provide a method for detecting visual features and measuring the posture of an autonomous aerial refueling drogue, comprising:

[0007] S1, defining and calibrating the relative relationship between coordinate systems required for the pose measurement of the autonomous aerial refueling drogue, wherein the required coordinate systems include a world coordinate system, a camera coordinate system, an image coordinate system, and a pixel coordinate system;

[0008] S2, collect the motion video of the cone sleeve, label the key points of the cone sleeve picture and construct a cone sleeve dataset; wherein the motion video of the cone sleeve is collected at a fixed video frame rate and image resolution;

[0009] S3, build a basic framework of a basic cone key point detection model, train the basic cone key point detection model to form a deep learning-based cone key point detection model, and the cone key point detection model is used for sub-pixel level key point pixel detection results of key point features on a non-cooperative target cone;

[0010] S4, based on the sub-pixel level key point detection coordinate results of the key point features output by the cone key point detection model, fitting the ellipse equation of the inner and outer rings of the cone; based on the ellipse equation of the inner and outer rings of the cone and the relative relationship between the coordinate systems, the initial value of the cone pose is solved by a double-space cone pose measurement method;

[0011] S5, based on the double-space cone pose measurement optimization method of the chamfer distance loss, the initial value of the cone pose is optimized by key point feature pose re-projection.

[0012] Preferably, the S1 comprises:

[0013] S11, define the coordinate origin and direction of the world coordinate system, the camera coordinate system, the image coordinate system and the pixel coordinate system; the coordinate origin of the world coordinate system is defined at the position of the oil receiving machine refueling pipe cone head, and the three-axis coordinate directions are x-axis, y-axis and z-axis three basic coordinate axes, which are perpendicular to each other and together constitute a three-dimensional rectangular coordinate system; wherein: x-axis: usually indicates the horizontal direction, right for positive, left for negative; y-axis: usually indicates the vertical direction, downward for positive, upward for negative; z-axis: usually indicates the depth direction, forward for positive, backward for negative; the origin of the camera coordinate system is located at the camera optical center, and the three-axis coordinate directions are parallel to the image plane respectively, with the camera optical axis perpendicular to the image plane as the Z c axis, the X c and Y c axes conform to the right-hand coordinate system; the origin of the image coordinate system is the image center, and the two-axis coordinate directions are x-axis positive direction to the right and y-axis positive direction to the down; the coordinate origin of the pixel coordinate system is the upper left corner of the image as the coordinate origin, and the two-axis coordinate directions are u-axis positive direction to the right and v-axis positive direction to the down;

[0014] S12, calibrate the relative position relationship between the world coordinate system, the camera coordinate system, the image coordinate system and the pixel coordinate system; including:

[0015] (1) First, calibrate the camera intrinsic parameters;

[0016] (2) Then, according to the camera intrinsic parameter imaging model principle and the spatial transformation relationship, the camera extrinsic parameters are calibrated based on the principle of minimizing the reprojection error between the two-dimensional detection points and the three-dimensional space points optimized by LM;

[0017] (3) Finally, the relative positional relationship among the world coordinate system, camera coordinate system, image coordinate system, and pixel coordinate system is determined based on formula (1), which is as follows:

[0018]

[0019] Wherein, R, T represent the relative position relationship between the world coordinate system and the camera coordinate system; u0 and v0 are the origin coordinates of the image coordinate system after converting the image coordinate system to the pixel coordinate system; f x and f y are the normalized focal lengths of the x-axis and y-axis of the image coordinate system, u and v are the pixel plane coordinates, and x, y, and z are the world coordinate system coordinates.

[0020] Preferably, the S2 includes:

[0021] S21, using a camera fixed on the nose of the receiving aircraft to capture multiple motion videos of the drogue and extract original images of the drogue, wherein the multiple motion videos include motion videos of multiple scenes, multiple scales, and different drogue positions, and the camera captures the multiple motion videos at a fixed video frame rate and image resolution;

[0022] S22, marking key edge point features of the inner ring and outer ring of the taper sleeve and generating corresponding marking files;

[0023] S23, constructing a cone sleeve dataset based on the arrangement of the cone sleeve original images and the annotated files, wherein the cone sleeve dataset includes a training set, a validation set, and a test set, and the proportions of the number of images in the training set, the validation set, and the test set are 60%, 20%, and 20%, respectively.

[0024] Preferably, the S3 includes:

[0025] S31, construct a basic cone sleeve key point detection model; the basic cone sleeve key point detection model includes 1 Steam, 4 Stages, 3 Downsample modules and 1 Neck module; adopt the structure reparameterization technology to merge the convolution blocks into a 3*3 convolution, and adopt the sub-pixel key point output head based on coordinate classification to encode and decode the key point coordinates. The specific formulas of key point encoding and decoding are shown in formulas (2) and (3) respectively:

[0026] The key point encoding formula is shown in formula (2):

[0027]

[0028] Among them, σ is the standard deviation of the coordinate Gaussian distribution encoding, i, j are row vector index and column vector index respectively, are the horizontal and vertical coordinates of the key points before encoding, x i ,y j They are Gaussian distribution on the horizontal and vertical axes after encoding respectively.

[0029] The key point decoding formula is shown in formula (3):

[0030]

[0031] Where α is the scaling factor of the coordinate encoding and decoding, argmax() is the function that outputs the maximum value index in the Gaussian distribution; x (i),o y (j) are the Gaussian distribution of the horizontal coordinate and the Gaussian distribution of the vertical coordinate before the cone key point detection model predicts the output decoding, are the horizontal and vertical coordinates of the key points after decoding.

[0032] S32, constructing a loss function for the cone sleeve key point detection model, training the basic cone sleeve key point detection model based on the training set, the validation set, and the loss function until convergence to obtain the cone sleeve key point detection model; the loss function includes a key point classification loss function, a KL divergence loss function, an MSE regression loss function, and a chamfer distance loss function. The formulas for each loss function are shown in equations (4)-(7):

[0033] The key point classification loss function is shown in formula (4):

[0034]

[0035] Among them, N is the number of training samples, M is the number of key point categories, and y ij is the label category of the jth key point of the i-th sample, p ij is the predicted category probability of the jth key point of the i-th sample.

[0036] The KL divergence loss function is shown in formula (5):

[0037]

[0038] Among them, M is the number of key point categories, Y(ij) is the labeled Gaussian distribution of the i-th key point in the j-th coordinate direction, and X(ij) is the predicted Gaussian distribution of the i-th key point in the j-th coordinate direction.

[0039] The MSE regression loss function is shown in formula (6):

[0040]

[0041] in, are the marked coordinates and predicted coordinates of the i-th key point on the inner ring ellipse of the cone sleeve, are the labeled coordinates and predicted coordinates of the i-th key point on the outer ring ellipse of the cone sleeve.

[0042] The chamfer distance loss function is shown in formula (7):

[0043]

[0044] in, is the set of labeled points and predicted points on the j-th ellipse of the cone sleeve, D chamfer is the chamfer distance loss function.

[0045] In summary, considering the above loss functions, the designed total loss function is shown in formula (8):

[0046]

[0047] Among them, L cls ,η1 is the classification loss and its corresponding weight, L KL ,η2 is the KL divergence loss and its corresponding weight, L MSE ,η3 is the MSE regression loss and its corresponding weight, L chamfer ,η4 is the chamfer distance loss and its corresponding weight, L total is the total classification loss;

[0048] S33, performing inference based on the cone-set key point detection model and outputting sub-pixel key point detection pixel coordinate results, including:

[0049] (1) using each frame of the test video in the test set as input to the cone sleeve key point detection model;

[0050] (2) The cone-set key point detection model performs inference and outputs the sub-pixel key point detection coordinate results of the corresponding frame image.

[0051] Preferably, the S4 includes:

[0052] S41, based on the determination that the inner and outer rings of the cone sleeve are annular and are imaged as ellipses on the image plane, and the sub-pixel key point detection coordinate results of the key point features output by the cone sleeve key point detection model, an ellipse fitting algorithm is used to calculate the ellipse equations of the inner and outer rings of the cone sleeve in the image plane pixel coordinate system; wherein the ellipse fitting algorithm is implemented based on the least squares method, and the ellipse equations of the inner and outer rings of the cone sleeve are shown in formula (9):

[0053] au2 +bv 2 +cuv+du+ev+f=0 (9);

[0054] Among them, u, v are pixel plane coordinates, a, b, c, d, e, f are parameters of the general equation of the ellipse;

[0055] S42, based on the ellipse equations of the inner and outer rings of the cone sleeve and the relative relationship between the coordinate systems, solving the initial value of the cone sleeve posture by a dual-space cone sleeve posture measurement method.

[0056] Preferably, the S42 includes:

[0057] (1) The general elliptical equation corresponding to the elliptical equations of the inner and outer rings of the cone sleeve is transformed into an elliptical quadratic equation as shown in the following equation (10):

[0058] [uv 1]E[uv 1] T =0 (10)

[0059] Where u and v are pixel plane coordinates, and E is the elliptic quadratic equation matrix. According to the general ellipse equation (9) and the elliptic quadratic equation (10), the elliptic quadratic equation matrix E is specifically shown in the following equation (11):

[0060]

[0061] Among them, a, b, c, d, e, and f are the parameters of the general equation of the ellipse;

[0062] (2) According to the camera imaging principle and perspective projection principle, the camera imaging formula corresponding to the mapping relationship between the three-dimensional point in space and the two-dimensional point in the plane is determined as shown in the following formula (12):

[0063]

[0064] Among them, K is the camera internal parameter matrix, f x ,f y , u0, v0 represent the relative position relationship between the camera coordinate system, the image coordinate system and the pixel coordinate system; u0 and v0 are the origin coordinates of the image coordinate system after converting the image coordinate system to the pixel coordinate system; f x and f y are the normalized focal lengths of the x-axis and y-axis of the image coordinate system, u and v are the pixel plane coordinates, and x, y, and z are the world coordinate system coordinates;

[0065] (3) According to the elliptic quadratic formulas (10), (11) and the camera imaging formula (12), the spatial equation of the cone-in-cone in the camera coordinate system is obtained as shown in the following formula (13):

[0066] [xyz]KT EK[xyz]T=0 (13);

[0067] Among them, K is the camera intrinsic parameter matrix, x, y, z are the world coordinate system coordinates, and E is the elliptic quadratic equation matrix; for the convenience of expression, define Q = K T EK;

[0068] (4) Determine the transformation matrix P and transform the view cone equation from the camera coordinate system to the standard coordinate system. The spatial equation of the view cone in the standard coordinate system is shown in the following equation (14):

[0069] P T QP=P -1 QP=diag{λ1,λ2,λ3} (14);

[0070] Where P is the transformation matrix, Q = K T EK, λ1, λ2, λ3 are matrices P T The characteristic value of QP;

[0071] (5) calculating the coordinates of the center point of the cone sleeve in the standard coordinate system and the plane normal vector based on the spatial equation of the viewing cone in the standard coordinate system;

[0072] The coordinates of the center point of the cone sleeve in the standard coordinate system are shown in the following formula (15):

[0073]

[0074] Among them, R is the radius of the cone sleeve space circle, λ1, λ2, λ3 are the matrix P T The characteristic value of QP, is the coordinate of the center point of the cone sleeve space circle in the standard coordinate system.

[0075] The plane normal vector of the cone sleeve in the standard coordinate system is shown in the following formula (16):

[0076]

[0077] Among them, λ1, λ2, λ3 are matrices P T The characteristic value of QP, is the plane normal vector of the cone sleeve space circle in the standard coordinate system;

[0078] (6) The center point coordinates and the plane normal vector are converted from the standard coordinate system back to the camera coordinate system through the transformation matrix P. The center point coordinates and the plane normal vector of the cone sleeve in the camera coordinate system are shown in the following formula (17):

[0079]

[0080] Where P is the transformation matrix, is the coordinate of the center point of the cone sleeve space circle in the standard coordinate system, and x, y, z are the coordinates of the center point of the cone sleeve space circle in the camera coordinate system; is the plane normal vector of the cone sleeve space circle in the standard coordinate system, n x ,n y ,n z is the plane normal vector of the cone sleeve space circle in the camera coordinate system;

[0081] At the same time, the plane normal vector of the cone sleeve in the standard coordinate system is converted into the pitch angle and yaw angle of the cone sleeve in the camera coordinate system as shown in formula (18):

[0082]

[0083] Among them, n x ,n y ,n z is the plane normal vector of the cone sleeve space circle in the camera coordinate system, φ is the pitch angle of the cone sleeve in the camera coordinate system, and β is the yaw angle of the cone sleeve in the camera coordinate system.

[0084] Preferably, the S5 includes:

[0085] S51, based on the initial value of the cone sleeve posture obtained in S42, according to the camera perspective projection formula and the three-dimensional digital-analog point coordinates of the cone sleeve, calculate the pixel coordinates of the two-dimensional point of the cone sleeve posture reprojection on the pixel plane;

[0086] S52, constructing a minimization target optimization function based on the chamfer distance loss function of the two-dimensional detection points and the pose reprojection two-dimensional points, wherein the minimization target optimization function is shown in the following formula (19):

[0087]

[0088] Among them, P i is the pixel coordinate point set of the two-dimensional detection point, is the pixel coordinate point set of the pose reprojected two-dimensional point, are the optimized rotation matrix and translation vector, argmin() is the Levenberg-Marquardt optimization algorithm;

[0089] The chamfer distance loss function is shown in formula (20):

[0090]

[0091] Among them, P i is the pixel coordinate point set of the two-dimensional detection point, is the pixel coordinate point set of the pose reprojected two-dimensional point; |P i | is the number of corresponding point sets, For the two-dimensional detection point set Pi Middle detection point p ij and the pixel coordinate point set of the reprojected two-dimensional point Center projection point The minimum distance between.

[0092] A second aspect of the present invention is to provide a system for autonomous aerial refueling drogue visual feature detection and posture measurement, which is used to implement the method of the first aspect, including:

[0093] A system calibration module (101) is used to define and calibrate the relative relationship between coordinate systems required for the posture measurement of the autonomous aerial refueling drogue, wherein the required coordinate systems include a world coordinate system, a camera coordinate system, an image coordinate system, and a pixel coordinate system;

[0094] An image acquisition module (102) is used to acquire motion video of the cone sleeve, mark key points of the cone sleeve image and construct a cone sleeve data set; wherein the motion video of the cone sleeve is acquired at a fixed video frame rate and image resolution;

[0095] A feature detection module (103) is used to build an infrastructure of a basic cone sleeve key point detection model, train the basic cone sleeve key point detection model to form a cone sleeve key point detection model based on deep learning, and the cone sleeve key point detection model is used for sub-pixel key point pixel detection results of key point features on non-cooperative target cone sleeves;

[0096] A posture measurement module (104) is used to fit the cone sleeve inner and outer circle ellipse equations based on the sub-pixel key point detection coordinate results of the key point features output by the cone sleeve key point detection model; and to solve the cone sleeve initial posture value by a dual-space cone sleeve posture measurement method based on the relative relationship between the cone sleeve inner and outer circle ellipse equations and the coordinate system;

[0097] A posture optimization module (105) is used for a dual-space circular posture measurement optimization method based on chamfer distance loss, and optimizes the initial value of the cone sleeve posture by reprojecting key point feature postures.

[0098] A third aspect of the present invention provides an electronic device, comprising a processor and a memory, wherein the memory stores a plurality of instructions, and the processor is configured to read the instructions and execute the method described in the first aspect.

[0099] A fourth aspect of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a plurality of instructions, and the plurality of instructions can be read by a processor to execute the method described in the first aspect.

[0100] Beneficial effects of the method and system of the present invention:

[0101] The visual feature detection method can realize real-time detection of sub-pixel key point features of non-cooperative target drogues. At the same time, the posture measurement method can realize visual posture measurement based on the key point features of the drogue, providing robust and accurate measurement results for the autonomous aerial refueling system, improving the efficiency of autonomous aerial refueling, and thus solving the problems of difficult positioning of key point features of non-cooperative targets, long reasoning time, and low measurement accuracy in the existing technology, so as to achieve the purpose of robust and accurate measurement of non-cooperative target drogues. BRIEF DESCRIPTION OF THE DRAWINGS

[0102] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in related technologies, the following briefly introduces the drawings required for use in the specific embodiments or related technical descriptions. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0103] Figure 1 Flowchart of a method for detecting visual features and measuring the posture of an autonomous aerial refueling drogue according to an embodiment of the present invention;

[0104] Figure 2 This is a flowchart of step S1 of the autonomous aerial refueling drogue visual feature detection and posture measurement method provided by an embodiment of the present invention;

[0105] Figure 3 A schematic diagram of the relative relationship between various coordinate systems of the aerial refueling measurement system provided according to an embodiment of the present invention;

[0106] Figure 4 This is a flowchart of step S2 of the autonomous aerial refueling drogue visual feature detection and posture measurement method provided by an embodiment of the present invention;

[0107] Figure 5 This is a flowchart of step S3 of the autonomous aerial refueling drogue visual feature detection and posture measurement method provided by an embodiment of the present invention;

[0108] Figure 6 Schematic diagram of a basic cone sleeve key point detection model provided according to an embodiment of the present invention;

[0109] Figure 7 A schematic diagram of key point encoding and decoding based on coordinate classification according to an embodiment of the present invention;

[0110] Figure 8 This is a flowchart of step S4 of the autonomous aerial refueling drogue visual feature detection and posture measurement method provided by an embodiment of the present invention;

[0111] Figure 9This is a flowchart of step S5 of the autonomous aerial refueling drogue visual feature detection and posture measurement method provided by an embodiment of the present invention;

[0112] Figure 10 Autonomous aerial refueling drogue visual feature detection and pose measurement system architecture diagram;

[0113] Figure 11 A structural diagram of an electronic device provided according to an embodiment of the present invention. DETAILED DESCRIPTION

[0114] The technical solution of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0115] In the description of the present invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings and are intended solely to facilitate and simplify the description of the present invention. They are not intended to indicate or imply that the devices or components referred to must have, be constructed, or operate in a specific orientation, and therefore should not be construed as limitations on the present invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0116] In the description of the present invention, it should be noted that, unless otherwise expressly specified or limited, the terms "mounted," "connected," and "connected" should be understood in a broad sense. For example, they may refer to fixed, detachable, or integral connections; mechanical or electrical connections; direct or indirect connections through an intermediate medium; and internal communication between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on the specific circumstances.

[0117] Example 1

[0118] like Figure 1 As shown, this embodiment provides a method for detecting visual features and measuring the posture of an autonomous aerial refueling drogue, including:

[0119] S1, defining and calibrating the relative relationship between coordinate systems required for the pose measurement of the autonomous aerial refueling drogue, wherein the required coordinate systems include a world coordinate system, a camera coordinate system, an image coordinate system, and a pixel coordinate system;

[0120] like Figure 2As shown, as a preferred embodiment, the S1 includes:

[0121] S11, defining the coordinate origins and directions of the world coordinate system, the camera coordinate system, the image coordinate system, and the pixel coordinate system;

[0122] In this embodiment, the coordinate origin of the world coordinate system is defined at the location of the refueling pipe cone of the receiving aircraft. Its three-axis coordinate directions are the three basic coordinate axes of x-axis, y-axis, and z-axis. These three coordinate axes are perpendicular to each other and together form a three-dimensional rectangular coordinate system. Among them: x-axis: usually represents the horizontal direction, with positive to the right and negative to the left; y-axis: usually represents the vertical direction, with positive downward and negative upward; z-axis: usually represents the depth direction, with positive forward and negative backward;

[0123] The origin of the camera coordinate system is located at the optical center of the camera, and the three-axis coordinate direction is the camera optical axis perpendicular to the image plane and pointing outward as Z. c Axis, X c and Y c Following the right-hand coordinate system, they are parallel to the image plane;

[0124] The origin of the image coordinate system is the center of the image, and the directions of the two axes are that the positive direction of the x-axis is rightward and the positive direction of the y-axis is downward;

[0125] The coordinate origin of the pixel coordinate system is the upper left corner of the image, and the two-axis coordinate directions are that the positive direction of the u axis is rightward and the positive direction of the v axis is downward.

[0126] like Figure 3 As shown, point p in the image coordinate system is the projection point of point P in the camera coordinate system, and the corresponding relationship between the image coordinate system and the pixel coordinate system is as follows Figure 3 As shown, u0 and v0 are the origin coordinates of the image coordinate system after the image coordinate system is converted to the pixel coordinate system.

[0127] S12, calibrating the relative positional relationships among the world coordinate system, the camera coordinate system, the image coordinate system, and the pixel coordinate system.

[0128] In this embodiment, the S12 includes:

[0129] (1) First, the camera’s internal parameters are calibrated. The specific solution method is described in detail in Zhang Zhengyou’s article “A flexible new technique for camera calibration[R]. Microsoft Corporation, NSRTR-98-71, 1998”.

[0130] (2) Then, according to the camera intrinsic parameter imaging model principle and the spatial transformation relationship, the camera extrinsic parameters are calibrated based on the principle of minimizing the reprojection error between the two-dimensional detection points and the three-dimensional space points optimized by LM.

[0131] (3) Finally, the relative positional relationship among the world coordinate system, camera coordinate system, image coordinate system, and pixel coordinate system is determined based on formula (1), which is as follows:

[0132]

[0133] Wherein, R, T represent the relative position relationship between the world coordinate system and the camera coordinate system; u0 and v0 are the origin coordinates of the image coordinate system after converting the image coordinate system to the pixel coordinate system; f x and f y are the normalized focal lengths of the x-axis and y-axis of the image coordinate system, u and v are the pixel plane coordinates, and x, y, and z are the world coordinate system coordinates.

[0134] S2, collecting motion videos of the cone cover, marking key points of the cone cover image and constructing a cone cover dataset; wherein the motion videos of the cone cover are collected at a fixed video frame rate and image resolution;

[0135] See also Figure 4 As a preferred embodiment, S2 includes:

[0136] S21, using a camera fixed on the nose of the receiving aircraft to capture multiple motion videos of the drogue and extract original images of the drogue, wherein the multiple motion videos include motion videos of multiple scenes, multiple scales, and different drogue positions, and the camera captures the multiple motion videos at a fixed video frame rate and image resolution;

[0137] S22, marking key edge point features of the inner ring and outer ring of the taper sleeve and generating corresponding marking files;

[0138] Specifically, in this embodiment, the inner ring of the cone sleeve includes 12 edge key points, and the outer ring includes 36 edge key points. All key points are evenly distributed on the edges of the inner and outer rings of the cone sleeve according to angles to form edge key point features;

[0139] S23, constructing a cone sleeve dataset based on the arrangement of the cone sleeve original images and the annotated files, wherein the cone sleeve dataset includes a training set, a validation set, and a test set, and the proportions of the number of images in the training set, the validation set, and the test set are 60%, 20%, and 20%, respectively.

[0140] S3, building an infrastructure of a basic cone-in-shell key point detection model, training the basic cone-in-shell key point detection model to form a cone-in-shell key point detection model based on deep learning, wherein the cone-in-shell key point detection model is used for sub-pixel key point pixel detection results of key point features on the cone-in-shell of a non-cooperative target;

[0141] As a preferred embodiment, Figure 5 As shown, the S3 includes:

[0142] S31, cf. Figure 6 The basic cone sleeve key point detection model schematic diagram shown in the figure is used to construct the basic cone sleeve key point detection model; the basic cone sleeve key point detection model includes 1 Steam, 4 Stages, 3 Downsample modules and 1 Neck module; it is used to realize the extraction and fusion of low-level details, high-level semantics and multi-scale features; among them, in order to achieve lightweight model and real-time processing, the structural re-parameterization technology is used to merge the convolution blocks into a 3*3 convolution, which effectively reduces the number of model parameters and computational cost, and improves the inference speed and real-time performance of the detection model in airborne embedded devices and edge computing devices; at the same time, see Figure 7 In order to improve the accuracy of cone-set key point detection, a sub-pixel key point output head based on coordinate classification is used to encode and decode key point coordinates. By converting the coordinate regression task into a coordinate classification task, the one-dimensional Gaussian distribution reduces the amount of output head calculation compared to the two-dimensional heat map method of Heatmap, and facilitates the training convergence of the cone-set key point detection model. At the same time, when the coordinate encoding and decoding scaling factor is greater than 1, the key point encoding and decoding accuracy based on coordinate classification can reach the sub-pixel level. The specific formulas for key point encoding and decoding are shown in Equation (2) and Equation (3), respectively:

[0143] The key point encoding formula is shown in formula (2):

[0144]

[0145] Among them, σ is the standard deviation of the coordinate Gaussian distribution encoding, i, j are row vector index and column vector index respectively, are the horizontal and vertical coordinates of the key points before encoding, x i ,y j They are Gaussian distributions of the horizontal and vertical axes after encoding, respectively.

[0146] The key point decoding formula is shown in formula (3):

[0147]

[0148] Where α is the scaling factor of the coordinate encoding and decoding, argmax() is the function that outputs the maximum value index in the Gaussian distribution; x (i),oy (j) are the Gaussian distributions of the horizontal and vertical coordinates before the cone key point detection model predicts the output decoding, are the horizontal and vertical coordinates of the key points after decoding.

[0149] S32, constructing a loss function for the cone sleeve key point detection model, training the basic cone sleeve key point detection model based on the training set, the validation set, and the loss function until convergence to obtain the cone sleeve key point detection model; the loss function includes a key point classification loss function, a KL divergence loss function, an MSE regression loss function, and a chamfer distance loss function. The formulas for each loss function are shown in equations (4)-(7):

[0150] The key point classification loss function is shown in formula (4):

[0151]

[0152] Among them, N is the number of training samples, M is the number of key point categories, and y ij is the label category of the jth key point of the i-th sample, p ij is the predicted category probability of the jth key point of the i-th sample.

[0153] The KL divergence loss function is shown in formula (5):

[0154]

[0155] Among them, M is the number of key point categories, Y(ij) is the labeled Gaussian distribution of the i-th key point in the j-th coordinate direction, and X(ij) is the predicted Gaussian distribution of the i-th key point in the j-th coordinate direction.

[0156] The MSE regression loss function is shown in formula (6):

[0157]

[0158] in, are the marked coordinates and predicted coordinates of the i-th key point on the inner ring ellipse of the cone sleeve, are the labeled coordinates and predicted coordinates of the i-th key point on the outer ring ellipse of the cone sleeve.

[0159] The chamfer distance loss function is shown in formula (7):

[0160]

[0161] in, is the set of labeled points and predicted points on the j-th ellipse of the cone sleeve, D chamfer is the chamfer distance loss function.

[0162] In summary, considering the above loss functions, the designed total loss function is shown in formula (8):

[0163]

[0164] Among them, L cls ,η1 is the classification loss and its corresponding weight, L KL ,η2 is the KL divergence loss and its corresponding weight, L MSE ,η3 is the MSE regression loss and its corresponding weight, L chamfer ,η4 is the chamfer distance loss and its corresponding weight, L total is the total classification loss;

[0165] S33, performing inference based on the cone-set key point detection model and outputting sub-pixel key point detection pixel coordinate results;

[0166] In this embodiment, the S33 includes:

[0167] (1) using each frame of the test video in the test set as input to the cone sleeve key point detection model;

[0168] (2) The cone-set key point detection model performs inference and outputs the sub-pixel key point detection coordinate results of the corresponding frame image.

[0169] S4, fitting the ellipse equations of the inner and outer circles of the cone sleeve based on the sub-pixel key point detection coordinate results of the key point features output by the cone sleeve key point detection model; solving the initial value of the cone sleeve posture using a dual-space cone sleeve posture measurement method based on the relative relationship between the ellipse equations of the inner and outer circles of the cone sleeve and the coordinate system;

[0170] As a preferred embodiment, Figure 8 As shown, the S4 includes:

[0171] S41, based on the determination that the inner and outer rings of the cone sleeve are annular and are imaged as ellipses on the image plane, and the sub-pixel key point detection coordinate results of the key point features output by the cone sleeve key point detection model, an ellipse fitting algorithm is used to calculate the ellipse equations of the inner and outer rings of the cone sleeve in the image plane pixel coordinate system; wherein the ellipse fitting algorithm is implemented based on the least squares method, and the ellipse equations of the inner and outer rings of the cone sleeve are shown in formula (9):

[0172] au 2 +bv 2 +cuv+du+ev+f=0 (9);

[0173] Among them, u and v are pixel plane coordinates, and a, b, c, d, e, and f are parameters of the general equation of the ellipse.

[0174] S42, based on the cone inner and outer ring ellipse equation and the relative relationship between the coordinate systems, the cone pose initial value is solved by a double-space cone pose measurement method, comprising:

[0175] (1) The ellipse general equation corresponding to the cone inner and outer ring ellipse equation is converted into an ellipse quadratic form equation as shown in the following formula (10):

[0176] [u v 1]E[u v 1] T = 0 (10)

[0177] Wherein, u, v are pixel plane coordinates, and E is an ellipse quadratic form equation matrix. According to the ellipse general equation (9) and the ellipse quadratic form equation (10), the ellipse quadratic form equation matrix E is specifically as shown in the following formula (11):

[0178]

[0179] Wherein, a, b, c, d, e, f are ellipse general equation parameters.

[0180] (2) According to the camera imaging principle and the perspective projection principle, the mapping relationship corresponding to the space three-dimensional point and the plane two-dimensional point is determined as shown in the following formula (12):

[0181]

[0182] Wherein, K is a camera intrinsic matrix, f x ,f y ,u0,v0 represents the relative position relationship between the camera coordinate system and the image coordinate system, pixel coordinate system; u0 and v0 are the origin coordinates of the image coordinate system after the image coordinate system is converted to the pixel coordinate system; f x and f y are the normalized focal length of the image coordinate system x axis and y axis respectively, u, v are pixel plane coordinates, and x, y, z are world coordinate system coordinates.

[0183] (3) According to the ellipse quadratic form formula (10), (11) and the camera imaging formula (12), the space equation of the cone frustum in the camera coordinate system is obtained as shown in the following formula (13):

[0184] [x y z]K T EK[x y z] T = 0 (13);

[0185] Wherein, K is a camera intrinsic matrix, x, y, z are world coordinate system coordinates, and E is an ellipse quadratic form equation matrix; in order to facilitate expression, Q = K T EK.

[0186] (4) According to the matrix principle, the matrix Q is a real symmetric matrix. Therefore, there exists a transformation matrix P that can diagonalize the matrix Q. At the same time, in order to facilitate matrix calculation and solution, the transformation matrix P can transform the cone equation from the camera coordinate system to the standard coordinate system. The spatial equation of the cone in the standard coordinate system is shown in the following formula (14):

[0187] P T QP=P -1 QP=diag{λ1,λ2,λ3} (14);

[0188] Where P is the transformation matrix, Q = K T EK, λ1, λ2, λ3 are matrices P T Characteristic value of QP.

[0189] (5) calculating the coordinates of the center point of the cone sleeve in the standard coordinate system and the plane normal vector based on the spatial equation of the viewing cone in the standard coordinate system;

[0190] The coordinates of the center point of the cone sleeve in the standard coordinate system are shown in the following formula (15):

[0191]

[0192] Among them, R is the radius of the cone sleeve space circle, λ1, λ2, λ3 are the matrix P T The characteristic value of QP, is the coordinate of the center point of the cone sleeve space circle in the standard coordinate system.

[0193] The plane normal vector of the cone sleeve in the standard coordinate system is shown in the following formula (16):

[0194]

[0195] Among them, λ1, λ2, λ3 are matrices P T The characteristic value of QP, is the plane normal vector of the cone sleeve space circle in the standard coordinate system;

[0196] (6) The center point coordinates and the plane normal vector are converted from the standard coordinate system back to the camera coordinate system through the transformation matrix P. The center point coordinates and the plane normal vector of the cone sleeve in the camera coordinate system are shown in the following formula (17):

[0197]

[0198] Where P is the transformation matrix, is the coordinate of the center point of the cone sleeve space circle in the standard coordinate system, and x, y, z are the coordinates of the center point of the cone sleeve space circle in the camera coordinate system; is the plane normal vector of the cone sleeve space circle in the standard coordinate system, n x ,ny ,n z is the plane normal vector of the cone sleeve space circle in the camera coordinate system.

[0199] At the same time, the plane normal vector of the cone sleeve in the standard coordinate system is converted into the pitch angle and yaw angle of the cone sleeve in the camera coordinate system as shown in formula (18):

[0200]

[0201] Among them, n x ,n y ,n z is the plane normal vector of the cone sleeve space circle in the camera coordinate system, φ is the pitch angle of the cone sleeve in the camera coordinate system, and β is the yaw angle of the cone sleeve in the camera coordinate system.

[0202] The technical effects of implementing step S42 include: the key points of the inner ring and outer ring of the cone sleeve are not on the same spatial plane, so the pose solution based on the inner ring and outer ring of the cone sleeve can effectively avoid the ambiguity of the measurement results.

[0203] S5, a dual-space circular pose measurement optimization method based on chamfer distance loss, optimizes the initial value of the cone sleeve pose by reprojecting the key point feature pose.

[0204] like Figure 9 As shown, as a preferred embodiment, the S5 includes:

[0205] S51, based on the initial value of the cone sleeve posture obtained in S42, according to the camera perspective projection formula and the three-dimensional digital-analog point coordinates of the cone sleeve, calculate the pixel coordinates of the two-dimensional point of the cone sleeve posture reprojection on the pixel plane.

[0206] S52, constructing a minimization target optimization function based on the chamfer distance loss function of the two-dimensional detection points and the pose reprojection two-dimensional points, wherein the minimization target optimization function is shown in the following formula (19):

[0207]

[0208] Among them, P i is the pixel coordinate point set of the two-dimensional detection point, is the pixel coordinate point set of the pose reprojected two-dimensional point, are the optimized rotation matrix and translation vector, argmin() is the Levenberg-Marquardt optimization algorithm;

[0209] The chamfer distance loss function is shown in formula (20):

[0210]

[0211] Among them, P ia pixel coordinate point set of a two-dimensional detection point, a pixel coordinate point set of a two-dimensional pose re-projection point, i a number of corresponding point sets, a two-dimensional detection point set P i a detection point p ij in a two-dimensional detection point set P a pixel coordinate point set of a two-dimensional pose re-projection point, a projection point

[0212] Technical effect of S5: After obtaining the initial value of the cone pose based on the double-space cone pose measurement method, the double-space circle pose measurement optimization method based on the chamfer distance loss is used to improve the accuracy of the cone pose measurement by optimizing the key point feature re-projection, and the optimized result of the cone pose measurement is output.

[0213] Embodiment two

[0214] As shown in Figure 10 , the embodiment provides a self-contained aerial refueling cone visual feature detection and pose measurement system, which is used to implement the method of embodiment one, comprising:

[0215] A system calibration module 101 is configured to define and calibrate the relative relationship between the coordinate systems required for the pose measurement of the self-contained aerial refueling cone, wherein the required coordinate systems include a world coordinate system, a camera coordinate system, an image coordinate system, and a pixel coordinate system.

[0216] An image acquisition module 102 is configured to acquire a motion video of the cone, label key points of the cone picture, and construct a cone dataset; wherein the motion video of the cone is acquired at a fixed video frame rate and image resolution.

[0217] A feature detection module 103 is configured to build a basic framework of a basic cone key point detection model, train the basic cone key point detection model to form a deep learning-based cone key point detection model, and use the cone key point detection model to detect sub-pixel level key point pixels of key point features on a non-cooperative target cone.

[0218] A pose measurement module 104 is configured to fit an inner and outer ring ellipse equation based on the sub-pixel level key point detection coordinate results of the key point features output by the cone key point detection model, and solve an initial value of the cone pose by a double-space cone pose measurement method based on the inner and outer ring ellipse equation and the relative relationship between the coordinate systems.

[0219] A pose optimization module 105 is configured to optimize the initial value of the cone pose by a key point feature pose re-projection based on a double-space circle pose measurement optimization method based on the chamfer distance loss.

[0220] The application further provides a memory which stores a plurality of instructions for implementing the method of embodiment one.

[0221] As shown in Figure 11 The application further provides an electronic device which comprises a processor 301 and a memory 302 connected with the processor 301, and the memory 302 stores a plurality of instructions which can be loaded and executed by the processor to enable the processor to execute the method of embodiment one.

[0222] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the application, but not to limit it; although the application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the application.

Claims

1. A method for visual feature detection and pose measurement of an autonomous aerial refueling drogue, characterized in that: include: S1, defining and calibrating the relative relationship between coordinate systems required for the pose measurement of the autonomous aerial refueling drogue, wherein the required coordinate systems include a world coordinate system, a camera coordinate system, an image coordinate system, and a pixel coordinate system; S2, collecting motion videos of the cone cover, marking key points of the cone cover image and constructing a cone cover dataset; wherein the motion videos of the cone cover are collected at a fixed video frame rate and image resolution; the cone cover dataset includes a training set, a validation set and a test set; S3: Building a basic framework for a basic cone-in-a-box key point detection model, training the basic cone-in-a-box key point detection model to form a cone-in-a-box key point detection model based on deep learning, wherein the cone-in-a-box key point detection model is used for sub-pixel key point pixel detection results of key point features on the cone-in-a-box of a non-cooperative target; S3 includes: S31, construct a basic cone sleeve key point detection model; the basic cone sleeve key point detection model includes 1 Steam, 4 Stages, 3 Downsample modules and 1 Neck module; the convolution blocks are merged into a 3*3 convolution using structural reparameterization technology; the sub-pixel key point output head based on coordinate classification is used to encode and decode the key point coordinates, and the specific formulas for key point encoding and decoding are shown in Equation (2) and Equation (3), respectively: The key point encoding formula is shown in formula (2): Among them, σ is the standard deviation of the coordinate Gaussian distribution encoding, i, j are row vector index and column vector index respectively, are the horizontal and vertical coordinates of the key points before encoding, x i ,y j They are Gaussian distribution of horizontal coordinate and Gaussian distribution of vertical coordinate after encoding respectively; The key point decoding formula is shown in formula (3): Where α is the scaling factor of the coordinate encoding and decoding, argmax() is the function that outputs the maximum value index in the Gaussian distribution; x (i),o y (j) are the Gaussian distribution of the horizontal coordinate and the Gaussian distribution of the vertical coordinate before the cone key point detection model predicts the output decoding, are the horizontal and vertical coordinates of the key points after decoding; S32, constructing a loss function of a cone sleeve key point detection model, training the basic cone sleeve key point detection model based on the training set, the validation set and the loss function until convergence to obtain the cone sleeve key point detection model; the loss function includes a key point classification loss function KL divergence loss function MSE regression loss function And the first chamfer distance loss function S33, performing inference based on the cone-set key point detection model and outputting sub-pixel key point detection pixel coordinate results, including: (1) using each frame of the test video in the test set as input to the cone sleeve key point detection model; (2) The cone-set key point detection model performs inference and outputs the sub-pixel key point detection coordinate results of the corresponding frame image; S4, fitting the ellipse equations of the inner and outer circles of the cone sleeve based on the sub-pixel key point detection coordinate results of the key point features output by the cone sleeve key point detection model; solving the initial value of the cone sleeve posture using a dual-space cone sleeve posture measurement method based on the relative relationship between the ellipse equations of the inner and outer circles of the cone sleeve and the coordinate system; S5, a dual-space circular pose measurement optimization method based on the second chamfer distance loss, optimizes the initial value of the cone sleeve pose by reprojecting the key point feature pose.

2. The method for detecting visual features and measuring the posture of an autonomous aerial refueling drogue according to claim 1 is characterized in that: Said S1 comprises: S11, define the coordinate origin and direction of the world coordinate system, the camera coordinate system, the image coordinate system and the pixel coordinate system; the coordinate origin of the world coordinate system is defined at the position of the refueling pipe cone of the refueling aircraft, and its three-axis coordinate directions are the three basic coordinate axes of x-axis, y-axis and z-axis. These three coordinate axes are perpendicular to each other and together constitute a rectangular coordinate system in three-dimensional space; among which: x-axis: usually represents the horizontal direction, with positive to the right and negative to the left; y-axis: usually represents the vertical direction, with positive downward and negative upward; z-axis: usually represents the depth direction, with positive forward and negative backward; the origin of the camera coordinate system is located at the optical center of the camera, and its three-axis coordinate directions are the camera optical axis perpendicular to the image plane and outward as Z axis. c Axis, X c and Y c The right-hand coordinate system is parallel to the image plane. The origin of the image coordinate system is the center of the image, and the directions of its two axes are: the positive direction of the x-axis is rightward, and the positive direction of the y-axis is downward. The origin of the pixel coordinate system is the upper left corner of the image, and the directions of its two axes are: the positive direction of the u-axis is rightward, and the positive direction of the v-axis is downward. S12, calibrating the relative positional relationships among the world coordinate system, the camera coordinate system, the image coordinate system, and the pixel coordinate system; including: (1) First, calibrate the camera's internal parameters; (2) Then, according to the camera intrinsic parameter imaging model principle and the spatial transformation relationship, the camera extrinsic parameters are calibrated based on the principle of minimizing the reprojection error between the two-dimensional detection points and the three-dimensional space points optimized by LM; (3) Finally, the relative positional relationship among the world coordinate system, camera coordinate system, image coordinate system, and pixel coordinate system is determined based on formula (1), which is as follows: Wherein, R, T represent the relative position relationship between the world coordinate system and the camera coordinate system; u0 and v0 are the origin coordinates of the image coordinate system after converting the image coordinate system to the pixel coordinate system; f x and f v are the normalized focal lengths of the x-axis and y-axis of the image coordinate system, u and v are the pixel plane coordinates, and x, y, and z are the world coordinate system coordinates.

3. The method for detecting visual features and measuring the posture of an autonomous aerial refueling drogue according to claim 2 is characterized in that: The S2 includes: S21, using a camera fixed on the nose of the receiving aircraft to capture multiple motion videos of the drogue and extract original images of the drogue, wherein the multiple motion videos include motion videos of multiple scenes, multiple scales, and different drogue positions, and the camera captures the multiple motion videos at a fixed video frame rate and image resolution; S22, marking key edge point features of the inner ring and outer ring of the taper sleeve and generating corresponding marking files; S23, constructing a cone sleeve dataset based on the arrangement of the cone sleeve original images and the annotated files, wherein the ratios of the number of images in the training set, the validation set, and the test set are 60%, 20%, and 20%, respectively.

4. The method for detecting visual features and measuring the position and posture of an autonomous aerial refueling drogue according to claim 3 is characterized in that: The loss function formulas are shown in equations (4)-(7): The key point classification loss function is shown in formula (4): Among them, N is the number of training samples, M is the number of key point categories, and y ij is the label category of the jth key point of the i-th sample, p ij is the predicted category probability of the jth key point of the i-th sample; The KL divergence loss function is shown in formula (5): Where M is the number of key point categories, Y(ij) is the labeled Gaussian distribution of the jth coordinate direction of the i-th key point, and X(ij) is the predicted Gaussian distribution of the jth coordinate direction of the i-th key point; The MSE regression loss function is shown in formula (6): in, are the marked coordinates and predicted coordinates of the i-th key point on the inner ring ellipse of the cone sleeve, are the marked coordinates and predicted coordinates of the i-th key point on the outer ring ellipse of the cone sleeve; The first chamfer distance loss function is shown in formula (7): in, is the set of labeled points and predicted points on the j-th ellipse of the cone sleeve, is the second chamfer distance loss function; In summary, considering the above loss functions, the designed total loss function is shown in formula (8): Among them, L cls ,η1 is the classification loss and its corresponding weight, L KL ,η2 is the KL divergence loss and its corresponding weight, L MSE ,η3 is the MSE regression loss and its corresponding weight, L chamfer ,η4 is the chamfer distance loss and its corresponding weight, L total is the total classification loss; S33, performing inference based on the cone-set key point detection model and outputting sub-pixel key point detection pixel coordinate results, including: (1) using each frame of the test video in the test set as input to the cone sleeve key point detection model; (2) The cone-set key point detection model performs inference and outputs the sub-pixel key point detection coordinate results of the corresponding frame image.

5. The method for detecting visual features and measuring the posture of an autonomous aerial refueling drogue according to claim 4 is characterized in that: The S4 includes: S41, based on the determination that the inner and outer rings of the cone sleeve are annular and are imaged as ellipses on the image plane, and the sub-pixel key point detection coordinate results of the key point features output by the cone sleeve key point detection model, an ellipse fitting algorithm is used to calculate the ellipse equations of the inner and outer rings of the cone sleeve in the image plane pixel coordinate system; wherein the ellipse fitting algorithm is implemented based on the least squares method, and the ellipse equations of the inner and outer rings of the cone sleeve are shown in formula (9): oh 2 +bv 2 +cuv+du+ev+f=0 (9); Among them, u, v are pixel plane coordinates, a, b, c, d, e, f are parameters of the general equation of the ellipse; S42, based on the ellipse equations of the inner and outer rings of the cone sleeve and the relative relationship between the coordinate systems, solving the initial value of the cone sleeve posture by a dual-space cone sleeve posture measurement method.

6. The method for detecting visual features and measuring the position and posture of an autonomous aerial refueling drogue according to claim 5 is characterized in that: The S42 includes: (1) The general ellipse equation corresponding to the ellipse equation of the inner and outer rings of the cone sleeve is converted into the quadratic ellipse equation as shown below (10): [uv 1]E[uv 1] T =0 (10); Where u and v are pixel plane coordinates, and E is the elliptic quadratic equation matrix. According to the general ellipse equation (9) and the elliptic quadratic equation (10), the elliptic quadratic equation matrix E is specifically shown in the following equation (11): Among them, a, b, c, d, e, and f are the parameters of the general equation of the ellipse; (2) According to the camera imaging principle and perspective projection principle, the camera imaging formula corresponding to the mapping relationship between the three-dimensional point in space and the two-dimensional point in the plane is determined as shown in the following formula (12): Among them, K is the camera internal parameter matrix, f x ,f y , u0, v0 represent the relative position relationship between the camera coordinate system, the image coordinate system and the pixel coordinate system; u0 and v0 are the origin coordinates of the image coordinate system after converting the image coordinate system to the pixel coordinate system; f x and f y are the normalized focal lengths of the x-axis and y-axis of the image coordinate system, u and v are the pixel plane coordinates, and x, y, and z are the world coordinate system coordinates; (3) According to the elliptic quadratic equation (10), the elliptic quadratic equation matrix E (11) and the camera imaging formula (12), the spatial equation of the cone-in-cone in the camera coordinate system is obtained as shown in the following equation (13): [xyz]K T EK[xyz] T =0 (13); Among them, K is the camera intrinsic parameter matrix, x, y, z are the world coordinate system coordinates, and E is the elliptic quadratic equation matrix; for the convenience of expression, define Q = K T EK; (4) Determine the transformation matrix P and transform the view cone equation from the camera coordinate system to the standard coordinate system. The spatial equation of the view cone in the standard coordinate system is shown in the following equation (14): P T QP=P -1 QP=diag{λ1,λ2,λ3} (14); Where P is the transformation matrix, Q = K T EK, λ1, λ2, λ3 are matrices P T The characteristic value of QP; (5) calculating the coordinates of the center point of the cone sleeve in the standard coordinate system and the plane normal vector based on the spatial equation of the viewing cone in the standard coordinate system; The coordinates of the center point of the cone sleeve in the standard coordinate system are shown in the following formula (15): Among them, R is the radius of the cone sleeve space circle, λ1, λ2, λ3 are the matrix P T The characteristic value of QP, is the coordinate of the center point of the cone sleeve space circle in the standard coordinate system; The plane normal vector of the cone sleeve in the standard coordinate system is shown in the following formula (16): Among them, λ1, λ2, λ3 are matrices P T The characteristic value of QP, is the plane normal vector of the cone sleeve space circle in the standard coordinate system; (6) The center point coordinates and the plane normal vector are converted from the standard coordinate system back to the camera coordinate system through the transformation matrix P. The center point coordinates and the plane normal vector of the cone sleeve in the camera coordinate system are shown in the following formula (17): Where P is the transformation matrix, is the coordinate of the center point of the cone sleeve space circle in the standard coordinate system, and x, y, z are the coordinates of the center point of the cone sleeve space circle in the camera coordinate system; is the plane normal vector of the cone sleeve space circle in the standard coordinate system, n x ,n y ,n z is the plane normal vector of the cone sleeve space circle in the camera coordinate system; At the same time, the plane normal vector of the cone sleeve in the standard coordinate system is converted into the pitch angle and yaw angle of the cone sleeve in the camera coordinate system as shown in formula (18): Among them, n x ,n y ,n z is the plane normal vector of the cone sleeve space circle in the camera coordinate system, φ is the pitch angle of the cone sleeve in the camera coordinate system, and β is the yaw angle of the cone sleeve in the camera coordinate system.

7. The method for detecting visual features and measuring the position and posture of an autonomous aerial refueling drogue according to claim 6, wherein: The S5 includes: S51, based on the initial value of the cone sleeve posture obtained in S42, according to the camera perspective projection formula and the three-dimensional digital-analog point coordinates of the cone sleeve, calculate the pixel coordinates of the two-dimensional point of the cone sleeve posture reprojection on the pixel plane; S52, constructing a minimization target optimization function based on the second chamfer distance loss function of the two-dimensional detection point and the pose reprojection two-dimensional point, wherein the minimization target optimization function is shown in the following formula (19): Among them, P i is the pixel coordinate point set of the two-dimensional detection point, is the pixel coordinate point set of the pose reprojected two-dimensional point, are the optimized rotation matrix and translation vector, argmin() is the Levenberg-Marquardt optimization algorithm; The second chamfer distance loss function is shown in formula (20): Among them, P i is the pixel coordinate point set of the two-dimensional detection point, is the pixel coordinate point set of the pose reprojected two-dimensional point; |P i | is the number of corresponding point sets, For the two-dimensional detection point set P i Middle detection point p ij and the pixel coordinate point set of the reprojected two-dimensional point Center projection point The minimum distance between.

8. An autonomous aerial refueling drogue visual feature detection and posture measurement system, used to implement the method according to any one of claims 1 to 7, characterized in that: include: A system calibration module (101) is used to define and calibrate the relative relationship between coordinate systems required for the posture measurement of the autonomous aerial refueling drogue, wherein the required coordinate systems include a world coordinate system, a camera coordinate system, an image coordinate system, and a pixel coordinate system; An image acquisition module (102) is used to acquire motion video of the cone sleeve, mark key points of the cone sleeve image and construct a cone sleeve data set; wherein the motion video of the cone sleeve is acquired at a fixed video frame rate and image resolution; A feature detection module (103) is used to build an infrastructure of a basic cone sleeve key point detection model, train the basic cone sleeve key point detection model to form a cone sleeve key point detection model based on deep learning, and the cone sleeve key point detection model is used for sub-pixel key point pixel detection results of key point features on non-cooperative target cone sleeves; A posture measurement module (104) is used to output the sub-pixel key point detection coordinate results of the key point features based on the cone sleeve key point detection model, and fit the ellipse equations of the inner and outer circles of the cone sleeve; Based on the elliptical equations of the inner and outer rings of the cone sleeve and the relative relationship between the coordinate systems, the initial value of the cone sleeve posture is solved by a dual-space cone sleeve posture measurement method; A posture optimization module (105) is used for a dual-space circular posture measurement optimization method based on a second chamfer distance loss, and optimizes the initial value of the cone sleeve posture by reprojecting key point feature postures.

9. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores a plurality of instructions, and the processor is configured to read the instructions and execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a plurality of instructions, and the plurality of instructions can be read by a processor to execute the method according to any one of claims 1 to 7.

Images