Visual alignment method between refueling probe and refueling drogue for autonomous aerial refueling of UAV

Abstract

The present invention discloses a visual alignment method for an autonomous aerial refueling refueling probe and a refueling cone sleeve of a UAV. The optical sensor device located directly above and behind the refueling probe obtains data, and after processing, outputs the relative spatial distance measurement alignment information of the refueling refueling probe and the refueling cone sleeve in three-dimensional space. The present invention uses the structure of the refueling cone sleeve to simplify the refueling cone sleeve model into a key point model. There is no need to arrange artificial marks or active light sources on the refueling cone sleeve in advance. The inner and outer contours of the cone sleeve can be detected, and the three-dimensional alignment information of the relative position of the refueling cone sleeve and the refueling probe can be measured. It has the characteristics of high real-time performance, strong reliability and high precision, and can meet the requirements of the soft autonomous aerial refueling of UAVs.

Description

Technical Field

[0001] The invention relates to the technical field of aerial refueling, and in particular to a method for visually aligning a soft autonomous aerial refueling receiving probe of an unmanned aerial vehicle and a refueling cone sleeve. Background Art

[0002] With the development of aviation technology, drones are playing an increasingly important role in both military and civilian fields. Sufficient fuel capacity is a necessary condition for maximizing the performance of drones, and aerial refueling technology is a key technology to meet this requirement. Aerial refueling technology is divided into hard refueling technology and soft refueling technology. Soft refueling is more in line with the single, small, multi-frequency, and decentralized aerial refueling mode of drones. In the actual process of soft aerial refueling, the refueling drogue is easily affected by unstable airflow, which can cause its position and angle to change rapidly. For the task of effectively docking the refueling probe to the position of the refueling drogue, great requirements are placed on the real-time and accuracy of the relative position of the ranging refueling drogue. In recent years, relevant research teams have proposed a method of extracting the characteristics of the active light source arranged in the refueling drogue to obtain the target and achieve ranging. However, the problem is that the active light source is easily affected by harsh conditions such as strong light and rain and fog interference, which seriously affects the ranging effect at long distances under these conditions, and it is difficult to meet the environmental adaptability requirements of soft autonomous aerial refueling. In addition, some researchers have used machine learning methods to identify the characteristics of the entire refueling drogue. The model trained with a large amount of data can mitigate the impact of harsh environments compared to active light sources. However, directly relying on the extracted results for visual ranging is difficult to meet the accuracy requirements of soft autonomous aerial refueling docking, resulting in a large deviation between its center position and the refueling port at the center of the refueling drogue. In particular, during aerial refueling, there are multiple relative posture relationships between the camera and the refueling drogue. Summary of the invention

[0003] In order to overcome the shortcomings of the prior art, the present invention provides a method for visually aligning a refueling probe and a refueling cone sleeve of an autonomous aerial refueling unmanned aerial vehicle. The data is obtained by an optical sensor device located directly above and behind the refueling probe, and the data is processed by the method to output the relative spatial distance measurement alignment information of the refueling probe and the refueling cone sleeve in three-dimensional space. The present invention simplifies the refueling cone sleeve model into a key point model by utilizing the structure of the refueling cone sleeve itself. There is no need to arrange artificial markers or active light sources on the refueling cone sleeve in advance. The inner and outer contours of the cone sleeve can be detected, and the three-dimensional alignment information of the relative position of the refueling cone sleeve and the refueling probe can be measured. The method alternately uses target detection and target tracking algorithms, adds the discrimination of successful alignment and abnormal state, and has the characteristics of high real-time performance, strong reliability and high precision, and can meet the requirements of soft autonomous aerial refueling of unmanned aerial vehicles.

[0004] The technical solution adopted by the present invention to solve the technical problem includes the following steps:

[0005] Step 1: During autonomous aerial refueling, use the optical sensor located just above and behind the refueling probe to acquire data; set the camera resolution to the maximum supported resolution;

[0006] Step 2: Perform distortion correction and noise removal on the data acquired by the optical sensor device, use Zhang's camera calibration method to correct the distortion of the obtained camera internal distortion parameters, and use Gaussian blur method to reduce image noise;

[0007] Step 3: Detect and track the local area where the refueling drogue is located;

[0008] Step 4) simplifying the refueling drogue model into a key point model, and measuring the relative spatial distance between the refueling drogue and the refueling probe;

[0009] Step 5) calculating the inner and outer contour information of the refueling cone sleeve;

[0010] The coordinate set A obtained in step 4) is directly divided into two sets according to the order in which the key points appear. For the first set, the major axis, minor axis, center coordinates and rotation angle of the ellipse are obtained by ellipse fitting, and the ellipse information is used as the inner contour information; for the second set, the major axis, minor axis, center coordinates and rotation angle of the ellipse are obtained by ellipse fitting, and the ellipse information is used as the outer contour information; it is determined whether the inner contour and the outer contour intersect, if so, the relative position three-dimensional alignment information of the outer contour of the refueling cone sleeve is calculated and output; if not, the relative position three-dimensional alignment information of the inner contour is calculated and output;

[0011] Step 6) Determine the alignment success status;

[0012] Within 1 second, if the relative straight-line distance between the refueling probe and the refueling cone sleeve is equal to 0, and the refueling end of the refueling probe is located within the inner contour of the refueling cone sleeve, the alignment success status is output; otherwise, the alignment fails and the process goes to step 7);

[0013] Step 7) Determine the abnormal state;

[0014] If the relative straight-line distance between the refueling probe and the refueling cone sleeve is greater than 0 and less than or equal to a pre-set docking distance, and the refueling end of the refueling probe is outside the area where the outer contour of the refueling cone sleeve is located or the refueling cone sleeve moves a distance greater than the set threshold within 0.2 seconds, an abnormal message is output.

[0015] The specific steps of detecting and tracking the local area where the refueling drogue is located in step 3 are:

[0016] 3.1) The training set used for the pre-trained YOLO v3 object detection includes scenes of the refueling drogue in various weather conditions (e.g., sunny days, direct sunlight on the refueling drogue, cloudy days, rainy and foggy days), scenes of the refueling probe and the refueling drogue in various positions and angles, and scenes of opaque objects blocking different areas of the refueling drogue (e.g., the entire front area of ​​the refueling drogue is blocked by 10%, 30%, and 50%). The resolution of the images in the training set is the maximum resolution supported by the camera, and the training is performed until the number of iterations reaches more than 10,000;

[0017] 3.2) Use the target detection algorithm to detect the entire image range, and only retain the results within the target detection area. In the target detection result set, sort from large to small according to the detection confidence. If the target detection result set is not empty, it is considered that the target detection is successful; if the target detection result set is empty, it is considered that the target detection fails;

[0018] If the detection is successful, the result with the highest detection confidence (the size and position of the refueling drogue target in the image) is taken as the first result that meets the requirements. Let the generated area ratio be scale. Since it is the area where the target detection result is enlarged, scale is greater than 1. For the size and position of the refueling drogue target in the image, let the coordinates of the upper left vertex of the target image be (left, top), the width be w, and the height be h. The enlargement scale size = scale*(w+h) / 2. In order to obtain the enlarged target detection area, the target detection area is enlarged according to the enlargement scale size. In order to prevent the enlarged area from exceeding the four boundaries of the image, the coordinates of the upper left vertex of the enlarged area are (max(0,left+w / 2-size / 2),max(0,top+h / 2-size / 2)), the width is min(the width of the entire image, size), and the height is min(the height of the entire image, size). The enlarged area is taken as the new target detection area.

[0019] If the target detection fails, the generated area scale is added by 1. For the current target detection area, the coordinates of the upper left vertex of the current target detection area are set to (left, top), the width is w, and the height is h. The magnification scale size = scale*(w+h) / 2. The target detection area is enlarged according to the magnification scale size. To prevent the enlarged area from exceeding the four boundaries of the image, the coordinates of the upper left vertex of the enlarged area are (max(0,left+w / 2-size / 2),max(0,top+h / 2-size / 2)), the width is min(the width of the entire image, size), and the height is min(the height of the entire image, size). This result is used as the new target detection area;

[0020] 3.3) Use the Kalman filter target tracking algorithm to obtain the position and size of the refueling cone sleeve in the image; when the Kalman filter target tracking algorithm is executed continuously for n frames of images, n is the number of tracking frames to be set in advance, then jump to step 4; and when executing n frames of images, the area where the refueling cone sleeve is located in the nth frame image is enlarged as the area to be detected by the next frame target detection algorithm. The specific steps are: first, let the scale of the generated area in the nth frame image be scale, set the coordinates of the upper left vertex of the generated area to (left, top), the width to w, and the height to h, and the enlargement scale size = scale*(w+h) / 2, and enlarge the target detection area according to the enlargement scale size. In order to prevent the enlarged area from exceeding the four boundaries of the image, the coordinates of the upper left vertex of the enlarged area are (max(0,left+w / 2-size / 2),max(0,top+h / 2-size / 2)), the width is min(the width of the entire image, size), and the height is min(the height of the entire image, size), and this result is used as the new target detection area.

[0021] The specific steps of step 4) are:

[0022] 4.1) The ribs of the refueling cone have obvious sharp corners. The refueling cone model is simplified into a key point model by using its own structure. The key points contained in the model are distributed at both ends of each rib. The specific simplification steps are as follows:

[0023] Get 16 key points of the refueling drogue, each key point is distributed at the two ends of each rib, and the marked key points are located at the outer contour and inner contour of the refueling drogue, with 8 key points respectively. For the outer contour of the drogue, start from the top position and go counterclockwise, and mark 8 key points at the positions of the key points with the same angle and equal proportion. For the inner contour of the drogue, start from the top position and go counterclockwise, and mark 8 key points at the positions of the key points with the same angle and equal proportion.

[0024] The supervised descent method (SDM) is used for training and key point detection. Let x 0 is the average position of all key points in the training set, x * is the position of the marked key point, h(x) uses the SIFT feature as the local feature of the key point to extract the key point feature, d(x) represents the position of all key points in the training set, and the training parameter △x is continuously adjusted so that x 0 Tends to x * , so as to minimize the following formula:

[0025] f(x 0 +△x)=||h(d(x0 +△x))-h(d(x * ))||

[0026] f(x 0 +△x) to make a second-order Taylor expansion, and take the derivative of △x and set it to 0, let φ 0 =h(d(x 0 +△x)),φ * =h(d(x * )),get:

[0027]

[0028] Among them, H is the Hessian matrix, J is the Jacobian matrix, let get:

[0029] x k =x k-1 +R k-1 φ k-1 +b k-1

[0030] The parameters to be found can be solved by the following formula:

[0031]

[0032] Get the trained training parameters △x;

[0033] 4.2) In the new target detection area, the supervised descent method is used to predict the positions of the 16 key points of the refueling drogue in this area, and the 16 predicted key point positions are stored in the result set in the order of the key point sequence numbers;

[0034] 4.3) Step 4.2) obtains the coordinate set A of the 16 key points in the image coordinate system, and the coordinate set corresponding to the 16 key points in the world coordinate system is the coordinate set B. The iterative pose solution method is adopted to use the coordinate set A and the coordinate set B to obtain the relative six-degree-of-freedom pose relationship between the camera and the refueling cone sleeve, and then the pose relationship is subtracted from the relative spatial distance of the refueling end of the refueling probe calculated in advance, and the difference is the final relative spatial distance information.

[0035] The beneficial effect of the present invention is that the refueling drogue model is simplified into a key point model by utilizing the structure of the refueling drogue itself, and no artificial markers or active light sources are required to be arranged in advance on the refueling drogue. The inner and outer contours of the drogue can be detected, and the three-dimensional alignment information of the relative position of the refueling drogue and the refueling probe can be measured. The present invention alternately uses target detection and target tracking algorithms, adds the discrimination of alignment success and abnormal state, and has the characteristics of high real-time performance, strong reliability and high precision, and can meet the requirements of UAV soft autonomous aerial refueling. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 It is a flow chart of a specific embodiment of the present invention.

[0037] Figure 2 It is a schematic diagram of the positions of the key points of the key point model of the present invention.

[0038] Figure 3 It is a schematic diagram of the outer contour and inner contour of the refueling cone sleeve of the present invention.

[0039] Among them, 1-outer contour, 2-inner contour. DETAILED DESCRIPTION

[0040] The present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0041] The present invention is further described below in conjunction with the accompanying drawings and embodiments. It should be noted that the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0042] The present invention provides a method for visually aligning a UAV soft autonomous aerial refueling receiving probe with a refueling drogue, referring to Figure 1 The specific steps include:

[0043] Step 1: During autonomous aerial refueling, use the optical sensor located just above and behind the refueling probe to acquire data; set the camera resolution to the maximum supported resolution;

[0044] Step 2: Perform distortion correction and noise removal on the data acquired by the optical sensor device, perform distortion correction on the camera internal distortion parameters obtained by Zhang's camera calibration method, and use Gaussian blur method to reduce image noise.

[0045] Step 3: Detect and track the local area where the refueling drogue is located.

[0046] 3.1) The training set used for pre-trained YOLO v3 object detection includes scenes of the refueling drogue in various weather conditions (e.g., sunny days, direct sunlight on the refueling drogue, cloudy days, rainy and foggy days), various positions and angles between the refueling probe and the refueling drogue, and scenes with opaque objects occluding different areas of the refueling drogue (e.g., the entire front area of ​​the refueling drogue is blocked by 10%, 30%, and 50%). The resolution of the images in the training set is the maximum resolution supported by the camera, and the training is performed until the number of iterations reaches more than 10,000.

[0047] 3.2) Use the target detection algorithm to detect the entire image range, and only retain the results within the target detection area. In the target detection result set, sort it from large to small according to the detection confidence. If the target detection result set is not empty, it is considered that the target detection is successful; if the target detection result set is empty, it is considered that the target detection fails; if the detection is successful, the result with the largest detection confidence (the size and position of the refueling drogue target in the image) is taken as the first result that meets the requirements, and the generated area ratio is set to scale. Since it is the area where the target detection result is magnified, scale is greater than 1. For the size and position of the refueling drogue target in the image, the coordinates of the upper left vertex of the target image are (left, top), the width is w, and the height is h. The magnification scale size = scale*(w+h) / 2. In order to obtain the enlarged target detection area, the target detection area is enlarged according to the magnification scale size. In order to prevent the enlarged area from exceeding the four boundaries of the image, the upper left vertex of the enlarged area is enlarged. The vertex coordinates are (max(0,left+w / 2-size / 2),max(0,top+h / 2-size / 2)), the width is min(the width of the entire image, size), and the height is min(the height of the entire image, size). The enlarged area is used as the new target detection area. If the target detection fails, the regional scale scale is generated plus 1. For the current target detection area, the upper left vertex coordinates of the current target detection area are set to (left, top), the width is w, and the height is h. The enlargement scale size=scale*(w+h) / 2, and the target detection area is enlarged according to the enlargement scale size. In order to prevent the enlarged area from exceeding the four boundaries of the image, the upper left vertex coordinates of the enlarged area are (max(0,left+w / 2-size / 2),max(0,top+h / 2-size / 2)), the width is min(the width of the entire image, size), and the height is min(the height of the entire image, size). This result is used as the new target detection area.

[0048] 3.3) Use the Kalman filter target tracking algorithm to obtain the position and size of the refueling cone sleeve in the image. When the Kalman filter target tracking algorithm is executed continuously for n frames, n is the number of tracking frames to be set in advance, then jump to step 4; and when executing n frames, the area where the refueling cone sleeve is located in the nth frame image is enlarged as the area to be detected by the next frame target detection algorithm. The specific steps are as follows: first, let the scale of the generated area in the nth frame image be scale, set the coordinates of the upper left vertex of the generated area to (left, top), the width to w, and the height to h, and the enlargement scale size = scale*(w+h) / 2, and enlarge the target detection area according to the enlargement scale size. In order to prevent the enlarged area from exceeding the four boundaries of the image, the coordinates of the upper left vertex of the enlarged area are (max(0,left+w / 2-size / 2),max(0,top+h / 2-size / 2)), the width is min(the width of the entire image, size), and the height is min(the height of the entire image, size). This result is used as the new target detection area.

[0049] Step 4) Simplify the refueling cone sleeve model into a key point model, and measure the relative spatial distance between the refueling cone sleeve and the refueling probe.

[0050] 4.1) The ribs of the refueling cone have obvious sharp corners. The refueling cone model is simplified into a key point model by using its own structure. The key points contained in the model are distributed at both ends of each rib. The specific simplification steps are as follows:

[0051] Get 16 key points of the refueling cone sleeve, each key point is distributed at both ends of each rib, and the marked key points are located at the outer contour and inner contour of the refueling cone sleeve, with 8 key points respectively. For the outer contour of the cone sleeve, start from the top position and follow the counterclockwise direction, and mark the positions of the key points at the same angle and equal proportion. A total of 8 key points are marked. For the inner contour of the cone sleeve, start from the top position and follow the counterclockwise direction, and mark the positions of the key points at the same angle and equal proportion. The positions of the 16 marked key points are described as follows Figure 2 As shown;

[0052] The supervised descent method (SDM) is used for training and key point detection. Let x 0 is the average position of all key points in the training set, x * is the position of the marked key point, h(x) uses the SIFT feature as the local feature of the key point to extract the key point feature, d(x) represents the position of all key points in the training set, and the training parameter △x is continuously adjusted so that x 0 Tends to x *, so as to minimize the following formula:

[0053] f(x 0 +△x)=||h(d(x 0 +△x))-h(d(x * ))||

[0054] f(x 0 +△x) to make a second-order Taylor expansion, and take the derivative of △x and set it to 0, let φ 0 =h(d(x 0 +△x)),φ * =h(d(x * )),get:

[0055]

[0056] Among them, H is the Hessian matrix, J is the Jacobian matrix, let get:

[0057] x k =x k-1 +R k-1 φ k-1 +b k-1

[0058] The parameters to be found can be solved by the following formula:

[0059]

[0060] Get the trained training parameters △x;

[0061] 4.2) In the new target detection area, the supervised descent method is used to predict the positions of the 16 key points of the refueling drogue in this area, and the 16 predicted key point positions are stored in the result set in the order of the key point sequence numbers;

[0062] 4.3) In step 4.2), the coordinate set A of the 16 key points in the image coordinate system is obtained, and the coordinate set B corresponding to the 16 key points in the world coordinate system is obtained. The iterative pose solution method is used to obtain the relative six-degree-of-freedom pose relationship between the camera and the refueling cone sleeve using the coordinate set A and the coordinate set B. Then, the pose relationship is subtracted from the relative spatial distance of the refueling end of the refueling probe calculated in advance, and the difference is the final relative spatial distance information;

[0063] Step 5) Calculate the inner and outer contour information of the refueling cone sleeve. The inner and outer contours described are as follows: Figure 3As shown, specifically, the coordinate set A obtained in step 4) is directly divided into two sets according to the order in which the key points appear. For the first set, the major axis, minor axis, center coordinates and rotation angle of the ellipse are obtained by ellipse fitting, and the ellipse information is used as the inner contour information; for the second set, the major axis, minor axis, center coordinates and rotation angle of the ellipse are obtained by ellipse fitting, and the ellipse information is used as the outer contour information; it is determined whether the inner contour and the outer contour intersect, if so, the relative position three-dimensional alignment information of the outer contour of the refueling cone sleeve is calculated and output; if not, the relative position three-dimensional alignment information of the inner contour is calculated and output;

[0064] Step 6) Determine the alignment success status;

[0065] Within 1 second, if the relative straight-line distance between the refueling probe and the refueling cone sleeve is equal to 0, and the refueling end of the refueling probe is located within the inner contour of the refueling cone sleeve, the alignment success status is output; otherwise, the alignment fails and the process goes to step 7);

[0066] Step 7) Determine the abnormal state;

[0067] If the relative straight-line distance between the refueling probe and the refueling cone sleeve is greater than 0 and less than or equal to a pre-set docking distance, and the refueling end of the refueling probe is outside the area where the outer contour of the refueling cone sleeve is located or the refueling cone sleeve moves a distance greater than the set threshold within 0.2 seconds, an abnormal message is output.

Claims

1. A method for visually aligning a refueling probe and a refueling drogue of an unmanned aerial vehicle autonomously, characterized in that The steps include: Step 1: During autonomous aerial refueling, use the optical sensor located just above and behind the refueling probe to acquire data; set the camera resolution to the maximum supported resolution; Step 2: Perform distortion correction and noise removal on the data acquired by the optical sensor device, use Zhang's camera calibration method to correct the distortion of the obtained camera internal distortion parameters, and use Gaussian blur method to reduce image noise; Step 3: Detect and track the local area where the refueling drogue is located; Step 4) simplifying the refueling drogue model into a key point model, and measuring the relative spatial distance between the refueling drogue and the refueling probe; The specific steps of step 4) are: 4.1) The ribs of the refueling cone have obvious sharp corners. The refueling cone model is simplified into a key point model by using its own structure. The key points contained in the model are distributed at both ends of each rib. The specific simplification steps are as follows: Get 16 key points of the refueling drogue, each key point is distributed at the two ends of each rib, and the marked key points are located at the outer contour and inner contour of the refueling drogue, with 8 key points respectively. For the outer contour of the drogue, start from the top position and go counterclockwise, and mark 8 key points at the positions of the key points with the same angle and equal proportion. For the inner contour of the drogue, start from the top position and go counterclockwise, and mark 8 key points at the positions of the key points with the same angle and equal proportion. The supervised descent method is used for training and key point detection. Let x0 be the average position of all key points in the training set, and x * is the position of the marked key point, h(x) uses the SIFT feature as the local feature of the key point to extract the key point feature, d(x) represents the position of all key points in the training set, and the training parameter △x is continuously adjusted so that x0 tends to x * , so as to minimize the following formula: f(x0+△x)=||h(d(x0+△x))-h(d(x * ))|| Perform a second-order Taylor expansion on f(x0+△x), take the derivative with respect to △x and set it to 0, set φ0=h(d(x0+△x)), φ * =h(d(x * )),get: Among them, H is the Hessian matrix, J is the Jacobian matrix, let get: x k =x k-1 +R k-1 f k-1 +b k-1 The parameters to be found can be solved by the following formula: Get the trained training parameters △x; 4.2) In the new target detection area, the supervised descent method is used to predict the positions of the 16 key points of the refueling drogue in this area, and the 16 predicted key point positions are stored in the result set in the order of the key point sequence numbers; 4.3) In step 4.2), the coordinate set A of the 16 key points in the image coordinate system is obtained, and the coordinate set B corresponding to the 16 key points in the world coordinate system is obtained. The iterative pose solution method is used to obtain the relative six-degree-of-freedom pose relationship between the camera and the refueling cone sleeve using the coordinate set A and the coordinate set B. Then, the pose relationship is subtracted from the relative spatial distance of the refueling end of the refueling probe calculated in advance, and the difference is the final relative spatial distance information; Step 5) calculating the inner and outer contour information of the refueling cone sleeve; The coordinate set A obtained in step 4) is directly divided into two sets according to the order in which the key points appear. For the first set, the first ellipse information of the major axis, minor axis, center coordinates and rotation angle of the ellipse is obtained by ellipse fitting, and the first ellipse information is used as the inner contour information; for the second set, the second ellipse information of the major axis, minor axis, center coordinates and rotation angle of the ellipse is obtained by ellipse fitting, and the second ellipse information is used as the outer contour information; it is determined whether the inner contour and the outer contour intersect, if so, the relative position three-dimensional alignment information of the outer contour of the refueling cone sleeve is calculated and output; if not, the relative position three-dimensional alignment information of the inner contour is calculated and output; Step 6) Determine the alignment success status; Within 1 second, if the relative straight-line distance between the refueling probe and the refueling cone sleeve is equal to 0, and the refueling end of the refueling probe is located within the inner contour of the refueling cone sleeve, the alignment success status is output; otherwise, the alignment fails and the process goes to step 7); Step 7) Determine the abnormal state; If the relative straight-line distance between the refueling probe and the refueling cone sleeve is greater than 0 and less than or equal to a pre-set docking distance, and the refueling end of the refueling probe is outside the area where the outer contour of the refueling cone sleeve is located or the refueling cone sleeve moves a distance greater than the set threshold within 0.2 seconds, an abnormal message is output.

2. The method for visually aligning a refueling probe and a refueling drogue of a UAV autonomous aerial refueling according to claim 1 is characterized by: The specific steps of detecting and tracking the local area where the refueling drogue is located in step 3 are: 3.1) The training set used by the pre-trained YOLO v3 object detection includes scenes of the refueling drogue in various weather conditions, various positions and angles between the refueling probe and the refueling drogue, and different areas of the refueling drogue blocked by opaque objects. The resolution of the images in the training set is the maximum resolution supported by the camera, and the training is performed until the number of iterations reaches more than 10,000; 3.2) Use the target detection algorithm to detect the entire image range, and only retain the results within the target detection area. In the target detection result set, sort it from large to small according to the detection confidence. If the target detection result set is not empty, it is considered that the target detection is successful; If the target detection result set is empty, it is considered that the target detection has failed; If the detection is successful, the result with the highest detection confidence is taken as the first result that meets the requirements. Let the generated area ratio be scale. Since it is the area where the target detection result is enlarged, scale is greater than 1. For the size and position of the obtained refueling drogue target in the image, let the coordinates of the upper left vertex of the target image be (left, top), the width be w, and the height be h. The enlargement scale size = scale*(w+h) / 2. In order to obtain the enlarged target detection area, the target detection area is enlarged according to the enlargement scale size. In order to prevent the enlarged area from exceeding the four boundaries of the image, the coordinates of the upper left vertex of the enlarged area are (max(0,left+w / 2-size / 2),max(0,top+h / 2-size / 2)), the width is min(the width of the entire image, size), and the height is min(the height of the entire image, size). The enlarged area is taken as the new target detection area. If the target detection fails, the generated area scale is added by 1. For the current target detection area, the coordinates of the upper left vertex of the current target detection area are set to (left, top), the width is w, and the height is h. The magnification scale size = scale*(w+h) / 2. The target detection area is enlarged according to the magnification scale size. To prevent the enlarged area from exceeding the four boundaries of the image, the coordinates of the upper left vertex of the enlarged area are (max(0,left+w / 2-size / 2),max(0,top+h / 2-size / 2)), the width is min(the width of the entire image, size), and the height is min(the height of the entire image, size). This result is used as the new target detection area; 3.3) Use the Kalman filter target tracking algorithm to obtain the position and size of the refueling cone sleeve in the image; when the Kalman filter target tracking algorithm is executed continuously for n frames of images, n is the number of tracking frames to be set in advance, then jump to step 4; and when executing n frames of images, the area where the refueling cone sleeve is located in the nth frame image is enlarged as the area to be detected by the next frame target detection algorithm. The specific steps are: first, let the scale of the generated area in the nth frame image be scale, set the coordinates of the upper left vertex of the generated area to (left, top), the width to w, and the height to h, and the enlargement scale size = scale*(w+h) / 2, and enlarge the target detection area according to the enlargement scale size. In order to prevent the enlarged area from exceeding the four boundaries of the image, the coordinates of the upper left vertex of the enlarged area are (max(0,left+w / 2-size / 2),max(0,top+h / 2-size / 2)), the width is min(the width of the entire image, size), and the height is min(the height of the entire image, size), and this result is used as the new target detection area.

Images