A method and system for positioning an aircraft refueling pipe refueling port based on monocular SLAM

Abstract

The present application relates to the technical field of positioning the fueling port of an aircraft fueling pipe, and discloses a method and system for positioning the fueling port of an aircraft fueling pipe based on monocular SLAM. The method utilizes a monocular camera installed on the fueling pipe, combined with SLAM technology for real-time environmental perception and positioning, to achieve rapid and accurate identification and positioning of the fueling port, thereby improving the efficiency and safety of aerial refueling. The present application aims to solve the problem of accurate alignment of the fueling pipe and the fuel receiving port during aerial refueling.

Description

Technical Field

[0001] The present application relates to the technical field of positioning an aircraft refueling pipe refueling port, and in particular to a monocular SLAM-based positioning method and system for an aircraft refueling pipe refueling port. Background Art

[0002] With the development of aviation technology, autonomous aerial refueling has become a trend. During autonomous aerial refueling, precise relative posture measurement and control are required between the receiving aircraft and the refueling aircraft to ensure that the refueling drogue is accurately docked with the refueling port of the receiving aircraft. This places extremely high demands on the accuracy, real-timeness and reliability of the positioning device and method.

[0003] Traditional aerial refueling methods mainly rely on manual operation and simple optical marker assistance, which is difficult to meet the high-precision and high-efficiency requirements of autonomous aerial refueling. Moreover, these methods are easily affected by environmental factors such as light and weather, resulting in reduced positioning accuracy.

[0004] In addition, some existing vision-based positioning methods, such as the visnav visual navigation system that uses multiple LED cursors for pose calculation, have improved positioning accuracy to a certain extent, but still have problems such as reliance on external markers. Especially in complex environments, such as during refueling, the circular features of the cone sleeve of the refueling port change due to the change in the opening and closing state of the cone sleeve, which limits the application of the visnav visual navigation system. Summary of the invention

[0005] In order to improve the accuracy of existing refueling port positioning methods, the present application provides an aircraft refueling pipe refueling port positioning method and system based on monocular SLAM.

[0006] In the first aspect, the present application provides a method for positioning an aircraft refueling pipe refueling port based on monocular SLAM, which adopts the following technical solution:

[0007] A method for positioning an aircraft refueling pipe refueling port based on monocular SLAM comprises the following steps:

[0008] Based on the monocular camera installed on the refueling pipe, the surrounding environment is photographed to obtain an environmental picture;

[0009] Extracting environmental feature points according to the environmental image;

[0010] Matching the environmental feature points with feature points in a preset reference image, and outputting feature matching results;

[0011] Constructing a three-dimensional map of the environment through a SLAM algorithm according to the feature matching results, calculating movement information of the environmental feature points in the environmental image, and updating the three-dimensional map of the environment according to the movement information;

[0012] photographing the fuel inlet to obtain a fuel inlet image, and identifying fuel inlet feature points of the fuel inlet from the fuel inlet image;

[0013] Based on the relative position relationship between the fuel filler port feature point and the environment feature point, calculating the position data of the fuel filler port feature point in the three-dimensional map of the environment as the fuel filler port position data;

[0014] Based on the relative distance data and relative position relationship between the fuel filler port feature point and the environmental feature point, the posture data of the fuel filler pipe is calculated as the fuel filler pipe posture data;

[0015] Calculate first location data based on a first positioning tag installed on the refueling pipe and a positioning base station installed next to the refueling port, and calculate second location data based on a second positioning tag installed on the refueling pipe and the positioning base station;

[0016] Calculate position reference data using a first algorithm according to the first position data and the second position data, and use the position reference data to correct the fuel filler port position data;

[0017] Calculate attitude reference data using a second algorithm according to the first position data and the second position data, and correct the attitude data of the refueling pipe using the attitude reference data;

[0018] calculating position deviation data according to preset fuel filler port standard position data and the fuel filler port position data, and adjusting the position of the fuel filler pipe based on the position deviation data;

[0019] The posture deviation data is calculated according to the preset standard posture data of the fuel filler pipe and the posture data of the fuel filler pipe, and the posture of the fuel filler pipe is adjusted based on the posture deviation data.

[0020] By adopting the above technical solutions, the monocular SLAM algorithm can reflect the changes in the surrounding environment in real time by extracting environmental feature points and constructing a three-dimensional map. It not only improves the ability to perceive the environment, but also ensures the accuracy and real-time nature of the map, providing a solid foundation for the subsequent positioning of the refueling port. By taking pictures of the refueling port with a high-definition camera and identifying the feature points of the refueling port, the position of the refueling port in the three-dimensional map can be accurately calculated. The positioning error caused by environmental factors in the traditional method is avoided, and the positioning accuracy is improved. Based on the relative distance data between the feature points of the refueling port and the feature points of the environment, the posture data of the refueling pipe can be accurately calculated. Not only the position of the refueling port, but also the posture of the refueling pipe is considered, thereby realizing the accurate measurement of the posture of the refueling pipe. By updating the three-dimensional map of the environment and the position data of the refueling port in real time, real-time monitoring and adjustment of the refueling process can be achieved. This not only improves the efficiency of the refueling process, but also ensures the safety and stability of the refueling process. Autonomous aerial refueling technology reduces manual intervention and reduces the difficulty and cost of operation. At the same time, since the system can automatically measure and adjust the posture, it can greatly improve the automation of the refueling process. This technical solution does not rely on external markers, reducing the impact of environmental factors (such as light and weather) on positioning accuracy. The system can operate stably in various complex environments and improve the environmental adaptability of the system. The accuracy of positioning data can be further improved by using multiple positioning tags and positioning base stations for auxiliary positioning and correction. It not only improves the reliability of the system, but also increases the fault tolerance of the system. Since the system can measure and adjust the posture in real time, it can promptly detect and correct deviations in the refueling process. This not only ensures the smooth progress of the refueling process, but also effectively avoids safety accidents caused by positioning errors. Therefore, the accuracy, efficiency and reliability of autonomous aerial refueling in this application will be significantly improved.

[0021] Optionally, the method further comprises the following steps:

[0022] The coefficient of position correction is adjusted according to the anti-correlation of the position deviation data; the larger the position deviation data, the smaller the coefficient of position correction; the smaller the position deviation data, the larger the coefficient of position correction; wherein the position correction adopts UWB positioning technology.

[0023] By adopting the above technical solution, although UWB positioning technology has the advantages of high precision and high anti-interference, its positioning error may increase in long distance situations; therefore, through the anti-correlation adjustment mechanism, the UWB positioning error caused by the increase in distance can be reduced, thereby improving the accuracy of the refueling process.

[0024] Optionally, the method further comprises the following steps:

[0025] Calculate shaking data representing shaking degree according to the fuel filler port posture data;

[0026] According to a preset shaking threshold, calculating the difference between the shaking data and the shaking threshold as a shaking difference;

[0027] The coefficient of the position correction is adjusted according to the positive correlation of the shake difference. The higher the shake difference is, the larger the coefficient of the position correction is; and the lower the shake difference is, the smaller the coefficient of the position correction is.

[0028] By adopting the above technical solution, when the sway difference is high, that is, when the degree of sway of the fuel filler port is large, the value of the position correction coefficient is increased to strengthen the dynamic adjustment and compensation of the fuel filler pipe position; when the sway difference is low, that is, when the degree of sway of the fuel filler port is small, the value of the position correction coefficient is reduced to avoid instability caused by excessive adjustment. According to the positive correlation between the sway difference and the degree of sway represented by the fuel filler port attitude data, the stability and docking accuracy of the refueling process can be significantly improved. Using sensors installed at or near the fuel filler port (such as gyroscopes, accelerometers, etc.), the attitude data of the fuel filler port, including angles, angular velocities, etc., are obtained in real time; based on the real-time acquired fuel filler port attitude data, the sway data of the fuel filler port can be calculated by analyzing the fluctuation range, frequency and other characteristics of the attitude data.

[0029] Optionally, the method further comprises the following steps:

[0030] Acquiring inertial attitude data based on an inertial attitude sensor arranged on a camera;

[0031] The posture is calculated by weighted averaging the inertial posture data, the posture reference data and the fuel pipe posture data, and the fuel pipe posture data is updated.

[0032] By adopting the above technical solution, the posture data calculated by the weighted average method will be used to update the posture data of the refueling pipe, so as to more accurately reflect the current posture of the refueling pipe. The updated posture data of the refueling pipe is applied to the control system of the refueling pipe, and the posture and position of the refueling pipe are adjusted to ensure that the refueling drogue can be accurately connected to the refueling port of the receiving aircraft.

[0033] Optionally, the method further comprises the following steps:

[0034] According to the positive correlation of the relative distance data, the weighted calculation coefficient of the inertial posture data is adjusted, the larger the relative distance data is, the larger the weighted calculation coefficient of the inertial posture data is; the smaller the relative distance data is, the smaller the weighted calculation coefficient of the inertial posture data is;

[0035] Get the flight speed of the tanker;

[0036] The weighted calculation coefficient of the refueling pipe attitude data is adjusted according to the positive correlation with the flight speed; the greater the flight speed, the greater the weighted calculation coefficient of the refueling pipe attitude data; the smaller the flight speed, the smaller the weighted calculation coefficient of the refueling pipe attitude data.

[0037] By adopting the above technical solution, the greater the relative distance, the more unstable the environment may be. Therefore, the weighting coefficient of the inertial attitude data will increase accordingly to ensure the accuracy of the attitude calculation. On the contrary, when the relative distance is small, the environment may be relatively stable, and the weight of the inertial attitude data can be appropriately reduced. The attitude data of the refueling hose is crucial to the aerial refueling process because it directly affects the docking accuracy and stability between the refueling hose and the receiving aircraft. Changes in flight speed will affect the attitude and stability of the refueling hose. The greater the flight speed, the more violent the dynamic response of the refueling hose may be. Therefore, the weighting coefficient of the attitude data of the refueling hose will increase accordingly to improve the sensitivity and accuracy of the attitude calculation. On the contrary, when the flight speed is low, the dynamic response of the refueling hose is relatively stable, and the weight of the attitude data of the refueling hose can be appropriately reduced.

[0038] Optionally, the method further comprises the following steps:

[0039] Extracting cone sleeve features from the environment image;

[0040] In the continuous frames at a set number of intervals, the key points of the cone sleeve feature are matched;

[0041] Calculating the overall scaling change of the cone sleeve feature according to the distance change between the key points;

[0042] The telescopic movement speed of the fuel filling pipe is adjusted inversely according to the overall scaling change amplitude. The larger the overall scaling change amplitude, the slower the telescopic movement speed of the fuel filling pipe; the smaller the overall scaling change amplitude, the faster the telescopic movement speed of the fuel filling pipe.

[0043] By adopting the above technical solution, when the overall scaling change range is large, it means that the motion state of the cone sleeve is unstable or subject to large external interference. At this time, the telescopic movement speed of the fuel pipe should be reduced to avoid docking failure or safety hazards caused by too fast adjustment. On the contrary, when the overall scaling change range is small, it means that the motion state of the cone sleeve is relatively stable. At this time, the telescopic movement speed of the fuel pipe can be increased to speed up the docking process and improve the docking accuracy.

[0044] Optionally, the method further comprises the following steps:

[0045] Extracting cone sleeve features from the environment image;

[0046] In the continuous frames at a set number of intervals, local features of the cone sleeve feature are matched;

[0047] Calculating the distortion change amplitude of the local feature;

[0048] The posture adjustment speed of the fuel filler port is adjusted inversely according to the distortion change amplitude. The larger the distortion change amplitude, the slower the posture adjustment speed of the fuel filler port; the smaller the distortion change amplitude, the faster the posture adjustment speed of the fuel filler port.

[0049] By adopting the above technical solution, the local distortion change amplitude of the cone sleeve characteristic reflects the dynamic stability of the cone sleeve in the air and the degree of interference by external forces. When the cone sleeve is disturbed by a large external force or its own stability is poor, its local distortion change amplitude will increase; otherwise, it will decrease. Therefore, the motion state and docking condition of the cone sleeve can be indirectly judged by monitoring this change amplitude.

[0050] In the second aspect, the present application provides an aircraft refueling pipe refueling port positioning system based on monocular SLAM, which adopts the following technical solutions:

[0051] An aircraft refueling pipe refueling port positioning system based on monocular SLAM, based on the above-mentioned aircraft refueling pipe refueling port positioning method based on monocular SLAM, includes a monocular camera, an image processing module, a feature extraction and matching module, a monocular SLAM construction module, a refueling pipe refueling port positioning module, an error correction and feedback module, and a control and execution module;

[0052] The monocular camera is connected to the image processing module and is used to collect image information of the refueling pipe and the surrounding environment;

[0053] The image processing module is used to receive and process image information, and output the processed image to the feature extraction and matching module;

[0054] The feature extraction and matching module is used to extract feature points from the image processed by the image processing module, and match the feature points in the current image with the feature points in the preset reference image to determine the position and posture of the camera in space; and transmit the matching results to the monocular SLAM construction module;

[0055] The monocular SLAM construction module is used to construct a three-dimensional map of the environment based on the feature matching result using a monocular SLAM algorithm, and simultaneously calculate the motion trajectory of the camera; transmit the constructed three-dimensional map of the environment and the motion trajectory information of the camera to the fuel pipe and fuel port positioning module;

[0056] The refueling pipe refueling port positioning module calculates the refueling pipe refueling port position data by using the constructed environment three-dimensional map and the position and posture of the camera in combination with the geometric features of the refueling pipe, and transmits the refueling pipe refueling port position data to the error correction and feedback module and the control and execution module;

[0057] The error correction and feedback module calculates the first position data based on the first positioning tag installed on the fuel pipe and the positioning base station installed next to the fuel receiving port, and calculates the second position data based on the second positioning tag installed on the fuel pipe and the positioning base station; calculates the position reference data using the first position data and the first algorithm according to the first position data and the second position data, and corrects the fuel filling port position data using the position reference data; calculates the attitude reference data using the second algorithm according to the first position data and the second position data, and corrects the fuel filling pipe attitude data using the attitude reference data; and transmits the corrected information to the control and execution module;

[0058] The control and execution module is used to adjust the position of the fuel filler pipe according to the preset fuel filler port standard position data and the position deviation data calculated from the fuel filler port position data; and to adjust the posture of the fuel filler pipe according to the preset fuel filler pipe standard posture data and the posture data of the fuel filler pipe calculated.

[0059] In summary, the present application includes at least one of the following beneficial technical effects:

[0060] Improve positioning accuracy: Use SLAM technology for environmental perception and map construction, combined with image processing and computer vision technology to achieve precise positioning of the refueling port, improving the accuracy and safety of aerial refueling.

[0061] Reduce operational difficulty: Through autonomous alignment technology, the pilot's operating burden is reduced, and the operational difficulty and risk of aerial refueling are reduced.

[0062] Enhanced adaptability: The method of the present invention is not affected by ambient lighting, weather and other conditions and has strong adaptability and robustness.

[0063] Improved efficiency: The automated alignment process reduces manual intervention and improves the efficiency of aerial refueling. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 This is a step-by-step diagram of the aircraft refueling pipe refueling port positioning method based on monocular SLAM.

[0065] Figure 2 This is the structural diagram of the aircraft refueling pipe and refueling port positioning system based on monocular SLAM.

[0066] Figure 3 It is a method of adjusting the coefficient of position correction based on the degree of shaking.

[0067] Figure 4 The method is to adjust the retracting and extending speed of the fuel filling pipe based on the overall scaling range of the cone sleeve.

[0068] Figure 5 The invention is a method for adjusting the speed of the attitude adjustment of the fuel filling pipe based on the twisting variation of the cone sleeve. DETAILED DESCRIPTION

[0069] Embodiments of the present application are described in detail below, examples of which are shown in the accompanying drawings.

[0070] In the description of this specification, the description with reference to the terms "certain embodiments", "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples" or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiments or examples are included in at least one embodiment or example of the present application. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0071] The present application embodiment discloses a method for locating an aircraft refueling pipe refueling port based on monocular SLAM, referring to Figure 1 and Figure 2 , including the following steps:

[0072] Based on the monocular camera installed on the fuel pipe, the surrounding environment is photographed to obtain the environment picture, and the fuel port is photographed to obtain the fuel port picture. The monocular camera is installed at the end of the fuel pipe to ensure that the camera can clearly capture the fuel port and its surroundings. When used for the first time, the camera is initialized, including calibrating the camera parameters and setting the image resolution. The captured environment picture and fuel port picture need to be preprocessed, including denoising, contrast enhancement, edge detection and other operations to improve the image quality.

[0073] The monocular camera can be rotated or adjusted in posture by a driving mechanism. In addition, the refueling pipe can also be retracted or adjusted in posture, and the driving mechanism is such as a pan / tilt or one or more steering gears.

[0074] Extract environmental feature points from the environmental image, match them with the feature points in the preset reference image, and output the feature matching results. Use image processing and computer vision technology to identify environmental feature points, such as corners, edges, etc. Then, through the feature matching algorithm, match the feature points in the current image with the feature points in the existing map or reference image to determine the position and posture of the monocular camera in space.

[0075] Based on the feature matching results, the SLAM algorithm is used to build a three-dimensional map of the environment, calculate the movement information of the environmental feature points in the environmental image, that is, the movement trajectory of the monocular camera, and then update the three-dimensional map of the environment based on the movement information. During the refueling process, the camera takes real-time images of the refueling port and its surroundings. Using SLAM technology, the captured images are processed, feature points are extracted, and a three-dimensional map of the environment is built. At the same time, based on the movement information of the camera, the map is updated in real time to achieve accurate tracking of the position of the refueling pipe.

[0076] The fuel filler port feature points of the fuel filler port are identified from the fuel filler port image; based on the relative position relationship between the fuel filler port feature points and the environmental feature points, the position data of the fuel filler port feature points in the environmental three-dimensional map are calculated as the fuel filler port position data.

[0077] In the constructed environment map, image processing and computer vision technology are used to identify the features of the fuel filler port. The location of the fuel filler port and the posture of the fuel pipe are determined by comparing the fuel filler port features with the preset template. A template image of the fuel filler port is preset, and the template image contains the feature information of the fuel filler port. In the constructed three-dimensional map, image processing and computer vision technology are used to match the real-time acquired image with the template image to identify the location of the fuel filler port. Position determination: Once the fuel filler port is identified, the spatial position information provided by the SLAM algorithm can be used to accurately determine the position of the fuel filler port in the three-dimensional map.

[0078] Based on the relative distance data and relative position relationship between the feature points of the refueling port and the feature points of the environment, the posture data of the refueling pipe is calculated as the posture data of the refueling pipe. By using the constructed environment map and the posture information of the monocular camera, combined with the geometric features of the refueling pipe, such as the center and radius of the circular features, the position of the refueling port of the refueling pipe is accurately calculated. Some optimization algorithms, such as the least squares method, can be used to further improve the positioning accuracy. Therefore, based on the real-time image and position information of the monocular camera, the relative position and posture between the end of the refueling pipe and the refueling port can be calculated.

[0079] The first location data is calculated based on the first positioning tag installed on the fuel pipe and the positioning base station installed next to the fuel receiving port, and the second location data is calculated based on the second positioning tag installed on the fuel pipe and the positioning base station next to the fuel receiving port. Among them, the first positioning tag and the second positioning tag installed on the fuel pipe are both UWB positioning tags; the positioning base station installed next to the fuel receiving port and the positioning base station installed on the fuel pipe are both UWB positioning base stations. The UWB positioning tag can transmit UWB signals and communicate with the UWB positioning base station.

[0080] There are multiple positioning base stations installed beside the oil receiving port, such as four positioning base stations, which are symmetrically distributed beside the oil receiving port to form a square or rectangular shape, and the centers of the four positioning base stations coincide with the center of the oil receiving port.

[0081] UWB technology, UWB (Ultra-Wideband) technology is a wireless carrier communication technology. UWB technology uses a frequency bandwidth of more than 1GHz and uses nanosecond non-sinusoidal narrow pulses to transmit data, so it occupies a large spectrum range. It does not use a sinusoidal carrier, but directly modulates an impulse pulse with a very steep rise and fall time.

[0082] The first algorithm is used to calculate the position reference data according to the first position data and the second position data, and the position reference data is used to correct the fuel port position data. The first algorithm is a weighted average algorithm, which improves the accuracy and robustness of positioning through sensor fusion. The monocular camera IMU inertial sensor is fused. Through multi-sensor fusion technology, richer environmental information can be obtained, and the accuracy and stability of the SLAM algorithm can be improved.

[0083] The attitude reference data is calculated using the second algorithm according to the first position data and the second position data, and the attitude reference data is used to correct the attitude data of the refueling pipe. The second algorithm is a weighted average algorithm, which improves the accuracy and robustness of positioning through sensor fusion.

[0084] The position deviation data is calculated based on the preset standard position data of the fuel filler port and the position data of the fuel filler port, and the position of the fuel filler pipe is adjusted based on the position deviation data. The way to adjust the position of the fuel filler pipe is to directly extend or shorten the fuel filler pipe; or to adjust the position of the fuel dispenser.

[0085] The attitude deviation data is calculated based on the preset standard attitude data of the fueling hose and the attitude data of the fueling hose, and the attitude of the fueling hose is adjusted based on the attitude deviation data. By introducing information from other sensors, such as using UWB technology or using sensors such as GNSS, the positioning results of the monocular SLAM can be corrected. At the same time, according to the actual refueling docking situation, the positioning parameters are continuously fed back and adjusted to adapt to different refueling scenarios and conditions. Adjust the attitude of the fueling hose, such as realizing the attitude rotation of the fueling hose through a gimbal or one or more servos.

[0086] The monocular SLAM algorithm can reflect the changes in the surrounding environment in real time by extracting environmental feature points and constructing a three-dimensional map. It not only improves the ability to perceive the environment, but also ensures the accuracy and real-time nature of the map, providing a solid foundation for the subsequent positioning of the refueling port. By taking pictures of the refueling port with a high-definition camera and identifying the feature points of the refueling port, the position of the refueling port in the three-dimensional map can be accurately calculated. The positioning error caused by environmental factors in the traditional method is avoided, and the positioning accuracy is improved. Based on the relative distance data between the feature points of the refueling port and the environmental feature points, the posture data of the refueling pipe can be accurately calculated. Not only the position of the refueling port, but also the posture of the refueling pipe is considered, thereby realizing the accurate measurement of the posture of the refueling pipe. By updating the three-dimensional map of the environment and the position data of the refueling port in real time, real-time monitoring and adjustment of the refueling process can be achieved. This not only improves the efficiency of the refueling process, but also ensures the safety and stability of the refueling process. Autonomous aerial refueling technology reduces manual intervention and reduces the difficulty and cost of operation. At the same time, since the system can automatically measure and adjust the posture, it can greatly improve the automation of the refueling process. This technical solution does not rely on external markers, reducing the impact of environmental factors (such as light and weather) on positioning accuracy. The system can operate stably in various complex environments and improve the environmental adaptability of the system. The accuracy of positioning data can be further improved by using multiple positioning tags and positioning base stations for auxiliary positioning and correction. It not only improves the reliability of the system, but also increases the fault tolerance of the system. Since the system can measure and adjust the posture in real time, it can promptly detect and correct deviations in the refueling process. This not only ensures the smooth progress of the refueling process, but also effectively avoids safety accidents caused by positioning errors. Therefore, the accuracy, efficiency and reliability of autonomous aerial refueling in this application will be significantly improved.

[0087] Among them, monocular SLAM (Simultaneous Localization and Mapping) technology has the advantages of simple structure, easy calibration, convenient maintenance, and large effective field of view. It obtains environmental image information through a single camera, and uses image processing and computer vision algorithms to estimate its own position and reconstruct the surrounding environment in three dimensions.

[0088] In the field of aircraft refueling pipe refueling port positioning, monocular SLAM technology can make full use of the structural features of the refueling pipe itself (such as circular features), combined with advanced image processing and posture solution algorithms, to achieve accurate positioning of the refueling port. This technology can not only overcome the limitations of traditional methods, but also improve the accuracy and real-time performance of positioning, providing strong support for the development of autonomous aerial refueling technology.

[0089] The background technology of the aircraft refueling pipe refueling port positioning device and method based on monocular SLAM technology is to solve the limitations of traditional positioning methods in autonomous aerial refueling scenarios, give full play to the advantages of monocular SLAM technology, and achieve accurate and real-time positioning of the refueling port to meet the needs of the development of modern aviation technology.

[0090] The method further comprises the steps of:

[0091] Set an initial position correction coefficient; adjust the position correction coefficient according to the anti-correlation of the position deviation data; the larger the position deviation data, the farther the current position is from the standard position, and the smaller the position correction coefficient is, so as to avoid system instability or overshoot caused by excessive correction. It means that the current position is close to the standard position, and the position correction coefficient should be increased to speed up the convergence speed. Among them, the position correction adopts UWB positioning technology. The adjusted position correction coefficient is applied to the position data provided by the UWB positioning technology to obtain the corrected refueling port position data. According to the corrected refueling port position data, adjust the position of the refueling pipe to ensure that the end of the refueling pipe is accurately connected with the refueling port.

[0092] Although UWB positioning technology has the advantages of high precision and high anti-interference, its positioning error may increase over long distances; therefore, the anti-correlation adjustment mechanism can reduce the UWB positioning error caused by the increase in distance and improve the accuracy of the refueling process. Dynamically adjusting the position correction coefficient can avoid system instability or overshoot caused by excessive correction and enhance system stability.

[0093] Reference Figure 3 , the method further comprises the following steps:

[0094] The shaking data representing the shaking degree is calculated based on the posture data of the fuel filler port. The posture data of the fuel filler port, including angle, angular velocity and other data, is acquired in real time by using sensors installed at or near the fuel filler port, such as a gyroscope and an accelerometer. The shaking data of the fuel filler port is calculated by analyzing the fluctuation range, frequency and other characteristics of the posture data. This can be achieved by filtering, derivation, integration and other processing of the posture data.

[0095] A shaking threshold is preset, and the threshold can be adjusted according to the stability of the refueling process and the requirements of docking accuracy. According to the preset shaking threshold, the difference between the shaking data and the shaking threshold is calculated as the shaking difference.

[0096] The coefficient of position correction is adjusted according to the positive correlation of the sway difference. The higher the sway difference, the greater the sway of the fuel filler port. At this time, the position correction coefficient should be increased to strengthen the dynamic adjustment and compensation of the fuel filler pipe position; the lower the sway difference, the smaller the sway of the fuel filler port. At this time, the position correction coefficient should be reduced to avoid instability caused by over-adjustment. The coefficient of position correction is adjusted according to the positive correlation of the sway difference of the sway degree represented by the fuel filler port posture data, which can significantly improve the stability and docking accuracy of the refueling process.

[0097] The adjusted position correction coefficient is applied to the position data provided by the UWB positioning technology (or other positioning technology) to obtain the corrected fuel port position data. According to the corrected fuel port position data, the position of the fuel pipe is adjusted to ensure that the end of the fuel pipe is accurately connected to the fuel port. By introducing a dynamic adjustment mechanism for the shaking data, the impact of the fuel port shaking on the refueling process can be more effectively dealt with, and the stability of the system can be improved. By adjusting the position correction coefficient with positive correlation, a more precise control of the fuel pipe position can be achieved, thereby improving the docking accuracy between the fuel port and the end of the fuel pipe. It can be adjusted and optimized according to different refueling environments and conditions to meet different refueling needs.

[0098] The method further comprises the steps of:

[0099] The inertial attitude data, including angular velocity, acceleration, etc., are obtained based on the IMU inertial attitude sensor set on the camera.

[0100] The attitude is calculated and updated by weighted average based on the inertial attitude data, attitude reference data and fuel pipe attitude data. The attitude reference data can come from other stable sensors or preset reference attitudes; the fuel pipe attitude data comes from the sensor or control system on the fuel pipe. The current comprehensive attitude is calculated by weighted average based on the inertial attitude data, attitude reference data and fuel pipe attitude data. The weight of the weighted average can be adjusted according to factors such as data reliability and accuracy.

[0101] By fusing the monocular camera and IMU inertial sensor, through multi-sensor fusion technology, more abundant environmental information can be obtained, the accuracy and stability of the SLAM algorithm can be improved, and the accuracy and robustness of positioning can be improved. The posture data calculated by the weighted average method will be used to update the posture data of the refueling pipe, so as to more accurately reflect the current posture of the refueling pipe. The updated posture data of the refueling pipe is applied to the control system of the refueling pipe, and the posture and position of the refueling pipe are adjusted to ensure that the refueling cone sleeve can be accurately connected to the refueling port of the receiving aircraft.

[0102] In other implementations, the IMU inertial attitude sensor can also be replaced by a laser radar, or the laser radar and the IMU inertial attitude sensor can be used in combination. Multi-sensor fusion technology can obtain richer environmental information and improve the accuracy and stability of the SLAM algorithm.

[0103] The method further comprises the steps of:

[0104] The weighted calculation coefficient of the inertial attitude data is adjusted according to the positive correlation of the relative distance data. The larger the relative distance data, the larger the weighted calculation coefficient of the inertial attitude data; the smaller the relative distance data, the smaller the weighted calculation coefficient of the inertial attitude data. In the three-dimensional map of the environment, the relative distance data can be calculated based on the position of the coordinates. The larger the relative distance, the more unstable the environment is. Therefore, the weighted coefficient of the inertial attitude data will increase accordingly to ensure the accuracy of the attitude calculation. On the contrary, when the relative distance is small, the environment is relatively stable, and the weight of the inertial attitude data can be appropriately reduced. The attitude data of the refueling hose is crucial to the aerial refueling process because it directly affects the docking accuracy and stability between the refueling hose and the receiving aircraft.

[0105] Changes in flight speed will affect the attitude and stability of the refueling hose; obtain the flight speed of the tanker, and adjust the weighted calculation coefficient of the refueling hose attitude data according to the positive correlation with the flight speed. The higher the flight speed, the more violent the dynamic response of the refueling hose may be. Therefore, the weighted coefficient of the refueling hose attitude data will increase accordingly to improve the sensitivity and accuracy of the attitude calculation. On the contrary, when the flight speed is low, the dynamic response of the refueling hose is relatively stable, and the weight of the refueling hose attitude data can be appropriately reduced. Among them, the flight speed of the refueling aircraft is obtained in real time using a pitot tube, GPS or other flight parameter sensors.

[0106] Reference Figure 4 , the method further comprises the following steps:

[0107] Extract the features of the cone sleeve from the environment picture: Use image processing algorithms, such as edge detection, feature extraction, etc., to extract the features of the cone sleeve from the environment picture captured by the camera. These features can be the contour, color, texture, etc. of the cone sleeve.

[0108] In the continuous frames with a set number of intervals, the key points of the cone sleeve feature are matched; in the continuous frames with a set number of intervals, the key points of the cone sleeve feature are matched using feature matching algorithms such as SIFT, SURF, ORB, etc. These key points can be significant points, corner points or edge points in the cone sleeve feature.

[0109] According to the distance change between key points, the overall scale change of the cone sleeve feature is calculated; according to the distance change between the matched key points, the overall scale change of the cone sleeve feature in consecutive frames is calculated. This can be achieved by calculating the mean or standard deviation of the distance between key points.

[0110] The telescopic movement speed of the fuel pipe is adjusted in anti-correlation according to the overall scaling change amplitude. When the overall scaling change amplitude is large, it means that the movement state of the cone sleeve is unstable or is subject to large external interference. At this time, the telescopic movement speed of the fuel pipe should be reduced to avoid docking failure or safety hazards caused by too fast adjustment. When the overall scaling change amplitude is small, it means that the movement state of the cone sleeve is relatively stable. At this time, the telescopic movement speed of the fuel pipe can be increased to speed up the docking process and improve the docking accuracy.

[0111] Combined with the attitude data of the refueling pipe obtained by multi-sensor fusion technology and the position information of the drogue obtained by image processing, the attitude and position of the refueling pipe are adjusted to ensure that the refueling drogue can be accurately docked to the refueling port of the receiving aircraft. By dynamically adjusting the telescopic movement speed of the refueling pipe, the docking strategy can be adjusted in real time according to the motion state of the drogue, thereby improving the stability of the docking. Increasing the telescopic movement speed of the refueling pipe when the motion state of the drogue is stable can speed up the docking process. At the same time, combined with multi-sensor fusion technology, the accuracy of docking can be further improved. This method combines image processing and dynamic control strategies, can adapt to docking requirements under different environments and flight conditions, and enhances the robustness of the system.

[0112] Reference Figure 5 , the method further comprises the following steps:

[0113] Extract the features of the cone sleeve from the environmental image; use image processing algorithms (such as edge detection, feature extraction, etc.) to extract the overall and local features of the cone sleeve from the environmental image captured by the camera. These features can be the contour, color, texture, and specific shape or pattern of the cone sleeve.

[0114] In the continuous frames with a set number of intervals, the local features of the cone sleeve features are matched; in the continuous frames with a set number of intervals, the local features of the cone sleeve features are matched using a feature matching algorithm (such as SIFT, SURF, ORB, etc. or their improved versions). These local features can be specific points, lines or areas on the cone sleeve, and they can maintain a relatively stable correspondence in different frames.

[0115] Calculate the distortion change amplitude of local features; Based on the matched local features, calculate their distortion change amplitude in consecutive frames. This can be achieved by comparing the relative position, angle or shape differences between feature points. The distortion change amplitude reflects the dynamic stability of the drogue in the air and the degree of interference by external forces.

[0116] The attitude adjustment speed of the refueling port is adjusted inversely according to the distortion change amplitude. When the distortion change amplitude is large, it means that the cone sleeve is subject to large external force interference or its own stability is poor. At this time, the attitude adjustment speed of the refueling port should be reduced to avoid docking failure or safety hazards caused by too fast adjustment. When the distortion change amplitude is small, it means that the movement state of the cone sleeve is relatively stable. At this time, the attitude adjustment speed of the refueling port can be increased to speed up the docking process and improve the docking accuracy.

[0117] Combined with the attitude data of the refueling pipe obtained by multi-sensor fusion technology, the position information of the drogue obtained by image processing, and the distortion change amplitude of the local features, the attitude and position of the refueling pipe are comprehensively adjusted to ensure that the refueling drogue can be accurately docked to the refueling port of the receiving aircraft. By dynamically adjusting the attitude adjustment speed of the refueling port, the docking strategy can be adjusted in real time according to the local distortion change amplitude of the drogue, thereby improving the stability of the docking. Increasing the attitude adjustment speed of the refueling port when the motion state of the drogue is stable can speed up the docking process. At the same time, combining multi-sensor fusion technology and image processing technology can further improve the accuracy of docking. This method combines image processing, feature matching and dynamic control strategies, can adapt to docking requirements under different environments and flight conditions, and enhances the robustness of the system.

[0118] The implementation principle of the present application embodiment of a method for locating the refueling port of an aircraft refueling pipe based on monocular SLAM is as follows: monocular SLAM technology has the advantages of simple structure, easy calibration, convenient maintenance, and large effective field of view. It obtains environmental image information through a single camera, and uses image processing and computer vision algorithms to estimate its own position and three-dimensional reconstruction of the surrounding environment. In the field of positioning the refueling port of an aircraft refueling pipe, monocular SLAM technology can make full use of the structural features of the refueling pipe itself, such as circular features, combined with advanced image processing and posture solution algorithms to achieve accurate positioning of the refueling port. This technology can not only overcome the limitations of traditional methods, but also improve the accuracy and real-time performance of positioning, providing strong support for the development of autonomous aerial refueling technology. The background technology of the aircraft refueling pipe refueling port positioning device and method based on monocular SLAM technology is to solve the limitations of traditional positioning methods in autonomous aerial refueling scenarios, give full play to the advantages of monocular SLAM technology, and achieve accurate and real-time positioning of the refueling port to meet the needs of the development of modern aviation technology.

[0119] The present invention is a monocular SLAM technology-based aircraft refueling pipe refueling port positioning device and method, which realizes accurate and real-time positioning of the refueling pipe refueling port by utilizing advanced computer vision technology and sensor fusion technology, and provides strong technical support for autonomous aerial refueling. This method not only overcomes the limitations of traditional positioning methods and improves positioning accuracy and reliability, but also has the advantages of simple structure and easy implementation, and has broad application prospects and market value.

[0120] The embodiment of the present application also discloses a monocular SLAM-based aircraft refueling pipe refueling port positioning system, based on the above-mentioned monocular SLAM-based aircraft refueling pipe refueling port positioning method, including a monocular camera, an image processing module, a feature extraction and matching module, a monocular SLAM construction module, a refueling pipe refueling port positioning module, an error correction and feedback module, and a control and execution module;

[0121] A monocular camera is connected to the image processing module to collect image information of the refueling pipe and the surrounding environment;

[0122] An image processing module is used to receive and process image information and output the processed image to a feature extraction and matching module;

[0123] The feature extraction and matching module is used to extract feature points from the image processed by the image processing module, and match the feature points in the current image with the feature points in the preset reference image to determine the position and posture of the camera in space; the matching results are transmitted to the monocular SLAM construction module;

[0124] A monocular SLAM construction module is used to construct a three-dimensional map of the environment based on the feature matching results using a monocular SLAM algorithm, and simultaneously calculate the motion trajectory of the camera; the constructed three-dimensional map of the environment and the motion trajectory information of the camera are transmitted to the fuel pipe and fuel port positioning module;

[0125] The fuel pipe refueling port positioning module calculates the position data of the fuel pipe refueling port by using the constructed three-dimensional environment map and the position and posture of the camera, combined with the geometric characteristics of the fuel pipe, and transmits the position data of the fuel pipe refueling port to the error correction and feedback module and the control and execution module;

[0126] The error correction and feedback module calculates the first position data based on the first positioning tag installed on the fuel pipe and the positioning base station installed next to the fuel receiving port, and calculates the second position data based on the second positioning tag installed on the fuel pipe and the positioning base station; calculates the position reference data using the first position data and the first algorithm according to the first position data and the second position data, and corrects the fuel filling port position data using the position reference data; calculates the attitude reference data using the second algorithm according to the first position data and the second position data, and corrects the fuel filling pipe attitude data using the attitude reference data; and transmits the corrected information to the control and execution module;

[0127] The control and execution module is used to control the movement of the aircraft according to the position information of the refueling port to achieve autonomous aerial refueling docking; and during the docking process, it is necessary to monitor the refueling status in real time and make adjustments as needed to ensure the safety and smooth progress of the refueling process.

[0128] The implementation principle of the aircraft refueling pipe refueling port positioning system based on monocular SLAM in the embodiment of the present application is: suppose that in a certain aerial refueling mission, the receiving aircraft needs to dock with the refueling pipe of the refueling aircraft. First, the device of the present invention is installed on the refueling pipe of the refueling aircraft to ensure that the monocular camera can clearly capture the refueling port and its surrounding environment. During the refueling process, the monocular camera collects image information in real time and preprocesses it through the image processing module. Then, the feature extraction and matching module extracts key feature points from the image and matches them with the existing map or reference image to preliminarily determine the relative position and posture of the receiving aircraft and the refueling pipe. Then, the monocular SLAM construction module constructs an environmental map based on the result of feature matching and estimates the motion trajectory of the receiving aircraft. The refueling pipe refueling port positioning module uses the constructed map and posture information, combined with the geometric features of the refueling pipe, to accurately calculate the position of the refueling pipe refueling port. The error correction and feedback module corrects the positioning result through UWB technology. In other embodiments, the positioning result can also be corrected by introducing sensor information such as GNSS. After the positioning result is corrected, feedback and adjustment are performed according to the actual docking situation. Finally, the control and execution module controls the movement of the receiving aircraft according to the positioning results to achieve accurate docking with the refueling pipe. Throughout the process, each module works together to ensure that the receiving aircraft can accurately find the refueling port of the refueling pipe and complete the aerial refueling mission.

[0129] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A method for positioning an aircraft refueling pipe refueling port based on monocular SLAM, characterized in that: The steps include: Based on the monocular camera installed on the refueling pipe, the surrounding environment is photographed to obtain an environmental picture; Extracting environmental feature points according to the environmental image; Matching the environmental feature points with feature points in a preset reference image, and outputting feature matching results; Constructing a three-dimensional map of the environment through a SLAM algorithm according to the feature matching results, calculating movement information of the environmental feature points in the environmental image, and updating the three-dimensional map of the environment according to the movement information; photographing the fuel inlet to obtain a fuel inlet image, and identifying fuel inlet feature points of the fuel inlet from the fuel inlet image; Based on the relative position relationship between the fuel filler port feature point and the environment feature point, calculating the position data of the fuel filler port feature point in the three-dimensional map of the environment as the fuel filler port position data; Based on the relative distance data and relative position relationship between the fuel filler port feature point and the environmental feature point, the posture data of the fuel filler pipe is calculated as the fuel filler pipe posture data; Calculate first location data based on a first positioning tag installed on the refueling pipe and a positioning base station installed next to the refueling port, and calculate second location data based on a second positioning tag installed on the refueling pipe and the positioning base station; Calculate position reference data using a first algorithm according to the first position data and the second position data, and use the position reference data to correct the fuel filler port position data; Calculate attitude reference data using a second algorithm according to the first position data and the second position data, and correct the attitude data of the refueling pipe using the attitude reference data; calculating position deviation data according to preset fuel filler port standard position data and the fuel filler port position data, and adjusting the position of the fuel filler pipe based on the position deviation data; Calculating posture deviation data according to preset standard posture data of the fuel filling pipe and the posture data of the fuel filling pipe, and adjusting the posture of the fuel filling pipe based on the posture deviation data; According to the anti-correlation of the position deviation data, the coefficient of the position correction is adjusted; the larger the position deviation data, the smaller the coefficient of the position correction; the smaller the position deviation data, the larger the coefficient of the position correction; wherein the position correction adopts UWB positioning technology; Calculate shaking data representing shaking degree according to the fuel filler port posture data; According to a preset shaking threshold, calculating the difference between the shaking data and the shaking threshold as a shaking difference; The coefficient of position correction is adjusted according to the positive correlation of the shake difference, the higher the shake difference is, the larger the coefficient of position correction is; the lower the shake difference is, the smaller the coefficient of position correction is; Acquiring inertial attitude data based on an inertial attitude sensor arranged on a camera; The posture is calculated by weighted averaging the inertial posture data, the posture reference data and the fuel pipe posture data, and the fuel pipe posture data is updated.

2. The method for positioning the aircraft refueling pipe refueling port based on monocular SLAM according to claim 1 is characterized in that: The method further comprises the steps of: According to the positive correlation of the relative distance data, the weighted calculation coefficient of the inertial posture data is adjusted, the larger the relative distance data is, the larger the weighted calculation coefficient of the inertial posture data is; the smaller the relative distance data is, the smaller the weighted calculation coefficient of the inertial posture data is; Get the flight speed of the tanker; The weighted calculation coefficient of the refueling pipe attitude data is adjusted according to the positive correlation with the flight speed; the greater the flight speed, the greater the weighted calculation coefficient of the refueling pipe attitude data; the smaller the flight speed, the smaller the weighted calculation coefficient of the refueling pipe attitude data.

3. The method for positioning the aircraft refueling pipe refueling port based on monocular SLAM according to claim 2 is characterized in that: The method further comprises the steps of: Extracting cone sleeve features from the environment image; In the continuous frames at a set number of intervals, the key points of the cone sleeve feature are matched; Calculating the overall scaling change of the cone sleeve feature according to the distance change between the key points; The telescopic movement speed of the fuel filling pipe is adjusted inversely according to the overall scaling change amplitude. The larger the overall scaling change amplitude, the slower the telescopic movement speed of the fuel filling pipe; the smaller the overall scaling change amplitude, the faster the telescopic movement speed of the fuel filling pipe.

4. The method for positioning the aircraft refueling pipe refueling port based on monocular SLAM according to claim 2 is characterized in that: The method further comprises the steps of: Extracting cone sleeve features from the environment image; In the continuous frames at a set number of intervals, local features of the cone sleeve feature are matched; Calculating the distortion change amplitude of the local feature; The posture adjustment speed of the fuel filler port is adjusted inversely according to the distortion change amplitude. The larger the distortion change amplitude, the slower the posture adjustment speed of the fuel filler port; the smaller the distortion change amplitude, the faster the posture adjustment speed of the fuel filler port.

5. An aircraft refueling pipe refueling port positioning system based on monocular SLAM, based on the aircraft refueling pipe refueling port positioning method based on monocular SLAM according to any one of claims 1 to 4, characterized in that: It includes a monocular camera, an image processing module, a feature extraction and matching module, a monocular SLAM construction module, a fuel pipe and fuel port positioning module, an error correction and feedback module, and a control and execution module; The monocular camera is connected to the image processing module and is used to collect image information of the refueling pipe and the surrounding environment; The image processing module is used to receive and process image information, and output the processed image to the feature extraction and matching module; The feature extraction and matching module is used to extract feature points from the image processed by the image processing module, and match the feature points in the current image with the feature points in the preset reference image to determine the position and posture of the camera in space; and transmit the matching results to the monocular SLAM construction module; The monocular SLAM construction module is used to construct a three-dimensional map of the environment based on the feature matching result using a monocular SLAM algorithm, and simultaneously calculate the motion trajectory of the camera; transmit the constructed three-dimensional map of the environment and the motion trajectory information of the camera to the fuel pipe and fuel port positioning module; The refueling pipe refueling port positioning module calculates the refueling pipe refueling port position data by using the constructed environment three-dimensional map and the position and posture of the camera in combination with the geometric features of the refueling pipe, and transmits the refueling pipe refueling port position data to the error correction and feedback module and the control and execution module; The error correction and feedback module calculates the first position data based on the first positioning tag installed on the fuel pipe and the positioning base station installed next to the fuel receiving port, and calculates the second position data based on the second positioning tag installed on the fuel pipe and the positioning base station; calculates the position reference data using the first position data and the first algorithm according to the first position data and the second position data, and corrects the fuel port position data using the position reference data; calculates the attitude reference data using the second algorithm according to the first position data and the second position data, and corrects the fuel pipe attitude data using the attitude reference data; and transmits the corrected information to the control and execution module; The control and execution module is used to adjust the position of the fuel filler pipe according to the preset fuel filler port standard position data and the position deviation data calculated from the fuel filler port position data; And it is used to adjust the posture of the fuel filler pipe according to the preset fuel filler pipe standard posture data and the posture deviation data calculated from the fuel filler pipe posture data.

Images