Butt joint positioning method between tractor and airplane wheel, tractor and medium

Abstract

The invention discloses a butt joint positioning method between a tractor and airplane wheels, the tractor and a medium, and the method comprises the steps: obtaining the position description information of the tractor according to source point cloud data collected by a vehicle-mounted laser radar sensor, so as to control the tractor to move to a to-be-dragged airplane; according to the color image and the depth image shot by the vehicle-mounted depth camera, front wheel point cloud information of the front wheel to be dragged is calculated, so that the course angle of the front wheel to be dragged relative to the tractor is calculated, and course angle alignment operation is achieved; according to an alignment color image which is collected by the vehicle-mounted depth camera and contains the front airplane wheel to be dragged, the current pixel coordinate of the center point of the front airplane wheel to be dragged in the alignment color image is calculated, and the horizontal relative pose between the tractor and the front airplane wheel to be dragged is solved, so that horizontal alignment operation between the tractor and the front airplane wheel to be dragged is carried out. According to the technical scheme of the embodiment of the invention, the butt joint positioning accuracy, the environmental adaptability, the automation degree and the operation degree in the traction operation are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of aircraft towing operations, and more particularly relates to a docking and positioning method between a tow tractor and an aircraft wheel, a tow tractor, and a medium. Background Art

[0002] When manually operating a tow tractor, the docking accuracy of the driver is restricted by various factors such as personal experience, skills, reaction speed, and visual judgment ability under complex weather conditions (such as rain, snow, haze, and strong light), which may cause deviations when the tow tractor is docked with the aircraft wheel. In the scenarios of special aircraft models or narrow parking positions, the lack of operation accuracy will significantly reduce the docking success rate, and traditional manual operation is difficult to meet the real-time requirements of modern airport operations. Especially in low visibility or extreme weather conditions, the manual operation efficiency is even lower.

[0003] In the prior art, aircraft tow tractors adopt the manual driving mode and require at least two guides to complete the docking through visual inspection. This mode has problems of high labor consumption and low operation accuracy, which easily leads to aircraft wear and flight delays. Due to relying on manual judgment of aircraft parameters and real-time poses, not only the training cost is high, but the risk of errors also increases significantly under complex working conditions. In addition, although some improvement schemes attempt to equip tow tractors with sensors such as infrared or laser rangefinders, these sensors are easily interfered by environmental factors and the measurement accuracy is unstable, resulting in their unreliable application in actual towing operations. Summary of the Invention

[0004] Embodiments of the present invention provide a docking and positioning method between a tow tractor and an aircraft wheel, a tow tractor, and a medium, which can quickly and accurately realize the docking and positioning between an automatically driven tow tractor and the nose landing gear of an aircraft to be towed in various complex scenarios.

[0005] According to one aspect of the embodiments of the present invention, a docking and positioning method between a tow tractor and an aircraft wheel is provided, which is executed by an automatically driven tow tractor. The method includes:

[0006] According to the source point cloud data collected by the on-vehicle lidar sensor at the current position, obtain the position description information of the vehicle in the point cloud map coordinate system; and according to the position description information, combined with the use of static and dynamic path planning algorithms, control the tow tractor to move to the working starting point where the aircraft to be towed is located;

[0007] When the tow tractor moves to the working starting point, according to the original color image and the original depth image including the nose landing gear captured by the on-vehicle depth camera, fuse and calculate the front wheel point cloud information of the front wheel of the aircraft to be towed;

[0008] Calculate the heading angle of the front landing gear to be towed relative to the tractor according to the point cloud information of the front landing gear, so as to perform the heading angle alignment operation between the tractor and the front landing gear to be towed;

[0009] After completing the heading angle alignment, calculate the current pixel coordinates of the center point of the front landing gear to be towed in the aligned color image according to the aligned color image containing the front landing gear to be towed collected by the on-vehicle depth camera;

[0010] Solve the horizontal relative pose between the tractor and the front landing gear to be towed according to the current pixel coordinates and the preset pixel coordinates of the calibration point, so as to perform the horizontal alignment operation between the tractor and the front landing gear to be towed.

[0011] According to another aspect of the embodiments of the present invention, there is also provided a docking and positioning device between a tractor and an aircraft landing gear, which is configured in an automatically driven tractor. The device includes:

[0012] A radar positioning and automatic driving module, which is used to obtain the position description information of the vehicle itself in the point cloud map coordinate system according to the source point cloud data collected by the on-vehicle lidar sensor at the current position; and according to the position description information, combine the use of static and dynamic path planning algorithms to control the tractor to move to the working starting point where the aircraft to be towed is located;

[0013] A depth vision perception module, which is used to fuse and calculate the point cloud information of the front landing gear to be towed according to the original color image and the original depth image containing the front landing gear taken by the on-vehicle depth camera when the tractor moves to the working starting point;

[0014] A heading angle calculation module, which is used to calculate the heading angle of the front landing gear to be towed relative to the tractor according to the point cloud information of the front landing gear, so as to perform the heading angle alignment operation between the tractor and the front landing gear to be towed;

[0015] A central pixel coordinate calculation module, which is used to calculate the current pixel coordinates of the center point of the front landing gear to be towed in the aligned color image according to the aligned color image containing the front landing gear to be towed collected by the on-vehicle depth camera after completing the heading angle alignment;

[0016] A horizontal pose adjustment module, which is used to solve the horizontal relative pose between the tractor and the front landing gear to be towed according to the current pixel coordinates and the preset pixel coordinates of the calibration point, so as to perform the horizontal alignment operation between the tractor and the front landing gear to be towed.

[0017] According to another aspect of the embodiments of the present invention, there is also provided an automatically driven tractor, and the tractor includes:

[0018] An on-vehicle lidar sensor, which is used to collect the source point cloud data of the surrounding environment;

[0019] A vehicle-mounted depth camera, used to collect color images and / or depth images of the surrounding environment;

[0020] at least one processor; and

[0021] a memory communicatively connected to the at least one processor; wherein,

[0022] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the docking and positioning method between the tractor and the aircraft wheel as described in any one of the embodiments of the present invention.

[0023] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the docking and positioning method between a tractor and an aircraft wheel as described in any one of the embodiments of the present invention when executed.

[0024] The technical solution of the embodiment of the present invention uses the vehicle-mounted laser radar to collect environmental point cloud data in real time, and then intelligently matches it with the pre-built high-precision apron point cloud map to accurately obtain the three-dimensional posture information of the tractor in the global coordinate system. Then, a hybrid algorithm combining static global path planning and dynamic local obstacle avoidance is used to intelligently plan the optimal driving route, and to perceive and avoid dynamic obstacles in real time. After the tractor arrives at the work site, the high-precision depth camera synchronously collects the color image and depth information of the aircraft's front landing gear, and the image processing algorithm is used to perform data fusion to construct a complete three-dimensional point cloud model of the front wheels, and the relative position of the front wheels to the tractor is obtained. The system detects heading angle deviation and automatically controls the tractor to adjust its azimuth to achieve precise heading alignment. Finally, the high-definition image obtained by the depth camera is used to locate the center coordinates of the front wheel using visual detection technology and compared with the preset ideal calibration points. The posture deviation in the horizontal plane is calculated and the docking method is realized. The system cleverly uses multi-sensor data fusion and hierarchical progressive control strategies to implement an intelligent processing flow from global positioning to local perception, from rough adjustment to fine alignment. This not only ensures the safety and reliability of the docking process, but also greatly improves the operating efficiency and accuracy in various complex docking operation scenarios, realizing the automation upgrade of aircraft towing operations.

[0025] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] To more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the accompanying drawings required for the description of the embodiments. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can be obtained based on these drawings.

[0027] Figure 1 It is a flowchart of a method for docking and positioning between a tractor and an aircraft wheel according to Embodiment 1 of the present invention;

[0028] Figure 2 It is a flowchart of another method for docking and positioning between a tractor and an aircraft wheel according to Embodiment 2 of the present invention;

[0029] Figure 3 It is a flowchart of the specific steps of radar positioning and automatic navigation in the docking and positioning between a tractor and an aircraft wheel according to Embodiment 3 of the present invention;

[0030] Figure 4 It is a flowchart of the specific steps of aligning the course angle in the docking and positioning between a tractor and an aircraft wheel according to Embodiment 3 of the present invention;

[0031] Figure 5 It is a flowchart of the specific steps of horizontal alignment in the docking and positioning between a tractor and an aircraft wheel according to Embodiment 3 of the present invention;

[0032] Figure 6 It is a schematic structural diagram of a device for the method of docking and positioning between a tractor and an aircraft wheel according to Embodiment 4 of the present invention;

[0033] Figure 7 It is a schematic structural diagram of a tractor for implementing the method of docking and positioning between a tractor and an aircraft wheel in the embodiments of the present invention. Detailed implementation manners

[0034] To enable those skilled in the art to better understand the solutions of the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0035] It should be noted that the terms "first", "second", etc. in the description, claims and above-mentioned drawings of the present invention are used to distinguish similar objects, and do not necessarily describe a specific order or sequence. It should be understood that such data used can be interchanged under appropriate circumstances, so that the embodiments of the present invention described here can be implemented in an order other than those illustrated or described here. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device comprising a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0036] Embodiment 1

[0037] Figure 1 FIG. is a flowchart of a docking and positioning method between a tractor and an aircraft wheel provided in Embodiment 1 of the present invention. This embodiment is applicable to the situation of automatic docking and positioning between an autonomous tractor and an aircraft wheel. This method can be executed by a docking and positioning device between the tractor and the aircraft wheel. The docking and positioning device between the tractor and the aircraft wheel is generally configured in an autonomous tractor and is executed by a controller in the tractor.

[0038] Correspondingly, as Figure 1 shown, the method includes:

[0039] S110. According to the source point cloud data collected by the on-vehicle lidar sensor at the current position, obtain the position description information of the vehicle in the point cloud map coordinate system, and according to the position description information, combine the use of static and dynamic path planning algorithms to control the tractor to move to the working starting point where the aircraft to be towed is located.

[0040] In the embodiment of the present invention, in order to move the tractor to the working starting point where the aircraft to be towed is located more accurately and efficiently, it is first necessary to accurately determine the real-time position of the tractor in the airport, that is, accurately locate the coordinate position of the tractor in the entire airport point cloud map.

[0041] Due to the requirements of the special working environment of the airport (presence of large metal structures, dynamic obstacles, and strict safety requirements, etc.), traditional position detection methods have obvious limitations: GPS (Global Positioning System) signals are easily blocked by buildings and aircraft fuselages, resulting in positioning drift; UWB (Ultra-Wideband) base stations are complex to deploy and are easily interfered; visual marker recognition is unreliable under bad weather or night conditions.

[0042] In contrast, on-vehicle lidar sensors can acquire high-density three-dimensional point cloud data in real time and match it with a pre-constructed high-precision point cloud map of the airport. With the characteristics of active detection, high precision, and strong anti-interference, they have become an ideal position detection solution in the airport environment. This technical solution based on point cloud matching can not only overcome environmental interference but also achieve centimeter-level positioning accuracy, effectively solving the deficiencies of traditional methods and ensuring that the tractor can reach the designated starting point of work efficiently and safely, meeting the stringent requirements of aircraft towing operations.

[0043] Correspondingly, before the tractor executes the towing and docking task of the aircraft, it can be located at any position in the airport. When the tractor needs to execute the towing and docking task, it can use the on-vehicle lidar sensor fixedly configured on the vehicle to acquire the source point cloud data of the surrounding environment by emitting laser beams and receiving reflected signals. After that, based on the source point cloud data, the position coordinate information and orientation information of the vehicle in the point cloud map coordinate system can be accurately obtained as the position description information.

[0044] After the tractor obtains the position description information of the vehicle, combined with the starting point of the aircraft to be towed that needs to be towed, a driving path can be planned starting from the position description information and ending at the starting point of the work according to a preset path planning algorithm, so that the tractor moves along the driving path to the starting point to complete the aircraft towing task.

[0045] Among them, considering the complexity and dynamics of the airport environment, static and dynamic path planning algorithms can be combined. First, a static path is planned, and during the process of the tractor moving along the static path, the static path is dynamically optimized in real time according to the surrounding environmental information.

[0046] In an optional implementation manner of this embodiment, after the tractor obtains accurate pose information (that is, position description information) based on the lidar sensor, a hybrid path planning strategy combining A* and D* can be adopted to achieve optimal navigation. Among them, the A* algorithm is based on the principle of heuristic search and can efficiently plan the global optimal path in a static environment. It guides the search direction by comprehensively evaluating the path cost and the heuristic function. At the same time, the D* algorithm uses the reverse search and incremental update mechanism. When the lidar detects a sudden obstacle, it can quickly adjust the local path. This "static global planning + dynamic local optimization" solution not only ensures the optimality of the path but also can respond to environmental changes in real time. While effectively improving the planning efficiency, it can effectively shorten the obstacle avoidance response time, perfectly meeting the dual requirements of navigation accuracy and real-time performance for airport towing operations.

[0047] S120. When the tractor moves to the working starting point, based on the original color image and the original depth image of the front landing gear captured by the on-vehicle depth camera, fuse and calculate the front wheel point cloud information of the front wheel to be towed.

[0048] In the embodiment of the present invention, the tractor synchronously acquires the original color image and the original depth image of the front landing gear of the aircraft to be towed through an on-vehicle RGB-D camera (i.e., a depth camera), realizing multi-modal data fusion.

[0049] Among them, the original color image provides rich texture features for target recognition, and the original depth image provides accurate three-dimensional space information. After the two are spatio-temporally aligned through the camera calibration parameters, they jointly construct the front wheel point cloud information of the front wheel. As a high-precision point cloud model.

[0050] This fusion perception scheme has significant advantages: on the one hand, it can maintain stable recognition performance under strong light or weak light conditions. The color information can make up for the deficiencies of the depth data, and the depth information can correct the influence of light on vision; on the other hand, it effectively improves the perception robustness in complex environments through data complementarity, providing a reliable three-dimensional environment representation for subsequent precise docking. Compared with the single-sensor scheme, this technology significantly improves the adaptability and reliability of the system under special working conditions at the airport.

[0051] In an optional implementation manner of this embodiment, when the tractor moves to the working starting point, based on the original color image and the original depth image of the front landing gear captured by the on-vehicle depth camera, fusing and calculating the front wheel point cloud information of the front wheel to be towed may include:

[0052] When the tractor moves to the working starting point, capture the original color image and the original depth image of the front landing gear through the on-vehicle depth camera; input the original color image into a pre-trained front wheel recognition model to segment out the color sub-image matching the front wheel to be towed; perform fusion processing on the segmented color sub-image and the original depth image to obtain the front wheel point cloud information of the front wheel to be towed; input the front wheel point cloud information of the front wheel to be towed into a pre-trained point cloud segmentation model to obtain the front wheel point cloud information of the front wheel to be towed.

[0053] In this optional implementation manner, a front wheel recognition model for two-dimensional image segmentation can be first trained to identify and segment out the color sub-image matching the front wheel to be towed that needs to be towed and docked in the two-dimensional original color image.

[0054] After obtaining the pixel positions of the color sub-image in the original color image, the depth information of each pixel at the same pixel position can be obtained in the original depth image, and the depth information of the above-mentioned pixels is fused with the RGB information of each pixel in the color sub-image to obtain the point cloud information of the front landing gear before towing.

[0055] Considering that the point cloud information of the front landing gear before towing is extracted based on the two-dimensional image recognition result, the resolution or accuracy of the point cloud information of the front landing gear before towing may be relatively rough. Furthermore, the inventor further trains a point cloud segmentation model for three-dimensional point cloud segmentation to perform refined three-dimensional segmentation on the point cloud information of the front landing gear before towing, and obtain the point cloud information of the front landing gear that accurately matches the actual front landing gear before towing.

[0056] Based on the above embodiments, after the tractor obtains the aircraft towing task, it can synchronously obtain the model description information of the aircraft to be towed. Based on this model description information, the tractor can obtain the standard front landing gear images of this type of aircraft under different standardized environmental parameters (visibility, temperature, light, etc.) from the pre-constructed two-dimensional image database. Then, combined with the current real-time environmental parameters, a most suitable target standard front landing gear image is obtained, and the target standard front landing gear image and the original color image are jointly input into the front landing gear recognition model to segment a more accurate color sub-image.

[0057] Similarly, based on this model description information, the tractor can also obtain the standard front landing gear point cloud data of this type of aircraft under different standardized environmental parameters (visibility, temperature, light, etc.) from the pre-constructed three-dimensional point cloud database. Then, combined with the current real-time environmental parameters, the most suitable target standard front landing gear point cloud data is obtained, and the target standard front landing gear point cloud data and the fused point cloud information of the front landing gear before towing are jointly input into the point cloud segmentation model to segment a more accurate point cloud information of the front landing gear.

[0058] Based on the above embodiments, the front landing gear recognition model can be trained based on the first improved YOLOv8 network, and the point cloud segmentation model can be trained based on the improved PointNet++ network, where:

[0059] The first improved YOLOv8 network is constructed by adding Space-to-Depth Convolution (SPD-Conv) modules to the front ends of the third, sixth, ninth, and twelfth layers of the backbone network of the standard YOLOv8 network, and adding a Squeeze-and-Excitation Attention (SE Attention) module to the last layer of the backbone network of the standard YOLOv8 network;

[0060] The improved PointNet++ network is constructed by adding a self-attention mechanism module to the standard PointNet++ network, modifying the feature extraction network in the standard PointNet++ network into a graph convolutional network, and increasing the sampling radii of different scales for different feature extraction modules in the feature aggregation method of the standard PointNet++ network; and

[0061] In the first target dataset used for model training of the first improved YOLOv8 network, it contains a real dataset of the front landing gear of an aircraft and a computer-aided design virtual dataset under various types of meteorological scenarios.

[0062] Correspondingly, the original color image can be obtained by an RGB-D camera and input into the improved YOLOv8 algorithm network for the recognition of the front landing gear. The point cloud information can be obtained through the original depth image, and the color point cloud is obtained by fusing RGB and the depth point cloud, that is, the point cloud information of the front landing gear to be towed.

[0063] Among them, the original color image collected synchronously by the RGB-D camera is used to be input into the improved YOLOv8 network, and the original depth image is used to generate point cloud information. The YOLOv8 algorithm used by the standard YOLOv8 network, as a single-stage object detection algorithm, has a very strong timeliness function. By adding the SPD-Conv module to the third, sixth, ninth, and twelfth layers of the backbone network of the standard YOLOv8 network, the input size of The feature map X is sliced in the spatial dimension, S represents the length and width of the feature map, represents the number of channels of the input feature map, and it is divided into intermediate feature maps of size . scale represents the downsampling factor, and its size determines the downsampling ratio of the SPD-Conv module to the feature map. In order to retain as much information of the feature map as possible, the SPD layer concatenates the divided sub-feature maps along the channel dimension to generate a sub-feature map of size , and for the intermediate feature map ​Perform the operation of non-stride convolution, and finally output a feature map with a size of the output feature map , which represents the number of channels of the output feature map.

[0064] Furthermore, after adding the SE Attention module to the last layer of the backbone network of the standard YOLOv8 network, through the squeeze (i.e., compression) operation in the SE Attention module, global average pooling is performed on the output feature map to reduce the feature values of each channel to a global vector, capturing the global information of each channel. Subsequently, through the Excitation (i.e., excitation) operation composed of two fully connected layers, a ReLU activation function, and a Softmax activation function in the SE Attention module, dimensionality reduction is first performed and then dimensionality increase is carried out. A weight vector is generated through the sigmoid function in the SE Attention module to ensure that their sum is 1. Finally, through the Scale (scaling) operation in the SE Attention module, the channel attention weights obtained in the previous step are multiplied by the original input feature map to adjust the feature values of each channel, emphasizing the information of important channels and suppressing unimportant information.

[0065] The first improved YOLOv8 network is obtained by improving the standard YOLOv8 network. First, the SPD-Conv module is used to perform spatial segmentation and channel splicing operations, effectively retaining the small target feature information and solving the problem of small target feature loss caused by traditional downsampling. Furthermore, the SE Attention module is used for attention weighting, enhancing the expression ability of key features. This improvement significantly improves the detection accuracy of the algorithm for the nose landing gear in the complex environment of the airport, especially performing excellently in challenging scenarios such as target occlusion and lighting changes.

[0066] Correspondingly, in order to construct a more robust first target dataset for model training of the first improved YOLOv8 network, the embodiments of the present invention further adopt a multi-source data fusion strategy: on the one hand, collect the real image data of the nose landing gear under various meteorological conditions (sunny, rainy, snowy, and foggy, etc.) and lighting environments (daytime, night, and backlight, etc.) at different airports; on the other hand, generate a virtual dataset containing different aircraft models and different postures through computer-aided design (CAD, Computer Aided Design) modeling, and use physics-based rendering (PBR) technology to simulate various complex environment effects. Optimize the anchor box size distribution through the Kmeans++ algorithm, and further expand the data diversity by means of data augmentation (such as geometric transformation, noise injection, and simulation of weather special effects, etc.).

[0067] This multi-source data fusion strategy significantly improves the generalization ability of the model in extreme weather, fills in the extreme situations that are difficult to cover with real data, and ensures data consistency through domain adaptation training. The rich dataset not only enhances the detection robustness of the YOLOv8 network, but also lays a solid foundation for subsequent point cloud segmentation and pose estimation modules, enabling this solution to adapt to the complex operating environments of various airports.

[0068] Similarly, in order to further improve the segmentation accuracy of the front landing gear point cloud information, the embodiment of the present invention trains a point cloud segmentation model based on an improved PointNet++ network obtained by improving the standard PointNet++ network.

[0069] Specifically, the inventor made three key improvements to the limitations of the traditional PointNet++ network in complex scenarios: First, a self-attention mechanism (implemented through PyTorch) was introduced, enabling the PointNet++ network to dynamically focus on the key geometric features of the front landing gear; Second, the original feature extraction network in the PointNet++ network was replaced with a graph convolutional network (GCN), using the graph structure to better model the topological relationship of the local point sets in the point cloud; Finally, the feature aggregation method in the PointNet++ network was improved, setting multi-scale sampling radii in each layer of the feature extraction module to simultaneously capture local features at different granularity levels.

[0070] The inventor found through multiple experiments that these improvements significantly enhance the segmentation performance of the PointNet++ network for the front landing gear point cloud in the complex airport environment: The self-attention mechanism enhances the recognition ability of key parts, the GCN structure improves the expression ability of local geometric features, and multi-scale sampling ensures the complete extraction of different-sized structures. While maintaining real-time performance, the segmentation accuracy of the improved model has been improved, showing stronger robustness in challenging scenarios such as partial occlusion of the landing gear and strong light reflection.

[0071] S130. Calculate the heading angle of the front landing gear to be towed relative to the tractor according to the front landing gear point cloud information, so as to perform the heading angle alignment operation between the tractor and the front landing gear to be towed.

[0072] In the embodiment of the present invention, the heading angle is the included angle between the longitudinal axis of the tractor and the main axis of the front landing gear on the horizontal plane, and is the core parameter describing the relative spatial pose of the two. The heading angle directly determines the alignment accuracy between the tractor and the front wheel of the aircraft. When the heading deviation exceeds 1°, it may lead to docking failure or the risk of equipment collision.

[0073] It can be understood that after acquiring the front wheel point cloud information through the on-vehicle depth camera fixedly installed on the tractor, based on this front wheel point cloud information, the heading angle of the front wheel to be towed relative to the tractor can be accurately identified. Based on the calculated heading angle, the tractor can be controlled to rotate in the direction of eliminating this heading angle, so as to finally achieve the alignment of the heading angles between the tractor and the front wheel to be towed.

[0074] Optionally, on the basis of the above embodiments, before calculating the heading angle of the front wheel to be towed relative to the tractor according to the front wheel point cloud information, it may further include:

[0075] The obtained front wheel point cloud information after segmentation is denoised by a statistical outlier removal algorithm. By analyzing the distance distribution characteristics within the neighborhood of each point, isolated noise points and abnormal measurement values are intelligently filtered out; subsequently, the moving least squares method (MLS) is used for smooth surface fitting to achieve smooth optimization of the point cloud surface while maintaining the original geometric features.

[0076] It can be understood that the processes of denoising and smoothing will bring significant advantages to the embodiments of the present invention: statistical denoising effectively eliminates the abnormal points caused by sensor noise and environmental interference, improving the data quality; MLS smoothing repairs the discontinuous surface caused by occlusion or measurement error while maintaining the key geometric features of the wheel. The processed high-precision front wheel point cloud is converted to the camera coordinate system, which not only provides a more accurate three-dimensional space representation but also significantly improves the accuracy and stability of the subsequent heading angle calculation, enabling the system to still achieve reliable heading alignment in a complex working environment and laying a solid foundation for automated towing operations.

[0077] S140. After completing the heading angle alignment, calculate the current pixel coordinates of the center point of the front wheel to be towed in the aligned color image collected by the on-vehicle depth camera.

[0078] In the embodiments of the present invention, after completing the heading angle alignment, the on-vehicle depth camera on the tractor is controlled again to collect a new color image as the aligned color image.

[0079] In an optional implementation manner of this embodiment, based on this aligned color image, an improved morphological image processing technique can be used to calculate the pixel coordinates of the center point of the front wheel to be towed.

[0080] Specifically, the wheel area can be first segmented and extracted from the aligned color image through an adaptive threshold, and then combined with the Canny edge detection and least squares circle fitting algorithms to determine the hub center position at the sub-pixel level of accuracy. This technical solution takes advantage of the frontal viewing angle of the image after heading alignment, effectively eliminates the influence of perspective distortion, and can achieve accurate positioning under complex lighting conditions, providing an accurate two-dimensional projection reference for the calculation of the two-dimensional pixel coordinates of the center point.

[0081] S150. Solve the horizontal relative pose between the tractor and the front wheel to be towed according to the current pixel coordinates and the preset pixel coordinates of the calibration point, so as to perform the horizontal alignment operation between the tractor and the front wheel to be towed.

[0082] In this embodiment, after obtaining the current pixel coordinates of the center point of the front wheel to be towed in the aligned color image, the current pixel coordinates can be compared with the preset pixel coordinates of the calibration point to solve the horizontal relative pose between the tractor and the front wheel to be towed.

[0083] Among them, the pixel coordinates of the calibration point can be understood as the pixel position of the center point of the front wheel to be towed in the color image obtained by a fixedly installed depth camera after the tractor reaches the working starting point where the aircraft to be towed is located and achieves heading angle alignment and horizontal alignment with the aircraft to be towed.

[0084] Furthermore, after obtaining the current pixel coordinates, by comparing the pixel position differences between the current pixel coordinates and the preset pixel coordinates of the calibration point, the horizontal relative pose between the tractor and the front wheel to be towed in free space can be calculated. Then, by controlling the tractor to move in the direction of eliminating the horizontal relative pose, the horizontal alignment between the tractor and the front wheel to be towed can be achieved. After completing the horizontal docking, the tractor can be controlled to accurately tow and dock with the front wheel to be towed, and then the towing control of the aircraft to be towed can be completed.

[0085] The technical solution of the embodiment of the present invention uses the on-board laser radar to collect environmental point cloud data in real time, and then intelligently matches it with the pre-built high-precision apron point cloud map to accurately obtain the three-dimensional posture information of the tractor in the global coordinate system. Then, a hybrid algorithm combining static global path planning and dynamic local obstacle avoidance is used to intelligently plan the optimal driving route, and to perceive and avoid dynamic obstacles in real time. After the tractor arrives at the work site, the high-precision depth camera synchronously collects the color image and depth information of the front landing gear of the aircraft, and the image processing algorithm is used to perform data fusion to construct a complete three-dimensional point cloud model of the front wheels, and obtain the front wheels relative to the tractor. The system detects the heading angle deviation of the vehicle and automatically controls the tractor to adjust its direction to achieve precise heading alignment. Finally, the high-definition image obtained by the depth camera is used, and the visual detection technology is used to locate the center coordinates of the front wheel. After comparing it with the preset ideal calibration point, the posture deviation in the horizontal plane is calculated and the docking method is realized. The system cleverly uses multi-sensor data fusion and hierarchical progressive control strategies to implement an intelligent processing flow from global positioning to local perception, from rough adjustment to fine alignment. This not only ensures the safety and reliability of the docking process, but also greatly improves the operating efficiency and accuracy in complex weather environments, realizing the automation upgrade of aircraft towing operations.

[0086] Embodiment 2

[0087] Figure 2 This is a flow chart of a docking positioning method between a tractor and an aircraft wheel provided in the second embodiment of the present invention. This embodiment is optimized based on the above embodiments. In this embodiment, the operations of "the source point cloud data collected by the vehicle-mounted laser radar sensor at the current position, obtaining the position description information of the vehicle in the point cloud map coordinate system" and "calculating the current pixel coordinates of the center point of the front wheel to be towed in the alignment color image" are specifically refined.

[0088] Correspondingly, such as Figure 2 As shown, the method includes:

[0089] S210, collecting source point cloud data of the surrounding environment at the current position through the vehicle-mounted laser radar sensor, and obtaining a target point cloud map pre-established for the airport apron area.

[0090] When the tractor receives an aircraft towing command at any location in the airport, it can directly turn on the on-board laser radar sensor to collect source point cloud data of the surrounding environment at the current location. The source point cloud data can be point cloud data collected by the on-board laser radar sensor within a set angle range, or point cloud data collected by the on-board laser radar sensor in 360° omnidirectional directions, which is not limited in this embodiment.

[0091] Specifically, in order to accurately locate the tractor based on the source point cloud data, it is necessary to further combine and use a target point cloud map that matches the airport apron area. By using a dedicated lidar device to collect data throughout the airport apron area, the target point cloud map can be accurately constructed. Furthermore, the tractor can be accurately located by performing point cloud registration of the source point cloud data on the target point cloud map.

[0092] In an alternative implementation manner of this embodiment, before collecting the source point cloud data of the surrounding environment at the current position by the vehicle-mounted lidar sensor, it may further include:

[0093] Obtain at least one piece of meteorological data that matches the current position in real time, and set at least one sensor parameter of the vehicle-mounted lidar sensor according to each piece of meteorological data.

[0094] The inventor found through research that for different types of meteorological conditions, the data collection accuracy of the vehicle-mounted lidar sensor is also different. For example, raindrops or snowflakes will reflect the laser beam, generating noise points and reducing the quality of the point cloud data. Particles in the haze will scatter the laser beam, resulting in sparse or inaccurate point cloud data. Strong sunlight will interfere with the receiver of the lidar, reducing the signal intensity. Dust or sand will reflect the laser beam, generating false point clouds, etc. Based on this, in order to improve the accuracy of the point cloud data collected by the vehicle-mounted lidar sensor and thus improve the positioning accuracy of the tractor, it is considered to combine the real-time meteorological data at the location of the tractor and set at least one sensor parameter of the vehicle-mounted lidar sensor.

[0095] Among them, the meteorological data may include: temperature value, meteorological type (sunny, rainy, snowy, hazy, and sandy, etc.), and degree parameters under the meteorological type (for example, light intensity, rainfall, snowfall, haze content, and sand content, etc.). The sensor parameters may include: signal transmission power, working band, scanning frequency, resolution, and point cloud density of the lidar sensor.

[0096] Specifically, different sensor parameters of the lidar sensor can be used to perform actual collection tests under different types of meteorological data. Furthermore, according to the collection test results, the optimal sensor parameter configuration methods under various types of meteorological data can be clustered. After obtaining at least one piece of meteorological data that matches the current position in real time, determine the cluster to which the real-time obtained meteorological data belongs. Furthermore, the optimal sensor parameter configuration method corresponding to this cluster can be obtained, and at least one sensor parameter of the vehicle-mounted lidar sensor can be set.

[0097] Through the above settings, parameters such as the scanning frequency and resolution of the lidar can be dynamically adjusted according to real-time meteorological data, ensuring that source point cloud data of the best quality can be obtained under different weather conditions. This adaptive sensing technology significantly improves the data acquisition ability of the embodiments of the present invention in harsh weather conditions such as rain, snow, haze, etc., laying a good foundation for subsequent tractor positioning processing.

[0098] Correspondingly, after collecting source point cloud data of the surrounding environment at the current position through the vehicle-mounted lidar sensor, it may further include:

[0099] Determine the meteorological type according to each piece of the meteorological data, and obtain a denoising algorithm matching the meteorological type to perform denoising processing on the source point cloud data.

[0100] Specifically, a specially optimized combination of denoising algorithms is adopted for different meteorological conditions: in rainy and snowy weather, a statistical outlier removal algorithm is mainly used. This algorithm analyzes the distance distribution characteristics of points within the neighborhood of each point, calculates the mean μ and standard deviation σ, and automatically identifies and removes abnormal points whose distances exceed a reasonable range (such as μ±3σ). This method is particularly suitable for eliminating random noise points caused by rain and snow. For the haze environment, a radius-based statistical filtering is enabled by the system. This algorithm sets a reasonable search radius (usually 5-10 cm), statistics the point density within this radius, and filters out discrete noise points whose densities are significantly lower than the normal values, effectively solving the problems of point cloud sparsity and drift caused by haze. For strong wind weather, a temporal filtering technology is adopted to eliminate dynamic noise through the temporal consistency analysis of multiple frames of point clouds.

[0101] This set of adaptive denoising solutions intelligently switches the processing algorithm through the environmental data real-time feedback by the meteorological sensor, ensuring that clean and reliable point cloud data can be obtained under various complex meteorological conditions, providing high-quality input for subsequent feature extraction and matching.

[0102] S220. Through the nearest neighbor search algorithm, determine the local neighborhood of each source point cloud point in the source point cloud data, and calculate the local geometric attributes and FPFH (FastPoint Feature Histograms) corresponding to each source point cloud point according to the local neighborhood of each source point cloud point.

[0103] In this embodiment, high-precision point cloud registration is achieved through innovative nearest neighbor search and feature extraction technologies. First, a KD-tree accelerated nearest neighbor search algorithm is used to establish local spatial relationships for each point cloud point, and geometric attributes such as surface normal and curvature are accurately calculated. Based on this, an FPFH descriptor is further generated. This descriptor constructs a 33-dimensional feature vector with strong discrimination by quantifying geometric relationships such as the normal angle, projection distance, and azimuth angle between neighboring points.

[0104] This multi-level feature expression method effectively improves the robustness of point cloud processing. While ensuring the operation efficiency, the FPFH descriptor provides rich local structure information and shows excellent adaptability to point cloud data with different densities and resolutions.

[0105] S230. Evaluate the significance score of each source point cloud point according to the local geometric attributes of each source point cloud point, and identify the source ISS (Intrinsic Shape Signatures) key points among the source point cloud points according to the significance scores.

[0106] In this embodiment, the ISS key points are identified by calculating the significance score through analyzing the local geometric attributes of the point cloud. This method first evaluates the structural importance of each point based on geometric features such as point cloud curvature and normal change, and then filters out the most representative feature points through non-maximum suppression.

[0107] This recognition technology based on intrinsic shape features can effectively capture the significant structural features in the point cloud, and can stably detect highly distinguishable key points even in the case of noise interference or partial occlusion. Its advantage lies in automatically identifying regions with rich features such as corners and edges by quantifying the degree of change in local surface geometric characteristics, while filtering out redundant points in flat regions, which not only greatly improves the quality of feature points, but also significantly reduces the subsequent matching calculation amount, providing a more robust and efficient feature basis for point cloud registration.

[0108] S240. Match the FPFH of each source ISS key point with the FPFH of each target ISS key point in the target point cloud map to obtain the paired points between the source point cloud data and the target point cloud map.

[0109] In this embodiment, efficient and accurate point cloud registration is achieved by matching the FPFH feature descriptors of the ISS key points in the source point cloud and the target point cloud. This method first extracts the ISS key points with significant geometric features in the two point clouds, then calculates the FPFH feature descriptor around each key point to characterize its local geometric structure, and finally establishes the corresponding relationship of point pairs through the nearest neighbor search in the feature space.

[0110] This matching method based on feature descriptors can effectively overcome the problems of noise interference and density change in point cloud data. Its advantage lies in significantly improving the operation efficiency while maintaining a high matching accuracy by combining the significance of ISS key points and the discrimination ability of FPFH descriptors. Even in the case of partial occlusion, a reliable matching effect can still be achieved, providing a high-quality corresponding point pair basis for subsequent precise pose calculation.

[0111] S250. Calculate the transformation matrix from the source point cloud data to the target point cloud map according to each pair of matching points, and obtain the current position coordinates and current attitude orientation of the vehicle in the point cloud map coordinate system according to the transformation matrix.

[0112] In this embodiment, the transformation matrix from the source point cloud to the target point cloud is calculated through the matched feature point pairs, and the robust estimation algorithm is used to solve the optimal rigid body transformation parameters, so as to determine the accurate pose of the tractor in the global map. This method constructs an error function based on the paired points, and simultaneously solves the rotation and translation transformations through iterative optimization, effectively overcoming the interference of sensor noise and matching errors. Its technical advantage lies in being able to derive the complete six-degree-of-freedom pose information using sparse but accurate matching point pairs, maintaining good computational efficiency while ensuring the positioning accuracy, enabling the system to continuously output stable pose data in a dynamic environment, providing a reliable positioning reference for automated towing operations, and significantly enhancing the adaptability and accuracy of the navigation system in a complex airport environment.

[0113] S260. According to the position description information, combine the use of static and dynamic path planning algorithms to control the tractor to move to the working starting point where the aircraft to be towed is located.

[0114] Furthermore, it is possible to obtain in real time a control area (which can also be called a detour area) that suddenly appears in the airport apron area, for example, an oil spill area. Then, wirelessly send the coordinate information of the oil spill area in the target point cloud map to each moving tractor for the tractor to dynamically plan a new navigation route to avoid the control area. Or, when the control area suddenly appears very close to the tractor and the preparation time for the aforementioned avoidance method is insufficient, a manual-assisted route planning method can be used, for example: immediately send a voice instruction such as "Avoid the oil spill area XX on the left front" to the currently moving tractor for the tractor to quickly avoid.

[0115] S270. When the tractor moves to the working starting point, fuse and calculate the front wheel point cloud information of the front landing gear to be towed according to the original color image and original depth image captured by the on-vehicle depth camera.

[0116] S280. Calculate the heading angle of the front landing gear to be towed relative to the tractor according to the front wheel point cloud information to perform the heading angle alignment operation between the tractor and the front landing gear to be towed.

[0117] Based on the above embodiments, after obtaining the front-wheel point cloud information, a new front-wheel point cloud information can be obtained again after waiting for a period of time (e.g., 10s) while keeping the current motion state of the tractor unchanged. According to the front-wheel depth information in the above two front-wheel point cloud information, it is detected whether the aircraft to be towed moves abnormally slowly. If so, the tractor needs to send an acoustic warning and stay away from the aircraft to be towed according to the preset avoidance strategy to prevent unnecessary collision risks.

[0118] S290. After completing the heading angle alignment, use the on-vehicle depth camera to collect the aligned color image containing the front wheel to be towed.

[0119] S2100. Input the aligned color image into the pre-trained front-wheel bounding box detection model to obtain the target detection box calibrated in the aligned color image.

[0120] Among them, the front-wheel bounding box detection model only outputs a detection box with a single target size, and the target size is consistent with the size of the minimum circumscribed rectangle of the front wheel to be towed photographed by the on-vehicle depth camera on the tractor at the standard position.

[0121] Specifically, after completing the heading angle alignment, use the on-vehicle depth camera to collect the aligned color image containing the front wheel to be towed. The key to this step is that the image obtained after heading alignment has a standard front view angle, effectively avoiding the influence of perspective distortion on the subsequent detection accuracy. The camera uses the same resolution and aspect ratio as during model training (e.g., 1028×1028) to ensure that the input data is consistent with the training data distribution. This standardized acquisition scheme significantly improves the stability of detection, especially maintaining reliable imaging quality under complex lighting conditions.

[0122] Based on the above embodiments, the front-wheel bounding box detection model can be trained based on the second improved YOLOv8 network, where:

[0123] After setting the default resolution in the standard YOLOv8 network to the target resolution and setting the loss function in the standard YOLOv8 network to the CloU function, the second improved YOLOv8 network is constructed;

[0124] Among them, the CloU Loss is used to replace the traditional IoU loss function. This loss function makes the bounding box positioning accuracy reach the sub-pixel level by simultaneously considering multiple key geometric factors such as the distance between the centers of the detection boxes, the aspect ratio matching degree, and the overlapping area. It shows significant advantages especially when dealing with difficult samples with partial occlusion or blurred edges.

[0125] In the second target dataset used for model training of the second improved YOLOv8 network, it contains a real dataset of the front landing gear of an aircraft and a computer-aided design virtual dataset under various types of meteorological scenarios, and each training image in the second target dataset has the same aspect ratio of the target image; and

[0126] The target resolution is consistent with the resolution of the images captured by the on-vehicle depth camera on the tractor, and the aspect ratio of the target image is consistent with the aspect ratio of the images captured by the on-vehicle depth camera on the tractor.

[0127] S2110. According to the pixel coordinates of the four corner points in the target detection box, calculate the current pixel coordinates of the center point of the front wheel to be towed in the aligned color image.

[0128] In this embodiment, a pre-trained deep learning model is used to obtain a rectangular detection box and its corner point coordinates that are strictly matched with the standard size. Subsequently, the geometric midpoint intersection method is used to calculate the center position: connect the diagonal vertices of the detection box to form two midlines, and the intersection point is the theoretical center coordinate. To improve the accuracy, the system adopts a standardized aiming box design to eliminate the influence of perspective distortion, and can be combined with sub-pixel edge refinement technology.

[0129] This solution combines geometric principles with deep learning. While ensuring millimeter-level positioning accuracy, it also has high computational performance and excellent engineering practicability, and can meet the stringent requirements of towing operations for real-time performance and reliability. It is more adaptable to the stable operation needs of industrial scenarios than complex key point detection or instance segmentation methods.

[0130] S2120. According to the current pixel coordinates and the preset pixel coordinates of the calibration points, solve the horizontal relative pose between the tractor and the front wheel to be towed, so as to perform the horizontal alignment operation between the tractor and the front wheel to be towed.

[0131] The technical solution of this embodiment uses the vehicle-mounted laser radar to collect environmental point cloud data in real time, and quickly establishes the spatial topological relationship between point clouds through the nearest neighbor search algorithm. The system calculates the local surface normal, curvature and other geometric features, and constructs a FPFH descriptor with strong discrimination. The descriptor forms a 33-dimensional feature vector by quantifying the geometric relationships such as the normal angle, projection distance and azimuth between the neighborhood points. Based on the ISS (intrinsic shape feature) key point detection technology, the system can automatically identify feature areas with significant geometric characteristics in the point cloud, such as structural features such as edges and corners. While maintaining computational efficiency, this method achieves efficient characterization of complex geometric structures. It can not only accurately capture the local detail features of the point cloud, but also adapt to point cloud data with different densities and noise levels, providing a stable and reliable feature basis for subsequent registration and positioning. Afterwards, a hybrid algorithm combining static global path planning and dynamic local obstacle avoidance is used to intelligently plan the optimal driving route. After the tractor arrives at the work site, the high-precision depth camera synchronously collects the color image and depth information of the aircraft's front landing gear, and uses the image processing algorithm to perform data fusion to construct a complete three-dimensional point cloud model of the front wheel, obtain the heading angle deviation of the front wheel relative to the tractor, and automatically control the tractor to adjust its orientation to achieve precise heading alignment. Finally, the high-definition image obtained by the depth camera is used to locate the center coordinates of the front wheel using visual detection technology, and compared with the preset ideal calibration point, to calculate the posture deviation in the horizontal plane and realize the docking method. The multi-sensor data fusion and hierarchical progressive control strategy are cleverly used to implement the intelligent processing flow from global positioning to local perception, from rough adjustment to fine alignment, which not only ensures the safety and reliability of the docking process, but also greatly improves the operating efficiency and operation accuracy in complex weather environments, realizing the automation upgrade of aircraft towing operations.

[0132] Embodiment 3

[0133] For ease of understanding, the specific application scenarios applicable to each embodiment of the invention are described. In this specific application scenario, in order to make the docking between the tractor and the aircraft wheel more accurate, the embodiment of the present invention designs a complete docking and positioning solution between the tractor and the aircraft wheel.

[0134] In this embodiment, three steps are used to ensure accurate docking between the tractor and the aircraft wheels:

[0135] Specifically, in Figure 3 FIG. 2 shows a flowchart of the specific steps of radar positioning and automatic navigation in docking positioning between a tractor and an aircraft wheel according to an embodiment of the present invention. Figure 3As shown in the figure, in this embodiment, a lidar positioning module is used to construct a three-dimensional point cloud map of the airport apron, and an improved ICP (Iterative Closest Point) algorithm is used to achieve real-time precise positioning. During the operation of the tractor, the lidar continuously collects the current environmental point cloud data and inputs it into the improved ICP algorithm network using ISS and FPFH feature descriptors. Through normal estimation and feature matching, high-precision registration with the target point cloud map is completed, and the precise pose of the vehicle in the global coordinate system is output. The obtained positioning information is input into the navigation network integrating A* and D* algorithms. Among them, the A* algorithm is responsible for the global optimal path planning in the static environment, and the D* algorithm processes the local path adjustment of dynamic obstacles. The two work together to ensure the planning efficiency and real-time performance. At the same time, the system integrates the lidar point cloud and the depth information of the RGB-D camera to construct a dynamic obstacle map, and finally, the vision-driven positioning system guides the tractor to accurately reach the starting point of the target work. Through multi-sensor data fusion and hierarchical planning strategies, this solution realizes the full-process automation of the towing operation in a complex airport environment.

[0136] Further, Figure 4 shows a specific step flowchart of the heading angle alignment in the docking positioning between a tractor and an aircraft wheel in an embodiment of the present invention. As Figure 4 shown, multi-sensor fusion technology can be used to achieve precise identification and positioning of the aircraft's nose landing gear. When the tractor reaches the starting point of work, the nose landing gear is detected through an improved YOLOv8 network, which significantly improves the detection accuracy and real-time performance by introducing the SPD-Conv module and the SE Attention mechanism. At the same time, the depth information collected by the RGB-D camera is fused with the color image to generate a color point cloud, which is input into the optimized PointNet++ network for segmentation. This network enhances the point cloud feature extraction ability through improvements such as adding self-attention mechanism and GCN feature extraction. After point cloud denoising and smoothing processing, the system accurately calculates the three-dimensional coordinates and relative heading angle of the front wheel, and finally completes the heading alignment. Through the optimization of deep learning networks and multi-source data fusion, this solution realizes the high-precision identification and positioning of the nose landing gear in a complex environment, providing a reliable technical guarantee for automated towing operations.

[0137] Further, Figure 5 shows a specific step flowchart of the horizontal alignment in the docking positioning between a tractor and an aircraft wheel in an embodiment of the present invention. As Figure 5As shown in the figure, after the automated tractor completes the heading angle alignment, the improved YOLOv8 algorithm is used to accurately calculate and align the horizontal posture. The specific process is as follows: first, a high-resolution RGB image is collected and input into the optimized YOLOv8 network (using 1024×1024 input resolution, CloU loss function, and Anchor ratio adapted to the size of the front wheel) for detection to obtain a high-precision front wheel detection frame; then, the center point pixel position is calculated based on the coordinates of the four corner points of the detection frame, and compared and analyzed with the preset calibration points, and finally the horizontal relative posture of the tractor and the front wheel is obtained through perspective geometry calculation. Through algorithm optimization and multi-level geometric calculation, this solution achieves sub-pixel detection accuracy and millimeter-level posture alignment, ensuring the accuracy and reliability of automated traction operations.

[0138] Furthermore, through the ingenious coordination of the above steps, the precise positioning and alignment between the automated tractor and the aircraft wheels can be finally achieved, which can achieve the following effective effects:

[0139] (1) Through the collaborative work of lidar positioning, visual recognition and autonomous navigation technology, the whole process from environmental perception to final docking is automated. The intelligent algorithm can autonomously complete map construction, path planning and precise positioning, significantly reducing manual operation links, making the entire traction process more efficient and reliable, and greatly improving the automation level of airport ground operations.

[0140] (2) Based on the improved deep learning algorithm and multi-sensor fusion technology, the 3D position of the aircraft wheels can be accurately identified. Through the optimized point cloud registration and visual detection algorithms, sub-millimeter measurement accuracy is achieved, which fully meets the stringent accuracy requirements of the civil aviation field for towing operations and ensures the safety and reliability of the docking process.

[0141] (3) The system adopts an innovative anti-interference algorithm design and is equipped with all-weather working sensors, which can effectively cope with various complex weather conditions such as rain, snow, fog, haze, and strong light. The intelligent compensation algorithm can automatically correct the measurement errors caused by environmental factors, ensuring stable operation under different meteorological conditions, greatly expanding the application scenarios of the system.

[0142] (4) Adopt parallel computing architecture and algorithm optimization to achieve millisecond-level response. Through real-time operating system and hardware acceleration, high-frequency data collection and processing are maintained to meet the timeliness requirements of airport operations. Excellent real-time performance ensures smooth processes and timely response to emergencies. Intelligent task scheduling dynamically allocates resources to ensure that key tasks are executed first.

[0143] Embodiment 4

[0144] Figure 6 This is a schematic diagram of a docking positioning method and device between a tractor and an aircraft wheel provided in Embodiment 4 of the present invention.Figure 6 As shown, the device includes: a radar positioning and autonomous driving module 610, a depth vision perception module 620, a heading angle calculation module 630, a central pixel coordinate calculation module 640, and a horizontal pose adjustment module 650, where:

[0145] The radar positioning and autonomous driving module 610 is used to obtain the position description information of the vehicle in the point cloud map coordinate system according to the source point cloud data collected by the on-vehicle lidar sensor at the current position; and according to the position description information, combined with the use of static and dynamic path planning algorithms, control the tractor to move to the working starting point where the aircraft to be towed is located;

[0146] The depth vision perception module 620 is used to, when the tractor moves to the working starting point, fuse and calculate the front wheel point cloud information of the front landing gear to be towed according to the original color image and the original depth image containing the front landing gear captured by the on-vehicle depth camera;

[0147] The heading angle calculation module 630 is used to calculate the heading angle of the front landing gear to be towed relative to the tractor according to the front wheel point cloud information, so as to perform the heading angle alignment operation between the tractor and the front landing gear to be towed;

[0148] The central pixel coordinate calculation module 640 is used to calculate the current pixel coordinates of the center point of the front landing gear to be towed in the aligned color image according to the aligned color image containing the front landing gear to be towed collected by the on-vehicle depth camera after the heading angle alignment is completed;

[0149] The horizontal pose adjustment module 650 is used to solve the horizontal relative pose between the tractor and the front landing gear to be towed according to the current pixel coordinates and the preset calibration point pixel coordinates, so as to perform the horizontal alignment operation between the tractor and the front landing gear to be towed.

[0150] In the technical solution of the embodiment of the present invention, after using the vehicle-mounted lidar to collect environmental point cloud data in real time, by intelligently matching with a pre-constructed high-precision apron point cloud map, the three-dimensional pose information of the tractor in the global coordinate system is accurately obtained. Then, a hybrid algorithm combining static global path planning and dynamic local obstacle avoidance is adopted to intelligently plan the optimal driving route and real-time sense and avoid dynamic obstacles. After the tractor reaches the working location, the high-precision depth camera synchronously collects the color image and depth information of the front landing gear of the aircraft, and through the image processing algorithm for data fusion, a complete three-dimensional point cloud model of the front landing gear is constructed, the heading angle deviation of the front landing gear relative to the tractor is obtained, and the tractor is automatically controlled to adjust its orientation to achieve precise heading alignment. Finally, through the high-definition image obtained by the depth camera, the vision detection technology is used to locate the center coordinates of the front landing gear, and compared with the preset ideal calibration points, the pose deviation in the horizontal plane is calculated and the docking is realized. This method cleverly uses multi-sensor data fusion and hierarchical progressive control strategies to ensure the safety and reliability of the docking process through an intelligent processing flow from global positioning to local perception and from rough adjustment to fine alignment, and also greatly improves the operation efficiency and operation accuracy in complex weather environments, realizing the automatic upgrade of aircraft towing operations.

[0151] Based on the above embodiments, the radar positioning and autonomous driving module 610 is specifically used for:

[0152] Collect the source point cloud data of the surrounding environment at the current position through the vehicle-mounted lidar sensor, and obtain the target point cloud map established in advance for the airport apron area;

[0153] Through the nearest neighbor search algorithm, determine the local neighborhood of each source point cloud point in the source point cloud data, and calculate the local geometric attributes and FPFH corresponding to each source point cloud point according to the local neighborhood of each source point cloud point;

[0154] Evaluate the significance score of each source point cloud point according to the local geometric attributes of each source point cloud point, and identify the source ISS key points among the source point cloud points according to the significance scores;

[0155] Match the FPFH of each source ISS key point with the FPFH of each target ISS key point in the target point cloud map to obtain the paired points between the source point cloud data and the target point cloud map;

[0156] According to each paired point, calculate the transformation matrix from the source point cloud data to the target point cloud map, and according to the transformation matrix, obtain the current position coordinates and current attitude orientation of the vehicle in the point cloud map coordinate system.

[0157] Further, based on the above embodiments, the radar positioning and autonomous driving module 610 may include: a parameter setting unit and a denoising processing unit, where:

[0158] The parameter setting unit is configured to, before collecting the source point cloud data of the surrounding environment at the current position through the vehicle-mounted lidar sensor, obtain at least one meteorological data matching the current position in real time, and perform parameter setting on at least one sensor parameter of the vehicle-mounted lidar sensor according to each of the meteorological data; and

[0159] The denoising processing unit is configured to, after collecting the source point cloud data of the surrounding environment at the current position through the vehicle-mounted lidar sensor, determine the meteorological type according to each of the meteorological data, and obtain a denoising algorithm matching the meteorological type to perform denoising processing on the source point cloud data.

[0160] Based on the above embodiments, the depth vision perception module 620 is specifically configured to:

[0161] When the tractor moves to the working starting point, capture the original color image and the original depth image including the front landing gear through the vehicle-mounted depth camera;

[0162] Input the original color image into the pre-trained front wheel recognition model to segment the color sub-image matching the front wheel to be towed;

[0163] Fuse the segmented color sub-image with the original depth image to obtain the point cloud information of the front wheel to be towed;

[0164] Input the point cloud information of the front wheel to be towed into the pre-trained point cloud segmentation model to obtain the point cloud information of the front wheel of the front wheel to be towed.

[0165] Based on the above embodiments, the front wheel recognition model is trained based on the first improved YOLOv8 network, and the point cloud segmentation model is trained based on the improved PointNet++ network, where:

[0166] The first improved YOLOv8 network is constructed by adding a space-to-depth convolution module to the front ends of the third, sixth, ninth, and twelfth layers of the backbone network of the standard YOLOv8 network, and adding a squeeze-and-excitation attention mechanism module to the last layer of the backbone network of the standard YOLOv8 network;

[0167] The improved PointNet++ network is constructed by adding a self-attention mechanism module to the standard PointNet++ network, modifying the feature extraction network in the standard PointNet++ network into a graph convolution network, and adding different-scale sampling radii to different feature extraction modules in the feature aggregation method of the standard PointNet++ network; and

[0168] In the first target dataset for model training of the first improved YOLOv8 network, it includes the real dataset of the aircraft nose landing gear and the computer-aided design virtual dataset under various types of meteorological scenarios.

[0169] Based on the above embodiments, the central pixel coordinate calculation module 640 is specifically used for:

[0170] After completing the alignment of the heading angle, use the on-vehicle depth camera to collect the aligned color image containing the nose wheel to be towed;

[0171] Input the aligned color image into the pre-trained nose wheel bounding box detection model to obtain the target detection box calibrated in the aligned color image;

[0172] Among them, the nose wheel bounding box detection model only outputs a detection box with a single target size, and the target size is consistent with the size of the minimum circumscribed rectangle of the nose wheel to be towed photographed by the on-vehicle depth camera on the tractor at the standard position;

[0173] According to the pixel coordinates of the four corner points in the target detection box, calculate the current pixel coordinates of the center point of the nose wheel to be towed in the aligned color image.

[0174] Based on the above embodiments, the nose wheel bounding box detection model is trained based on the second improved YOLOv8 network, where:

[0175] After setting the default resolution in the standard YOLOv8 network to the target resolution and setting the loss function in the standard YOLOv8 network to the CloU function, the second improved YOLOv8 network is constructed;

[0176] In the second target dataset for model training of the second improved YOLOv8 network, it includes the real dataset of the aircraft nose landing gear and the computer-aided design virtual dataset under various types of meteorological scenarios, and each training image in the second target dataset has the same target image aspect ratio; and

[0177] The target resolution is consistent with the resolution of the image photographed by the on-vehicle depth camera on the tractor, and the target image aspect ratio is consistent with the image aspect ratio of the image photographed by the on-vehicle depth camera on the tractor.

[0178] The positioning device between the tractor and the aircraft wheel provided by the embodiments of the present invention can execute the positioning method between the tractor and the aircraft wheel provided by any embodiment of the present invention, and has the corresponding functional modules and beneficial effects for executing the method.

[0179] In the technical solution of the present disclosure, the processing of the user's personal information, such as collection, storage, use, processing, transmission, provision, and disclosure, complies with the provisions of relevant laws and regulations and does not violate public order and good customs.

[0180] Embodiment Five

[0181] Figure 7 Fig. shows a schematic structural diagram of a tractor 10 that can be used to implement the embodiments of the present invention. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0182] As Figure 7 shown, the autonomous tractor 10 includes: an on-vehicle lidar sensor 11 for collecting source point cloud data of the surrounding environment; an on-vehicle depth camera 12 for collecting color images and / or depth images of the surrounding environment;

[0183] The tractor 10 includes at least one lidar sensor 11, one depth camera sensor 12, and one processor 13, as well as a memory communicatively connected to at least one processor 13, such as a read-only memory (ROM) 14, a random access memory (RAM) 15, etc. Among them, the memory stores a computer program executable by at least one processor. The processor 13 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 14 or the computer program loaded from the storage unit 20 into the random access memory (RAM) 15. In the RAM 15, various programs and data required for the operation of the tractor 11 can also be stored. The processor 13, the ROM 14, and the RAM 15 are connected to each other through a bus 16. The input / output (I / O) interface 17 is also connected to the bus 16.

[0184] Multiple components in the tractor 10 are connected to the I / O interface 17, including: an input unit 18, such as a keyboard, a mouse, etc.; an output unit 19, such as various types of displays, speakers, etc.; a storage unit 20, such as a disk, an optical disc, etc.; and a communication unit 21, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 21 allows the tractor 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0185] The processor 13 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the processor 13 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The processor 13 executes the various methods and processes described above, such as a docking positioning method between a tractor and an aircraft wheel, that is:

[0186] Obtain the position description information of the vehicle itself in the point cloud map coordinate system according to the source point cloud data collected by the on-vehicle lidar sensor at the current position; and according to the position description information, combined with the use of static and dynamic path planning algorithms, control the tractor to move to the working starting point where the aircraft to be towed is located; when the tractor moves to the working starting point, fuse and calculate the front wheel point cloud information of the front landing gear to be towed according to the original color image and the original depth image of the front landing gear captured by the on-vehicle depth camera; according to the front wheel point cloud information, calculate the heading angle of the front landing gear to be towed relative to the tractor to perform a heading angle alignment operation between the tractor and the front landing gear to be towed; after completing the heading angle alignment, calculate the current pixel coordinates of the center point of the front landing gear to be towed in the aligned color image according to the aligned color image of the front landing gear to be towed collected by the on-vehicle depth camera; according to the current pixel coordinates and the preset calibrated point pixel coordinates, solve the horizontal relative pose between the tractor and the front landing gear to be towed to perform a horizontal alignment operation between the tractor and the front landing gear to be towed.

[0187] In some embodiments, the docking positioning method between the tractor and the aircraft wheel can be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as the storage unit 20. In some embodiments, part or all of the computer program can be loaded and / or installed onto the tractor 10 via the ROM 14 and / or the communication unit 21. When the computer program is loaded into the RAM 15 and executed by the processor 13, one or more steps of the docking positioning method between the tractor and the aircraft wheel described above can be executed. Alternatively, in other embodiments, the processor 13 can be configured to execute the docking positioning method between the tractor and the aircraft wheel in any other suitable manner (e.g., by means of firmware).

[0188] To provide interaction with a user, the systems and techniques described herein can be implemented on a tractor. The electronic device has: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) through which the user can provide input to the electronic device. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and the input received from the user can be in any form (including acoustic input, voice input, or tactile input).

[0189] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps recited in the present invention can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of the present invention can be achieved, and no limitation is imposed herein.

[0190] The above specific embodiments do not constitute a limitation on the protection scope of the present invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A method for docking and positioning a tractor and an aircraft wheel, characterized in that: The method is performed by an autonomous tractor, and includes: According to the source point cloud data collected by the on-board laser radar sensor at the current position, the position description information of the vehicle in the point cloud map coordinate system is obtained, and according to the position description information, the static and dynamic path planning algorithms are combined to control the tractor to move to the working starting point where the aircraft to be towed is located; When the tractor moves to the work starting point, the front wheel point cloud information of the front wheel to be towed is fused and calculated based on the original color image and original depth image including the front landing gear taken by the on-board depth camera; According to the front wheel point cloud information, the heading angle of the front wheel to be towed relative to the tractor is calculated to align the heading angle between the tractor and the front wheel to be towed; After the heading angle alignment is completed, the current pixel coordinates of the center point of the front wheel to be towed in the alignment color image are calculated according to the alignment color image including the front wheel to be towed acquired by the vehicle-mounted depth camera; According to the current pixel coordinates and the preset calibration point pixel coordinates, the horizontal relative posture between the tractor and the front wheel to be towed is solved to perform the horizontal alignment operation between the tractor and the front wheel to be towed.

2. The method according to claim 1, characterized in that According to the source point cloud data collected by the vehicle-mounted LiDAR sensor at the current position, the vehicle's position description information in the point cloud map coordinate system is obtained, including: The vehicle-mounted laser radar sensor collects source point cloud data of the surrounding environment at the current location, and obtains a pre-established target point cloud map for the airport apron area; The local neighborhood of each source point cloud point is determined in the source point cloud data through the nearest neighbor search algorithm, and the local geometric attributes and fast feature histogram FPFH corresponding to each source point cloud point are calculated according to the local neighborhood of each source point cloud point; According to the local geometric properties of each source point cloud point, the significance score of each source point cloud point is evaluated, and according to each significance score, the source intrinsic shape feature descriptor ISS key point is identified in each source point cloud point; Match the FPFH of each source ISS key point with the FPFH of each target ISS key point in the target point cloud map to obtain each pairing point between the source point cloud data and the target point cloud map; According to each paired point, the transformation matrix from the source point cloud data to the target point cloud map is calculated, and according to the transformation matrix, the current position coordinates and current posture orientation of the vehicle in the point cloud map coordinate system are obtained.

3. The method according to claim 2, characterized in that Before collecting source point cloud data of the surrounding environment at the current position through the on-board laser radar sensor, it also includes: Acquire at least one item of meteorological data matching the current location in real time, and set at least one sensor parameter of the vehicle-mounted laser radar sensor according to each of the meteorological data; and After collecting the source point cloud data of the surrounding environment at the current position through the vehicle-mounted LiDAR sensor, it also includes: The meteorological type is determined according to each of the meteorological data, and a denoising algorithm matching the meteorological type is obtained to perform denoising processing on the source point cloud data.

4. The method according to claim 1, characterized in that: When the tractor moves to the starting point of the work, the front wheel point cloud information of the front wheel to be towed is calculated based on the original color image and original depth image including the front landing gear taken by the on-board depth camera, including: When the tractor moves to the work starting point, the original color image and original depth image containing the front landing gear are captured by the on-board depth camera; Input the original color image into the pre-trained front wheel recognition model to segment the color sub-image matching the front wheel to be towed; The segmented color sub-image is fused with the original depth image to obtain the point cloud information of the front wheel to be towed; The point cloud information of the front wheel to be towed is input into a pre-trained point cloud segmentation model to obtain the point cloud information of the front wheel to be towed.

5. The method according to claim 4, characterized in that The front wheel recognition model is obtained based on the first improved YOLOv8 network training, and the point cloud segmentation model is obtained based on the improved PointNet++ network training, where: The first improved YOLOv8 network is constructed by adding spatial to depth convolution modules to the front end of the third, sixth, ninth and twelfth layers of the backbone network of the standard YOLOv8 network, and adding compression and incentive attention mechanism modules to the last layer of the backbone network of the standard YOLOv8 network; The improved PointNet++ network is constructed by adding a self-attention mechanism module to the standard PointNet++ network, modifying the feature extraction network in the standard PointNet++ network to a graph convolutional network, and adding sampling radii of different scales to different feature extraction modules in the feature aggregation method in the standard PointNet++ network; and The first target data set used for model training of the first improved YOLOv8 network includes real data sets and computer-aided design virtual data sets of aircraft front landing gear in various types of weather scenarios.

6. The method according to claim 1, characterized in that After the heading angle alignment is completed, the current pixel coordinates of the center point of the front wheel to be towed in the alignment color image collected by the vehicle-mounted depth camera and including the front wheel to be towed are calculated, including: After completing the heading angle alignment, the on-board depth camera is used to collect an aligned color image containing the front wheels to be towed; Inputting the alignment color image into a pre-trained front wheel frame detection model to obtain a target detection frame calibrated in the alignment color image; Among them, the front wheel frame detection model only outputs a detection frame of a single target size, and the target size is consistent with the size of the minimum circumscribed rectangle of the front wheel to be towed photographed by the on-board depth camera on the tractor at the standard position; According to the pixel coordinates of the four corner points in the target detection frame, the current pixel coordinates of the center point of the front wheel to be towed in the alignment color image are calculated.

7. The method according to claim 6, characterized in that The front wheel frame detection model is trained based on the second improved YOLOv8 network, where: After setting the default resolution in the standard YOLOv8 network to the target resolution and setting the loss function in the standard YOLOv8 network to the CloU function, a second improved YOLOv8 network is constructed; A second target data set used for model training of a second improved YOLOv8 network includes a real data set of aircraft nose landing gear in various types of weather scenes and a computer-aided design virtual data set, and each training image in the second target data set has the same target image aspect ratio; and The target resolution is consistent with the resolution of the image captured by the on-board depth camera on the tractor, and the target image aspect ratio is consistent with the image aspect ratio of the image captured by the on-board depth camera on the tractor.

8. A docking and positioning device between a tractor and an aircraft wheel, characterized in that: Configured in an autonomous driving tractor, the device comprises: The radar positioning automatic driving module is used to obtain the location description information of the vehicle in the point cloud map coordinate system according to the source point cloud data collected by the on-board laser radar sensor at the current position, and control the tractor to move to the working starting point where the to-be-towed aircraft is located based on the location description information by combining static and dynamic path planning algorithms; The depth vision perception module is used to fuse and calculate the front wheel point cloud information of the front wheel to be towed based on the original color image and original depth image including the front landing gear taken by the on-board depth camera when the tractor moves to the work starting point; The heading angle calculation module is used to calculate the heading angle of the front wheel to be towed relative to the tractor according to the front wheel point cloud information, so as to align the heading angle between the tractor and the front wheel to be towed; The center pixel coordinate calculation module is used to calculate the current pixel coordinates of the center point of the front wheel to be towed in the alignment color image after the heading angle alignment is completed, based on the alignment color image containing the front wheel to be towed acquired by the vehicle-mounted depth camera; The horizontal posture adjustment module is used to solve the horizontal relative posture between the tractor and the front wheel to be towed according to the current pixel coordinates and the preset calibration point pixel coordinates, so as to perform the horizontal alignment operation between the tractor and the front wheel to be towed.

9. An automatic driving tractor, characterized in that: The tractor comprises: On-vehicle LiDAR sensor, used to collect source point cloud data of the surrounding environment; A vehicle-mounted depth camera, used to collect color images and / or depth images of the surrounding environment; at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the docking and positioning method between the tractor and the aircraft wheel according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the docking and positioning method between a tractor and an aircraft wheel according to any one of claims 1 to 7 when executed.

Images