Docking and positioning method between a tractor and an aircraft wheel, tractor and medium

Abstract

The present invention discloses a docking and positioning method between a tractor and an aircraft wheel, a tractor, and a medium. The method includes: obtaining the vehicle's position description information based on source point cloud data collected by an on-board laser radar sensor to control the tractor to move to the aircraft to be towed; calculating the front wheel point cloud information of the front wheel to be towed based on the color image and depth image captured by the on-board depth camera to calculate the heading angle of the front wheel to be towed relative to the tractor, thereby achieving a heading angle alignment operation; calculating the current pixel coordinates of the center point of the front wheel to be towed in the alignment color image captured by the on-board depth camera and containing the front wheel to be towed, and solving the horizontal relative posture between the tractor and the front wheel to be towed to perform a horizontal alignment operation between the tractor and the front wheel to be towed. The technical solutions of the embodiments of the present invention improve the docking and positioning accuracy, environmental adaptability, and degree of automation in towing operations.

Description

Technical Field

[0001] The present invention belongs to the technical field of aircraft towing operations, and in particular relates to a docking and positioning method between a tractor and aircraft wheels, a tractor, and a medium. Background Art

[0002] When manually operating a tractor, docking accuracy is limited by a variety of factors, including personal experience, skills, reaction speed, and visual judgment in complex weather conditions (such as rain, snow, fog, and strong sunlight). This can lead to misalignment between the tractor and the aircraft's wheels. In the case of special aircraft or confined parking spaces, insufficient operational precision significantly reduces docking success rates. Furthermore, traditional manual operation struggles to meet the real-time demands of modern airport operations. Manual operation efficiency is particularly low in low visibility or extreme weather conditions.

[0003] In existing technologies, aircraft tugs are manually driven and require at least two guides to complete the docking through visual collaboration. This model has the problems of high manpower consumption and low operating accuracy, which can easily lead to aircraft wear and flight delays. Since it relies on manual judgment of aircraft parameters and real-time posture, not only is the training cost high, but the risk of error is significantly increased under complex working conditions. In addition, although some improvement plans have attempted to equip tugs with sensors such as infrared or laser ranging, these sensors are easily interfered with by environmental factors and have unstable measurement accuracy, which makes them unreliable in actual tug operations. Summary of the Invention

[0004] The embodiments of the present invention provide a method for docking and positioning between a tractor and aircraft wheels, a tractor, and a medium, which can quickly and accurately achieve docking and positioning between an autonomously driven tractor and the front landing gear of an aircraft to be towed in various complex scenarios.

[0005] According to one aspect of an embodiment of the present invention, a method for docking and positioning a tractor with an aircraft wheel is provided, the method being performed by an autonomously driven tractor, the method comprising:

[0006] Based on the source point cloud data collected by the on-board lidar sensor at the current position, the vehicle's position description information in the point cloud map coordinate system is obtained. Based on this position description information, a combination of static and dynamic path planning algorithms is used to control the tractor to move to the starting point where the aircraft to be towed is located.

[0007] When the tractor moves to the work starting point, the front wheel point cloud information of the front wheel to be towed is calculated based on the original color image and original depth image including the front landing gear taken by the on-board depth camera;

[0008] Calculate the heading angle of the to-be-towed front wheel relative to the towing vehicle based on the front wheel point cloud information, so as to align the heading angles of the towing vehicle and the front wheel;

[0009] After completing the heading angle alignment, the current pixel coordinates of the center point of the front wheel to be towed in the alignment color image captured by the vehicle-mounted depth camera and including the front wheel to be towed are calculated;

[0010] According to the current pixel coordinates and the preset calibration point pixel coordinates, the horizontal relative position between the tractor and the front wheel to be towed is solved to perform the horizontal alignment operation between the tractor and the front wheel to be towed.

[0011] According to another aspect of an embodiment of the present invention, there is provided a docking and positioning device between a tractor and an aircraft wheel, which is configured in an autonomous tractor, and includes:

[0012] The radar positioning autopilot module is used to obtain the vehicle's position description information in the point cloud map coordinate system based on the source point cloud data collected by the on-board lidar sensor at the current position. Based on this position description information, it combines static and dynamic path planning algorithms to control the tractor to move to the working starting point where the aircraft to be towed is located.

[0013] The depth vision perception module is used to fuse and calculate the front wheel point cloud information of the front wheel to be towed based on the original color image and original depth image including the front landing gear taken by the on-board depth camera when the towing vehicle moves to the work starting point;

[0014] The heading angle calculation module is used to calculate the heading angle of the front wheel to be towed relative to the tractor based on the front wheel point cloud information, so as to align the heading angle between the tractor and the front wheel to be towed;

[0015] A center pixel coordinate calculation module is used to calculate the current pixel coordinates of the center point of the front wheel to be towed in the alignment color image captured by the vehicle-mounted depth camera after the heading angle alignment is completed;

[0016] The horizontal posture adjustment module is used to solve the horizontal relative posture between the tractor and the front wheel to be towed based on the current pixel coordinates and the preset calibration point pixel coordinates, so as to perform the horizontal alignment operation between the tractor and the front wheel to be towed.

[0017] According to another aspect of an embodiment of the present invention, there is further provided an autonomous driving tractor, the tractor comprising:

[0018] On-vehicle lidar sensor, used to collect source point cloud data of the surrounding environment;

[0019] An onboard depth camera, used to capture color images and / or depth images of the surrounding environment;

[0020] at least one processor; and

[0021] a memory communicatively connected to the at least one processor; wherein,

[0022] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the docking and positioning method between a tractor and an aircraft wheel as described in any one of the embodiments of the present invention.

[0023] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the docking and positioning method between a tractor and an aircraft wheel as described in any one of the embodiments of the present invention when executed.

[0024] The technical solution of the embodiment of the present invention uses a vehicle-mounted lidar to collect environmental point cloud data in real time, and then intelligently matches it with a pre-built high-precision apron point cloud map to accurately obtain the three-dimensional pose information of the tractor in the global coordinate system. Then, a hybrid algorithm combining static global path planning and dynamic local obstacle avoidance is used to intelligently plan the optimal driving route and perceive and avoid dynamic obstacles in real time. After the tractor arrives at the work site, a high-precision depth camera synchronously collects color images and depth information of the aircraft's front landing gear. The data is fused through an image processing algorithm to construct a complete three-dimensional point cloud model of the front wheels, and the relative position of the front wheels to the tractor is obtained. The system detects heading angle deviation and automatically controls the tractor to adjust its orientation to achieve precise heading alignment. Finally, through the high-definition image obtained by the depth camera, the visual detection technology is used to locate the center coordinates of the front wheel and compare it with the preset ideal calibration point. The posture deviation in the horizontal plane is calculated and the docking method is realized. The system cleverly uses multi-sensor data fusion and hierarchical progressive control strategies to implement an intelligent processing flow from global positioning to local perception, from rough adjustment to fine alignment. It not only ensures the safety and reliability of the docking process, but also greatly improves the operating efficiency and operation accuracy in various complex docking operation scenarios, realizing the automation upgrade of aircraft towing operations.

[0025] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present invention, nor is it intended to limit the scope of the present invention. Other features of the present invention will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0027] Figure 1 This is a flow chart of a method for docking and positioning a tractor and an aircraft wheel according to a first embodiment of the present invention;

[0028] Figure 2 This is a flow chart of another method for docking and positioning between a tractor and an aircraft wheel provided according to a second embodiment of the present invention;

[0029] Figure 3 This is a flowchart of specific steps of radar positioning and automatic navigation in docking positioning between a tractor and an aircraft wheel provided in accordance with the third embodiment of the present invention;

[0030] Figure 4 This is a flowchart of specific steps for aligning the heading angle in docking positioning between a tractor and an aircraft wheel according to a third embodiment of the present invention;

[0031] Figure 5 This is a flowchart of specific steps for horizontal alignment in docking positioning between a tractor and an aircraft wheel according to a third embodiment of the present invention;

[0032] Figure 6 2 is a schematic structural diagram of a method and apparatus for docking and positioning a tractor and an aircraft wheel according to a fourth embodiment of the present invention;

[0033] Figure 7 The present invention is a schematic structural diagram of a tractor for implementing a method for docking and positioning a tractor with aircraft wheels according to an embodiment of the present invention. DETAILED DESCRIPTION

[0034] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0035] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0036] Example 1

[0037] Figure 1 This is a flow chart of a method for docking and positioning a tractor and an aircraft wheel, provided in accordance with a first embodiment of the present invention. This embodiment is applicable to situations where an autonomous tractor and an aircraft wheel are used for automated docking and positioning. The method can be performed by a docking and positioning device between the tractor and the aircraft wheel. The docking and positioning device between the tractor and the aircraft wheel can generally be configured in an autonomous tractor and executed by a controller in the tractor.

[0038] Correspondingly, such as Figure 1 As shown, the method includes:

[0039] S110. Obtain the location description information of the vehicle in the point cloud map coordinate system based on the source point cloud data collected by the vehicle-mounted laser radar sensor at the current position, and control the tractor to move to the working starting point where the aircraft to be towed is located by combining static and dynamic path planning algorithms based on the location description information.

[0040] In an embodiment of the present invention, in order to more accurately and efficiently move the tractor to the work starting point where the aircraft to be towed is located, it is first necessary to accurately determine the real-time position of the tractor in the airport, that is, to accurately locate the coordinate position of the tractor in the entire airport point cloud map.

[0041] Due to the special working environment requirements of airports (such as the presence of large metal structures, dynamic obstacles, and strict safety requirements), traditional location detection methods have obvious limitations: GPS (Global Positioning System) signals are easily blocked by buildings and aircraft fuselages, resulting in positioning drift; UWB (Ultra-Wideband) base stations are complex to deploy and susceptible to interference; and visual marker recognition is not reliable in bad weather or at night.

[0042] In contrast, vehicle-mounted LiDAR sensors, through real-time collection of high-density 3D point cloud data and matching it with pre-built, high-precision point cloud maps of airports, offer an ideal location detection solution for airport environments thanks to their active detection, high precision, and strong anti-interference capabilities. This point cloud matching-based solution not only overcomes environmental interference but also achieves centimeter-level positioning accuracy, effectively addressing the shortcomings of traditional methods and ensuring that tractors can efficiently and safely reach their designated starting points, meeting the stringent requirements of aircraft towing operations.

[0043] Accordingly, the tractor can be located anywhere on the airport before performing an aircraft towing and docking mission. When the tractor needs to perform the towing and docking mission, it can use its fixed onboard lidar sensor to emit a laser beam into the surrounding environment and receive the reflected signal, thereby acquiring source point cloud data of the surrounding environment. Based on this source point cloud data, the tractor can then accurately obtain the vehicle's position coordinates and orientation in the point cloud map coordinate system as a location description.

[0044] After the tractor obtains its own location description information, combined with the work starting point of the towed aircraft, it can use the location description information as the path planning starting point and the work starting point as the path planning end point to plan a driving path according to the preset path planning algorithm, so that the tractor moves along the driving path to the work starting point to complete the aircraft towing task.

[0045] Among them, considering the complexity and dynamic nature of the airport environment, static and dynamic path planning algorithms can be used in combination. First, a static path is planned. Then, while the tractor moves along the static path, the static path is dynamically optimized in real time based on the surrounding environmental information.

[0046] In an optional implementation of this embodiment, after the tractor acquires precise pose information (i.e., position description information) from its lidar sensor, it can employ a hybrid path planning strategy combining A* and D* to achieve optimal navigation. The A* algorithm, based on heuristic search principles, efficiently plans a globally optimal path in a static environment, guiding the search direction by comprehensively evaluating path costs and heuristic functions. Meanwhile, the D* algorithm utilizes a reverse search and incremental update mechanism to rapidly adjust the local path when the lidar detects an unexpected obstacle. This "static global planning + dynamic local optimization" approach ensures path optimality while enabling real-time response to environmental changes. This approach significantly improves planning efficiency while significantly reducing obstacle avoidance response time, perfectly meeting the dual navigation accuracy and real-time requirements of airport towing operations.

[0047] S120. When the tractor moves to the work starting point, the front wheel point cloud information of the front wheel to be towed is calculated by fusing the original color image and the original depth image including the front landing gear captured by the vehicle-mounted depth camera.

[0048] In an embodiment of the present invention, the tractor uses an onboard RGB-D camera (i.e., a depth camera) to synchronously capture the original color image and original depth image of the front landing gear of the towed aircraft to achieve multimodal data fusion.

[0049] The original color image provides rich texture features for target recognition, while the original depth image provides accurate 3D spatial information. After the two are aligned in time and space using camera calibration parameters, they jointly construct the front wheel point cloud information as a high-precision point cloud model.

[0050] This fusion sensing solution offers significant advantages: First, it maintains stable recognition performance in both strong and low-light conditions. Color information compensates for the lack of depth data, while depth information corrects for the effects of illumination on vision. Second, data complementation effectively enhances perception robustness in complex environments, providing reliable three-dimensional environmental representation for subsequent precise docking. Compared to single-sensor solutions, this technology significantly improves the system's adaptability and reliability under the unique operating conditions of airports.

[0051] In an optional implementation of this embodiment, when the tractor moves to the work starting point, the front wheel point cloud information of the front wheel to be towed is calculated by fusing the original color image and the original depth image including the front landing gear captured by the vehicle-mounted depth camera, which may include:

[0052] When the tractor moves to the work starting point, the on-board depth camera captures the original color image and original depth image containing the front landing gear. The original color image is input into a pre-trained front wheel recognition model to segment a color sub-image matching the front wheel to be towed. The segmented color sub-image is fused with the original depth image to obtain the point cloud information of the front wheel to be towed. The point cloud information of the front wheel to be towed is input into the pre-trained point cloud segmentation model to obtain the front wheel point cloud information of the front wheel to be towed.

[0053] In this optional embodiment, a front wheel recognition model for two-dimensional image segmentation can be first trained to identify and segment a color sub-image that matches the front wheel to be towed that is required for towing and docking in the towed aircraft in the two-dimensional original color image.

[0054] After obtaining the pixel position of the color sub-image in the original color image, the depth information of each pixel at the same pixel position in the original depth image can be obtained, and the depth information of each pixel is fused with the RGB information of each pixel in the color sub-image to obtain the point cloud information of the front wheel to be towed.

[0055] Considering that the point cloud information of the front wheel to be towed is extracted based on the results of two-dimensional image recognition, the resolution or accuracy of the point cloud information of the front wheel to be towed may be relatively rough. Therefore, the inventors further trained a point cloud segmentation model for three-dimensional point cloud segmentation to perform refined three-dimensional segmentation of the point cloud information of the front wheel to be towed, and obtain front wheel point cloud information that accurately matches the actual front wheel to be towed.

[0056] Based on the above embodiments, after the tractor obtains an aircraft towing task, it can simultaneously obtain model description information of the aircraft to be towed. Based on this model description information, the tractor can obtain standard nose wheel images of this model of aircraft under different standardized environmental parameters (visibility, temperature, lighting, etc.) from a pre-built two-dimensional image database. Then, combined with the current real-time environmental parameters, the tractor obtains a most suitable target standard nose wheel image, and inputs this target standard nose wheel image and the original color image into the nose wheel recognition model to obtain a more accurate color sub-image segmentation.

[0057] Similarly, based on the model description information, the tractor can also obtain the standard front wheel point cloud data of the aircraft model under different standardized environmental parameters (visibility, temperature, lighting, etc.) from the pre-built three-dimensional point cloud database. Then, combined with the current real-time environmental parameters, the most suitable target standard front wheel point cloud data is obtained, and the target standard front wheel point cloud data and the fused front wheel point cloud information to be towed are input into the point cloud segmentation model to obtain more accurate front wheel point cloud information.

[0058] Based on the above embodiments, the front wheel recognition model can be obtained based on the first improved YOLOv8 network training, and the point cloud segmentation model can be obtained based on the improved PointNet++ network training, wherein:

[0059] The first improved YOLOv8 network is constructed by adding Space-to-Depth Convolution (SPD-Conv) modules to the front end of the third, sixth, ninth, and twelfth layers of the standard YOLOv8 backbone network, and adding a Squeeze-and-Excitation Attention (SE Attention) module to the last layer of the standard YOLOv8 backbone network.

[0060] The improved PointNet++ network is constructed by adding a self-attention mechanism module to the standard PointNet++ network, modifying the feature extraction network in the standard PointNet++ network to a graph convolutional network, and adding sampling radii of different scales to different feature extraction modules in the feature aggregation method in the standard PointNet++ network; and

[0061] The first target data set used for model training of the first improved YOLOv8 network includes real data sets and computer-aided design virtual data sets of aircraft nose landing gear in various types of weather scenarios.

[0062] Correspondingly, the original color image can be obtained through the RGB-D camera and input into the improved YOLOv8 algorithm network to identify the front wheel, and the point cloud information can be obtained through the original depth image. The color point cloud is obtained by fusing the RGB and depth point clouds, that is, the point cloud information of the front wheel to be towed.

[0063] The original color image collected synchronously by the RGB-D camera is used to input into the improved YOLOv8 network, and the original depth image is used to generate point cloud information. The YOLOv8 algorithm used by the standard YOLOv8 network is a single-stage target detection algorithm with strong time-efficiency. By adding the SPD-Conv module to the third, sixth, ninth, and twelfth layers of the standard YOLOv8 network backbone, the input size can be converted to The feature map X is sliced in the spatial dimension, S represents the length and width of the feature map, Represents the number of channels of the input feature map, which is divided into The size is The scale represents the downsampling factor, and its size determines the downsampling ratio of the feature map by the SPD-Conv module. In order to preserve the information of the feature map as much as possible, the SPD layer splices the segmented sub-feature maps along the channel dimension to generate a sub-feature map of size Sub-feature map of , in the intermediate feature map Perform non-step convolution operations and finally output a size The output feature map of , Represents the number of channels of the output feature map.

[0064] Furthermore, after adding the SE Attention module to the final layer of the standard YOLOv8 network's backbone, the SE Attention module's squeeze operation performs global average pooling on the output feature map, reducing the eigenvalues of each channel to a global vector and capturing the global information of each channel. Subsequently, the SE Attention module uses an Excitation operation, consisting of two fully connected layers, a ReLU activation function, and a Softmax activation function, to perform dimensionality reduction and then dimensionality increase. The SE Attention module uses the sigmoid function to generate a weight vector, ensuring that its sum is 1. Finally, the SE Attention module uses a Scale operation to multiply the channel attention weights obtained in the previous step by the original input feature map, adjusting the eigenvalues of each channel, emphasizing the information of important channels and suppressing unimportant information.

[0065] The first improved YOLOv8 network was developed by improving the standard YOLOv8 network. First, the SPD-Conv module was used to perform spatial segmentation and channel concatenation, effectively preserving the features of small objects and addressing the problem of small object feature loss caused by traditional downsampling. Furthermore, the SE Attention module was used to perform attention weighting, enhancing the representation of key features. This improvement significantly improved the algorithm's detection accuracy for nose landing gear in complex airport environments, particularly in challenging scenarios such as target occlusion and changing lighting.

[0066] To construct a more robust first target dataset for training the first improved YOLOv8 network, various embodiments of the present invention further employ a multi-source data fusion strategy. First, real image data of nose landing gear from different airports under various weather conditions (sunny, rainy, snowy, and foggy) and lighting environments (daytime, nighttime, and backlighting) is collected. Second, a virtual dataset containing different aircraft models and poses is generated through computer-aided design (CAD) modeling. Physically based rendering (PBR) is employed to simulate various complex environmental effects. The Kmeans++ algorithm is used to optimize the size distribution of anchor boxes, and data augmentation techniques (such as geometric transformation, noise injection, and simulated weather effects) are employed to further expand data diversity.

[0067] This multi-source data fusion strategy significantly improves the model's generalization capabilities in extreme weather conditions, addresses extreme conditions that are difficult to capture with real-world data, and ensures data consistency through domain adaptation training. This rich dataset not only enhances the detection robustness of the YOLOv8 network but also lays a solid foundation for subsequent point cloud segmentation and pose estimation modules, enabling this solution to adapt to the complex operating environments of various airports.

[0068] Similarly, in order to further improve the segmentation accuracy of the front wheel point cloud information, an embodiment of the present invention trains a point cloud segmentation model based on an improved PointNet++ network obtained by improving the standard PointNet++ network.

[0069] Specifically, the inventors made three key improvements to address the limitations of the traditional PointNet++ network in complex scenarios: first, they introduced a self-attention mechanism (implemented through PyTorch) to enable the PointNet++ network to dynamically focus on the key geometric features of the front landing gear; second, they replaced the original feature extraction network in the PointNet++ network with a graph convolutional network (GCN), using the graph structure to better model the topological relationship of local point sets in the point cloud; finally, they improved the feature aggregation method in the PointNet++ network and set multi-scale sampling radius in each layer of the feature extraction module to simultaneously capture local features at different granularity levels.

[0070] Through multiple experiments, the inventors found that these improvements significantly enhance the PointNet++ network's performance in segmenting nose landing gear point clouds in complex airport environments: the self-attention mechanism enhances the recognition of key features, the GCN architecture improves the representation of local geometric features, and multi-scale sampling ensures the complete extraction of structures of varying sizes. The improved model maintains real-time performance while achieving improved segmentation accuracy, demonstrating enhanced robustness in challenging scenarios such as partial wheel occlusion and strong light reflection.

[0071] S130 . Calculate the heading angle of the front wheel to be towed relative to the tractor based on the front wheel point cloud information, so as to align the heading angles of the tractor and the front wheel to be towed.

[0072] In this embodiment of the present invention, the heading angle is the horizontal angle between the longitudinal axis of the tractor and the main axis of the front wheel. It is a core parameter that describes the relative spatial orientation of the two. The heading angle directly determines the alignment accuracy between the tractor and the aircraft's front wheel. A heading deviation exceeding 1° can result in docking failure or equipment collision risk.

[0073] It is understood that after acquiring front wheel point cloud information using a fixed depth camera mounted on the tractor, the heading angle of the towed front wheel relative to the tractor can be accurately identified based on this front wheel point cloud information. Based on the calculated heading angle, the tractor can be controlled to rotate in a direction that eliminates this heading angle, ultimately achieving heading alignment between the tractor and the towed front wheel.

[0074] Optionally, based on the above embodiments, before calculating the heading angle of the front wheel to be towed relative to the tractor based on the front wheel point cloud information, the following steps may be further included:

[0075] The nose wheel point cloud information obtained after segmentation is denoised based on a statistical outlier removal algorithm. By analyzing the distance distribution characteristics within the neighborhood of each point, isolated noise points and abnormal measurement values are intelligently filtered out. Subsequently, the moving least squares method (MLS) is used for smooth surface fitting to achieve smooth optimization of the point cloud surface while maintaining the original geometric features.

[0076] It is understandable that the denoising and smoothing processes provide significant advantages to the embodiments of the present invention: statistical denoising effectively eliminates outliers caused by sensor noise and environmental interference, improving data quality; MLS smoothing maintains the key geometric features of the wheel while repairing surface discontinuities caused by occlusion or measurement errors. The processed high-precision nose wheel point cloud is converted to the camera coordinate system, providing not only a more accurate three-dimensional spatial representation but also significantly improving the accuracy and stability of subsequent heading angle calculations. This enables the system to achieve reliable heading alignment even in complex operating environments, laying a solid foundation for automated towing operations.

[0077] S140 . After the heading angle alignment is completed, the current pixel coordinates of the center point of the front wheel to be towed in the alignment color image captured by the vehicle-mounted depth camera and including the front wheel to be towed are calculated.

[0078] In the embodiment of the present invention, after the heading angle alignment is completed, the onboard depth camera on the tractor is controlled again to capture a new color image as the alignment color image.

[0079] In an optional implementation of this embodiment, based on the aligned color image, an improved morphological image processing technique may be used to calculate the pixel coordinates of the center point of the front wheel to be towed.

[0080] Specifically, an adaptive threshold is first used to segment and extract the wheel region from the aligned color image. Then, a Canny edge detection and least-squares circle fitting algorithm are combined to determine the wheel hub center position with sub-pixel accuracy. This technical solution leverages the orthographic angle of the heading-aligned image to effectively eliminate the effects of perspective distortion, enabling precise positioning under complex lighting conditions and providing an accurate 2D projection reference for calculating the 2D pixel coordinates of the center point.

[0081] S150. Calculate the horizontal relative position between the tractor and the front wheel to be towed based on the current pixel coordinates and the preset pixel coordinates of the calibration point, so as to perform a horizontal alignment operation between the tractor and the front wheel to be towed.

[0082] In this embodiment, after obtaining the current pixel coordinates of the center point of the front wheel to be towed in the alignment color image, the current pixel coordinates can be compared with the pre-calibrated pixel coordinates of the calibration point to solve the horizontal relative position between the tractor and the front wheel to be towed.

[0083] The pixel coordinates of the calibration point can be understood as the pixel position of the center point of the front wheel of the to-be-towed aircraft in a color image acquired by a fixed depth camera at the current position after the towing vehicle reaches the working starting point where the to-be-towed aircraft is located and achieves heading angle alignment and horizontal alignment with the to-be-towed aircraft.

[0084] After obtaining the current pixel coordinates, the horizontal relative position between the tractor and the front wheel of the aircraft to be towed in free space can be calculated by comparing the pixel position difference between the current pixel coordinates and the pixel coordinates of the preset calibration points. Subsequently, by controlling the tractor to move in a direction that eliminates this horizontal relative position, horizontal alignment between the tractor and the front wheel of the aircraft to be towed can be achieved. After achieving this horizontal alignment, the tractor can be controlled to accurately dock with the front wheel of the aircraft to be towed, thereby completing towing control of the aircraft to be towed.

[0085] The technical solution of the embodiment of the present invention uses a vehicle-mounted laser radar to collect environmental point cloud data in real time, and then intelligently matches it with a pre-built high-precision apron point cloud map to accurately obtain the three-dimensional pose information of the tractor in the global coordinate system. Then, a hybrid algorithm combining static global path planning and dynamic local obstacle avoidance is used to intelligently plan the optimal driving route and perceive and avoid dynamic obstacles in real time. After the tractor arrives at the work site, a high-precision depth camera synchronously collects color images and depth information of the aircraft's front landing gear. The data is fused through an image processing algorithm to construct a complete three-dimensional point cloud model of the front wheels, and the front wheels relative to the tractor are obtained. The system can detect the heading angle deviation of the vehicle and automatically control the tractor to adjust its orientation to achieve precise heading alignment. Finally, the high-definition image obtained by the depth camera is used to locate the center coordinates of the front wheel using visual detection technology, and compared with the preset ideal calibration point to calculate the posture deviation in the horizontal plane and realize docking. It cleverly uses multi-sensor data fusion and hierarchical progressive control strategies through an intelligent processing flow from global positioning to local perception, from rough adjustment to fine alignment. It not only ensures the safety and reliability of the docking process, but also greatly improves the operating efficiency and operation accuracy in complex weather environments, realizing the automation upgrade of aircraft towing operations.

[0086] Example 2

[0087] Figure 2 This is a flowchart of a docking and positioning method for a tractor and an aircraft wheel, provided in Example 2 of the present invention. This example is an optimization of the previous examples. This example specifically details the operations of "obtaining a vehicle's position description in the point cloud map coordinate system from the source point cloud data collected by the vehicle-mounted lidar sensor at its current location" and "calculating the current pixel coordinates of the center point of the towed front wheel within the alignment color image."

[0088] Correspondingly, such as Figure 2 As shown, the method includes:

[0089] S210 , collecting source point cloud data of the surrounding environment at the current position through the vehicle-mounted laser radar sensor, and obtaining a pre-established target point cloud map for the airport apron area.

[0090] When a tractor receives an aircraft towing command at any location in the airport, it can directly activate its onboard LiDAR sensor to collect source point cloud data of the surrounding environment at the current location. The source point cloud data can be point cloud data collected by the onboard LiDAR sensor within a set angle range, or point cloud data collected by the onboard LiDAR sensor in a 360° omnidirectional direction, which is not limited in this embodiment.

[0091] Specifically, to accurately locate the tractor based on the source point cloud data, it is necessary to further combine it with a target point cloud map that matches the airport apron area. By using specialized LiDAR equipment to collect data across the entire airport apron area, this target point cloud map can be accurately constructed. Furthermore, the tractor can be precisely located by registering the source point cloud data with the target point cloud map.

[0092] In an optional implementation of this embodiment, before collecting source point cloud data of the surrounding environment at the current position by the vehicle-mounted laser radar sensor, the following steps may also be included:

[0093] At least one item of meteorological data matching the current location is acquired in real time, and at least one sensor parameter of the vehicle-mounted lidar sensor is set according to each item of meteorological data.

[0094] Through research, the inventors discovered that the data collection accuracy of on-board LiDAR sensors varies depending on different weather conditions. For example, raindrops or snowflakes can reflect the laser beam, generating noise points and reducing the quality of the point cloud data. Particulate matter in haze can scatter the laser beam, resulting in sparse or inaccurate point cloud data. Strong sunlight can interfere with the LiDAR receiver, reducing signal strength. Dust or sand can reflect the laser beam, generating false point clouds. Based on this, to improve the accuracy of the point cloud data collected by the on-board LiDAR sensor and, by extension, the accuracy of the tractor's positioning, it is considered to configure at least one sensor parameter of the on-board LiDAR sensor in conjunction with real-time weather data at the tractor's location.

[0095] The meteorological data may include: temperature, weather type (sunny, rainy, snowy, foggy, dusty, etc.), and degree parameters of each weather type (such as light intensity, rainfall, snowfall, foggy, dusty, etc.). The sensor parameters may include: the LiDAR sensor's signal transmission power, operating band, scanning frequency, resolution, and point cloud density.

[0096] Specifically, different sensor parameters of the lidar sensor can be used to conduct actual data acquisition tests under different types of meteorological data. Furthermore, based on the acquisition test results, the optimal sensor parameter configuration for various types of meteorological data can be clustered. After obtaining at least one item of meteorological data that matches the current location in real time, the cluster to which the real-time meteorological data belongs is determined. Furthermore, the optimal sensor parameter configuration corresponding to the cluster can be obtained and used to set at least one sensor parameter of the vehicle-mounted lidar sensor.

[0097] This setup allows the LiDAR's scanning frequency and resolution to be dynamically adjusted based on real-time meteorological data, ensuring optimal quality source point cloud data is acquired in all weather conditions. This adaptive sensing technology significantly enhances the data acquisition capabilities of this embodiment of the present invention in adverse weather conditions such as rain, snow, fog, and haze, laying a solid foundation for subsequent tractor positioning processing.

[0098] Accordingly, after collecting the source point cloud data of the surrounding environment at the current position by the vehicle-mounted laser radar sensor, the following steps may also be included:

[0099] The meteorological type is determined according to each of the meteorological data, and a denoising algorithm matching the meteorological type is obtained to perform denoising processing on the source point cloud data.

[0100] Specifically, a combination of specially optimized denoising algorithms is used for different meteorological conditions: In rainy and snowy weather, a statistical outlier removal algorithm is primarily used. This algorithm analyzes the distance distribution characteristics of points in the neighborhood around each point, calculates the mean μ and standard deviation σ, and automatically identifies and removes outliers whose distances fall outside a reasonable range (e.g., μ±3σ). This method is particularly suitable for eliminating random noise points caused by rain and snow. For haze environments, the system enables radius-based statistical filtering. This algorithm sets a reasonable search radius (usually 5-10cm), calculates the point density within this radius, and filters out discrete noise points whose density is significantly lower than normal, effectively addressing the sparse and drifting point cloud caused by haze. For strong winds, temporal filtering technology is used to eliminate dynamic noise through temporal consistency analysis of multi-frame point clouds.

[0101] This adaptive denoising solution intelligently switches processing algorithms based on real-time feedback from meteorological sensors, ensuring that clean and reliable point cloud data can be obtained under various complex meteorological conditions, providing high-quality input for subsequent feature extraction and matching.

[0102] S220. Determine the local neighborhood of each source point cloud point in the source point cloud data by using a nearest neighbor search algorithm, and calculate the local geometric attributes and FPFH (Fast Point Feature Histograms) corresponding to each source point cloud point based on the local neighborhood of each source point cloud point.

[0103] In this embodiment, high-precision point cloud registration is achieved through innovative nearest neighbor search and feature extraction techniques. First, a KD-tree-accelerated nearest neighbor search algorithm is used to establish local spatial relationships for each point cloud point, accurately calculating geometric properties such as surface normals and curvature. Based on this, an FPFH descriptor is generated. This descriptor quantifies geometric relationships between neighboring points, such as normal angles, projected distances, and azimuths, to construct a highly discriminative 33-dimensional feature vector.

[0104] This multi-level feature representation effectively improves the robustness of point cloud processing. The FPFH descriptor provides rich local structural information while ensuring computational efficiency, and exhibits excellent adaptability to point cloud data of varying densities and resolutions.

[0105] S230 , evaluating the saliency score of each source point cloud point based on the local geometric attributes of each source point cloud point, and identifying source ISS (Intrinsic Shape Signatures) key points in each source point cloud point based on the saliency scores.

[0106] In this embodiment, ISS key points are identified by calculating the significance score by analyzing the local geometric properties of the point cloud. This method first evaluates the structural importance of each point based on geometric features such as point cloud curvature and normal change, and then selects the most representative feature points through non-maximum suppression.

[0107] This recognition technology based on inherent shape features can effectively capture significant structural features in point clouds and can stably detect key points with high discrimination even in the presence of noise interference or partial occlusion. Its advantage lies in quantifying the degree of change in local surface geometric characteristics, automatically identifying feature-rich areas such as corners and edges, and filtering out redundant points in flat areas. This not only greatly improves the quality of feature points, but also significantly reduces the amount of subsequent matching calculations, providing a more robust and efficient feature basis for point cloud registration.

[0108] S240 , matching the FPFH of each source ISS key point with the FPFH of each target ISS key point in the target point cloud map to obtain each pairing point between the source point cloud data and the target point cloud map.

[0109] In this example, efficient and accurate point cloud registration is achieved by matching the FPFH feature descriptors of ISS keypoints in the source and target point clouds. This method first extracts ISS keypoints with significant geometric features from both point clouds. It then calculates the FPFH feature descriptors around each keypoint to characterize its local geometric structure. Finally, point pair correspondences are established through nearest neighbor search in the feature space.

[0110] This feature descriptor-based matching method can effectively overcome the problems of noise interference and density variation in point cloud data. Its advantage lies in that by combining the saliency of ISS key points and the distinguishing ability of FPFH descriptors, it significantly improves computational efficiency while maintaining high matching accuracy. Even in the presence of partial occlusion, reliable matching results can still be achieved, providing a high-quality corresponding point pair foundation for subsequent accurate pose solution.

[0111] S250 , calculating the transformation matrix from the source point cloud data to the target point cloud map based on each paired point, and obtaining the current position coordinates and current posture orientation of the vehicle in the point cloud map coordinate system based on the transformation matrix.

[0112] In this embodiment, the transformation matrix from the source point cloud to the target point cloud is calculated using matched feature point pairs, and a robust estimation algorithm is used to solve the optimal rigid body transformation parameters to determine the precise position and posture of the tractor within the global map. This method constructs an error function based on paired points and simultaneously solves for rotation and translation transformations through iterative optimization, effectively overcoming the interference of sensor noise and matching errors. Its technical advantage lies in its ability to derive complete six-degree-of-freedom pose information from sparse but accurate matching point pairs, maintaining good computational efficiency while ensuring positioning accuracy. This enables the system to continuously output stable pose data in dynamic environments, providing a reliable positioning benchmark for automated traction operations and significantly enhancing the adaptability and accuracy of the navigation system in complex airport environments.

[0113] S260: Based on the position description information, a static and dynamic path planning algorithm is used in combination to control the tractor to move to the work starting point where the aircraft to be towed is located.

[0114] Furthermore, the system can acquire in real time any unplanned restricted areas (also known as detour areas) within the airport apron, such as oil-spattered areas. The coordinates of these oil-spattered areas within the target point cloud map can then be wirelessly transmitted to each moving tractor, allowing the tractor to dynamically plan a new navigation route to avoid the restricted area. Alternatively, if a restricted area suddenly appears very close to a tractor and there's insufficient time to prepare for the aforementioned evasive maneuvers, manual route planning can be implemented. For example, a voice instruction, such as "Avoid oil-spattered area XX ahead on the left," can be sent to the moving tractor, allowing the tractor to quickly avoid the restricted area.

[0115] S270: When the tractor moves to the work starting point, the front wheel point cloud information of the front wheel to be towed is calculated by fusing the original color image and the original depth image including the front landing gear taken by the vehicle-mounted depth camera.

[0116] S280: Calculate the heading angle of the front wheel to be towed relative to the tractor based on the front wheel point cloud information, so as to align the heading angles of the tractor and the front wheel to be towed.

[0117] Based on the above embodiments, after acquiring nose wheel point cloud information, the tractor can maintain its current motion state, wait for a period of time (e.g., 10 seconds), and then acquire new nose wheel point cloud information again. Based on the nose wheel depth information in the two preceding nose wheel point cloud information, the tractor can detect whether the aircraft to be towed is moving abnormally slowly. If so, the tractor is required to issue an acoustic warning and, according to a pre-set avoidance strategy, move away from the aircraft to be towed to prevent unnecessary collision risks.

[0118] S290: After completing the heading angle alignment, collect an alignment color image including the front wheel to be towed through the vehicle-mounted depth camera.

[0119] S2100: Input the alignment color image into a pre-trained front wheel frame detection model to obtain a target detection frame calibrated in the alignment color image.

[0120] Among them, the front wheel frame detection model only outputs a detection frame of a single target size, and the target size is consistent with the size of the minimum circumscribed rectangle of the front wheel to be towed captured by the on-board depth camera on the tractor at the standard position.

[0121] Specifically, after completing the heading alignment, the vehicle's onboard depth camera captures an aligned color image of the front wheel to be towed. Key to this step is that the images obtained after heading alignment have a standard normal viewing angle, effectively preventing perspective distortion from affecting subsequent detection accuracy. The camera uses the same resolution and aspect ratio (e.g., 1028×1028) as used during model training, ensuring that the input data distribution aligns with the training data. This standardized acquisition scheme significantly improves detection stability, maintaining reliable image quality, especially under complex lighting conditions.

[0122] Based on the above embodiments, the front wheel frame detection model can be obtained based on the second improved YOLOv8 network training, where:

[0123] After setting the default resolution in the standard YOLOv8 network to the target resolution and the loss function in the standard YOLOv8 network to the CloU function, a second improved YOLOv8 network is constructed;

[0124] Among them, CloU Loss is used to replace the traditional IoU loss function. This loss function simultaneously considers multiple key geometric factors such as the distance between the center points of the detection box, the aspect ratio matching, and the area of the overlapping area, so that the border positioning accuracy reaches the sub-pixel level, especially when dealing with difficult samples with partial occlusion or blurred edges. It shows significant advantages.

[0125] A second target dataset used for model training of the second improved YOLOv8 network includes a real dataset of aircraft nose landing gear in various types of weather scenarios and a computer-aided design virtual dataset, and each training image in the second target dataset has the same target image aspect ratio; and

[0126] The target resolution is consistent with the resolution of the image captured by the onboard depth camera on the tractor, and the target image aspect ratio is consistent with the image aspect ratio of the image captured by the onboard depth camera on the tractor.

[0127] S2110. Calculate the current pixel coordinates of the center point of the front wheel to be towed in the alignment color image based on the pixel coordinates of the four corner points in the target detection frame.

[0128] In this embodiment, a pre-trained deep learning model is used to obtain a rectangular detection frame and its corner coordinates that strictly match standard dimensions. The center position is then calculated using the geometric midline intersection method: connecting the diagonal vertices of the detection frame to form two midlines, the intersection of which is the theoretical center coordinate. To improve accuracy, the system uses a standardized aiming frame design to eliminate the effects of perspective distortion and can incorporate sub-pixel edge refinement technology.

[0129] This solution combines geometric principles with deep learning to ensure millimeter-level positioning accuracy while also having efficient computing performance and excellent engineering practicality. It can meet the stringent real-time and reliability requirements of traction operations, and is more adaptable to the stable operation needs of industrial scenarios than complex key point detection or instance segmentation methods.

[0130] S2120. Calculate the horizontal relative position between the tractor and the front wheel to be towed based on the current pixel coordinates and the preset calibration point pixel coordinates, so as to perform a horizontal alignment operation between the tractor and the front wheel to be towed.

[0131] The technical solution of this embodiment utilizes a vehicle-mounted lidar to collect environmental point cloud data in real time and rapidly establishes spatial topological relationships between point clouds using a nearest neighbor search algorithm. The system calculates geometric features such as local surface normals and curvature to construct a highly discriminative FPFH descriptor. This descriptor quantifies geometric relationships such as normal angles, projection distances, and azimuths between neighboring points to form a 33-dimensional feature vector. Based on ISS (Intrinsic Shape Feature) keypoint detection technology, the system can automatically identify feature regions with significant geometric characteristics in the point cloud, such as structural features like edges and corners. This method achieves efficient characterization of complex geometric structures while maintaining computational efficiency. It can accurately capture local detail features of the point cloud while adapting to point cloud data of varying densities and noise levels, providing a stable and reliable feature foundation for subsequent registration and positioning. Afterwards, a hybrid algorithm combining static global path planning with dynamic local obstacle avoidance is used to intelligently plan the optimal driving route. After the tractor arrives at the work site, the high-precision depth camera synchronously collects color images and depth information of the aircraft's front landing gear, and fuses the data through image processing algorithms to construct a complete three-dimensional point cloud model of the front wheel, obtain the heading angle deviation of the front wheel relative to the tractor, and automatically control the tractor to adjust its orientation to achieve precise heading alignment. Finally, the high-definition image obtained by the depth camera is used to locate the center coordinates of the front wheel using visual detection technology, and compared with the preset ideal calibration point, to calculate the posture deviation in the horizontal plane and realize the docking method. The clever use of multi-sensor data fusion and hierarchical progressive control strategy through an intelligent processing process from global positioning to local perception, from rough adjustment to fine alignment not only ensures the safety and reliability of the docking process, but also greatly improves the operating efficiency and operation accuracy in complex weather environments, realizing the automation upgrade of aircraft towing operations.

[0132] Example 3

[0133] For ease of understanding, the specific application scenarios applicable to each embodiment of the invention are described below. In this specific application scenario, in order to make the docking between the tractor and the aircraft wheel more precise, the embodiment of the present invention designs a complete docking and positioning solution between the tractor and the aircraft wheel.

[0134] In this embodiment, three steps are used to ensure precise docking between the tractor and the aircraft wheels:

[0135] Specifically, in Figure 3 FIG1 shows a flowchart of the specific steps of radar positioning and automatic navigation in docking positioning between a tractor and an aircraft wheel according to an embodiment of the present invention. Figure 3As shown, this embodiment uses a lidar positioning module to construct a 3D point cloud map of the airport apron and achieves real-time, precise positioning using an improved ICP (Iterative Closest Point) algorithm. During tractor operation, the lidar continuously collects point cloud data from the current environment. This data is fed into an improved ICP algorithm network using ISS and FPFH feature descriptors. Normal estimation and feature matching achieve high-precision registration with the target point cloud map, outputting the vehicle's precise pose in the global coordinate system. The resulting positioning information is fed into a navigation network that integrates the A* and D* algorithms. The A* algorithm is responsible for global optimal path planning in static environments, while the D* algorithm handles local path adjustments for dynamic obstacles. These two algorithms work together to ensure efficient and real-time planning. Simultaneously, the system integrates lidar point cloud data with RGB-D camera depth information to construct a dynamic obstacle map. Finally, a vision-driven positioning system guides the tractor to its target starting point. This solution, through multi-sensor data fusion and a hierarchical planning strategy, achieves full automation of tractor operations in complex airport environments.

[0136] Further, in Figure 4 FIG. 1 shows a flowchart of the specific steps for aligning the heading angle in docking positioning between a tractor and an aircraft wheel according to an embodiment of the present invention. Figure 4 As shown in the figure, multi-sensor fusion technology can be used to accurately identify and locate an aircraft's nose landing gear. When the tractor arrives at the work starting point, the nose landing gear is detected using an improved YOLOv8 network. This network significantly improves detection accuracy and real-time performance by introducing the SPD-Conv module and SE Attention mechanism. Simultaneously, depth information captured by the RGB-D camera is fused with the color image to generate a color point cloud, which is then input into an optimized PointNet++ network for segmentation. This network enhances point cloud feature extraction capabilities by incorporating self-attention mechanisms and GCN feature extraction. After point cloud denoising and smoothing, the system accurately calculates the three-dimensional coordinates and relative heading angle of the front wheels, ultimately completing heading alignment. This solution, through deep learning network optimization and multi-source data fusion, achieves high-precision identification and positioning of the nose landing gear in complex environments, providing reliable technical support for automated towing operations.

[0137] Further, in Figure 5 FIG1 shows a flowchart of the specific steps of horizontal alignment in docking positioning between a tractor and an aircraft wheel according to an embodiment of the present invention. Figure 5As shown in the figure, after the automated tractor completes heading angle alignment, an improved YOLOv8 algorithm is used to accurately calculate and align the horizontal pose. The specific process is as follows: First, a high-resolution RGB image is acquired and fed into an optimized YOLOv8 network (using a 1024×1024 input resolution, CloU loss function, and an anchor ratio adapted to the front wheel size) for detection, obtaining a highly accurate front wheel detection frame. The center pixel position is then calculated based on the coordinates of the four corner points of the detection frame and compared with pre-set calibration points. Finally, perspective geometry is used to determine the horizontal relative pose of the tractor and front wheel. Through algorithmic optimization and multi-level geometric calculations, this solution achieves sub-pixel detection accuracy and millimeter-level pose alignment, ensuring the accuracy and reliability of automated tractor operations.

[0138] Furthermore, through the ingenious coordination of the above steps, precise positioning and alignment between the automated tractor and the aircraft wheels can be achieved, achieving the following effective results:

[0139] (1) Through the collaborative work of lidar positioning, visual recognition, and autonomous navigation technology, the entire process from environmental perception to final docking is automated. Intelligent algorithms can independently complete map construction, path planning, and precise positioning, significantly reducing manual operation links, making the entire traction process more efficient and reliable, and greatly improving the automation level of airport ground operations.

[0140] (2) Based on improved deep learning algorithms and multi-sensor fusion technology, the 3D position of aircraft wheels can be accurately identified. Through optimized point cloud registration and visual inspection algorithms, sub-millimeter measurement accuracy is achieved, fully meeting the stringent precision requirements of towing operations in the civil aviation field and ensuring the safety and reliability of the docking process.

[0141] (3) The system adopts an innovative anti-interference algorithm design and is equipped with all-weather working sensors, which can effectively cope with various complex weather conditions such as rain, snow, fog, haze, and strong light. The intelligent compensation algorithm can automatically correct the measurement errors caused by environmental factors, ensuring stable operation under different meteorological conditions, greatly expanding the application scenarios of the system.

[0142] (4) Parallel computing architecture and algorithm optimization are used to achieve millisecond-level response. Through real-time operating systems and hardware acceleration, high-frequency data collection and processing are maintained to meet the timeliness requirements of airport operations. Excellent real-time performance ensures smooth processes and timely response to emergencies. Intelligent task scheduling dynamically allocates resources to ensure that critical tasks are executed first.

[0143] Example 4

[0144] Figure 6 This is a structural diagram of a method and device for docking and positioning a tractor and an aircraft wheel provided in the fourth embodiment of the present invention. Figure 6 The device shown includes: a radar positioning automatic driving module 610, a depth visual perception module 620, a heading angle calculation module 630, a center pixel coordinate calculation module 640 and a real-time horizontal posture adjustment module 650, wherein:

[0145] The radar positioning automatic driving module 610 is used to obtain the vehicle's position description information in the point cloud map coordinate system based on the source point cloud data collected by the vehicle-mounted lidar sensor at the current position; and based on the position description information, it uses a combination of static and dynamic path planning algorithms to control the tractor to move to the working starting point where the aircraft to be towed is located;

[0146] The depth vision perception module 620 is used to fuse and calculate the front wheel point cloud information of the front wheel to be towed based on the original color image and original depth image of the front landing gear taken by the on-board depth camera when the towing vehicle moves to the work starting point;

[0147] The heading angle calculation module 630 is used to calculate the heading angle of the front wheel to be towed relative to the tractor based on the front wheel point cloud information, so as to align the heading angle between the tractor and the front wheel to be towed;

[0148] The center pixel coordinate calculation module 640 is used to calculate the current pixel coordinates of the center point of the front wheel to be towed in the alignment color image captured by the vehicle-mounted depth camera after the heading angle alignment is completed;

[0149] The horizontal posture adjustment module 650 is used to solve the horizontal relative posture between the tractor and the front wheel to be towed based on the current pixel coordinates and the preset calibration point pixel coordinates, so as to perform a horizontal alignment operation between the tractor and the front wheel to be towed.

[0150] The technical solution of the embodiment of the present invention uses a vehicle-mounted laser radar to collect environmental point cloud data in real time, and then intelligently matches it with a pre-built high-precision apron point cloud map to accurately obtain the three-dimensional pose information of the tractor in the global coordinate system. Then, a hybrid algorithm combining static global path planning and dynamic local obstacle avoidance is used to intelligently plan the optimal driving route and perceive and avoid dynamic obstacles in real time. After the tractor arrives at the work site, a high-precision depth camera synchronously collects color images and depth information of the aircraft's front landing gear. The data is fused through an image processing algorithm to construct a complete three-dimensional point cloud model of the front wheels, and the front wheels relative to the tractor are obtained. The system can detect the heading angle deviation of the vehicle and automatically control the tractor to adjust its orientation to achieve precise heading alignment. Finally, the high-definition image obtained by the depth camera is used to locate the center coordinates of the front wheel using visual detection technology, and compared with the preset ideal calibration point to calculate the posture deviation in the horizontal plane and realize docking. It cleverly uses multi-sensor data fusion and hierarchical progressive control strategies through an intelligent processing flow from global positioning to local perception, from rough adjustment to fine alignment. It not only ensures the safety and reliability of the docking process, but also greatly improves the operating efficiency and operation accuracy in complex weather environments, realizing the automation upgrade of aircraft towing operations.

[0151] Based on the above embodiments, the radar positioning automatic driving module 610 is specifically used to:

[0152] The vehicle-mounted lidar sensor collects source point cloud data of the surrounding environment at the current location and obtains a pre-established target point cloud map for the airport apron area;

[0153] The local neighborhood of each source point cloud point is determined in the source point cloud data through the nearest neighbor search algorithm, and the local geometric attributes and FPFH corresponding to each source point cloud point are calculated based on the local neighborhood of each source point cloud point.

[0154] According to the local geometric properties of each source point cloud point, the significance score of each source point cloud point is evaluated, and the source ISS key points are identified in each source point cloud point according to the significance score;

[0155] Match the FPFH of each source ISS key point with the FPFH of each target ISS key point in the target point cloud map to obtain the paired points between the source point cloud data and the target point cloud map;

[0156] Based on each paired point, the transformation matrix from the source point cloud data to the target point cloud map is calculated, and based on the transformation matrix, the current position coordinates and current posture orientation of the vehicle in the point cloud map coordinate system are obtained.

[0157] Furthermore, based on the above embodiments, the radar positioning automatic driving module 610 may include: a parameter setting unit and a denoising processing unit, wherein:

[0158] a parameter setting unit, configured to obtain in real time at least one item of meteorological data matching the current position before collecting source point cloud data of the surrounding environment at the current position through the vehicle-mounted laser radar sensor, and parameterize at least one sensor parameter of the vehicle-mounted laser radar sensor according to each of the meteorological data; and

[0159] The denoising processing unit is used to determine the meteorological type according to each meteorological data after collecting the source point cloud data of the surrounding environment at the current position through the vehicle-mounted laser radar sensor, and obtain a denoising algorithm matching the meteorological type to denoise the source point cloud data.

[0160] Based on the above embodiments, the depth vision perception module 620 is specifically configured to:

[0161] When the tractor moves to the work starting point, the vehicle-mounted depth camera captures the original color image and original depth image containing the front landing gear;

[0162] The original color image is input into the pre-trained nose wheel recognition model to segment the color sub-image that matches the nose wheel to be towed.

[0163] The segmented color sub-image is fused with the original depth image to obtain the point cloud information of the front wheel to be towed;

[0164] The point cloud information of the front wheel to be towed is input into a pre-trained point cloud segmentation model to obtain the point cloud information of the front wheel to be towed.

[0165] Based on the above embodiments, the front wheel recognition model is obtained based on the first improved YOLOv8 network training, and the point cloud segmentation model is obtained based on the improved PointNet++ network training, wherein:

[0166] The first improved YOLOv8 network is constructed by adding spatial-to-depth convolution modules to the front end of the third, sixth, ninth, and twelfth layers of the standard YOLOv8 network backbone network, and adding a compression and incentive attention mechanism module to the last layer of the standard YOLOv8 network backbone network;

[0167] The improved PointNet++ network is constructed by adding a self-attention mechanism module to the standard PointNet++ network, modifying the feature extraction network in the standard PointNet++ network to a graph convolutional network, and adding sampling radii of different scales to different feature extraction modules in the feature aggregation method in the standard PointNet++ network; and

[0168] The first target data set used for model training of the first improved YOLOv8 network includes real data sets and computer-aided design virtual data sets of aircraft nose landing gear in various types of weather scenarios.

[0169] Based on the above embodiments, the center pixel coordinate calculation module 640 is specifically configured to:

[0170] After completing the heading angle alignment, the vehicle-mounted depth camera is used to collect an aligned color image containing the front wheel to be towed;

[0171] Inputting the alignment color image into a pre-trained front wheel frame detection model to obtain a target detection frame calibrated in the alignment color image;

[0172] The front wheel frame detection model only outputs a detection frame of a single target size, and the target size is consistent with the size of the minimum circumscribed rectangle of the front wheel to be towed, as captured by the onboard depth camera on the tractor at a standard position.

[0173] According to the pixel coordinates of the four corner points in the target detection frame, the current pixel coordinates of the center point of the front wheel to be towed in the alignment color image are calculated.

[0174] Based on the above embodiments, the front wheel frame detection model is obtained based on the second improved YOLOv8 network training, where:

[0175] After setting the default resolution in the standard YOLOv8 network to the target resolution and the loss function in the standard YOLOv8 network to the CloU function, a second improved YOLOv8 network is constructed;

[0176] A second target dataset used for model training of the second improved YOLOv8 network includes a real dataset of aircraft nose landing gear in various types of weather scenarios and a computer-aided design virtual dataset, and each training image in the second target dataset has the same target image aspect ratio; and

[0177] The target resolution is consistent with the resolution of the image captured by the onboard depth camera on the tractor, and the target image aspect ratio is consistent with the image aspect ratio of the image captured by the onboard depth camera on the tractor.

[0178] The positioning device between a tractor and an aircraft wheel provided in an embodiment of the present invention can execute the positioning method between a tractor and an aircraft wheel provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0179] In the technical solutions disclosed herein, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0180] Example 5

[0181] Figure 7 A schematic diagram of a tractor 10 that can be used to implement an embodiment of the present invention is shown. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit implementations of the inventions described and / or claimed herein.

[0182] like Figure 7 As shown, the autonomous driving tractor 10 includes: an onboard laser radar sensor 11 for collecting source point cloud data of the surrounding environment; an onboard depth camera 12 for collecting color images and / or depth images of the surrounding environment;

[0183] The tractor 10 includes at least one lidar sensor 11, a depth camera sensor 12, and a processor 13. Furthermore, memory, such as a read-only memory (ROM) 14 and a random access memory (RAM) 15, is communicatively connected to the at least one processor 13. The memory stores computer programs executable by the at least one processor. The processor 13 can perform various appropriate actions and processes based on the computer programs stored in the ROM 14 or loaded from the storage unit 20 into the RAM 15. The RAM 15 may also store various programs and data required for the operation of the tractor 11. The processor 13, ROM 14, and RAM 15 are interconnected via a bus 16. An input / output (I / O) interface 17 is also connected to the bus 16.

[0184] Various components in the tractor 10 are connected to the I / O interface 17, including an input unit 18, such as a keyboard, mouse, etc.; an output unit 19, such as various types of displays, speakers, etc.; a storage unit 20, such as a magnetic disk, optical disk, etc.; and a communication unit 21, such as a network card, modem, wireless communication transceiver, etc. The communication unit 21 allows the tractor 10 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0185] The processor 13 can be any general-purpose and / or specialized processing component with processing and computing capabilities. Some examples of the processor 13 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 13 executes the various methods and processes described above, such as a docking positioning method between a tractor and an aircraft wheel, namely:

[0186] Based on the source point cloud data collected by the on-board lidar sensor at the current position, the position description information of the vehicle in the point cloud map coordinate system is obtained; and based on the position description information, the static and dynamic path planning algorithms are combined to control the tractor to move to the working starting point where the aircraft to be towed is located. When the tractor moves to the working starting point, the front wheel point cloud information of the front aircraft to be towed is calculated based on the original color image and original depth image containing the front landing gear taken by the on-board depth camera. Based on the front wheel point cloud information, the heading angle of the front aircraft to be towed relative to the tractor is calculated to perform the heading angle alignment operation between the tractor and the front aircraft to be towed. After the heading angle alignment is completed, the current pixel coordinates of the center point of the front aircraft to be towed in the alignment color image containing the front aircraft to be towed collected by the on-board depth camera are calculated. Based on the current pixel coordinates and the preset calibration point pixel coordinates, the horizontal relative position between the tractor and the front aircraft to be towed is solved to perform the horizontal alignment operation between the tractor and the front aircraft to be towed.

[0187] In some embodiments, the docking and positioning method between a tractor and an aircraft wheel can be implemented as a computer program tangibly embodied in a computer-readable storage medium, such as storage unit 20. In some embodiments, part or all of the computer program can be loaded and / or installed on the tractor 10 via ROM 14 and / or communication unit 21. When the computer program is loaded into RAM 15 and executed by processor 13, one or more steps of the docking and positioning method between a tractor and an aircraft wheel described above can be performed. Alternatively, in other embodiments, processor 13 can be configured to execute the docking and positioning method between a tractor and an aircraft wheel via any other suitable means (e.g., via firmware).

[0188] To provide user interaction, the systems and techniques described herein can be implemented on a tractor, where the electronic device has: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices can also be used to provide user interaction; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0189] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in the present invention can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of the present invention can be achieved. This is not limited herein.

[0190] The above specific embodiments do not limit the scope of protection of the present invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

Claims

1. A method for docking and positioning a tractor and an aircraft wheel, characterized in that: The method is performed by an autonomously driven tractor, comprising: Based on the source point cloud data collected by the on-board lidar sensor at the current position, the vehicle's position description information in the point cloud map coordinate system is obtained. Based on the position description information, a combination of static and dynamic path planning algorithms is used to control the tractor to move to the working starting point where the aircraft to be towed is located. When the tractor moves to the work starting point, the front wheel point cloud information of the front wheel to be towed is calculated based on the original color image and original depth image including the front landing gear taken by the on-board depth camera; Calculate the heading angle of the to-be-towed front wheel relative to the towing vehicle based on the front wheel point cloud information, so as to align the heading angles of the towing vehicle and the front wheel; After completing the heading angle alignment, the current pixel coordinates of the center point of the front wheel to be towed in the alignment color image captured by the vehicle-mounted depth camera and including the front wheel to be towed are calculated; According to the current pixel coordinates and the preset calibration point pixel coordinates, the horizontal relative position between the tractor and the front wheel to be towed is solved to perform the horizontal alignment operation between the tractor and the front wheel to be towed.

2. The method according to claim 1, characterized in that Based on the source point cloud data collected by the vehicle-mounted lidar sensor at the current position, obtain the vehicle's position description information in the point cloud map coordinate system, including: The vehicle-mounted lidar sensor collects source point cloud data of the surrounding environment at the current location and obtains a pre-established target point cloud map for the airport apron area; The local neighborhood of each source point cloud point is determined in the source point cloud data through the nearest neighbor search algorithm, and the local geometric attributes and fast feature histogram FPFH corresponding to each source point cloud point are calculated based on the local neighborhood of each source point cloud point. According to the local geometric properties of each source point cloud point, the significance score of each source point cloud point is evaluated, and according to each significance score, the source intrinsic shape feature descriptor ISS key points are identified in each source point cloud point; Match the FPFH of each source ISS key point with the FPFH of each target ISS key point in the target point cloud map to obtain the paired points between the source point cloud data and the target point cloud map; Based on each paired point, the transformation matrix from the source point cloud data to the target point cloud map is calculated, and based on the transformation matrix, the current position coordinates and current posture orientation of the vehicle in the point cloud map coordinate system are obtained.

3. The method according to claim 2, characterized in that Before collecting source point cloud data of the surrounding environment at the current position through the vehicle-mounted lidar sensor, it also includes: Acquire at least one item of meteorological data matching the current location in real time, and set at least one sensor parameter of the vehicle-mounted lidar sensor based on each item of meteorological data; and After collecting the source point cloud data of the surrounding environment at the current position through the vehicle-mounted lidar sensor, it also includes: The meteorological type is determined according to each of the meteorological data, and a denoising algorithm matching the meteorological type is obtained to perform denoising processing on the source point cloud data.

4. The method according to claim 1, wherein When the tractor moves to the work starting point, the front wheel point cloud information of the front wheel to be towed is calculated based on the original color image and original depth image of the front landing gear taken by the on-board depth camera, including: When the tractor moves to the work starting point, the vehicle-mounted depth camera captures the original color image and original depth image containing the front landing gear; The original color image is input into the pre-trained nose wheel recognition model to segment the color sub-image that matches the nose wheel to be towed. The segmented color sub-image is fused with the original depth image to obtain the point cloud information of the front wheel to be towed; The point cloud information of the front wheel to be towed is input into a pre-trained point cloud segmentation model to obtain the point cloud information of the front wheel to be towed.

5. The method according to claim 4, characterized in that The nose wheel recognition model is trained based on the first improved YOLOv8 network, and the point cloud segmentation model is trained based on the improved PointNet++ network. The first improved YOLOv8 network is constructed by adding spatial-to-depth convolution modules to the front end of the third, sixth, ninth, and twelfth layers of the standard YOLOv8 network backbone network, and adding a compression and incentive attention mechanism module to the last layer of the standard YOLOv8 network backbone network; The improved PointNet++ network is constructed by adding a self-attention mechanism module to the standard PointNet++ network, modifying the feature extraction network in the standard PointNet++ network to a graph convolutional network, and adding sampling radii of different scales to different feature extraction modules in the feature aggregation method in the standard PointNet++ network; and The first target data set used for model training of the first improved YOLOv8 network includes real data sets and computer-aided design virtual data sets of aircraft nose landing gear in various types of weather scenarios.

6. The method according to claim 1, characterized in that After completing the heading angle alignment, the current pixel coordinates of the center point of the front wheel to be towed in the alignment color image captured by the vehicle-mounted depth camera and including the front wheel to be towed are calculated, including: After completing the heading angle alignment, the vehicle-mounted depth camera is used to collect an aligned color image containing the front wheel to be towed; Inputting the alignment color image into a pre-trained front wheel frame detection model to obtain a target detection frame calibrated in the alignment color image; The front wheel frame detection model only outputs a detection frame of a single target size, and the target size is consistent with the size of the minimum circumscribed rectangle of the front wheel to be towed, as captured by the onboard depth camera on the tractor at a standard position. According to the pixel coordinates of the four corner points in the target detection frame, the current pixel coordinates of the center point of the front wheel to be towed in the alignment color image are calculated.

7. The method according to claim 6, characterized in that The front wheel frame detection model is based on the second improved YOLOv8 network training, where: After setting the default resolution in the standard YOLOv8 network to the target resolution and the loss function in the standard YOLOv8 network to the CloU function, a second improved YOLOv8 network is constructed; A second target dataset used for model training of the second improved YOLOv8 network includes a real dataset of aircraft nose landing gear in various types of weather scenarios and a computer-aided design virtual dataset, and each training image in the second target dataset has the same target image aspect ratio; and The target resolution is consistent with the resolution of the image captured by the onboard depth camera on the tractor, and the target image aspect ratio is consistent with the image aspect ratio of the image captured by the onboard depth camera on the tractor.

8. A docking and positioning device between a tractor and an aircraft wheel, characterized in that: Configured in an autonomous tractor, the device includes: The radar positioning autopilot module is used to obtain the vehicle's position description information in the point cloud map coordinate system based on the source point cloud data collected by the on-board lidar sensor at the current position. Based on this position description information, it combines static and dynamic path planning algorithms to control the tractor to move to the working starting point where the aircraft to be towed is located; The depth vision perception module is used to fuse and calculate the front wheel point cloud information of the front wheel to be towed based on the original color image and original depth image including the front landing gear taken by the on-board depth camera when the towing vehicle moves to the work starting point; The heading angle calculation module is used to calculate the heading angle of the front wheel to be towed relative to the tractor based on the front wheel point cloud information, so as to align the heading angle between the tractor and the front wheel to be towed; A center pixel coordinate calculation module is used to calculate the current pixel coordinates of the center point of the front wheel to be towed in the alignment color image captured by the vehicle-mounted depth camera after the heading angle alignment is completed; The horizontal posture adjustment module is used to solve the horizontal relative posture between the tractor and the front wheel to be towed based on the current pixel coordinates and the preset calibration point pixel coordinates, so as to perform the horizontal alignment operation between the tractor and the front wheel to be towed.

9. An automatic driving tractor, characterized in that: The tractor comprises: On-vehicle lidar sensor, used to collect source point cloud data of the surrounding environment; An onboard depth camera, used to capture color images and / or depth images of the surrounding environment; at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to perform the docking and positioning method between a tractor and an aircraft wheel according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the docking and positioning method between a tractor and an aircraft wheel according to any one of claims 1 to 7 when executed.

Images