Aircraft door posture locking system and method based on multi-sensor fusion

Abstract

The present application belongs to the technical field of boarding bridges, specifically, a system for locking the position of an aircraft door based on multi-sensor fusion, including a laser radar, a visual sensor, an obstacle avoidance sensor and a control module, wherein the control module includes a main control unit, a laser radar global locking unit, a visual local locking unit, and a safety obstacle avoidance unit; wherein the laser radar global locking unit globally locks the position of the door, the visual local locking unit is used to perform door detection and key point positioning of image data, the safety obstacle avoidance unit is used to process the safety obstacle avoidance sensor data and send the data to the main control unit, and the main control unit controls the boarding bridge to make a motion response. The present application also provides a method for locking the position of an aircraft door based on multi-sensor fusion. The present application realizes multi-modal autonomous aircraft door position locking from global to local; while ensuring the docking accuracy and efficiency of the boarding bridge, it also meets the requirements of unmanned pick-up operations.

Description

Technical Field

[0001] The present application belongs to the technical field of boarding bridges, and in particular, relates to a system and method for locking the position of an aircraft door based on multi-sensor fusion. Background Art

[0002] With the development of unmanned driving technology, the unmanned driving of special equipment at airports has also attracted the attention of the civil aviation industry. In 2021, the "Roadmap for the Application of Unmanned Driving Equipment at Airports (2021-2025)" issued by the Civil Aviation Administration of China guided the application and promotion of unmanned driving equipment at airports and pointed out the direction for the development of unmanned driving at airports.

[0003] As a key aviation ground equipment, the boarding bridge connects the terminal boarding gate with the aircraft door, making it convenient for passengers to enter and exit the cabin. The design and use of the boarding bridge not only provides an efficient and safe passage, but also has the flexibility to adapt to different aircraft models and airport layouts.

[0004] At present, the operation of boarding bridge docking with aircraft door mainly relies on the operator's visual observation to determine the position of aircraft door through the installation of sensor equipment. Not only does it affect the docking accuracy and efficiency of boarding bridge, but human visual observation cannot meet the requirements of unmanned boarding bridge docking.

[0005] In the door docking task, efficient and accurate locking of the aircraft door position is the key basis for achieving the task. At present, there is a technical route to lock the door by affixing a code on the aircraft fuselage, but the versatility of this method is poor, and there are great obstacles to the feasibility and operability of affixing a code on the fuselage. There is also a technical route to lock the door position by using a single sensor, but this method requires more information about the pre-parking position of the boarding bridge in advance. Summary of the invention

[0006] The present application provides a system and method for locking the posture of an aircraft door based on multi-sensor fusion, which solves the problem of low accuracy and low efficiency of docking a boarding bridge to an aircraft door.

[0007] The embodiment of the present application provides an aircraft cabin door posture locking system based on multi-sensor fusion, including a laser radar, a visual sensor, an obstacle avoidance sensor and a control module, wherein the control module includes a main control unit, a laser radar global locking unit, a visual local locking unit, and a safety obstacle avoidance unit; wherein,

[0008] The laser radar global locking unit is connected to the laser radar and is used to perform real-time processing and positioning of point cloud data and globally lock the position and posture of the cabin door;

[0009] The visual local locking unit is connected to the visual sensor and is used to perform door detection and key point positioning of the image data;

[0010] The safety obstacle avoidance unit is connected to the obstacle avoidance sensor and is used to process the safety obstacle avoidance sensor data and send the data to the main control unit;

[0011] The main control unit controls the boarding bridge to make a motion response according to the detection result.

[0012] Optionally, the control module further includes an aircraft parameter database unit and a data acquisition unit;

[0013] The aircraft parameter database unit is used to load and store the prior data in the laser radar global locking unit;

[0014] The data acquisition unit is used to acquire point cloud data and image data of the target door, and also includes a safety obstacle avoidance sensor to acquire distance information of obstacles on the ground.

[0015] Optionally, the radar, visual sensor, and obstacle avoidance sensor are all arranged on the boarding bridge; the obstacle avoidance sensor includes a ranging sensor, a multi-line laser radar, and a wide-beam radar.

[0016] The present application also provides a method for locking the position of an aircraft door based on multi-sensor fusion, comprising the following steps:

[0017] Step 1: Obtain point cloud data of the aircraft door and its surroundings, align the acquired point cloud data with the pre-established aircraft parameter database, and globally lock the door position;

[0018] Step 2: Acquire an image and use a deep convolutional neural network to perform target detection on the acquired image, and locally lock the position and posture of the hatch and its key points;

[0019] Step three: detect obstacles and control the boarding bridge to make motion response based on the detection results of obstacles by other sensors.

[0020] Optionally, in step 1, the point cloud data of the cabin door and its surroundings acquired by the laser radar global locking unit is used, and the laser radar global locking unit implements the global locking steps of the aircraft cabin door posture as follows:

[0021] Step (11), pre-establishing an aircraft parameter database;

[0022] Step (12), obtaining the original point cloud data of the target aircraft at the current viewing angle collected in real time by the laser radar, and preprocessing the original point cloud data to obtain preprocessed point cloud data;

[0023] Step (13) performs point cloud registration on the pre-processed point cloud data and the established aircraft parameter database to globally lock the position and posture of the cabin door.

[0024] Optionally, in step (12), the process of preprocessing the collected original point cloud data is as follows:

[0025] (a) Convert the format of the collected original point cloud data;.

[0026] (b) Cleaning and filtering of the original point cloud data.

[0027] Optionally, in step 2, a visual local locking unit is used to locally lock the position and posture of the cabin door and its key points. The steps for the visual local locking unit to locally lock the position and posture of the aircraft cabin door are:

[0028] Step (21), data acquisition, using a visual sensor to acquire image information of the target aircraft door at the current viewing angle in real time;

[0029] Step (22), cabin door detection, by inputting the image information of the current viewing angle acquired in real time in step (21) into a pre-trained target detection network model to perform cabin door detection and determine the rectangular area where the cabin door is located;

[0030] Step (23), key point extraction and positioning, extract the pixel coordinates of the lower left and right corners of the rectangular area where the hatch is located, obtain the three-dimensional world coordinate information of the corner points based on the binocular vision sensor, and complete local locking.

[0031] Optionally, in step (23), the binocular vision sensor realizes the positioning of the key points by acquiring depth information, and the process of acquiring the depth information is as follows:

[0032] First, two slightly offset images of the same scene are captured using two parallel cameras;

[0033] Then, by comparing the difference in pixel coordinates of the same physical point in the two images on the left and right camera images, that is, disparity;

[0034] Finally, the baseline length and focal length information of the two cameras are combined to calculate the depth information of the corresponding points in three-dimensional space.

[0035] Optionally, the depth information is calculated as follows:

[0036] set up is a point on the object to be measured. and They are the optical centers of the left and right cameras, and the points The imaging points on the two camera sensors are and , is the camera focal length, is the center distance between the two cameras, For depth information;

[0037] According to trigonometric relationship, we can get , rearrange the above formula, and we can get the depth information , where ( x L - x R ) represents parallax.

[0038] Optionally, the pixel coordinates of the corresponding points The calculation formula of the three-dimensional pose in the world coordinate system is: .

[0039] This application realizes multi-modal autonomous aircraft door posture locking from global to local through the coordination of laser radar, visual sensors and safety obstacle avoidance sensors; at the same time, while ensuring the accuracy and efficiency of boarding bridge docking, it also meets the requirements of unmanned aircraft pick-up operations and promotes the automation of large-scale special equipment at airports. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0041] Figure 1 This application is an aircraft door posture locking system based on multi-sensor fusion;

[0042] Figure 2 It is the structural block diagram of the control module;

[0043] Figure 3 It is a flow chart of the method for locking the position of aircraft door based on multi-sensor fusion;

[0044] Figure 4 This is the global locking flow chart of the aircraft door position;

[0045] Figure 5 A flowchart for processing the collected raw point cloud data;

[0046] Figure 6 Flowchart for point cloud registration;

[0047] Figure 7 It is a flowchart of partial locking of aircraft door position;

[0048] Figure 8 Flowchart for training the network model for aircraft door object detection. DETAILED DESCRIPTION

[0049] In order to make the technical problems, technical solutions and beneficial effects to be solved by the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0050] The aircraft door posture locking system and method based on multi-sensor fusion provided in the embodiments of the present application are now described.

[0051] See also Figure 1 The present application provides an aircraft door posture locking system based on multi-sensor fusion, including a laser radar, a visual sensor, an obstacle avoidance sensor and a control module.

[0052] The laser radar is installed on the boarding bridge to globally lock the relative position of the boarding bridge and the cabin door of the target aircraft.

[0053] The visual sensor is installed on the boarding bridge to partially lock the relative position between the boarding bridge and the cabin door of the target aircraft.

[0054] The obstacle avoidance sensor is installed on the boarding bridge so that when an obstacle is detected, the information can be transmitted to the main control unit, and the main control unit controls the boarding bridge to make a motion response according to the detection result, thus ensuring its safety when operating by aircraft.

[0055] The obstacle avoidance sensor includes but is not limited to a distance sensor, a multi-line laser radar, a wide beam radar, etc. Preferably, the distance sensor is installed at the bottom of the docking cabin floor and is arranged toward the cabin door. The obstacle avoidance sensor is arranged at the center of the docking cabin floor and is symmetrically distributed on the left and right. The selection and setting of the distance sensor can not only be used to measure the distance information between the docking cabin and the target cabin door, but also assist in leveling the docking cabin pedal. The multi-line laser radar is installed on the right side of the bottom of the docking cabin (i.e., the end close to the wing when docking the cabin door), and the detection and position determination of other parts of the aircraft (including but not limited to the aircraft wing) during the docking process are performed by multi-line laser detection. The wide beam radar is installed on the guardrail outside the running wheel or the running wheel base, facing the front and rear sides of the guardrail when the running wheel is facing the aircraft cabin door. The installation position of the wide beam radar can make the wide beam radar change its direction with the movement of the running wheel, and detect obstacles in front and behind the running wheel during the docking process of the boarding bridge. Of course, the obstacle avoidance sensor also includes other sensors that can realize obstacle avoidance detection, and the installation position and installation quantity are not limited to the above description.

[0056] See also Figure 2 The control module includes a main control unit, a laser radar global locking unit, a visual local locking unit, a safety obstacle avoidance unit, an aircraft parameter database unit and a data acquisition unit.

[0057] The laser radar global locking unit is connected to the laser radar. It is used for real-time processing and positioning of point cloud data. The laser radar global locking unit aligns the point cloud data of the cabin door and its surroundings obtained by the laser radar with the established aircraft parameter database to globally lock the cabin door posture.

[0058] The visual local locking unit is connected to the visual sensor. It is used to detect the hatch and locate key points of the image data. The visual local locking unit detects the target through a deep convolutional neural network based on the image obtained by the visual sensor, and locally locks the position and posture of the hatch and its key points with the help of binocular vision.

[0059] Safety obstacle avoidance unit, connected to obstacle avoidance sensor. Used to process safety obstacle avoidance sensor data and send data to the main control unit.

[0060] The main control unit controls the boarding bridge to make movement response according to the obstacle detection results of the obstacle avoidance sensor.

[0061] Aircraft parameter database unit, used to load and store prior data in the lidar global lock unit.

[0062] The data acquisition unit is used to obtain the point cloud data and image data of the target cabin door, and also includes a safety obstacle avoidance sensor to obtain the distance information of obstacles such as people or objects on the ground and the wing where the target cabin door is located.

[0063] The multi-sensor fusion aircraft door posture locking system first establishes or loads an aircraft parameter database unit as prior data in the global locking link of the laser radar unit for globally locking the aircraft door; the data acquisition unit acquires point cloud data, image data and distance information of the safety obstacle avoidance sensor of the aircraft door; the laser radar global locking unit processes the target door point cloud data acquired by the data acquisition unit in real time, matches the processed point cloud data with the aircraft parameter database, determines the relative posture, and completes the global locking of the target door; the visual local locking unit performs door detection on the image data acquired by the data acquisition unit according to the trained target detection model, determines the posture of the target docking point on the detection frame in the world coordinate system with the help of the visual sensor, and realizes the local locking of the target door; the safety obstacle avoidance unit processes the distance information of the safety obstacle avoidance sensor acquired by the data acquisition unit, and sends the distance information to the main control unit, and the main control unit controls the boarding bridge to make a motion response according to the data.

[0064] The system also includes a computing unit for solving relative positions and postures so as to generate instruction information acceptable to the motion mechanism.

[0065] The system also includes a sending unit, which is used to send the relative position and attitude solved by the calculation unit to the motion control device of the aviation ground equipment.

[0066] See also Figure 3 , an aircraft cabin door posture locking method based on multi-sensor fusion, comprising the following steps:

[0067] Step 1: Obtain point cloud data of the aircraft door and its surroundings, align the acquired point cloud data with the pre-established aircraft parameter database, and globally lock the door position;

[0068] Step 2: Acquire an image and use a deep convolutional neural network to perform target detection on the acquired image, and locally lock the position and posture of the hatch and its key points;

[0069] Step three: detect obstacles and control the boarding bridge to make motion response based on the detection results of obstacles by other sensors.

[0070] Specifically, in step 1, the point cloud data of the hatch and its surroundings obtained by the global locking unit of the laser radar is used. Figure 4 , the steps for the laser radar global locking unit to achieve global locking of the aircraft door posture are:

[0071] Step (11) pre-establishes an aircraft parameter database. The data contained in the database is not restricted to any source, for example, it can be established by collecting aircraft model data on-site when deploying boarding bridges at airports; it can also be established by importing aircraft model databases and other methods.

[0072] The data sources contained in the aircraft parameter database are not restricted, including on-site collection of aircraft data when deploying boarding bridges at airports; and also including establishment through various methods such as importing aircraft databases.

[0073] Step (12) obtains the original point cloud data of the target aircraft at the current viewing angle collected in real time by the laser radar, and preprocesses the original point cloud data to obtain preprocessed point cloud data.

[0074] See also Figure 5 , the collected original point cloud data is preprocessed, the specific process is as follows:

[0075] Point cloud data preprocessing refers to a series of processing and optimization of the point cloud collected by the lidar to improve the quality and availability of the data, thereby more effectively locking the target aircraft.

[0076] (1a) Convert the format of the collected raw point cloud data to ensure that the point cloud data can be read and processed. Format conversion requires converting the collected point cloud data into an appropriate data structure, such as storing it in a point cloud file format (such as PLY, PCD, etc.) or other common data formats.

[0077] (1b) Clean and filter the original point cloud data. The purpose of cleaning is to improve the quality of point cloud data and reduce the error and calculation amount of subsequent processing. Cleaning includes removing invalid or abnormal points, such as noise points, duplicate points, missing points, etc. Filtering refers to smoothing and denoising the point cloud data. Common methods include median filtering, Gaussian filtering, etc.

[0078] In this step, there is no single requirement for the brand and model of the LiDAR, but the selection process needs to ensure that equipment parameters such as accuracy and field of view are compatible with the actual scenario.

[0079] Step (13) performs point cloud registration on the pre-processed point cloud data and the established aircraft parameter database to globally lock the position and posture of the cabin door.

[0080] Determine whether the point cloud registration is successful. If the point cloud registration fails, re-register the point cloud;

[0081] If the point cloud registration is successful, the registration result Transformed into the position relationship between the pick-up port and the target aircraft door in the world coordinate system That is, after the point cloud is successfully registered, a transformation matrix can be obtained , which represents the relative position of the data collected by the current laser radar and the aircraft. Since the laser radar is fixed on the boarding bridge, the transformation matrix can be further used to determine the relative position of the boarding bridge and the target aircraft door. , that is, to achieve global locking of the target hatch.

[0082] Furthermore, by performing point cloud registration on the preprocessed point cloud data obtained in step (12) and the data in the aircraft parameter database pre-established in step (11), the relationship between the data currently collected by the laser radar and the position and posture of the aircraft can be obtained; since the laser radar is fixed on the boarding bridge, the relative position and posture of the boarding bridge and the cabin door of the target aircraft can be further locked.

[0083] The process of point cloud registration in step (13) may be carried out using different methods. This application describes the method based on feature matching as an example.

[0084] See also Figure 6 ,The method of point cloud registration is as follows: First, a feature extraction step is ,performed to extract features.

[0085] Then, the feature matching step is performed to calculate the transformation relationship between the point clouds by matching the extracted features to achieve point cloud alignment.

[0086] Feature extraction step, extracting discriminative features from the point cloud, such as SURF (Speeded Up Robust Features), SIFT (Scale Invariant Feature Transform), etc. These features can describe the geometry and appearance information of the point cloud and are robust to transformation and noise.

[0087] In the feature matching step, the extracted features are matched to find the corresponding feature point pairs in the two point clouds. The matching process is usually based on the geometric relationship between the feature points and the similarity of the descriptors, and is implemented through optimization or nearest neighbor search.

[0088] The above method for point cloud registration only provides a reference, and the actual registration process is not limited to the above method.

[0089] In step 2, the visual local locking unit is used to locally lock the position and posture of the hatch and its key points. Figure 7 The steps for the visual local locking unit to realize the local locking of the aircraft door posture are as follows:

[0090] Step (21), data acquisition: Use the visual sensor to obtain the image information of the target aircraft door in real time at the current viewing angle.

[0091] Step (22), cabin door detection. The cabin door detection is performed by inputting the image information of the current viewing angle acquired in real time in step (21) into a pre-trained target detection network model to determine the rectangular area where the cabin door is located.

[0092] See also Figure 8 , the training of the target detection network model, the specific steps are as follows:

[0093] (2a), Data acquisition. Use image acquisition equipment to photograph the cabin door and its surroundings to achieve target detection data acquisition. The distance between the image acquisition device and the target cabin door can be adjusted according to the requirements of the docking task. In order to help improve the generalization ability, robustness and practicality of the target detection network model, cabin door photos and background photos of different perspectives and different models can be appropriately added. There is no fixed restriction on the image acquisition device, for example, mobile phones, high-definition cameras, visual sensors, etc. can be used.

[0094] (2b) Data labeling: Label the collected data according to the target detection task, and use general labeling software to label the rectangular area where the hatch is located on the image. Figure 4 Two labeled images of the door target detection dataset are shown in the figure. For easy viewing, the labeled rectangular area is displayed in the original image.

[0095] (2c), model training. The door target detection data obtained in step (2b) is trained through a deep convolutional neural network to obtain a target detection model. The deep convolutional neural network used in this application is not limited to a single one. For example, network models such as Faster R-CNN, YOLO and SSD can be used. The following content only describes in detail the training process of a certain example network:

[0096] The deep convolutional neural network consists of four parts, namely the input end, Backbone (backbone network), Neck (neck network) and Head (head network).

[0097] At the input end, the dataset annotated in step (22) is first subjected to data augmentation. Data augmentation methods include random scaling, cropping, and arranging of the dataset. Data augmentation of the dataset not only enriches the dataset, but also improves the robustness of the network model.

[0098] Backbone (backbone network) processes the input image through a series of convolutional layers and pooling layers to complete the feature extraction of the data set.

[0099] Neck (neck network) fuses and upsamples the features extracted by Backbone (backbone network) to provide more advanced semantic information and the ability to adapt to images of different scales.

[0100] Head (head network) further processes the extracted features by adopting a multi-level feature fusion method to generate the final output result.

[0101] Step (23), key point extraction and positioning. Extract the pixel coordinates of the lower left and right corners of the rectangular area where the hatch is located, obtain the three-dimensional world coordinate information of the corner points based on the binocular vision sensor, and complete local locking.

[0102] The binocular vision sensor locates key points by acquiring depth information. The process of acquiring depth information is as follows:

[0103] First, two slightly offset images of the same scene are captured using two parallel cameras;

[0104] Then, by comparing the difference in pixel coordinates of the same physical point in the two images on the left and right camera images, i.e., disparity, the depth information is calculated as follows:

[0105] set up is a point on the object to be measured. and They are the optical centers of the left and right cameras, and the points The imaging points on the two camera sensors are and (The imaging plane of the camera is rotated and placed in front of the lens), is the camera focal length, is the center distance between the two cameras, is the depth information. According to the triangular geometric relationship, , rearrange the above formula, and we can get the depth information , where ( x L - x R ) represents parallax.

[0106] Finally, the baseline length and focal length information of the two cameras are combined to calculate the depth information of the corresponding point in three-dimensional space. Furthermore, the pixel coordinates in the left and right images are obtained by combining the camera principle and coordinate transformation. The three-dimensional pose in the world coordinate system is .

[0107] Through step (24), the pixel points on the hatch rectangular frame in step (23) can be mapped to three-dimensional coordinate points in the world coordinate system, and the position and posture of the target docking point can be determined.

[0108] This application realizes multi-modal autonomous aircraft door posture locking from global to local through the coordination of lidar, visual sensors and safety obstacle avoidance sensors; at the same time, while ensuring the accuracy and efficiency of boarding bridge docking, it also meets the requirements of unmanned aircraft pick-up operations, promotes the automation of large-scale special equipment at airports, and helps realize "smart airports".

[0109] In the description of the present application, it should be understood that terms such as "length", "width", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating orientation or positional relationships are based on the orientation or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present application.

[0110] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0111] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. Aircraft door posture locking system based on multi-sensor fusion, characterized in that: It includes a laser radar, a visual sensor, an obstacle avoidance sensor and a control module, wherein the control module includes a main control unit, a laser radar global locking unit, a visual local locking unit, and a safety obstacle avoidance unit; wherein, The laser radar global locking unit is connected to the laser radar and is used to perform real-time processing and positioning of point cloud data and globally lock the position and posture of the cabin door; The visual local locking unit is connected to the visual sensor and is used to perform door detection and key point positioning of the image data; The safety obstacle avoidance unit is connected to the obstacle avoidance sensor and is used to process the safety obstacle avoidance sensor data and send the data to the main control unit; The main control unit controls the boarding bridge to make a motion response according to the detection result; The steps for the laser radar global locking unit to achieve global locking of the aircraft door posture are as follows: Step (11), pre-establishing an aircraft parameter database; Step (12), obtaining the original point cloud data of the target aircraft at the current viewing angle collected in real time by the laser radar, and preprocessing the original point cloud data to obtain preprocessed point cloud data; Step (13), performing point cloud registration on the pre-processed point cloud data and the established aircraft parameter database, and globally locking the position and posture of the cabin door; The steps for the visual local locking unit to achieve local locking of the aircraft door posture are as follows: Step (21), data acquisition, using a visual sensor to acquire image information of the target aircraft door in real time at the current viewing angle; Step (22), cabin door detection, by inputting the image information of the current viewing angle acquired in real time in step (21) into a pre-trained target detection network model to perform cabin door detection and determine the rectangular area where the cabin door is located; Step (23), key point extraction and positioning, extract the pixel coordinates of the lower left and right corners of the rectangular area where the hatch is located, obtain the three-dimensional world coordinate information of the corner points based on the binocular vision sensor, and complete local locking.

2. The aircraft door position locking system based on multi-sensor fusion as claimed in claim 1, characterized in that: The control module also includes an aircraft parameter database unit and a data acquisition unit; The aircraft parameter database unit is used to load and store the prior data in the laser radar global locking unit; The data acquisition unit is used to acquire point cloud data and image data of the target door, and also includes a safety obstacle avoidance sensor to acquire distance information of obstacles.

3. The aircraft door position locking system based on multi-sensor fusion according to claim 1 or 2, characterized in that: The laser radar, visual sensor and obstacle avoidance sensor are all arranged on the boarding bridge; the obstacle avoidance sensor includes a ranging sensor, a multi-line laser radar and a wide beam radar.

4. Aircraft door posture locking method based on multi-sensor fusion, characterized in that: The following steps are involved: Step 1: Obtain point cloud data of the aircraft door and its surroundings, align the acquired point cloud data with the pre-established aircraft parameter database, and globally lock the door position; Step 2: Acquire an image and use a deep convolutional neural network to perform target detection on the acquired image, and locally lock the position and posture of the hatch and its key points; Step 3: Detect obstacles and control the boarding bridge to make motion response according to the detection result of the obstacle avoidance sensor; In step 1, the point cloud data of the cabin door and its surroundings are obtained by the laser radar global locking unit. The laser radar global locking unit implements the global locking steps of the aircraft cabin door posture as follows: Step (11), pre-establishing an aircraft parameter database; Step (12), obtaining the original point cloud data of the target aircraft at the current viewing angle collected in real time by the laser radar, and preprocessing the original point cloud data to obtain preprocessed point cloud data; Step (13), performing point cloud registration on the pre-processed point cloud data and the established aircraft parameter database, and globally locking the position and posture of the cabin door; In step 2, the visual local locking unit is used to locally lock the position and posture of the cabin door and its key points. The steps for the visual local locking unit to locally lock the position and posture of the aircraft cabin door are as follows: Step (21), data acquisition, using a visual sensor to acquire image information of the target aircraft door in real time at the current viewing angle; Step (22), cabin door detection, by inputting the image information of the current viewing angle acquired in real time in step (21) into a pre-trained target detection network model to perform cabin door detection and determine the rectangular area where the cabin door is located; Step (23), key point extraction and positioning, extract the pixel coordinates of the lower left and right corners of the rectangular area where the hatch is located, obtain the three-dimensional world coordinate information of the corner points based on the binocular vision sensor, and complete local locking.

5. The method for locking the position of an aircraft door based on multi-sensor fusion according to claim 4, characterized in that: In step (12), the process of preprocessing the collected original point cloud data is as follows: (a) Convert the format of the collected original point cloud data; (b) Cleaning and filtering of the original point cloud data.

6. The method for locking the position of an aircraft door based on multi-sensor fusion according to claim 4, characterized in that: In step (23), the binocular vision sensor locates the key points by acquiring depth information. The process of acquiring depth information is as follows: First, two slightly offset images of the same scene are captured using two parallel cameras; Then, by comparing the difference in pixel coordinates of the same physical point in the two images on the left and right camera images, that is, disparity; Finally, the baseline length and focal length information of the two cameras are combined to calculate the depth information of the corresponding points in three-dimensional space.

7. The method for locking the position of an aircraft door based on multi-sensor fusion as claimed in claim 6, characterized in that: The calculation process of depth information is: set up is a point on the object to be measured. and They are the optical centers of the left and right cameras, and the points The imaging points on the two camera sensors are and , is the camera focal length, is the center distance between the two cameras, For depth information; According to trigonometric relationship, we can get , rearrange the above formula, and we can get the depth information , where ( x L - x R ) represents parallax.

8. The method for locking the position of an aircraft door based on multi-sensor fusion according to claim 7, characterized in that: The pixel coordinates of the corresponding point The calculation formula of the three-dimensional pose in the world coordinate system is: .

Images