An aircraft parking guidance system and method based on HUD and machine vision

Abstract

This application provides an aircraft parking guidance system and method based on HUD and machine vision technology, belonging to the fields of machine vision, HUD technology, and airport surface guidance. It includes a HUD integration unit and a HUD projection and calculation unit. The integration unit is equipped with multiple cameras and radars. The integration unit sends detected external information to the HUD projection and calculation unit, which calculates the distance and azimuth deviation from the target parking space based on the features and projects this information onto the integration mirror of the HUD integration unit. This system and method can assist pilots in taxiing operations on taxiways and achieve precise aircraft parking. It can also handle complex weather conditions, improve parking efficiency, and enhance, optimize, or replace the work of aircraft guides, ground guidance vehicles, and airport automated parking equipment. Furthermore, the additional costs required to use this system are low, resulting in significant cost reduction and efficiency benefits.

Description

Technical Field

[0001] This application belongs to the fields of machine vision, HUD technology and aircraft parking guidance technology, specifically relating to an aircraft parking guidance system and method based on HUD and machine vision. Background Technology

[0002] After an aircraft lands, its movement to its taxiway parking position requires guidance from ground control vehicles and personnel. This process consumes significant manpower and resources, and can be particularly challenging in adverse weather conditions (such as extreme cold, heat, or heavy rain, which can significantly increase the workload for ground control personnel). While parking guidance terminals are now installed in airport parking areas, their installation and operation are costly and require manual operation. Currently, there is no aircraft parking guidance system or method based on HUD and machine vision technology. Summary of the Invention

[0003] To address the aforementioned issues, this application aims to provide an aircraft parking guidance system and method based on HUD and machine vision technology. It adds machine vision technology to the existing functions of HUD, acquiring relevant feature information about airport taxiways and parking positions, processing it, and then displaying it to the pilot. This enables pilots to achieve precise parking after landing with the assistance of H-VDGS.

[0004] To achieve the above objectives, this application provides an aircraft parking guidance system based on HUD and machine vision. The guidance system includes a HUD integration unit, a HUD projection unit, and a computing unit, wherein...

[0005] The HUD integration unit is used to detect external visual information and obstacle information and send the external visual information and obstacle information to the HUD projection unit. The HUD integration unit includes multiple cameras for capturing external visual information in front of the aircraft, a radar for detecting obstacle information in front, and a combination mirror.

[0006] The HUD projection component is used to calculate the distance and orientation deviation from the target berth based on the received external visual information and obstacle information, and to project the distance and orientation deviation of the target berth onto the HUD combination component.

[0007] The calculation component is used to receive the distance and orientation deviation from the target berth and draw it into a guide screen, which is then fed back to the HUD projection component. The HUD projection component projects the guide screen onto the HUD combination unit component.

[0008] The combined mirror is used to display the guide screen.

[0009] The aircraft parking guidance system based on HUD and machine vision technology provided in this application also has the following characteristics: multiple cameras with different focal lengths and different installation angles.

[0010] The aircraft parking guidance system based on HUD and machine vision technology provided in this application also has the following feature: the HUD projection component includes a feature recognition unit and an image projection unit.

[0011] The feature recognition unit uses a fusion feature recognition algorithm to detect the aircraft parking space ground markings and aircraft parking space signs outside the aircraft through obstacle information and external visual information, and then calculates the distance and orientation deviation from the target parking space.

[0012] The image projection unit is used to receive the guide screen of the computing component and project the guide screen onto the HUD combination unit.

[0013] The aircraft parking guidance system based on HUD and machine vision technology provided in this application also has the following feature: the computing component includes an image generation unit for drawing the distance and orientation deviation from the target parking space into a guidance screen.

[0014] The aircraft parking guidance system based on HUD and machine vision technology provided in this application also has the following feature: the guidance screen includes:

[0015] Scene guidance symbols are used to display taxiway guidance information;

[0016] Target detection symbols are used to indicate that the target location has not been identified;

[0017] Camera position identification symbols are used to indicate that the target camera position has been identified;

[0018] Orientation symbols are used to indicate and correct the aircraft's bearing.

[0019] Distance guidance symbols are used to indicate adjustments to the aircraft's ground speed;

[0020] The parking position symbol indicates that the aircraft has been parked correctly.

[0021] The aircraft parking guidance system based on HUD and machine vision technology provided in this application also has the following features: multiple cameras, including an external surveillance camera installed outside the aircraft to identify images of the aircraft's taxiing position; the image information acquired by the multiple external surveillance cameras is fused to obtain a parking taxiing image, which is displayed on the guidance screen.

[0022] The aircraft parking guidance system based on HUD and machine vision technology provided in this application also has the following feature: multiple external monitoring cameras, no fewer than three, are installed in the main landing gear area, the nose landing gear area and the APU device area, respectively.

[0023] The aircraft parking guidance system based on HUD and machine vision technology provided in this application also has the following feature: the HUD combination unit further includes an in-cabin voice recognition device, which is used to recognize in-cabin voice information and transmit the parking information indicated by the in-cabin voice information to the computing component and then draw it on the guidance screen.

[0024] Another objective of this application is to provide an aircraft parking guidance method based on HUD and machine vision technology, the method using the guidance system described in any of the foregoing claims, comprising the following steps:

[0025] S1: After the aircraft lands and taxis stably, the HUD automatically switches to the surface guidance mode.

[0026] S2: The HUD combination unit on the guidance system detects the characteristics of information such as taxiway, runway lights and pavement markings in front of the aircraft;

[0027] S3: The feature recognition unit in H-VDGS calculates the positional features of the taxiway centerline, taxiway side lights, and pavement signs in the taxiway area based on the detected features.

[0028] S4: H-VDGS draws a guidance screen for the pilot based on the information obtained, such as the taxiway centerline, taxiway edge lights, and taxiway markings, and the pilot performs taxiing-related operations.

[0029] S5: After the aircraft taxiway approaches the parking area, the pilot actively switches the HUD to parking guidance mode.

[0030] S6: The camera-radar combination on the H-VDGS detects the characteristics of parking position markings and signs in front of the aircraft.

[0031] S7: The feature recognition unit in H-VDGS calculates the orientation deviation, distance deviation, etc. from the ideal position of the parking position based on the detected features.

[0032] S8: H-VDGS draws a guidance screen for the pilot based on the acquired bearing deviation, distance deviation and other information. After observing the screen, the pilot then controls the aircraft to move according to the deviation information so that the aircraft can be accurately parked in the target parking position. At this time, the HUD displays a parking position symbol to indicate to the pilot that the aircraft has been parked accurately.

[0033] Beneficial effects

[0034] This invention proposes an aircraft parking guidance system and method based on HUD and machine vision technology, which can assist pilots in performing taxiing operations on taxiways and achieving precise parking. It can also cope with complex weather conditions (heavy rain, heavy snow, heavy fog, sandstorms, etc.), improve parking efficiency, and enhance, optimize or replace the work of ground aircraft guides, ground guidance vehicles and airport automatic parking equipment. Moreover, the additional cost required to use this system is low.

[0035] The beneficial effects are as follows:

[0036] (1) The system and method described in this invention can assist pilots in taxiing operations on the taxiway, reduce the workload of the guide vehicle, or replace this part of the work of the guide vehicle, and have significant cost reduction and efficiency improvement benefits.

[0037] (2) The system and method described in this invention can assist pilots in achieving precise parking positions, reduce the workload of aircraft ground guides, or replace this part of the work of aircraft ground guides, and have significant cost reduction and efficiency improvement benefits.

[0038] (3) The system and method described in this invention do not require changes to airport facilities, other systems or equipment on the aircraft except for HUD, or changes to the main structure of the aircraft. Moreover, the installation position selected on the HUD is very suitable for performing machine vision tasks. The modification cost caused by the modification work is very low, and it has significant cost reduction and efficiency improvement benefits. Attached Figure Description

[0039] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 This is a schematic diagram illustrating the system composition and working principle described in this invention;

[0041] Figure 2 This is a schematic diagram showing the installation positions of the camera and radar described in this invention;

[0042] Figure 3 This is a schematic diagram of the camera installation area according to the present invention;

[0043] Figure 4 This is a schematic diagram of the parking taxiing image of the system described in the parking guidance mode of the present invention;

[0044] Figure 5This is a schematic diagram of the parking and taxiing image of the system described in this invention in the field guidance mode;

[0045] Figure 6 This is a diagram illustrating the instructions given by the parking guide. Detailed Implementation

[0046] The present application will be further described in detail below with reference to the accompanying drawings and embodiments. However, it should be noted that these embodiments are not intended to limit the present application. Equivalent transformations or substitutions in function, method, or structure made by those skilled in the art based on these embodiments are all within the protection scope of the present application.

[0047] In the description of the embodiments of this application, it should be understood that the terms "center", "longitudinal", "lateral", "up", "down", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings. They are only for the convenience of describing the creation of this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the creation of this application.

[0048] Furthermore, the terms "first," "second," "third," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first," "second," etc., may explicitly or implicitly include one or more of that feature. In the description of this application, unless otherwise stated, "a plurality of" means two or more.

[0049] The terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to fixed connections, detachable connections, or integral connections; they can refer to mechanical connections or electrical connections; they can refer to direct connections or indirect connections through an intermediate medium; and they can refer to the internal connection between two components. Those skilled in the art can understand the specific meaning of these terms in this application based on the specific circumstances.

[0050] like Figures 1-6 As shown, an aircraft parking guidance system based on HUD and machine vision is provided. The guidance system includes a HUD integration unit, a HUD projection unit, and a computing unit, wherein...

[0051] The HUD integration unit is used to detect external visual information and obstacle information and send the external visual information and obstacle information to the HUD projection unit. The HUD integration unit includes multiple cameras for capturing external visual information in front of the aircraft, a radar for detecting obstacle information in front, and a combination mirror.

[0052] The HUD projection component is used to calculate the distance and orientation deviation from the target berth based on the received external visual information and obstacle information, and to project the distance and orientation deviation of the target berth onto the HUD combination component.

[0053] The calculation component is used to receive the distance and orientation deviation from the target berth and draw it into a guide screen, which is then fed back to the HUD projection component. The HUD projection component projects the guide screen onto the HUD combination unit component.

[0054] The combined mirror is used to display the guide screen.

[0055] In some embodiments, the multiple cameras have different focal lengths and different mounting angles. The cameras acquire data as follows: Figure 6 The image shown is of a parking guide.

[0056] In some embodiments, the HUD projection component includes a feature recognition unit and an image projection unit.

[0057] The feature recognition unit uses a fusion feature recognition algorithm to detect the aircraft parking space ground markings and aircraft parking space signs outside the aircraft through obstacle information and external visual information, and then calculates the distance and orientation deviation from the target parking space.

[0058] The image projection unit is used to receive the guide screen of the computing component and project the guide screen onto the HUD combination unit.

[0059] The feature recognition unit obtains the guide's guidance information by recognizing the guide's image information, and displays the guidance information on the guidance screen simultaneously.

[0060] In some embodiments, the computing component includes an image generation unit for drawing a guide screen based on the distance and orientation deviation from the target berth.

[0061] In some embodiments, the guide screen includes:

[0062] Scene guidance symbols are used to display taxiway guidance information;

[0063] Target detection symbols are used to indicate that the target location has not been identified;

[0064] Camera position identification symbols are used to indicate that the target camera position has been identified;

[0065] Orientation symbols are used to indicate and correct the aircraft's bearing.

[0066] Distance guidance symbols are used to indicate adjustments to the aircraft's ground speed;

[0067] The parking position symbol indicates that the aircraft has been parked correctly.

[0068] In some embodiments, the multiple cameras include an external surveillance camera installed outside the aircraft to identify images of the aircraft's taxiing position. The image information acquired by the multiple external surveillance cameras is fused to obtain a parking taxiing image, which is displayed in the guidance screen.

[0069] In some embodiments, there are at least three external surveillance cameras, which are respectively installed in the main landing gear area, the nose landing gear area, and the APU device area.

[0070] In some embodiments, the HUD assembly further includes an in-cabin voice recognition device, which is used to recognize in-cabin voice information and transmit the parking information indicated by the in-cabin voice information to the computing component and then display it on the guidance screen.

[0071] In some embodiments, an aircraft parking guidance method based on HUD and machine vision technology is provided. The method uses the guidance system described in any of the foregoing embodiments and includes the following steps:

[0072] S1: After the aircraft lands and taxis stably, the HUD automatically switches to the surface guidance mode.

[0073] S2: The HUD combination unit on the guidance system detects the characteristics of information such as taxiway, runway lights and pavement markings in front of the aircraft;

[0074] S3: The feature recognition unit in H-VDGS calculates the positional features of the taxiway centerline, taxiway side lights, and pavement signs in the taxiway area based on the detected features.

[0075] S4: H-VDGS draws a guidance screen for the pilot based on the information obtained, such as the taxiway centerline, taxiway edge lights, and taxiway markings, and the pilot performs taxiing-related operations.

[0076] S5: After the aircraft taxiway approaches the parking area, the pilot actively switches the HUD to parking guidance mode.

[0077] S6: The camera-radar combination on the H-VDGS detects the characteristics of parking position markings and signs in front of the aircraft.

[0078] S7: The feature recognition unit in H-VDGS calculates the orientation deviation, distance deviation, etc. from the ideal position of the parking position based on the detected features.

[0079] S8: H-VDGS draws a guidance screen for the pilot based on the acquired bearing deviation, distance deviation and other information. After observing the screen, the pilot then controls the aircraft to move according to the deviation information so that the aircraft can be accurately parked in the target parking position. At this time, the HUD displays a parking position symbol to indicate to the pilot that the aircraft has been parked accurately.

[0080] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the protection scope of this application. The above description is merely a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of this application, and these improvements and modifications should also be considered within the protection scope of this application.

Claims

1. An aircraft parking guidance system based on HUD and machine vision, characterized in that, The guidance system includes a HUD combination unit, a HUD projection unit, and a computing unit, wherein... The HUD integration unit is used to detect external visual information and obstacle information and send the external visual information and obstacle information to the HUD projection unit. The HUD integration unit includes multiple cameras for capturing external visual information in front of the aircraft, a radar for detecting obstacle information in front, and a combination mirror. The HUD projection component is used to calculate the distance and orientation deviation from the target berth based on the received external visual information and obstacle information, and to project the distance and orientation deviation of the target berth onto the HUD combination component. The calculation component is used to receive the distance and orientation deviation from the target berth and draw it into a guide screen, which is then fed back to the HUD projection component. The HUD projection component projects the guide screen onto the HUD combination unit component. The combined mirror is used to display the guide screen. The guide screen includes: Scene guidance symbols are used to display taxiway guidance information; Target detection symbols are used to indicate that the target location has not been identified; Camera position identification symbols are used to indicate that the target camera position has been identified; Orientation symbols are used to indicate and correct the aircraft's bearing. Distance guidance symbols are used to indicate adjustments to the aircraft's ground speed; The parking position symbol indicates that the aircraft has been parked correctly.

2. The aircraft parking guidance system based on HUD and machine vision technology according to claim 1, characterized in that, The multiple cameras have different focal lengths and different installation angles.

3. The aircraft parking guidance system based on HUD and machine vision technology according to claim 1, characterized in that, The HUD projection component includes a feature recognition unit and an image projection unit. The feature recognition unit uses a fusion feature recognition algorithm to detect the aircraft parking space ground markings and aircraft parking space signs outside the aircraft through obstacle information and external visual information, and then calculates the distance and orientation deviation from the target parking space. The image projection unit is used to receive the guide screen of the computing component and project the guide screen onto the HUD combination unit.

4. The aircraft parking guidance system based on HUD and machine vision technology according to claim 1, characterized in that, The computing component includes an image generation unit, used to draw the distance and orientation deviation from the target berth into a guide screen.

5. The aircraft parking guidance system based on HUD and machine vision technology according to claim 1, characterized in that, Multiple cameras include an external surveillance camera installed outside the aircraft to identify images of the aircraft's taxiing position. The image information acquired by the multiple external surveillance cameras is fused to obtain a parking taxiing image, which is displayed in the guidance screen.

6. The aircraft parking guidance system based on HUD and machine vision technology according to claim 5, characterized in that, There are at least three external surveillance cameras installed in the main landing gear area, the nose landing gear area, and the APU area, respectively.

7. The aircraft parking guidance system based on HUD and machine vision technology according to claim 1, characterized in that, The HUD assembly also includes an in-cabin voice recognition device, which is used to recognize in-cabin voice information and transmit the parking information indicated by the in-cabin voice information to the computing component and then display it on the guidance screen.

8. An aircraft parking guidance method based on HUD and machine vision technology, characterized in that, The method uses the guidance system as described in any one of claims 1-7, wherein the HUD projection component in the guidance system includes a feature recognition unit and an image projection unit, and the method includes the following steps: S1: After the aircraft lands and taxis stably, the HUD automatically switches to the surface guidance mode. S2: The HUD combination unit on the guidance system detects the characteristics of information such as taxiway, runway lights and pavement markings in front of the aircraft; S3: The feature recognition unit in H-VDGS calculates the positional features of the taxiway centerline, taxiway side lights, and pavement signs in the taxiway area based on the detected features; S4: H-VDGS draws a guidance screen for the pilot based on the information obtained, such as the taxiway centerline, taxiway edge lights, and taxiway markings, and the pilot performs taxiing-related operations. S5: After the aircraft taxiway approaches the parking area, the pilot actively switches the HUD to parking guidance mode. S6: The camera-radar combination on the H-VDGS detects the characteristics of the parking position markings and parking position signs in front of the aircraft. S7: The feature recognition unit in H-VDGS calculates the orientation deviation, distance deviation, etc. from the ideal position of the parking position based on the detected features; S8: H-VDGS draws a guidance screen for the pilot based on the acquired bearing deviation, distance deviation and other information. After observing the screen, the pilot then controls the aircraft to move according to the deviation information so that the aircraft can be accurately parked in the target parking position. At this time, the HUD displays a parking position symbol to indicate to the pilot that the aircraft has been parked accurately.

Images