A three-dimensional flight tube mapping method on a hwd for evtol

Abstract

The application belongs to the technical field of airborne visual navigation display. The application provides a three-dimensional flight tube drawing method for an airborne visual system. The method comprises the following steps: acquiring eVTOL information, HWD attitude information and waypoint coordinate information in a flight plan route; judging a flight phase in which the eVTOL is located; drawing a flight tube according to the flight phase, the eVTOL information, the HWD attitude information and the waypoint coordinate information; and displaying on the HWD. According to the method, the accurate position of the waypoint can be indicated by drawing the flight tube in real time. In addition, the off-axis situation of the head-mounted display is considered, so that the pilot is provided with super-large field of view flight guidance under the conditions of the whole flight phase and all-weather flight, the pilot's awareness of the external scene and situation awareness are improved, and the convenience and safety of flight operation are improved.

Description

TECHNICAL FIELD

[0001] The embodiment of the present disclosure relates to the technical field of airborne visual navigation display, in particular to a three-dimensional flight tube drawing method on HWD for eVTOL. BACKGROUND

[0002] eVTOL (Electric Vertical Take-off and Landing) is a popular urban aircraft at present, which is characterized by using vertical take-off and landing technology. Compared with a commercial helicopter, it uses electricity for hovering, vertical take-off and landing, and it has advantages such as high safety, low noise and high convenience.

[0003] Although the program control technology of the unmanned aerial vehicle is very developed at present, considering the acceptance of the local government and the public, the in-loop of the pilot is still an indispensable stage in the development process of eVTOL. For the pilot, it is almost impossible to independently control each power unit of the eVTOL with a distributed power configuration. The flight guidance provided to the pilot in the form of isometric during flight can greatly reduce the pilot's operating burden and improve the operation efficiency of the eVTOL.

[0004] In addition, the introduction of visual information processing in the vertical landing stage also solves the problem of low reliability of satellite navigation and inertial navigation, and efficient and rapid landing also helps to save electric energy and ensure the endurance.

[0005] Compared with the HUD (Head Up Display), the HWD (Head Worn Display) has the advantages of small size and light weight, and uses off-axis display design, so that the field of view of the HWD is larger than that of the HUD.

[0006] Therefore, it is necessary to improve one or more problems in the related technical solutions described above.

[0007] It should be noted that this part aims to provide background or context for the technical solutions of the present disclosure stated in the claims. The description herein is not admitted to be prior art merely because it is included in this part. SUMMARY

[0008] The purpose of the embodiment of the present disclosure is to provide a three-dimensional flight tube drawing method on HWD for eVTOL, thereby at least overcoming one or more problems caused by the limitations and defects of the related art.

[0009] According to the embodiment of the present disclosure, a three-dimensional flight tube drawing method on HWD for eVTOL is provided, which comprises:

[0010] obtaining eVTOL information, HWD attitude information, and waypoint coordinate information in a flight plan route;

[0011] judging a flight phase in which the eVTOL is located, and drawing a three-dimensional flight tube according to the flight phase, the eVTOL information, the HWD attitude information, and the waypoint coordinate information, and displaying on the HWD; wherein the flight phase includes a vertical take-off phase, a fixed-angle climb phase, a constant-height cruising phase, a fixed-angle approach phase, and a vertical descent phase.

[0012] Further, if the eVTOL is in the vertical take-off phase, a vertical upward climb arrow is drawn and displayed on the HWD.

[0013] Further, a red arrow is used to represent that the current vertical speed of the eVTOL is too large, suggesting to reduce the vertical speed; a green arrow is used to represent that the current vertical speed of the eVTOL is moderate, suggesting to keep; and a yellow arrow is used to represent that the current vertical speed of the eVTOL is low, suggesting to increase.

[0014] Further, if the eVTOL is in the fixed-angle climb phase, the constant-height cruising phase, or the fixed-angle approach phase,

[0015] linear interpolation with a first preset value is performed on the waypoint coordinate information to generate a new waypoint set; wherein the waypoint set includes a plurality of waypoints;

[0016] Based on the eVTOL information, the straight-line distance between each of the waypoints and the eVTOL is calculated in the BLH coordinate system, and it is judged whether the straight-line distance is greater than a second preset value; if yes, the waypoint does not participate in display calculation;

[0017] If not, the waypoint is converted from the BLH coordinate system to the OpenGL coordinate system;

[0018] The normal direction of each of the waypoint flight tube blocks in the symbolic drawing coordinate system is calculated to form a normal direction set;

[0019] According to the coordinates of the waypoints in the OpenGL coordinate system and the normal direction set, the three-dimensional flight tube at each of the waypoints in the OpenGL coordinate system is drawn.

[0020] Further, the step of converting the waypoint from the BLH coordinate system to the OpenGL coordinate system includes:

[0021] The waypoints are converted in the order of the BLH coordinate system, the north-east-sky coordinate system, the north-east-ground coordinate system, the body coordinate system and the OpenGL coordinate system; wherein,

[0022] The conversion formula from the north-east-ground coordinate system to the body coordinate system is:

[0023]

[0024] In the formula, is a conversion matrix of the north-east-ground coordinate system to the body coordinate system, is a conversion matrix function taking the attitude of the aircraft as a variable, respectively represent the magnetic heading angle, the pitch angle and the roll angle of the aircraft;

[0025] The conversion formula from the body coordinate system to the OpenGL coordinate system is:

[0026]

[0027]

[0028]

[0029] In the formula, , respectively are the horizontal coordinate and the vertical coordinate of the three-dimensional flight pipeline at the waypoint in the symbolic drawing coordinate system, is a depth coordinate, , , is a coordinate in the body coordinate system.

[0030] Further, the calculation formula of the normal direction is:

[0031]

[0032] In the formula, is a current waypoint coordinate vector, is a next waypoint coordinate vector, is a three-dimensional flight pipeline box normal direction at the current waypoint coordinate.

[0033] Further, if the eVTOL is in the vertical landing phase, then

[0034] An image is captured by using a downward-looking camera;

[0035] A target is identified and positioned on the image by using a deep learning algorithm to locate the relative position of the airport and the eVTOL;

[0036] According to the relative position, a horizontal direction offset indication arrow symbol is displayed in the HWD.

[0037] The technical solutions provided by the embodiments of the present disclosure can include the following beneficial effects:

[0038] In the embodiments of the present disclosure, by using the three-dimensional flight tube drawing method on the HWD for eVTOL, on the one hand, the accurate position of the waypoint can be indicated by drawing the three-dimensional flight tube in real time. On the other hand, considering the off-axis situation of the head-mounted display, super-large field of view flight guidance is provided for the pilot in the whole flight stage and all-weather flight conditions, which improves the pilot's awareness of the external situation and situation awareness, and improves the convenience and safety of flight operation. BRIEF DESCRIPTION OF DRAWINGS

[0039] The accompanying drawings, which are incorporated herein and form part of the specification, illustrate embodiments consistent with the present disclosure and, together with the description, further serve to explain the principles of the disclosure. It is apparent that the accompanying drawings described below are only some embodiments of the present disclosure, and other drawings can be obtained according to these drawings without creative labor for those skilled in the art.

[0040] Figure 1 A step diagram of a three-dimensional flight tube drawing method on the HWD for eVTOL in an exemplary embodiment of the present disclosure is shown;

[0041] Figure 2 A schematic diagram of an eVTOL flight stage in an exemplary embodiment of the present disclosure is shown;

[0042] Figure 3 A three-dimensional flight tube drawing operation concept flowchart in an exemplary embodiment of the present disclosure is shown;

[0043] Figure 4 A specific flowchart of a three-dimensional flight tube drawing method on the HWD for eVTOL in an exemplary embodiment of the present disclosure is shown;

[0044] Figure 5 A three-dimensional flight tube drawing effect diagram in an exemplary embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0045] Example implementations will now be described more fully with reference to the accompanying drawings. Example implementations, however, can be implemented in many different forms and should not be construed as limited to the examples set forth herein; rather, these implementations are provided so that this disclosure will be thorough and complete, and will fully convey the scope of example implementations to those skilled in the art. The described features, structures, or characteristics can be combined in one or more implementations.

[0046] In addition, the accompanying drawings are merely schematic and are not intended to be drawn to scale. Identical reference numerals denote like or similar parts throughout the several views, so that repeated description is omitted. Some of the block diagrams shown in the drawings are functional entities, which do not necessarily correspond to physically or logically independent entities.

[0047] A three-dimensional flight tube drawing method on a HWD for eVTOL is provided in the example embodiment. Referring to FIG. 1, the three-dimensional flight tube drawing method on a HWD for eVTOL can include steps S101-S102. Figure 1

[0048] Step S101: Obtain eVTOL information, HWD attitude information, and waypoint coordinate information in a flight plan route;

[0049] Step S102: Determine the flight phase of the eVTOL, draw a three-dimensional flight tube according to the flight phase, the eVTOL information, the HWD attitude information, and the waypoint coordinate information, and display it on the HWD; wherein the flight phase includes a vertical take-off phase, a fixed angle climb phase, a constant altitude cruise phase, a fixed angle approach phase, and a vertical descent phase.

[0050] Through the above-mentioned three-dimensional flight tube drawing method on a HWD for eVTOL, on the one hand, by drawing a three-dimensional flight tube in real time, the precise position of the waypoint can be indicated. On the other hand, considering the off-axis situation of the head-mounted display, the pilot is provided with super-large field of view flight guidance in all flight phases and all-weather flight conditions, which improves the pilot's awareness of the external situation and situation awareness, and improves the convenience and safety of flight operation.

[0051] The above-mentioned three-dimensional flight tube drawing method on a HWD for eVTOL in the example embodiment will be described in more detail. Figures 1 to 5 The above-mentioned three-dimensional flight tube drawing method on a HWD for eVTOL in the example embodiment will be described in more detail.

[0052] In step S101, the eVTOL position information and attitude information are obtained in real time using an on-board computing device; the HWD attitude information is obtained in real time using an on-board computing device; and the waypoint coordinate information in the flight plan route is obtained in real time using an on-board computing device during flight.

[0053] Specifically, the on-board computing device can obtain the longitude, latitude, and altitude position coordinates and attitude information of the eVTOL through the on-board bus, or independently obtain the above-mentioned data through external sensors.

[0054] ​The airborne computing device can obtain the current attitude information of the HWD through the bus connected with the HWD, and the generation of the HWD attitude information can use any one method or a combination of multiple methods including but not limited to SLAM positioning, visual positioning and MEMS positioning.

[0055] The airborne computing device reads the waypoint information in the flight plan route given by the airborne flight plan / task management system from the on-board bus, and selects the waypoint coordinates to be drawn in the field of view according to the current position of the eVTOL and the flight phase.

[0056] In step S102, as shown in the figure, Figure 2 The flight phase includes a vertical take-off phase, a fixed-angle climbing phase, a constant-altitude cruising phase, a fixed-angle approach phase and a vertical descent phase.

[0057] As shown in the figure, Figure 3 If the eVTOL is in the vertical take-off state, an arrow symbol indicating vertical take-off is displayed in the field of view of the HWD; if the eVTOL is in the fixed-angle climbing phase, a three-dimensional flight pipeline is displayed at an equal angle in the field of view of the HWD; if the eVTOL is in the constant-altitude cruising phase, a three-dimensional flight pipeline is displayed at an equal angle in the field of view of the HWD; if the eVTOL is in the fixed-angle approach phase, a three-dimensional flight pipeline is displayed at an equal angle in the field of view of the HWD; if the eVTOL is in the vertical descent phase, a horizontal direction offset indicating arrow symbol is drawn in the field of view of the HWD. The horizontal direction offset indicating arrow symbol stimulates the target detection and positioning of the off-board downward-looking camera on the image of the airport / airfield captured by the off-board downward-looking camera.

[0058] The drawing method of each box of the three-dimensional flight pipeline in the climbing, cruising and approach phases is as follows: the algorithm first performs linear interpolation on the flight route with an interval of 200 meters, and then calculates the position of the waypoint in the field of view of the pilot according to the conversion sequence of the world coordinate system (BLH system) -> north-east-sky coordinate system -> north-east-ground coordinate system -> body coordinate system -> symbol drawing coordinate system (OpenGL coordinate system); the length of the box is determined according to the projection length of the current airspace restriction in the symbol drawing coordinate system.

[0059] Specifically, as shown in the figure, Figure 4 If the eVTOL is in the vertical take-off phase, a vertical upward climbing arrow is drawn and displayed on the HWD. A red arrow indicates that the current vertical speed of the eVTOL is too high, suggesting that the vertical speed be reduced; a green arrow indicates that the current vertical speed of the eVTOL is moderate, suggesting that it be maintained; and a yellow arrow indicates that the current vertical speed of the eVTOL is low, suggesting that it be increased.

[0060] If the eVTOL is in the fixed angle climbing stage, the constant altitude cruising stage or the fixed angle approach stage, the linear interpolation with a first preset interval is performed on the waypoint coordinate information to generate a new waypoint set; for example, the onboard computing device performs linear interpolation on the flight plan route obtained from the eVTOL flight plan / task management system according to a straight line distance of 200 meters to generate a waypoint set. The waypoint set includes a plurality of waypoints;

[0061] Based on the eVTOL information, the straight line distance between each waypoint and the eVTOL is calculated in the BLH coordinate system, and it is determined whether the straight line distance is greater than a second preset value; for example, the straight line distance between each waypoint and the eVTOL is calculated in the world coordinate system (BLH coordinate system) according to the current position coordinates of the eVTOL, and it is determined whether the distance is greater than 1000 meters. If yes, the waypoint does not participate in the display calculation;

[0062] If no, the waypoint is converted from the BLH coordinate system to the OpenGL coordinate system; that is, the conversion of the waypoint is performed from the world coordinate system (BLH system) -> the North-East-Sky coordinate system -> the North-East-Ground coordinate system -> the body coordinate system -> the symbol drawing coordinate system (OpenGL coordinate system), and the normal direction of the three-dimensional flight pipeline box at the waypoint in the symbol drawing coordinate system is calculated.

[0063] The conversion formula from the North-East-Ground coordinate system to the body coordinate system is:

[0064]

[0065] In the formula, is the conversion matrix of the North-East-Ground coordinate system to the body coordinate system, is a conversion matrix function with the attitude of the aircraft as a variable, respectively represent the magnetic heading angle, the pitch angle and the roll angle of the aircraft.

[0066] The conversion formula from the body coordinate system to the OpenGL coordinate system is:

[0067]

[0068]

[0069]

[0070] In the formula, , respectively represent the horizontal coordinate and the vertical coordinate of the three-dimensional flight pipeline at the waypoint in the symbol drawing coordinate system, is the depth coordinate, and , , is the coordinate in the body coordinate system.

[0071] The normal directions of the three-dimensional flight tube at each waypoint are calculated in the symbol drawing coordinate system to form a set of normal directions.

[0072] The normal direction of the three-dimensional flight tube at a waypoint is calculated according to the following formula:

[0073]

[0074] In the formula, is the coordinate vector of the current waypoint, is the coordinate vector of the next waypoint, is the normal direction of the three-dimensional flight tube at the coordinate of the current waypoint.

[0075] According to the coordinates and the set of normal directions of the waypoints in the OpenGL coordinate system, the three-dimensional flight tube at each waypoint is drawn in the OpenGL coordinate system.

[0076] As shown in FIG. 1, it is a rendering effect diagram of the three-dimensional flight tube. Figure 5

[0077] If the eVTOL is in the vertical landing stage, an image is collected by using a downward-looking camera; target recognition and positioning are performed on the image by using a deep learning algorithm to locate the relative position of the airport and the eVTOL; and according to the relative position, a horizontal direction offset indicating arrow symbol is drawn and displayed in the HWD.

[0078] In one specific embodiment, the method for drawing a three-dimensional flight tube on the HWD of an eVTOL includes:

[0079] 1) determining the flight stage of the eVTOL according to the flight route stored in the airborne flight plan / task management system and the current position information of the eVTOL;

[0080] 2) if the eVTOL is in the vertical takeoff stage, displaying an upward arrow symbol on the HWD;

[0081] 3) if the eVTOL is in the constant angle climbing stage, displaying a climbing flight tube symbol on the HWD;

[0082] 4) if the eVTOL is in the constant altitude cruising stage, displaying a cruising flight tube symbol on the HWD;

[0083] 5) if the eVTOL is in the fixed angle approach stage, drawing an approach flight tube symbol on the HWD;

[0084] 6) if the eVTOL is in the vertical descent stage, displaying a horizontal direction offset indicating arrow symbol on the HWD.

[0085] ​Optionally, the flight plan / task management system in 1) can be an airborne device storing flight route information specific to any aircraft model, and the current position information of the eVTOL can be obtained in real time through an airborne position sensor or a modified position sensor.

[0086] Optionally, the determination of the flight phase of the eVTOL in 1) can be achieved by comparing the fitted current position coordinates with the flight plan route coordinates.

[0087] Optionally, in 2), the color of the upward arrow symbol should be able to represent the relative size of the current vertical speed, and red, green, and yellow are selected to represent three states of eVTOL vertical speed being too large, moderate, and small, respectively.

[0088] Optionally, in 2), the length of the upward arrow symbol should be able to represent the absolute size of the current vertical speed.

[0089] Optionally, in 3), 4), and 5), in the climb, cruise, and approach phases, a series of waypoints in the world coordinate system are linearly interpolated with an interpolation interval of 200 meters, and then converted through the world (BLH) coordinate system -> North-East-Sky coordinate system -> North-East-Ground coordinate system -> body coordinate system -> symbol drawing coordinate system (OpenGL) system.

[0090] Optionally, in 3), 4), and 5), in the climb, cruise, and approach phases, the size of the three-dimensional flight tube is set to 50 meters * 50 meters.

[0091] Optionally, in 6), in the vertical descent phase, the airborne computing device obtains video information of the target airport / airfield from the downward-looking camera, calculates the relative horizontal position of the eVTOL and the target airport / airfield using image processing algorithms, and guides the pilot to adjust the position of the eVTOL in the forward, backward, left, and right directions through the horizontal direction offset indicator arrow symbol.

[0092] Further, the present application also proposes an airborne device comprising a processor, a program memory, a head-mounted display, and a downward-looking camera. When the program of the memory is loaded by the processor, the video processing and three-dimensional flight tube symbol drawing algorithms described above are executed.

[0093] Through the above-mentioned three-dimensional flight tube drawing method on the HWD for eVTOL, on the one hand, the accurate position of the waypoints can be indicated by drawing the three-dimensional flight tube in real time. On the other hand, considering the off-axis situation of the head-mounted display, the pilot is provided with super-large field of view flight guidance in all flight phases and all-weather flight conditions, which improves the pilot's awareness of the external situation and situation awareness, and improves the convenience and safety of flight operation.

[0094] It should be understood that the terms "center", "longitudinal", "transverse", "length", "width", "thickness", "upper", "lower", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise" and the like indicating the orientation or positional relationship in the above description are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the embodiments of the present disclosure and simplifying the description, and do not indicate or imply that the device or element referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation on the embodiments of the present disclosure.

[0095] In addition, the terms "first", "second", "third" and the like are used only for the purpose of description, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the technical features indicated. Therefore, the features defined with "first", "second" can explicitly or implicitly include one or more of the features. In the description of the embodiments of the present disclosure, the meaning of "plurality" is two or more, unless otherwise explicitly specified and limited.

[0096] In the embodiments of the present disclosure, unless otherwise explicitly specified and limited, the terms "mounting", "connecting", "connecting", "fixing" and the like should be understood in a broad sense, for example, can be fixedly connected, can be detachably connected, or integrated; can be mechanically connected, or electrically connected; can be directly connected, or indirectly connected through an intermediate medium; can be the internal communication of two elements or the interaction relationship between two elements. For those skilled in the art, the specific meaning of the above terms in the present disclosure can be understood according to the specific circumstances.

[0097] In the embodiments of the present disclosure, unless otherwise explicitly specified and limited, the first feature "on" or "under" the second feature can include that the first and second features are in direct contact, or that the first and second features are not in direct contact but are in contact through another feature between them. Moreover, the first feature "on", "above" and "on" the second feature includes that the first feature is directly above and obliquely above the second feature, or only indicates that the horizontal height of the first feature is higher than that of the second feature. The first feature "under", "below" and "under" the second feature includes that the first feature is directly below and obliquely below the second feature, or only indicates that the horizontal height of the first feature is less than that of the second feature.

[0098] In the description of the disclosure, the description of the terms "one embodiment", "some embodiments", "an example", "a specific example" or "some examples" etc. means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the disclosure. In the description of the disclosure, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any appropriate manner in one or more embodiments or examples. In addition, those skilled in the art can combine and combine different embodiments or examples described in the specification.

[0099] Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses or adaptations of the disclosure that follow the general principles thereof and include the general principles thereof and include other known or customary features not specifically mentioned herein. The specification and examples are to be regarded as illustrative only, and the true scope and spirit of the disclosure are indicated by the appended claims.

Claims

1. A method for three-dimensional flight tube mapping on a HWD for eVTOL, characterized by, The method comprises: Obtaining eVTOL information, HWD attitude information, and waypoint coordinate information in a flight plan route; Determining the flight phase of the eVTOL, and drawing a three-dimensional flight tube according to the flight phase, the eVTOL information, the HWD attitude information, and the waypoint coordinate information, and displaying the three-dimensional flight tube on the HWD; wherein the flight phase comprises a vertical take-off phase, a fixed-angle climb phase, a constant-altitude cruise phase, a fixed-angle approach phase, and a vertical descent phase; If the eVTOL is in the vertical take-off phase, a vertical upward climb arrow is drawn and displayed on the HWD; If the eVTOL is in the fixed-angle climb phase, the constant-altitude cruise phase, or the fixed-angle approach phase, Linear interpolation with a first preset value is performed on the waypoint coordinate information to generate a new waypoint set; wherein the waypoint set comprises a plurality of waypoints; Based on the eVTOL information, the straight-line distance between each waypoint and the eVTOL is calculated in the BLH coordinate system, and it is determined whether the straight-line distance is greater than a second preset value; if yes, the waypoint does not participate in display calculation; If not, the waypoint is converted from the BLH coordinate system to the OpenGL coordinate system; The normal direction of each waypoint flight tube block in the symbol drawing coordinate system is calculated to form a normal direction set; According to the coordinates of the waypoints in the OpenGL coordinate system and the normal direction set, the three-dimensional flight tube at each waypoint in the OpenGL coordinate system is drawn; The step of converting the waypoint from the BLH coordinate system to the OpenGL coordinate system comprises: The waypoint is converted in the order of the BLH coordinate system, the North-East-Sky coordinate system, the North-East-Ground coordinate system, the body coordinate system, and the OpenGL coordinate system; wherein, The conversion formula from the North-East-Ground coordinate system to the body coordinate system is: wherein is a conversion matrix from the north-east-geodetic coordinate system to the body coordinate system, is a conversion matrix function with the aircraft attitude as a variable, respectively represent the magnetic heading angle, the pitch angle, the roll angle of the aircraft. The conversion formula from the body coordinate system to the OpenGL coordinate system is: wherein , are the horizontal and vertical coordinates of the three-dimensional flight tube at the waypoint in the symbol plot coordinate system, is the depth coordinate, and , , are the coordinates in the body coordinate system.

2. The method for 3D flight tube mapping on HWDs for eVTOLs of claim 1, wherein, The calculation formula of the normal direction is: wherein is the current waypoint coordinate vector, is the next waypoint coordinate vector, is the three-dimensional flight tube box normal direction at the current waypoint coordinate.

3. The method for 3D flight tube mapping on HWDs for eVTOLs of claim 1, wherein, If the eVTOL is in the vertical descent phase, An image is captured using a downward-looking camera; A target recognition and positioning algorithm is used to identify the relative position of the airport and the eVTOL based on the image; According to the relative position, a horizontal direction offset indicator arrow symbol is drawn and displayed in the HWD.

Images