Auxiliary display method, device and equipment of aircraft tractor and readable storage medium

Abstract

The application provides an airplane towing vehicle auxiliary display method, device, equipment and readable storage medium. The method comprises the following steps: collecting an image of an airplane during landing sliding; determining an airplane body reference line based on the image; wherein the airplane body reference line is used for identifying the position of the airplane; identifying a ground guide line from the image, and determining a guide reference line in the ground guide line according to the ground guide line and the airplane body reference line; the reference line is used for identifying a reference route for towing the airplane; determining the relative position between the airplane body reference line and the guide reference line; and displaying the airplane body reference line and the guide reference line on a towing vehicle display interface based on the relative position. The method of the application improves the accuracy of the driver in judging the relative position of the airplane body and the guide line, reduces the difficulty of correctly towing the airplane, and enhances the safety.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of aircraft towing technology, and in particular to an aircraft towing vehicle auxiliary display method, device, equipment and readable storage medium. BACKGROUND

[0002] Large aircrafts usually cannot complete reverse or consume high fuel cost by their own engines, so they usually need to complete reverse operation by aircraft towing vehicles. The aircraft towing vehicle is a working vehicle for towing or pushing the aircraft, and is an indispensable ground guarantee equipment in modern airports. Through the towing of the towing vehicle, the aircraft can realize the functions of entering the warehouse, moving to the runway, etc., that is, assisting the aircraft to the designated position of the apron. Due to the characteristics of large flow, many large equipment and heavy safety responsibility of the airport environment, in order to ensure the order and safety of the aircraft operation, guide lines will be drawn on the ground, and the aircraft is required to travel according to the guide lines.

[0003] In the prior art, when the aircraft is towed to the designated position by the towing vehicle, the relative position of the aircraft body and the guide line is usually judged by the driver of the towing vehicle relying on human experience.

[0004] However, since the aircraft body and the towing vehicle are not rigidly linked, and the aircraft body is large, there is a situation of visual obstruction, which leads to inaccurate judgment of the relative position of the aircraft body and the guide line by the driver, and brings certain difficulty and safety hazard to correctly tow the aircraft. SUMMARY

[0005] The present application provides an aircraft towing vehicle auxiliary display method, device, equipment and readable storage medium to solve the problem that the relative position of the aircraft body and the guide line judged by the driver through human experience is inaccurate in the prior art, causing difficulty in towing the aircraft.

[0006] In a first aspect, the present application provides an aircraft towing vehicle auxiliary display method, comprising:

[0007] Collecting images of the aircraft during landing and taxiing;

[0008] Determining an aircraft body reference line based on the images; wherein the aircraft body reference line is used to identify the position of the aircraft;

[0009] Identifying a ground guide line from the images, and determining a guide reference line in the ground guide line according to the ground guide line and the aircraft body reference line; the guide reference line is used to identify a reference route for towing the aircraft;

[0010] Determining the relative position between the aircraft body reference line and the guide reference line;

[0011] Based on the relative position, the fuselage reference line and the guide reference line are displayed on a tractor display interface.

[0012] In an alternative embodiment, the determining the aircraft fuselage reference line based on the image comprises:

[0013] Identifying an aircraft contour feature point in the image;

[0014] Determining the aircraft fuselage reference line according to the aircraft contour feature point.

[0015] In an alternative embodiment, the determining the aircraft fuselage reference line according to the aircraft contour feature point comprises:

[0016] Determining an aircraft contour feature straight line segment according to the aircraft contour feature point, and taking the feature straight line segment as the aircraft fuselage reference line.

[0017] In an alternative embodiment, the identifying the ground guide line from the image comprises:

[0018] Segmenting the collected image into a binary image through a semantic segmentation algorithm; the binary image is used to identify pixel values in the image;

[0019] Identifying the ground guide line from the binary image.

[0020] In an alternative embodiment, the determining the guide reference line in the ground guide line comprises:

[0021] Selecting a feature region containing the ground guide line according to the aircraft contour feature point;

[0022] Identifying a plurality of target guide lines in the feature region which do not intersect with the rest of the guide lines through a region growing algorithm;

[0023] Determining the guide reference line from the plurality of target guide lines.

[0024] In an alternative embodiment, the determining the guide reference line from the plurality of target guide lines comprises:

[0025] Calculating a perpendicular distance from a midpoint of a line connecting two rear wheels of the aircraft in the aircraft contour feature point to each of the target guide lines, and a residual rotation distance from the aircraft fuselage reference line to each of the target guide lines;

[0026] Determining the guide reference line according to the perpendicular distance and the residual rotation distance.

[0027] In an alternative embodiment, the relative position between the fuselage reference line and the guide reference line comprises:

[0028] an angle between the guide reference line and the fuselage reference line;

[0029] and / or a distance between the midpoints of the guide reference line and the fuselage reference line.

[0030] In a second aspect, the present application provides an auxiliary display device for a tug of an airplane, comprising:

[0031] a collection module configured to collect an image of the airplane during a landing taxiing process;

[0032] a first determination module configured to determine a fuselage reference line of the airplane based on the image, wherein the fuselage reference line is used to identify a position of the airplane;

[0033] an identification module configured to identify a ground guide line from the image;

[0034] a second determination module configured to determine a guide reference line in the ground guide line based on the ground guide line and the fuselage reference line of the airplane, wherein the guide reference line is used to identify a reference route for towing the airplane;

[0035] a display module configured to display the fuselage reference line and the guide reference line on a display interface of the tug based on the relative position.

[0036] In a third aspect, the present application provides an electronic device, comprising: a display, a memory, and a processor.

[0037] the display is configured to display the relative position between the fuselage reference line and the guide reference line and to display under the control of the processor;

[0038] the memory is configured to store executable instructions;

[0039] the processor is configured to run the instructions stored in the memory to execute the airplane tug auxiliary display method of any one of the first aspect.

[0040] In a fourth aspect, the present application provides a computer readable storage medium, wherein the computer readable storage medium stores computer execution instructions, and the computer execution instructions are executed by a processor to implement the airplane tug auxiliary display method of any one of the first aspect.

[0041] The application provides an airplane towing vehicle auxiliary display method, device, equipment and readable storage medium. The airplane towing vehicle auxiliary display method comprises the following steps: collecting an image of an airplane during landing and taxiing, and determining a fuselage reference line for identifying a position of the airplane based on the image. On the other hand, a ground guide line can be further identified from the image, and a guide reference line is determined in the ground guide line to identify a reference route for towing the airplane according to the ground guide line and the fuselage reference line of the airplane. After determining the relative position between the fuselage reference line and the guide reference line, the fuselage reference line and the guide reference line are displayed on a towing vehicle display interface. The method provided by the application improves the accuracy of the driver in judging the relative position between the airplane fuselage and the guide line, reduces the difficulty of correctly towing the airplane, and enhances the safety. BRIEF DESCRIPTION OF DRAWINGS

[0042] The accompanying drawings, which are incorporated herein and form a part of the specification, illustrate embodiments consistent with the present application and, together with the description, further serve to explain the principles of the application.

[0043] Figure 1A A scene schematic diagram in which a towing vehicle tows an airplane is applicable to an embodiment of the application.

[0044] Figure 1B A visual interface display schematic diagram is provided for an embodiment of the application.

[0045] Figure 2 A flowchart of an airplane towing vehicle auxiliary display method is provided for an embodiment of the application.

[0046] Figure 3 A flowchart of another airplane towing vehicle auxiliary display method is provided for an embodiment of the application.

[0047] Figure 4 A schematic diagram of an airplane towing vehicle auxiliary display device is provided for an embodiment of the application.

[0048] Figure 5 A structural schematic diagram of an electronic device is provided for an embodiment of the application.

[0049] The above-described drawings have shown the specific embodiments of the application, and the following will have a more detailed description. The drawings and the written description are not intended to limit the scope of the concept of the application by any means, but to illustrate the concept of the application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0050] The exemplary embodiments will be described in detail herein with reference to the attached drawings. In the following description, like reference numerals refer to like elements, unless indicated otherwise. The implementations described in the following exemplary embodiments are not meant to represent all implementations consistent with the present disclosure. Rather, they are simply examples of apparatuses and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0051] First, the terms involved in the present application are explained:

[0052] Yolo: refers to You Only Look Once, a real-time detection algorithm, which is used in the present application to detect images;

[0053] PaddleSeg: is a complete and easy-to-use industrial-grade segmentation model developed by Baidu based on its own PaddlePaddle, which is a complete and easy-to-use industrial-grade segmentation model;

[0054] HRnet: refers to High-Resoultion Net, a neural network model, which is used in the present application to implement image segmentation.

[0055] The application scenario of the present application can be applied to an aircraft tractor, as shown in Figure 1A , as shown in Figure 1A , which is a schematic diagram of a tractor towing an aircraft, including an aircraft 01 and a tractor 02. The driver determines the relative position of the aircraft body reference line and the ground guide line reference line through the visual interface, thereby assisting the driver to drive the tractor to tow the aircraft to the designated position. The visual interface display content is shown in Figure 1B , as shown in Figure 1B , which is a schematic diagram of a visual interface display, wherein the thick solid line represents the body reference line, and the thin solid line represents the guide reference line. The method of the present application aims to solve the technical problem of difficult aircraft towing caused by inaccurate relative position of the aircraft body and the guide line determined by human experience in the related art. Of course, the aircraft tractor provided by the present application can assist image display, including but not limited to the above application scenarios. As long as the scene involves aircraft towing, the method of the present application can be used.

[0056] The technical solutions of the present application and how the technical solutions of the present application solve the above technical problems will be described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of the present application will be described below with reference to the drawings.

[0057] Figure 2 is a flowchart of an aircraft tractor auxiliary display method provided by an embodiment of the present application, as shown inFigure 2 As shown, the method can be performed by a device mounted on a tractor, and can include the following steps:

[0058] S1, collecting images of the aircraft during the landing taxiing process.

[0059] When the image acquisition device collects images of the aircraft during the landing taxiing process, the image acquisition device can be any device or equipment that can collect images, for example, it can be a camera.

[0060] The main purpose of the aircraft tractor is to tow the aircraft to the designated position, so the tractor is generally located in front of the aircraft to ensure that the tractor driver can accurately determine the position of the aircraft and the tractor. In addition, in order to ensure that the collected images are complete and clear, the image acquisition device needs to be installed on the tractor at a position where it can collect the above images, i.e. to determine the installation position of the image acquisition device. For example, the image acquisition device can be installed on the top of the tractor.

[0061] Through the camera mounted on the top of the tractor, the airport area is monitored, the video image of the airport area is obtained, and the specific target is detected from the video image. In this embodiment, the collected images can include the aircraft fuselage, the aircraft wheels and the ground guide line. Through this step, the image collection process of the aircraft during the landing taxiing process is completed.

[0062] S2, determining the aircraft fuselage reference line based on the image; wherein the aircraft fuselage reference line is used to identify the position of the aircraft.

[0063] Optionally, the image collected in the previous step can transmit the data shot by the camera to the cloud through the Internet, and analyze and process the image data through cloud computing.

[0064] Optionally, the data shot by the camera can also be directly transmitted to the vehicle-mounted system, and the vehicle-mounted system uses its own computing resources to analyze and process the image data.

[0065] In this step, the aircraft fuselage reference line is determined by detecting the position of the aircraft contour. The aircraft fuselage reference line is used to identify the position of the aircraft.

[0066] S3, identifying the ground guide line from the image, and determining the guide reference line in the ground guide line according to the ground guide line and the aircraft fuselage reference line; the guide reference line is used to identify the reference route of towing the aircraft.

[0067] The ground guide line is a lane marking guiding the moving direction of the aircraft, and is used to indicate the direction in which the aircraft should move when being towed. The ground guide line is identified from the image. It can be understood that, due to the large area of the airport, when the aircraft is being towed, there can be situations such as aircraft turning, straight driving, turning, and merging. Therefore, there are numerous guide lines on the ground that are detected.

[0068] In order to correctly tow the aircraft, the guide reference line most suitable for towing the aircraft is selected from the numerous guide lines according to the ground guide line and the aircraft body reference line.

[0069] The ground guide reference line can be represented as the reference route most suitable for towing the aircraft to the specified position, that is, the reference route for identifying the towed aircraft.

[0070] S4, determining the relative position between the aircraft body reference line and the guide reference line.

[0071] After the aircraft body reference line and the ground guide reference line are determined, the relative position between them is obtained. The relative position relationship can be the angle and distance between the aircraft body reference line and the ground guide reference line.

[0072] S5, displaying the aircraft body reference line and the guide reference line on the towing vehicle display interface based on the relative position.

[0073] After step S4, the relative position relationship between the aircraft body reference line and the guide reference line is obtained, and then the calculation result is displayed on the towing vehicle visualization interface through the cloud or the vehicle-mounted system. The driver can monitor the current state of the aircraft body relative to the ground guide line in real time through the visualization interface to assist the driver in completing the aircraft towing operation.

[0074] The aircraft towing vehicle auxiliary display method provided in this embodiment can identify the position of the aircraft by collecting images of the aircraft during landing and taxiing and determining the aircraft body reference line based on the images. At the same time, the ground guide line is identified from the images, and the guide reference line is determined in the ground guide line based on the ground guide line and the aircraft body reference line, so as to identify the reference route for towing the aircraft. Then, after the relative position between the aircraft body reference line and the guide reference line is determined, the aircraft body reference line and the guide reference line are displayed on the towing vehicle display interface based on the relative position. The method of this embodiment can improve the accuracy of the driver in judging the relative position of the aircraft body and the guide line by visualizing the relative position of the aircraft body and the guide line, so as to assist the driver to tow the aircraft to the specified position more safely.

[0075] Figure 3 Another aircraft towing vehicle auxiliary display method flow chart provided in this embodiment is shown in Figure 3 Based on the above embodiment, the method can include the following steps to achieve:

[0076] First, the image collected by the camera is analyzed and processed:

[0077] In one possible implementation, the image collected in the previous step is uploaded to the cloud through internet technology, and the image data captured by the camera is uploaded to the cloud. The cloud service stores the image data in the server through functions such as real-time data synchronization and fast data loading, so as to facilitate the analysis and processing of the image data in the next step. When analyzing and processing the image data, cloud computing can be performed through the cloud. Cloud computing is centered on the Internet and provides fast and secure cloud computing services and data storage on the website. Users can select relevant cloud computing services according to their needs, and the cloud will feed back the results to the users, thereby completing the analysis and processing of the image data.

[0078] In another possible implementation, the image data can be directly transmitted to the vehicle-mounted system through the Internet. The vehicle-mounted system uses its own computing resources to analyze and process the image data through artificial intelligence algorithms.

[0079] The specific analysis and processing process is as follows:

[0080] In S2, the aircraft fuselage reference line is determined based on the image, which can include:

[0081] S21, identifying the aircraft contour feature points in the image;

[0082] S22, determining the aircraft fuselage reference line according to the aircraft contour feature points.

[0083] Specifically, the aircraft contour feature straight line segment is determined according to the aircraft contour feature points, and the feature straight line segment is taken as the aircraft fuselage reference line. In this application, any target detection algorithm can be used to detect the contour feature points. For example, the target detection algorithm used in this embodiment is the open source library Yolo target detection algorithm. The image is classified to identify the aircraft contour feature points in the image.

[0084] In this embodiment, the aircraft wheels are taken as the aircraft contour feature points for illustration, wherein the point of the aircraft front wheel in the image is taken as the first point, and the midpoint of the line connecting the centers of the two rear wheels is taken as the second point. Connecting the two points obtains a feature straight line segment, which represents the fuselage reference line. The fuselage reference line represents the position of the aircraft.

[0085] It should be noted that the target detection algorithm used in this embodiment is only used to detect the aircraft wheel image and does not limit the present application.

[0086] In addition, in the embodiment, the positions of the three wheels are detected to obtain position information of the three wheels, and the position of the fuselage is determined according to the relationship between the positions of the three wheels. It should be particularly noted that the method of determining the fuselage reference line by the aircraft wheels cannot be regarded as a limitation of the present application. The determination of the aircraft fuselage reference line can also be determined by selecting other parts of the aircraft, for example, taking the head of the aircraft as the first point and the tail of the aircraft as the second point, and connecting the two points to obtain a straight line segment which can be used as the aircraft fuselage reference line. For example, the midpoint of the line connecting the centers of the two wings of the aircraft is taken as the first point, and the tail of the aircraft is taken as the second point, and the straight line segment obtained by connecting the two points can also be used as the aircraft fuselage reference line. Since there are many possible ways, they will not be listed one by one here.

[0087] In S3, the ground guide line is identified from the image, and the guide reference line is determined in the ground guide line according to the ground guide line and the aircraft fuselage reference line, which can include:

[0088] S31, the collected image is segmented into a binary image by a semantic segmentation algorithm; the binary image is used to identify the pixel value in the image.

[0089] By any kind of semantic segmentation algorithm, the semantic segmentation algorithm in the embodiment uses HRnet in the open source library paddleseg. Semantic segmentation refers to the process of associating each pixel in an image to a class label. By semantic segmentation, the image is segmented into a binary image.

[0090] Among them, the binary image represents that each pixel on the image has only two possible values or gray states, and in the embodiment, the two possible values are 1 and 0.

[0091] S32, the ground guide line is identified from the binary image.

[0092] The pixel area containing the aircraft wheels obtained in the above steps is subjected to semantic segmentation, and the segmentation result is that the pixel value belonging to the guide line in the area is 1 and the others are 0. Through this step, the image content captured by the camera is classified, so that all the guide lines on the picture are identified.

[0093] It should be noted that due to the large area of the airport, there are situations such as aircraft turning angle, straight running, turning or merging when towing the aircraft, therefore, there are many guide lines on the ground detected.

[0094] The above-mentioned artificial intelligence algorithms, i.e. target detection algorithm and semantic segmentation algorithm, need to be pre-labeled and then learned for the aircraft wheels and the ground guide lines. The specific learning process belongs to the routine operation of the algorithm, and can be referred to related technologies, which will not be described here.

[0095] Further, in order to determine the relative position of the aircraft fuselage and the guide line, the next step is to determine the guide reference line. Specifically,

[0096] S321, according to the aircraft contour feature points, a feature region containing the ground guide line is selected.

[0097] Similarly, the aircraft contour feature points are taken as an example to illustrate the selection of the feature region containing the guide line according to the positions of the three aircraft wheels. The feature region is selected in order to filter out the redundant guide lines.

[0098] In the above binary image, the feature region selected by the embodiment is a rectangular region, which is centered on the three aircraft wheels and expanded, so as to obtain the ground guide line pixel points in the rectangular region.

[0099] The selection of the rectangular region is based on the center coordinates of the three aircraft wheels. Optionally, the coordinate system can adopt a Cartesian rectangular coordinate system, and the position of the origin can be specified by the user. In the embodiment, it is assumed that the center coordinates of the three aircraft wheels are in the same quadrant of the coordinate system. Among them, the center coordinates of the three aircraft wheels are (x1, y1), (x2, y2), and (x3, y3), respectively. The diagonal vertices of the rectangular region are (x min , y min ), (x max , y max ), x min is the minimum value in {x1, x2, x3}, x max is the maximum value in {x1, x2, x3}, y min is the minimum value in {y1, y2, y3}, and y max is the maximum value in {y1, y2, y3}.

[0100] The purpose of this step is to filter out redundant guide lines and ensure that the aircraft fuselage reference line and the ground guide reference line are in the same picture. It should be noted that the rectangular region centered on the three aircraft wheels can also be directly formed by the three aircraft wheels as the vertices to form a triangular region, or an arbitrary shaped region, as long as the above purpose is achieved. Therefore, the rectangular region in the embodiment cannot be regarded as a limitation of the present application.

[0101] Further,

[0102] S322, a plurality of target guide lines which do not intersect with the remaining guide lines in the feature region are identified by a region growing algorithm.

[0103] In the above step, when detecting the ground guide lines in the binary image, all the guide lines on the picture are obtained, but this step can only detect all the guide lines and cannot obtain the number of guide lines, and cannot separate the guide lines independently. Therefore, each guide line needs to be separated, and the separation means that each guide line is not intersected with each other.

[0104] In step S321, after the region is determined, a directional region growing algorithm is used to identify a plurality of target guide lines in the feature region which are not intersected with the remaining guide lines.

[0105] The specific method is:

[0106] According to the result of the semantic segmentation, a point with a pixel value of 1 in the result of the semantic segmentation is selected, and a point with the smallest pixel coordinate y value is selected as a seed point. In this embodiment, the seed point is not specified by a person, and only the starting point of the algorithm growth is provided, so other points can also be specified as seed points, and therefore cannot be regarded as a limitation on the present application. After the seed point is determined, the region growing direction is specified, and in this embodiment, five growing directions are specified, which are {(-1, 0), (1, 0), (0, 1), (1, 1), (-1, 1)}, and similarly, the growing direction can also be specified, and therefore cannot be regarded as a limitation on the present application.

[0107] After the seed point and the growing direction are determined, the pixel position is moved in five directions according to the specified region growing direction from the seed point. If the pixel value of the pixel position moved to is the same as the pixel value of the seed point, the point is taken as a new seed point, the original seed point is deleted, and the growth is performed. The above steps are iterated until no new seed point is added, and then the directional region growing is completed, and all the historical seed points are marked as a guide line.

[0108] Then, the directional region growing algorithm is continuously applied to the remaining points with a pixel value of 1 in the semantic segmentation result until all the points with a pixel value of 1 are marked as guide lines.

[0109] In this embodiment, by using the directional region growing algorithm, the seed point and the growing direction are specified, and a plurality of independent guide lines in the feature region are identified.

[0110] Further,

[0111] S323, from the plurality of target guide lines, a guide reference line is determined.

[0112] When the tractor is towing the airplane, a reference route cannot be randomly selected for towing, and a best guide route needs to be determined. In this embodiment, the determination process of the best guide reference line can be:

[0113] The perpendicular distance from the midpoint of the line connecting the two rear wheels of the aircraft to each target guide line and the relative rotation distance from the aircraft body reference line to each target guide line are calculated;

[0114] The guide reference line is determined according to the perpendicular distance and the relative rotation distance.

[0115] In the embodiment, the aircraft contour feature points are the aircraft wheels, the perpendicular distance d from the midpoint of the line connecting the two rear wheels of the aircraft to each guide line and the relative rotation distance d from the aircraft body reference line to each guide line are calculated. c It should be noted that the relative rotation distance d c is not the Euclidean distance, but the relative rotation according to the angle between the aircraft body reference line and each guide line.

[0116] According to the calculated d and d c, the multiple independent guide lines are scored and evaluated, and the guide line with the highest score is taken as the ground guide reference line.

[0117] The scoring formula is as follows:

[0118]

[0119] wherein d is the perpendicular distance from the midpoint of the line connecting the two rear wheels of the aircraft to each guide line;

[0120] d c is the relative rotation distance from the aircraft body reference line to each guide line.

[0121] Further,

[0122] After the aircraft body reference line and the guide reference line are determined, the relative position between the aircraft body reference line and the guide reference line is calculated.

[0123] The position relationship includes the angle between the guide reference line and the aircraft body reference line, and the distance between the midpoints of the guide reference line and the aircraft body reference line. The relative position is determined by calculating the angle between the guide reference line and the aircraft body reference line and the distance between the midpoints of the guide reference line and the aircraft body reference line.

[0124] The above position relationship is transmitted to the vehicle-mounted visualization device and displayed on the display screen of the vehicle-mounted visualization device, and the driver can obtain the current position of the aircraft body relative to the guide line in real time according to the display screen of the tractor, thereby assisting the driver to tow the aircraft to the specified position.

[0125] The embodiment provides an intelligent auxiliary image display method of an airplane towing vehicle, a fuselage reference line and a guide reference line are obtained through a related artificial intelligence algorithm, so that the relative position between the fuselage reference line and the ground guide reference line is obtained, and the detection result is transmitted to a vehicle-mounted visual device. The method shown in the embodiment can help the driver accurately judge the relative position of the airplane fuselage and the guide line through the vehicle-mounted visual interface, so that the difficulty of correctly towing the airplane is reduced, and the safety is improved.

[0126] Figure 4 An airplane towing vehicle auxiliary display device schematic diagram provided by the embodiment of the application, the device comprises:

[0127] The acquisition module 41 is configured to acquire images of the airplane during landing and taxiing;

[0128] The first determination module 42 is configured to determine a fuselage reference line of the airplane based on the images; wherein the fuselage reference line of the airplane is used to identify the position of the airplane;

[0129] The identification module 43 is configured to identify a ground guide line from the images;

[0130] The second determination module 44 is configured to determine a guide reference line in the ground guide line according to the ground guide line and the fuselage reference line of the airplane; the guide reference line is used to identify a reference route for towing the airplane;

[0131] The display module 45 is configured to display the fuselage reference line and the guide reference line on a display interface of the towing vehicle based on the relative position.

[0132] Figure 5 An electronic device structure schematic diagram is provided for the application. As shown in 5, the device can include at least one display 51, a processor 52 and a memory 53. Figure 5 An electronic device with one processor is shown.

[0133] The display 51 is configured to display the relative position of the fuselage reference line and the ground guide reference line, and display under the control of the processor 52;

[0134] The memory 53 is configured to store programs. Specifically, the programs can include program codes, and the program codes include computer operation instructions.

[0135] The memory 53 can include a high-speed RAM memory, and can also include a non-volatile memory, for example, at least one disk memory.

[0136] The processor 52 is configured to execute the computer execution instructions stored in the memory 53, so as to realize the airplane towing vehicle image display method;

[0137] The processor 52 can be a central processing unit (CPU) or an application specific integrated circuit (ASIC) or one or more integrated circuits configured to implement one or more embodiments of the present application. The processor 52 implements the method of assisting display of the aircraft tractor by running instructions stored in the memory 53.

[0138] Optionally, in a specific implementation, if the communication interface, the memory 53 and the processor 52 are implemented independently, the communication interface, the memory 53 and the processor 52 can be connected to each other through a bus and complete communication with each other. The bus can be an industry standard architecture (ISA) bus, a peripheral component (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc., but it does not mean that there is only one bus or one type of bus.

[0139] Optionally, in a specific implementation, if the communication interface, the display 51, the memory 53 and the processor 52 are integrated on a chip, the communication interface, the display 51, the memory 53 and the processor 52 can complete communication through an internal interface.

[0140] The present application also provides a computer readable storage medium, which can include a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk and various storage program codes. Specifically, the computer readable storage medium stores program information, and the program information is used for assisting display of the aircraft tractor.

[0141] Computer-readable media includes permanent and non-permanent, movable and non-movable media, which can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage device, or any other non-transmission medium that can be used to store information accessible by a computing device.

[0142] The embodiment of the present application also provides a program product, which, when executed by a processor, is used to execute the aircraft tractor auxiliary display method provided by the above method embodiment.

[0143] In the above embodiments, all or part of them can be realized by software, hardware, firmware or any combination thereof. When realized by software, all or part of them can be realized in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions according to the embodiments of the present application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) mode. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. integrated with one or more available media. The available media can be magnetic media (such as floppy disk, hard disk, magnetic tape), optical media (such as DVD), or semiconductor media (such as solid state disk (SSD)) and the like.

[0144] Other embodiments of the application will be apparent to those skilled in the art from consideration of the specification and practice of the application disclosed herein. It is intended that the specification and examples be considered as exemplary only, with the true scope and spirit of the application being indicated by the following claims.

[0145] It is to be understood that the application is not limited to the precise construction herein described and as shown in the attached drawings, and that various modifications and changes can be made by those skilled in the art without departing from the scope of the application. The scope of the application is to be indicated by the appended claims, rather than the description and examples.

Claims

1. An aircraft tug auxiliary display method, characterized by, include: Capture images of the aircraft during its landing and taxiing process; The aircraft fuselage reference line is determined based on the image; wherein, the aircraft fuselage reference line is used to identify the location of the aircraft; the aircraft fuselage reference line is obtained based on the straight line segment connecting the feature points of the aircraft outline; Ground guide lines are identified from the image, and a guide reference line is determined from the ground guide lines based on the ground guide lines and the aircraft fuselage reference lines; the guide reference line is used to identify the reference route for towing the aircraft. Determine the relative position between the fuselage reference line and the guide reference line; Based on the relative position, the fuselage reference line and the guide reference line are displayed on the tractor display interface; Determining the guide reference line in the ground guide line includes: Based on the aircraft's outline feature points, select the feature region containing the ground guide line; The region growing algorithm identifies multiple target guide lines within the feature region that do not intersect with the other guide lines. The guide reference line is determined from the plurality of target guide lines; Determining the guide reference line from the plurality of target guide lines includes: Calculate the vertical distance from the midpoint of the line connecting the two rear wheels of the aircraft to each of the target guide lines, and the cosine distance from the aircraft fuselage reference line to each of the target guide lines. The guide reference line is determined based on the vertical distance and the cosine distance; the cosine distance is calculated by taking the cosine of the angle between the fuselage reference line and each guide line.

2. The method of claim 1, wherein, The step of identifying ground guide lines from the image includes: The acquired image is segmented into a binary image using a semantic segmentation algorithm; the binary image is used to identify the pixel values ​​in the image. The ground guide line is identified from the binary image.

3. The method according to claim 1 or 2, characterized in that, The relative positions between the fuselage reference line and the guide reference line include: The angle between the guide reference line and the fuselage reference line; And / or, the distance between the midpoints of the guide reference line and the fuselage reference line.

4. An auxiliary display device for an aircraft tractor, characterized in that, include: The acquisition module is used to acquire images of the aircraft during its landing and taxiing process; The first determining module is used to determine an aircraft fuselage reference line based on the image; wherein, the aircraft fuselage reference line is used to identify the location of the aircraft; the aircraft fuselage reference line is obtained based on the straight line segment connecting the aircraft's contour feature points; The recognition module identifies the ground guide lines from the image; The second determining module is used to determine a guiding reference line in the ground guiding line based on the ground guiding line and the aircraft fuselage reference line; the guiding reference line is used to identify the reference route for towing the aircraft; and to determine the relative position between the fuselage reference line and the guiding reference line. The display module is used to display the fuselage reference line and the guide reference line on the tractor display interface based on the relative position; The second determining module is specifically used for: Based on the aircraft's outline feature points, select the feature region containing the ground guide line; The region growing algorithm identifies multiple target guide lines within the feature region that do not intersect with the other guide lines. The guide reference line is determined from the plurality of target guide lines; Determining the guide reference line from the plurality of target guide lines includes: Calculate the vertical distance from the midpoint of the line connecting the two rear wheels of the aircraft to each of the target guide lines, and the cosine distance from the aircraft fuselage reference line to each of the target guide lines. The guide reference line is determined based on the vertical distance and the cosine distance; the cosine distance is calculated by taking the cosine of the angle between the fuselage reference line and each guide line.

5. An auxiliary display device for an aircraft tractor, characterized in that, include: A display, a processor, and a memory communicatively connected to the processor; The display is used to show the relative positions of the fuselage reference line and the guide reference line, and the display is performed under the control of the processor. The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as described in any one of claims 1 to 3.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the aircraft tractor auxiliary display method as described in any one of claims 1 to 3.

Images