Method and system for assisting aircraft ground taxiing based on image comparison

Through image comparison technology, the position information and environmental identification information of the aircraft are used to solve the problem of positioning deviation caused by GPS signal abnormalities, improve the accuracy and reliability of airport guidance, and ensure the safety of aircraft taxiing.

CN113673417BActive Publication Date: 2024-11-22COMMERCIAL AIRCRAFT CORP OF CHINA LTD +1
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202110953659.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-08-19
Publication Date
2024-11-22
Estimated Expiration
2041-08-19

AI Technical Summary

Technical Problem

The existing airport guidance technology relies on GPS positioning. If the GPS signal is abnormal or the resolution is not high enough, it is likely to lead to positioning deviations, affecting the accuracy of the crew in determining whether the aircraft deviates from the taxiway.

Method used

Using an image comparison method, through image acquisition and database image comparison of the aircraft's surrounding environment, the aircraft's latitude and longitude information and scene altitude are used to determine whether the aircraft deviates from the predetermined taxi route to avoid positioning deviations caused by GPS signal abnormalities.

Benefits of technology

It improves the accuracy and reliability of airport guidance, ensures that the aircraft can accurately determine whether it deviates from the taxiway during taxiing, and reduces the crew's judgment burden.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113673417B_ABST
    Figure CN113673417B_ABST
Patent Text Reader

Abstract

The present invention discloses a method for assisting aircraft ground taxiing based on image comparison, comprising: collecting images of the aircraft's surrounding environment through an image acquisition device when the aircraft is performing ground taxiing; retrieving a database image corresponding to the collected image from a database based on identification information of the environment; performing image comparison between the collected image and the database image to obtain image similarity; if the image similarity is greater than or equal to a similarity threshold, determining that the aircraft has not deviated from a predetermined taxiing route; and if the image similarity is less than the similarity threshold, determining that the aircraft has deviated from the predetermined taxiing route.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to the field of civil aircraft cockpit-assisted airport guidance, and in particular to a method and system for assisting aircraft ground taxiing based on image comparison. Background Art

[0002] In the field of airport guidance, in order to improve airborne assisted ground taxiing, relevant systems are installed in the side display to improve the taxiing efficiency of the crew. Existing airport guidance technology mainly uses image recognition to determine whether the aircraft is in the middle of the taxiway, and prevents the aircraft from deviating from the taxiway through crew operation (even automatic system operation). Airport guidance technology is mainly based on GPS positioning. Once the GPS signal is abnormal or the resolution is not high enough, it may cause positioning deviation. This will result in providing the crew with a deviated position indication, affecting the crew's judgment, and requiring the crew to spend more energy to make a judgment.

[0003] In view of the shortcomings of existing airport guidance technology, it is desired to provide an improved method and system for assisting aircraft ground taxiing based on image comparison. Summary of the invention

[0004] A brief summary of one or more aspects is given below to provide a basic understanding of these aspects. This summary is not an exhaustive overview of all conceived aspects, and is neither intended to identify the key or critical elements of all aspects nor to define the scope of any or all aspects. Its only purpose is to give some concepts of one or more aspects in a simplified form as a prelude to a more detailed description that will be given later.

[0005] The present invention provides a method for assisting aircraft ground taxiing based on image comparison, comprising:

[0006] When the aircraft is taxiing on the ground, an image of the aircraft's surrounding environment is collected by an image collection device; a database image corresponding to the collected image is retrieved from a database based on identification information of the environment; an image comparison is performed between the collected image and the database image to obtain image similarity; if the image similarity is greater than or equal to a similarity threshold, it is determined that the aircraft has not deviated from a predetermined taxiing route; and if the image similarity is less than the similarity threshold, it is determined that the aircraft has deviated from the predetermined taxiing route.

[0007] In some embodiments, performing image comparison between the captured image and the database image to obtain image similarity further includes: extracting one or more visual elements in the captured image and assigning a corresponding comparison weight to each visual element; comparing each visual element of the captured image with the corresponding visual element of the database image respectively to obtain corresponding one or more visual element similarities; and combining the one or more visual element similarities based on the comparison weight of each visual element to obtain image similarity.

[0008] In some embodiments, the one or more visual elements include one or more of the following: a digital identification graphic of a runway number, a warning sign, a road surface width, a road surface shape at a fork, a shape of an object near a fork, a light sign, and a skyline outline.

[0009] In some embodiments, the identification information of the environment includes: runway or taxiway number information, longitude and latitude information of the aircraft, and ground altitude of the aircraft.

[0010] In some embodiments, capturing images through the image capture device further includes: when it is detected that the aircraft is in a predetermined position, triggering the image capture device to capture images, wherein the predetermined position includes one or more of the following: a jet bridge, an apron, a runway junction, a taxiway junction; and when it is detected that the aircraft is performing a predetermined operation, triggering the image capture device to capture images, wherein the predetermined operation includes one or more of the following: braking, turning.

[0011] In some embodiments, each database image in the database has an image credibility, and the method further includes: if the image similarity between the acquired image and the corresponding database image is greater than or equal to a similarity threshold, then increasing the image credibility of the database image; if the image similarity between the acquired image and the corresponding database image is less than the similarity threshold, then decreasing the image credibility of the database image; and when there is a database image in the database whose image credibility is less than the credibility threshold, updating the database by replacing the database image with the corresponding acquired image.

[0012] The present invention also provides a system for assisting aircraft ground taxiing based on image comparison, comprising: an image acquisition unit, which is configured to acquire images of the aircraft's surrounding environment when the aircraft is performing ground taxiing; an image retrieval unit, which is configured to retrieve a database image corresponding to the acquired image from a database based on identification information of the environment; an image comparison unit, which is configured to perform image comparison between the acquired image and the database image to obtain image similarity; and a judgment unit, which is configured to: if the image similarity is greater than or equal to a similarity threshold, determine that the aircraft has not deviated from a predetermined taxiing route; if the image similarity is less than the similarity threshold, determine that the aircraft has deviated from the predetermined taxiing route.

[0013] In some embodiments, the image comparison unit is further configured to: extract one or more visual elements in the captured image and assign a corresponding comparison weight to each visual element; compare each visual element of the captured image with the corresponding visual element of the database image to obtain corresponding one or more visual element similarities; and combine the one or more visual element similarities based on the comparison weight of each visual element to obtain image similarity.

[0014] In some embodiments, the image acquisition unit is further configured to: automatically perform image acquisition when it is detected that the aircraft is in a predetermined position, wherein the predetermined position includes one or more of the following: a jet bridge, an apron, a runway junction, a taxiway junction; automatically perform image acquisition when it is detected that the aircraft is performing a predetermined operation, wherein the predetermined operation includes one or more of the following: braking, turning.

[0015] In some embodiments, each database image in the database has an image credibility, and the system further includes a database updating unit, which is configured to: if the image similarity between the acquired image and the corresponding database image is greater than or equal to a similarity threshold, then increase the image credibility of the database image; if the image similarity between the acquired image and the corresponding database image is less than the similarity threshold, then decrease the image credibility of the database image; and when there is a database image in the database whose image credibility is less than the credibility threshold, update the database by replacing the database image with the corresponding acquired image.

[0016] The present invention also provides a computer-readable storage medium storing a computer program, wherein the computer program can be executed by a processor to perform the aforementioned method for assisting aircraft ground taxiing based on image comparison.

[0017] The technical solution of the present invention uses the aircraft's latitude and longitude information, field height and other identification information to mark the collected image and use the identification information to search the database, compare the collected image with the image in the database to obtain image similarity, and judge whether the aircraft deviates from the predetermined taxiing route based on the image similarity. The technical solution of the present invention that assists aircraft ground taxiing based on image comparison can avoid positioning deviation caused by abnormal GPS signals and improve the accuracy and reliability of airport guidance. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The features, nature and advantages of the present invention will become more apparent when the detailed description set forth below is understood in conjunction with the accompanying drawings. In the accompanying drawings, the same reference numerals are always used for corresponding identification. It should be noted that the drawings described are only schematic and non-limiting. In the drawings, the sizes of some components may be exaggerated and are not drawn to scale for illustrative purposes.

[0019] Figure 1 A logical schematic diagram of a system for assisting aircraft ground taxiing based on image comparison according to the present invention is shown.

[0020] Figure 2 A flow chart showing an exemplary method of assisting aircraft ground taxiing based on image comparison according to the present invention.

[0021] Figure 3A flowchart of an exemplary method of updating a database based on image credibility according to the present invention is shown.

[0022] Figure 4 A block diagram of a system for assisting aircraft ground taxiing based on image comparison according to the present invention is shown.

[0023] Figure 5 The device block diagram of the system for assisting aircraft ground taxiing based on image comparison according to the present invention is shown. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical scheme and advantages of the present invention clearer, the present invention is further described in detail below in conjunction with specific embodiments and with reference to the accompanying drawings. In the following detailed description, many specific details are set forth to provide a thorough understanding of the described exemplary embodiments. However, it is obvious to those skilled in the art that the described embodiments can be practiced without some or all of these specific details. In other exemplary embodiments, well-known structures are not described in detail to avoid unnecessarily obscuring the concepts of the present disclosure. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. At the same time, in the absence of conflict, the various aspects described in the embodiments can be combined arbitrarily.

[0025] Conventional airport guidance technology based on GPS positioning is easily affected by GPS signals. When the GPS signal is abnormal or the resolution is not high enough, it may cause positioning deviation, thus affecting the crew's judgment.

[0026] The present invention provides an improved method and system for assisting aircraft ground taxiing based on image comparison, which avoids the shortcomings of airport guidance technology based on GPS positioning and realizes more accurate and reliable airport guidance.

[0027] Figure 1 FIG. 1 is a logical schematic diagram of a system 100 for assisting aircraft ground taxiing based on image comparison according to the present invention.

[0028] The aircraft can obtain its own position through the aircraft navigation signal (102) while taxiing at the airport. When the system detects that the aircraft is at a specific location of the airport (such as a jet bridge, apron, runway or taxiway intersection, etc.), the image acquisition device (106) on the aircraft (such as a camera installed at the tail, wing top, nose, etc.) can be triggered to collect images of the aircraft's surroundings. In addition, when the system detects that the aircraft is performing a specific aircraft action (104), the image acquisition device can also be triggered to collect images of the aircraft's surroundings.

[0029] The collected image can be obtained through image acquisition. At the same time, the collected image can be annotated with identification information of the environment around the aircraft. For example, the collected image can be annotated with the position information of the aircraft (for example, the latitude and longitude information of the aircraft, the altitude of the scene, etc.), thereby obtaining an image with position information (108).

[0030] The position information of the image can then be used to perform an image search in the database (110) to obtain a database image corresponding to the captured image. The two images can then be compared. Details on image search and image comparison will be described in detail in Figure 2 Described in detail in.

[0031] The comparison result can be sent to the information processing center (112), and the information processing center determines whether the aircraft deviates from the scheduled taxiing route based on the image comparison result. At the same time, the information processing center can also determine whether the database needs to be updated based on the image comparison result. Details about database updates will be described in Figure 3 Described in detail in.

[0032] Figure 2 A flow chart of an exemplary method 200 of assisting aircraft ground taxiing based on image comparison according to the present invention is shown.

[0033] The method 200 starts at 202. At 202, an image of the environment around the aircraft is collected by an image collection device when the aircraft is taxiing on the ground. For example, the image collection can be performed by a camera installed on the aircraft.

[0034] In some embodiments, image acquisition may be triggered by a specific location. Specifically, when the aircraft is detected to be at a specific location (such as a jet bridge, apron, a fork in a runway or taxiway, etc.), the image acquisition device may be triggered to perform image acquisition.

[0035] In other embodiments, image acquisition may be triggered by a specific action. Specifically, when it is detected that the aircraft is performing a specific action (such as braking, turning, etc.), the image acquisition device may be triggered to perform image acquisition.

[0036] In some embodiments, when collecting images of the environment around the aircraft, identification information of the environment may also be recorded. The identification information may include runway or taxiway number information, aircraft latitude and longitude information, and aircraft altitude. In addition, the identification information may also include time of day, weather conditions, etc. In a preferred embodiment, the collected images may be annotated with the identification information and the two may be stored together.

[0037] At 204, a database image corresponding to the acquired image is retrieved from the database based on the identification information of the environment. Specifically, the database image corresponding to the acquired image can be retrieved from the database based on the latitude and longitude information and the surface altitude of the aircraft. If multiple database images (i.e., database images having the same latitude and longitude information and surface altitude as the acquired image) are retrieved from the database based on the latitude and longitude information and the surface altitude, a database image having the closest runway or taxiway number information, time of day, weather conditions and other identification information to the acquired image can be selected.

[0038] At 206, an image comparison is performed between the collected image and the database image to obtain image similarity. The image similarity indicates the degree of similarity between the collected image and the database image.

[0039] The image similarity between the collected image and the database image can be obtained in different ways. In a preferred embodiment, the image similarity between the collected image and the database image can be obtained based on visual elements. Specifically, the image similarity can be obtained in the following way.

[0040] First, extract one or more visual elements in the collected image at 208, and assign corresponding comparison weights to each visual element. Visual elements may include one or more of the following: digital identification graphics of runway numbers, warning signs, road width, fork road surface shape, shape of objects near fork roads, light signs, skyline outlines, etc. In some implementations, comparison weights may be assigned to visual elements based on the type of visual elements. For example, the comparison results of graphics with numbers are generally more reliable, while graphics with fork road shapes may make the comparison results less reliable due to reasons such as shooting angles. Therefore, higher comparison weights may be assigned to visual elements such as digital identification graphics, while relatively lower comparison weights may be assigned to visual elements such as fork road shapes and shape of objects near fork roads.

[0041] Next, each visual element of the captured image is compared with the corresponding visual element of the database image to obtain the corresponding one or more visual element similarities at 210. Various methods can be used to compare the visual elements, such as histogram comparison method, perceptual hash algorithm, key point matching method, etc.

[0042] Finally, the one or more visual element similarities are combined based on the comparison weight of each visual element to obtain image similarity at 212. For example, the image similarity can be obtained by multiplying each visual element similarity with the corresponding comparison weight and summing the multiplication results.

[0043] After obtaining the image similarity, method 200 proceeds to 214. At 214, it is determined whether the image similarity is greater than or equal to a similarity threshold, wherein the similarity threshold can be predetermined based on actual experience or obtained through theoretical calculation. If the image similarity is greater than or equal to the similarity threshold ("yes" at 214), it is determined that the aircraft has not deviated from the scheduled taxi route (216). If the image similarity is less than the similarity threshold ("no" at 214), it is determined that the aircraft has deviated from the scheduled taxi route (218).

[0044] Additionally, the image similarity and the judgment result of whether the aircraft deviates from the predetermined taxiing route can be provided to the display device on the aircraft for reference by the crew, or provided to other aircraft ( Figure 2 not shown).

[0045] Figure 3 A flow chart of an exemplary method 300 of updating a database based on image credibility according to the present invention is shown.

[0046] The method 300 starts at 302, where each database image in the database is assigned a corresponding initial image credibility. For example, the initial image credibility may be assigned to each database image based on factors such as recognition speed and reliability, light and dark effects, and the like.

[0047] At 304 , it is determined whether the image similarity between the captured image and the corresponding database image is greater than or equal to a similarity threshold.

[0048] If the image similarity between the acquired image and the corresponding database image is greater than or equal to the similarity threshold (“yes” at 304 ), the image confidence of the database image is incremented ( 306 ).

[0049] If the image similarity between the acquired image and the corresponding database image is less than the similarity threshold (“No” at 304 ), the image confidence of the database image is decremented ( 308 ).

[0050] At 310 , it is determined whether the image credibility of the database image is greater than or equal to a credibility threshold, wherein the credibility threshold may be predetermined based on actual measured experience.

[0051] If the image confidence of all database images in the database is greater than or equal to the confidence threshold ("yes" at 310), the existing database is maintained (312).

[0052] If the image confidence of the database image in the database is less than the confidence threshold (“No” at 310 ), the database is updated. Specifically, the database may be updated by replacing the database image with the corresponding acquired image ( 314 ).

[0053] For example, when the image similarity between the image collected by the first aircraft at a certain geographical location and the corresponding database image is less than the similarity threshold, the image credibility of the database image can be decreased. Subsequently, when the image similarity between the image collected by the second aircraft at the same geographical location and the database image is still less than the similarity threshold, the image credibility of the database image can continue to be decreased. Conversely, if the image similarity between the image collected by the second aircraft at the same geographical location and the database image is greater than or equal to the similarity threshold, the image credibility of the database image is increased, and so on. When the image similarity between the images collected by enough aircraft at the same geographical location and the database image is less than the similarity threshold, the image credibility of the database image is less than the credibility threshold, and the corresponding collected image can be used to replace the database image to update the database. In this way, it can be ensured that the images in the database are sufficiently credible so that the image comparison results are sufficiently reliable.

[0054] It should be noted that Figure 3 The method is only an example of database update. In different implementations, other methods can also be used to update the database. For example, if an aircraft is determined to deviate from the scheduled taxi route when passing a certain location, the aircraft can actively request (for example, request to the information processing center) to update the database. After receiving the request, the information processing center can mark the corresponding database image in the database, but temporarily keep the database unchanged. If another aircraft is determined to deviate from the scheduled taxi route when passing the same location, the other aircraft will update the database to the information processing center again. At this time, the information processing center will determine that the corresponding database image is incorrect, and update the database by replacing the corresponding database image with the latest acquired image.

[0055] Figure 4 A block diagram of a system 400 for assisting aircraft ground taxiing based on image comparison according to the present invention is shown.

[0056] See also Figure 4 The system 400 may include an image acquisition unit 402, an image retrieval unit 404, an image comparison unit 406, a determination unit 408, and a database update unit 410. Each of these units may be directly or indirectly connected or communicated with each other on one or more buses 412.

[0057] In various embodiments of the present invention, the image acquisition unit 402 may be configured to: acquire images of the aircraft's surroundings when the aircraft is taxiing on the ground. In some embodiments, the image acquisition unit 402 may be further configured to: automatically perform image acquisition when the aircraft is detected to be in a predetermined position, wherein the predetermined position includes one or more of the following: a jet bridge, an apron, a runway junction, a taxiway junction; and automatically perform image acquisition when the aircraft is detected to be performing a predetermined operation, wherein the predetermined operation includes one or more of the following: braking, turning.

[0058] The image retrieval unit 404 may be configured to retrieve a database image corresponding to the acquired image from the database based on the identification information of the environment. The identification information of the environment may include runway or taxiway number information, aircraft latitude and longitude information, aircraft surface altitude, and the like.

[0059] The image comparison unit 406 may be configured to: perform image comparison on the collected image and the database image to obtain image similarity. In some embodiments, the image comparison unit 406 may be further configured to: extract one or more visual elements in the collected image and assign a corresponding comparison weight to each visual element; compare each visual element of the collected image with the corresponding visual element of the database image to obtain corresponding one or more visual element similarities; combine the one or more visual element similarities based on the comparison weight of each visual element to obtain image similarity.

[0060] The judgment unit 408 may be configured to: if the image similarity is greater than or equal to a similarity threshold, determine that the aircraft has not deviated from the scheduled taxi route; if the image similarity is less than the similarity threshold, determine that the aircraft has deviated from the scheduled taxi route.

[0061] In some embodiments, each database image in the database has an image credibility. In such embodiments, the database updating unit 410 may be configured to: if the image similarity between the acquired image and the corresponding database image is greater than or equal to a similarity threshold, then increase the image credibility of the database image; if the image similarity between the acquired image and the corresponding database image is less than the similarity threshold, then decrease the image credibility of the database image; when there is a database image in the database whose image credibility is less than the credibility threshold, update the database by replacing the database image with the corresponding acquired image.

[0062] Although Figure 4Specific units of system 400 are shown in the figure, but it should be understood that these units are exemplary and non-restrictive. In different implementations, one or more of these units can be combined, split, removed, or additional units can be added. For example, in some implementations, image comparison unit 406 and judgment unit 408 can be combined into a single unit. In some implementations, system 400 may include a display unit (not shown) to display the image comparison results and prompt the crew whether the aircraft deviates from the predetermined taxi route.

[0063] Figure 5 A block diagram of a device 500 including a system for assisting aircraft ground taxiing based on image comparison according to the present invention is shown.

[0064] The apparatus illustrates a general hardware environment in which the present invention may be applied according to its exemplary embodiments.

[0065] Now refer to Figure 5 Device 500 is described, which is an exemplary embodiment of a hardware device that can be applied to various aspects of the present invention. Device 500 can be any machine configured to perform processing and / or computing, and can be, but is not limited to, a workstation, a server, a desktop computer, a laptop computer, a tablet computer, a personal digital assistant (PDA), a smart phone, or any combination thereof. The above system can be implemented in whole or in part by device 500 or similar devices or systems.

[0066] Device 500 may include components that may be connected or in communication with bus 512 via one or more interfaces. For example, device 500 may include bus 512, processor 502, memory 504, input device 508, and output device 510, among other things.

[0067] The processor 502 may be any type of processor and may include, but is not limited to, a general purpose processor and / or a dedicated processor (e.g., a special processing chip), an intelligent hardware device (e.g., a general purpose processor, a DSP, a CPU, a microcontroller, an ASIC, an FPGA, a programmable logic device, a discrete gate or transistor logic component, a discrete hardware component, or any combination thereof). In some cases, the processor 502 may be configured to operate a memory array using a memory controller. In other cases, a memory controller (not shown) may be integrated into the processor 502. The processor 502 may be responsible for managing the bus and general processing, including executing software stored on the memory. The processor 502 may also be configured to perform various functions described herein related to assisting aircraft ground taxiing based on image comparison. For example, the processor 502 may be configured to: capture images of the aircraft's surroundings through an image capture device when the aircraft is taxiing on the ground; retrieve a database image corresponding to the captured image from a database based on identification information of the environment; perform image comparison between the captured image and the database image to obtain image similarity; if the image similarity is greater than or equal to a similarity threshold, determine that the aircraft has not deviated from the scheduled taxi route; and if the image similarity is less than the similarity threshold, determine that the aircraft has deviated from the scheduled taxi route.

[0068] The memory 504 may be any storage device that enables data storage. The memory 504 may include, but is not limited to, a disk drive, an optical storage device, a solid-state memory, a floppy disk, a floppy disk, a hard disk, a tape or any other magnetic medium, an optical disk or any other optical medium, a ROM (read-only memory), a RAM (random access memory), a cache memory, and / or any other memory chip or cartridge, and / or any other medium from which a computer can read data, instructions, and / or code. The memory 504 may store computer executable software 506 including computer readable instructions that, when executed, cause the processor to perform various functions described herein related to assisting aircraft ground taxiing based on image comparison.

[0069] Input device 508 may be any type of device that can be used to input information.

[0070] Output device 510 may be any type of device for outputting information. In one embodiment, output device 510 may be any type of output device that can display information.

[0071] The detailed description described above in conjunction with the accompanying drawings describes examples and does not represent all examples that can be implemented or fall within the scope of the claims. The terms "example" and "exemplary" when used in this specification mean "used as an example, instance or illustration" and do not mean "better or better than other examples".

[0072] Reference throughout this specification to "one embodiment" or "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present invention. Thus, use of these phrases may refer to more than just one embodiment. Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0073] The previous description is provided to enable any person skilled in the art to practice the various aspects described herein. Various modifications to these aspects will be readily apparent to those skilled in the art, and the universal principles defined herein may be applied to other aspects. Therefore, the claims are not intended to be limited to the aspects shown herein, but should be granted the full scope consistent with the claims in language, wherein the singular reference to the elements is not intended to mean "there is and only one", but "one or more", unless otherwise specifically stated. Unless otherwise specifically stated, the term "some" refers to one or more. The elements of the various aspects described throughout the present invention are all structurally and functionally equivalent solutions currently or hereafter known to those of ordinary skill in the art, and are expressly incorporated herein by reference, and are intended to be covered by the claims.

[0074] It should also be noted that these embodiments may be described as a process depicted as a flow chart, flow diagram, structure diagram, or block diagram. Although the flow chart may describe the operations as sequential processes, many of these operations can be performed in parallel or concurrently. In addition, the order of these operations can be rearranged.

[0075] Although various embodiments have been illustrated and described, it should be understood that the embodiments are not limited to the precise configuration and components described above. Various modifications, substitutions and improvements obvious to those skilled in the art may be made in the arrangement, operation and details of the devices disclosed herein without departing from the scope of the claims.

Claims

1. A method for assisting aircraft ground taxiing based on image comparison, comprising: When the aircraft is taxiing on the ground, an image acquisition device is used to acquire an image of the environment surrounding the aircraft; Retrieving a database image corresponding to the acquired image from a database based on identification information of the environment; Extracting one or more visual elements from the captured image and assigning a corresponding comparison weight to each visual element; Comparing each visual element of the captured image with the corresponding visual element of the database image to obtain the corresponding one or more visual element similarities; combining the one or more visual element similarities based on the comparison weight of each visual element to obtain an image similarity; If the image similarity is greater than or equal to a similarity threshold, it is determined that the aircraft has not deviated from a predetermined taxi route; as well as If the image similarity is less than the similarity threshold, it is determined that the aircraft deviates from the predetermined taxi route.

2. The method according to claim 1, characterized in that The one or more visual elements include one or more of the following: a digital identification graphic of a runway number, a warning sign, a road surface width, a road surface shape at a fork, a shape of an object near a fork, a light sign, and a skyline outline.

3. The method according to claim 1, characterized in that The identification information of the environment includes: runway or taxiway number information, longitude and latitude information of the aircraft, and ground altitude of the aircraft.

4. The method according to claim 1, characterized in that Collecting images by the image acquisition device further includes: When it is detected that the aircraft is at a predetermined position, the image acquisition device is triggered to perform image acquisition, wherein the predetermined position includes one or more of the following: a jet bridge, an apron, a runway junction, a taxiway junction; and When it is detected that the aircraft is performing a predetermined operation, the image acquisition device is triggered to perform image acquisition, wherein the predetermined operation includes one or more of the following: braking, turning.

5. The method according to claim 1, characterized in that Each database image in the database has an image credibility, and the method further comprises: If the image similarity between the captured image and the corresponding database image is greater than or equal to the similarity threshold, then the image credibility of the database image is increased; If the image similarity between the acquired image and the corresponding database image is less than the similarity threshold, decreasing the image credibility of the database image; and When the image credibility of a database image existing in the database is less than a credibility threshold, the database is updated by replacing the database image with the corresponding acquired image.

6. A system for assisting aircraft ground taxiing based on image comparison, comprising: An image acquisition unit, configured to acquire images of the aircraft's surroundings when the aircraft is taxiing on the ground; an image retrieval unit configured to retrieve a database image corresponding to the acquired image from a database based on identification information of the environment; An image comparison unit, which is configured to: Extracting one or more visual elements from the captured image and assigning a corresponding comparison weight to each visual element; Comparing each visual element of the captured image with the corresponding visual element of the database image to obtain the corresponding one or more visual element similarities; combining the one or more visual element similarities based on the comparison weight of each visual element to obtain an image similarity; as well as a judgment unit configured to: determine that the aircraft has not deviated from a predetermined taxiing route if the image similarity is greater than or equal to a similarity threshold; If the image similarity is less than the similarity threshold, it is determined that the aircraft deviates from the predetermined taxi route.

7. The system according to claim 6, characterized in that The one or more visual elements include one or more of the following: a digital identification graphic of a runway number, a warning sign, a road surface width, a road surface shape at a fork, a shape of an object near a fork, a light sign, and a skyline outline.

8. The system according to claim 6, characterized in that The identification information of the environment includes: runway or taxiway number information, longitude and latitude information of the aircraft, and ground altitude of the aircraft.

9. The system according to claim 6, characterized in that The image acquisition unit is further configured to: When the aircraft is detected to be at a predetermined position, image acquisition is automatically performed, wherein the predetermined position includes one or more of the following: a jet bridge, an apron, a runway junction, or a taxiway junction; When it is detected that the aircraft is performing a predetermined operation, image acquisition is automatically performed, wherein the predetermined operation includes one or more of the following: braking, turning.

10. The system according to claim 6, characterized in that Each database image in the database has an image credibility, and the system further comprises a database updating unit, the database updating unit being configured to: If the image similarity between the captured image and the corresponding database image is greater than or equal to the similarity threshold, then the image credibility of the database image is increased; If the image similarity between the captured image and the corresponding database image is less than the similarity threshold, then the image credibility of the database image is decreased; as well as When the image credibility of a database image existing in the database is less than a credibility threshold, the database is updated by replacing the database image with the corresponding acquired image.

11. A computer-readable storage medium storing a computer program, wherein the computer program can be executed by a processor to perform the method according to any one of claims 1 to 5.

Citation Information

Cited By

  • Knotting clearing system for evaporator

    CN224219169U