Path classification for detection

Abstract

Method for classifying and managing vehicle paths, comprising: receiving a first dataset 208, (Fig.4) comprising a first set of images of vehicle paths (504, Figs.5&10); training an AI model 208, (Fig

Description

TECHNICAL FIELD

[0001] The invention is generally directed to path classification. In particular it provides a method, system, computer program product and a computer program for managing vehicle paths. Background Art

[0002] In aviation, a missed approach is a procedure followed by a pilot when an instrument approach cannot be completed to a full-stop landing. A "go-around” is an aborted landing of an aircraft that is on final approach. A go-around can either be initiated by the pilot or requested by air traffic control for various reasons, such as an un stabilized approach or an obstruction on the runway.

[0003] A flight dispatcher (also known as an airline dispatcher or flight operations officer) assists in planning flight paths, taking into account aircraft performance and loading, enroute winds, thunderstorm and turbulence forecasts, airspace restrictions, and airport conditions. Dispatchers also provide a flight following service and advise pilots if conditions change. In the United States and Canada, the flight dispatcher shares legal responsibility with the commander of the aircraft (joint responsibility dispatch system), so it is imperative that deviations from scheduled flight schedules are identified as soon as possible. A dispatcher may be responsible for a number of concurrent flights, so prompt identification of go-arounds is essential. The problem for dispatchers is different from air traffic control (ATC), because dispatchers usually work in the operations center of the airline, and not at a destination airport, in contrast with ATC.

[0004] A dispatcher during a duty shift, is assigned a number of flights. One of the problems from our customers is that as they managing this list of flights there is no way of knowing or being alerted that go-arounds are occurring unless the dispatcher pro-actively checks that the aircraft has landed or not. It not reported in any of the available data streams and would not be available to the dispatcher until much later if the pilot reports it to the airline or dispatcher. Knowing as early as possible when a go-around has occurred will give them a better situational awareness of what is happening with the flights they manage.

[0005] Operations management solutions exist that provide early insight into changing flight, airport, and airspace conditions. Some incorporate a streamlined workflow so flight dispatchers can make decisions with confidence and improve safety and efficiency of the airline operation. In this way, improvements in efficiency, safety, and reliability of aviation operations can be made.

[0006] However, existing solutions do not cover all scenarios.

[0007] Therefore, there is a need in the art to address the aforementioned problem. SUMMARY OF INVENTION

[0008] According to the present invention there are provided a method, a system, a computer program product, and a computer program according to the independent claims.

[0009] Viewed from a first aspect, the present invention provides a computer implemented method for managing vehicle paths, the method comprising: receiving a first dataset, the dataset comprising a first set of images of vehicle paths; training an Al model with the first set of images to determine a first model and a set of classifications for each of the images for a first location; identifying a first vehicle; creating a first image of a first path taken by the first vehicle; applying the first image to the model to determine a first classification of the set of classifications for the first image path; and based on the first classification, performing an action related to the first vehicle.

[0010] Viewed from a first aspect, the present invention provides a system for managing vehicle paths, the system operable for: receiving a first dataset, the dataset comprising a first set of images of vehicle paths; training an Al model with the first set of images to determine a first model and a set of classifications for each of the images for a first location; identifying a first vehicle; creating a first image of a first path taken by the first vehicle; applying the first image to the model to determine a first classification of the set of classifications for the first image path; and based on the first classification, performing an action related to the first vehicle.

[0011] Viewed from a further aspect, the present invention provides a computer program product for managing vehicle paths, the computer program product comprising a computer readable storage medium readable by a processing circuit and storing instructions for execution by the processing circuit for performing a method for performing the steps of the invention. Viewed from a further aspect, the present invention provides a computer program stored on a computer readable medium and loadable into the internal memory of a digital computer, comprising software code portions for performing the steps of the invention, when said program is run on a computer,.

[0012] Preferably, the present invention provides a method, system, computer program product and computer program, wherein receiving the first dataset comprises creating the first data set by gathering data, the data comprising at least one of vehicle position reports, vehicle images, and location data.

[0013] Preferably, the present invention provides a method, system, computer program product and computer program, wherein the vehicle comprises one of air transport, ground transport, and sea transport.

[0014] Preferably, the present invention provides a method, system, computer program product and computer program, wherein the first image is added to the first dataset to create a second set of images, and the Al model is retrained with the second set of images.

[0015] Preferably, the present invention provides a method, system, computer program product and computer program, further comprising: identifying a second vehicle; creating a second image of a second path taken by the second vehicle; comparing the first image with the second image to determine a composite image; applying the composite image to the model to determine a composite classification of the set of classifications for the composite image; based on the composite classification, performing an action related to at least one of the first vehicle and the second vehicle.

[0016] Preferably, the present invention provides a method, system, computer program product and computer program, wherein the set of classifications comprises at least one of a successful path, a go-around path, a runway miss path, a lane veer path, a collision path.

[0017] Preferably, the present invention provides a method, system, computer program product and computer program, wherein the Al model is taken from a list, the list comprising: a CNN; a GAN; a Transformer-Based Models; an Autoencoder; an RNN, a Large Language Models; and YOLO.

[0018] Preferably, the present invention provides a method, system, computer program product and computer program, wherein creating a first image of a first path taken by the first vehicle comprises creating timed images for the first vehicle over successive time slices, and collating the timed images to determine the first image.

[0019] Preferably, the present invention provides a method, system, computer program product and computer program, wherein applying the first image to the model to determine a first classification comprises determining a closest match of the first image from the dataset.

[0020] Preferably, the present invention provides a method, system, computer program product and computer program, wherein determining the closest match comprises matching the first image with partial images of the dataset.

[0021] This invention details a method to identify aircraft go-arounds by using aircraft position reports, flight data and airport data to first create datasets of flight path images that represent the flight path of an aircraft as it comes towards a landing at an airport along with the underlying details and image of the runways direction and length. In this manner a collection of historical aircraft approaches can be created in imagery with those representing go-arounds classified as such using for example a CNN. This model can then be created per airport and used to classify whether real-time incoming aircraft are in fact in a go-around scenario or not. Models can be retrained using actual go-around data after the fact and user feedback to train models based on different times of year at the airport in question.

[0022] Advantageously, the present invention enables a new feature that provides additional situational awareness to the dispatcher. When a "go-around” is detected, a dispatcher knows that there is still an aircraft that needs tracking and the opportunity to assist still exists.

[0023] This invention details how the use of a created image can be used to detect whether a go-around event has occurred or not. In addition, prediction of whether a go-around will occur can also be made. There is a period of time coming up towards an airport where data is gathered to create images, The classifier could predict paths if classification is in place and training data for "not yet past the runway” image scenarios.

[0024] The aircraft location over a period of time, along with the location of runways for the destination International Civil Aviation Organisation (ICAO) is used to create a synthetic image of the flight path of the aircraft along with overlays. A time period is calculated based on when the flight path should begin to be drawn and when it should stop being drawn. Using image classification methods, we then identify whether a flight has initiated a go-around.

[0025] Advantageously, the present invention identifies aircraft go-arounds by using aircraft position reports, flight data and airport data to first create datasets of flight path images that represent the flight path of an aircraft as it comes towards a landing at an airport along with the underlying details and image of the runways direction and length. In this manner a collection of historical aircraft approaches can be created in imagery with those representing go-arounds classified as such using, for example, a CNN. This model can then be created per airport and used to classify whether real-time incoming aircraft are in a go-around scenario or not. Models can be retrained using actual go-around data after the fact and user feedback to train models based on different times of year at the airport in question.

[0026] Advantageously, vehicle control is enhanced for human driven and autonomous vehicles in addition to existing sensors. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The present invention will now be described, by way of example only, with reference to preferred embodiments, as illustrated in the following figures:

[0028] FIG. 1 depicts a computing environment 100, according to an embodiment of the present invention;

[0029] FIG. 2 depicts a high-level exemplary schematic flow diagram 200 depicting operation methods steps identifying and actioning a path, according to a preferred embodiment of the present invention;

[0030] FIG. 3 depicts an aircraft system 300, according to a preferred embodiment of the present invention;

[0031] FIG. 4 depicts method steps for creating an archive of historical images, according to an embodiment of the present invention;

[0032] FIG. 5 depicts images 506 of flight paths 504, according to an embodiment of the present invention;

[0033] FIG.6 depicts method steps for analysing data, according to an embodiment of the present invention;

[0034] FIG. 7 depicts method steps for creating a classification model 906, according to an embodiment of the present invention;

[0035] FIG. 8 depicts method steps for creating a real-time images, according to an embodiment of the present invention;

[0036] FIG. 9 depicts software components 201, according to an embodiment of the present invention; and

[0037] FIG. 10 depicts a vehicle system 1000, according to an embodiment of the present invention. DETAILED DESCRIPTION

[0038] Various aspects of the present disclosure are described by narrative text, flowcharts, block diagrams of computer systems and / or block diagrams of the machine logic included in computer program product (GPP) embodiments. With respect to any flowcharts, depending upon the technology involved, the operations can be performed in a different order than what is shown in a given flowchart. For example, again depending upon the technology involved, two operations shown in successive flowchart blocks may be performed in reverse order, as a single integrated step, concurrently, or in a manner at least partially overlapping in time.

[0039] A computer program product embodiment ("GPP embodiment" or "CPP”) is a term used in the present disclosure to describe any set of one, or more, storage media (also called "mediums") collectively included in a set of one, or more, storage devices that collectively include machine readable code corresponding to instructions and / or data for performing computer operations specified in a given CPP claim. A "storage device" is any tangible device that can retain and store instructions for use by a computer processor. Without limitation, the computer readable storage medium may be an electronic storage medium, a magnetic storage medium, an optical storage medium, an electromagnetic storage medium, a semiconductor storage medium, a mechanical storage medium, or any suitable combination of the foregoing. Some known types of storage devices that include these mediums include: diskette, hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or Flash memory), static random access memory (SRAM), compact disc read-only memory (CD-ROM), digital versatile disk (DVD), memory stick, floppy disk, mechanically encoded device (such as punch cards or pits I lands formed in a major surface of a disc) or any suitable combination of the foregoing. A computer readable storage medium, as that term is used in the present disclosure, is not to be construed as storage in the form of transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide, light pulses passing through a fiber optic cable, electrical signals communicated through a wire, and / or other transmission media. As will be understood by those of skill in the art, data is typically moved at some occasional points in time during normal operations of a storage device, such as during access, de-fragmentation or garbage collection, but this does not render the storage device as transitory because the data is not transitory while it is stored.

[0040] FIG. 1 depicts a computing environment 100. Computing environment 100 contains an example of an environment for the execution of at least some of the computer code involved in performing the inventive methods, such as software functionality 201 for improved processing of vehicle paths. In addition to block 201, computing environment 100 includes, for example, computer 101, wide area network (WAN) 102, end user device (EUD) 103, remote server 104, public cloud 105, and private cloud 106. In this embodiment, computer 101 includes processor set 110 (including processing circuitry 120 and cache 121), communication fabric 111, volatile memory 112, persistent storage 113 (including operating system 122 and block 201, as identified above), peripheral device set 114 (including user interface (UI) device set 123, storage 124, and Internet of Things (loT) sensor set 125), and network module 115. Remote server 104 includes remote database 130. Public cloud 105 includes gateway 140, cloud orchestration module 141, host physical machine set 142, virtual machine set 143, and container set 144.

[0041] COMPUTER 101 may take the form of a desktop computer, laptop computer, tablet computer, smart phone, smart watch or other wearable computer, mainframe computer, quantum computer or any other form of computer or mobile device now known or to be developed in the future that is capable of running a program, accessing a network or querying a database, such as remote database 130. As is well understood in the art of computer technology, and depending upon the technology, performance of a computer-implemented method may be distributed among multiple computers and / or between multiple locations. On the other hand, in this presentation of computing environment 100, detailed discussion is focused on a single computer, specifically computer 101, to keep the presentation as simple as possible. Computer 101 may be located in a cloud, even though it is not shown in a cloud in FIG. 1. On the other hand, computer 101 is not required to be in a cloud except to any extent as may be affirmatively indicated.

[0042] PROCESSOR SET 110 includes one, or more, computer processors of any type now known or to be developed in the future. Processing circuitry 120 may be distributed over multiple packages, for example, multiple, coordinated integrated circuit chips. Processing circuitry 120 may implement multiple processor threads and / or multiple processor cores. Cache 121 is memory that is located in the processor chip package(s) and is typically used for data or code that should be available for rapid access by the threads or cores running on processor set 110. Cache memories are typically organized into multiple levels depending upon relative proximity to the processing circuitry. Alternatively, some, or all, of the cache for the processor set may be located "off chip.” In some computing environments, processor set 110 may be designed for working with qubits and performing quantum computing.

[0043] Computer readable program instructions are typically loaded onto computer 101 to cause a series of operational steps to be performed by processor set 110 of computer 101 and thereby effect a computer-implemented method, such that the instructions thus executed will instantiate the methods specified in flowcharts and / or narrative descriptions of computer-implemented methods included in this document (collectively referred to as "the inventive methods”). These computer readable program instructions are stored in various types of computer readable storage media, such as cache 121 and the other storage media discussed below. The program instructions, and associated data, are accessed by processor set 110 to control and direct performance of the inventive methods. In computing environment 100, at least some of the instructions for performing the inventive methods may be stored in block 201 in persistent storage 113.

[0044] COMMUNICATION FABRIC 111 is the signal conduction path that allows the various components of computer 101 to communicate with each other. Typically, this fabric is made of switches and electrically conductive paths, such as the switches and electrically conductive paths that make up busses, bridges, physical input I output ports and the like. Other types of signal communication paths may be used, such as fiber optic communication paths and / or wireless communication paths.

[0045] VOLATILE MEMORY 112 is any type of volatile memory now known or to be developed in the future. Examples include dynamic type random access memory (RAM) or static type RAM. Typically, volatile memory 112 is characterized by random access, but this is not required unless affirmatively indicated. In computer 101, the volatile memory 112 is located in a single package and is internal to computer 101, but, alternatively or additionally, the volatile memory may be distributed over multiple packages and / or located externally with respect to computer 101.

[0046] PERSISTENT STORAGE 113 is any form of non-volatile storage for computers that is now known or to be developed in the future. The non-volatility of this storage means that the stored data is maintained regardless of whether power is being supplied to computer 101 and / or directly to persistent storage 113. Persistent storage 113 may be a read only memory (ROM), but typically at least a portion of the persistent storage allows writing of data, deletion of data and re-writing of data. Some familiar forms of persistent storage include magnetic disks and solid state storage devices. Operating system 122 may take several forms, such as various known proprietary operating systems or open source Portable Operating System Interface-type operating systems that employ a kernel. The code included in block 201 typically includes at least some of the computer code involved in performing the inventive methods, for example in the client functionality 1200, and / or the server functionality 1300.

[0047] PERIPHERAL DEVICE SET 114 includes the set of peripheral devices of computer 101. Data communication connections between the peripheral devices and the other components of computer 101 may be implemented in various ways, such as Bluetooth connections, Near-Field Communication (NFC) connections, connections made by cables (such as universal serial bus (USB) type cables), insertion-type connections (for example, secure digital (SD) card), connections made through local area communication networks and even connections made through wide area networks such as the internet. In various embodiments, UI device set 123 may include components such as a display screen, speaker, microphone, wearable devices (such as goggles and smart watches), keyboard, mouse, printer, touchpad, game controllers, and haptic devices. Storage 124 is external storage, such as an external hard disk, or insertable storage, such as an SD card. Storage 124 may be persistent and / or volatile. In some embodiments, storage 124 may take the form of a quantum computing storage device for storing data in the form of qubits. In embodiments where computer 101 is required to have a large amount of storage (for example, where computer 101 locally stores and manages a large database) then this storage may be provided by peripheral storage devices designed for storing very large amounts of data, such as a storage area network (SAN) that is shared by multiple, geographically distributed computers. loT sensor set 125 is made up of sensors that can be used in Internet of Things applications. For example, one sensor may be a thermometer and another sensor may be a motion detector.

[0048] NETWORK MODULE 115 is the collection of computer software, hardware, and firmware that allows computer 101 to communicate with other computers through WAN 102. Network module 115 may include hardware, such as modems or Wi-Fi signal transceivers, software for packetizing and / or de-packetizing data for communication network transmission, and / or web browser software for communicating data over the internet. In some embodiments, network control functions and network forwarding functions of network module 115 are performed on the same physical hardware device. In other embodiments (for example, embodiments that utilize software-defined networking (SDN)), the control functions and the forwarding functions of network module 115 are performed on physically separate devices, such that the control functions manage several different network hardware devices. Computer readable program instructions for performing the inventive methods can typically be downloaded to computer 101 from an external computer or external storage device through a network adapter card or network interface included in network module 115.

[0049] WAN 102 is any wide area network (for example, the internet) capable of communicating computer data over non-local distances by any technology for communicating computer data, now known or to be developed in the future. In some embodiments, the WAN 102 may be replaced and / or supplemented by local area networks (LANs) designed to communicate data between devices located in a local area, such as a Wi-Fi network. The WAN and / or LANs typically include computer hardware such as copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and edge servers.

[0050] END USER DEVICE (EUD) 103 is any computer system that is used and controlled by an end user (for example, a customer of an enterprise that operates computer 101), and may take any of the forms discussed above in connection with computer 101. EUD 103 typically receives helpful and useful data from the operations of computer 101. For example, in a hypothetical case where computer 101 is designed to provide a recommendation to an end user, this recommendation would typically be communicated from network module 115 of computer 101 through WAN 102 to EUD 103. In this way, EUD 103 can display, or otherwise present, the recommendation to an end user. In some embodiments, EUD 103 may be a client device, such as thin client, heavy client, mainframe computer, desktop computer and so on.

[0051] REMOTE SERVER 104 is any computer system that serves at least some data and / or functionality to computer 101. Remote server 104 may be controlled and used by the same entity that operates computer 101. Remote server 104 represents the machine(s) that collect and store helpful and useful data for use by other computers, such as computer 101. For example, in a hypothetical case where computer 101 is designed and programmed to provide a recommendation based on historical data, then this historical data may be provided to computer 101 from remote database 130 of remote server 104.

[0052] PUBLIC CLOUD 105 is any computer system available for use by multiple entities that provides on-demand availability of computer system resources and / or other computer capabilities, especially data storage (cloud storage) and computing power, without direct active management by the user. Cloud computing typically leverages sharing of resources to achieve coherence and economies of scale. The direct and active management of the computing resources of public cloud 105 is performed by the computer hardware and / or software of cloud orchestration module 141. The computing resources provided by public cloud 105 are typically implemented by virtual computing environments that run on various computers making up the computers of host physical machine set 142, which is the universe of physical computers in and / or available to public cloud 105. The virtual computing environments (VCEs) typically take the form of virtual machines from virtual machine set 143 and / or containers from container set 144. It is understood that these VCEs may be stored as images and may be transferred among and between the various physical machine hosts, either as images or after instantiation of the VCE. Cloud orchestration module 141 manages the transfer and storage of images, deploys new instantiations of VCEs and manages active instantiations of VCE deployments. Gateway 140 is the collection of computer software, hardware, and firmware that allows public cloud 105 to communicate through WAN 102.

[0053] Some further explanation of virtualized computing environments (VCEs) will now be provided. VCEs can be stored as "images.” A new active instance of the VCE can be instantiated from the image. Two familiar types of VCEs are virtual machines and containers. A container is a VCE that uses operating-system-level virtualization. This refers to an operating system feature in which the kernel allows the existence of multiple isolated user-space instances, called containers. These isolated user-space instances typically behave as real computers from the point of view of programs running in them. A computer program running on an ordinary operating system can utilize all resources of that computer, such as connected devices, files and folders, network shares, CPU power, and quantifiable hardware capabilities. However, programs running inside a container can only use the contents of the container and devices assigned to the container, a feature which is known as containerization.

[0054] PRIVATE CLOUD 106 is similar to public cloud 105, except that the computing resources are only available for use by a single enterprise. While private cloud 106 is depicted as being in communication with WAN 102, in other embodiments a private cloud may be disconnected from the internet entirely and only accessible through a local / private network. A hybrid cloud is a composition of multiple clouds of different types (for example, private, community or public cloud types), often respectively implemented by different vendors. Each of the multiple clouds remains a separate and discrete entity, but the larger hybrid cloud architecture is bound together by standardized or proprietary technology that enables orchestration, management, and / or data / application portability between the multiple constituent clouds. In this embodiment, public cloud 105 and private cloud 106 are both part of a larger hybrid cloud.

[0055] The descriptions of the various embodiments of the present disclosure have been presented for purposes of illustration, but are not intended to be exhaustive or limited to the embodiments disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope of the described embodiments. The terminology used herein was chosen to best explain the principles of the embodiments, the practical application or technical improvement over technologies found in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein. It will be readily understood that the components of the application, as generally described and illustrated in the figures herein, may be arranged and designed in a wide variety of different configurations. Thus, the detailed description of the embodiments is not intended to limit the scope of the application as claimed but is merely representative of selected embodiments of the application.

[0056] One having ordinary skill in the art will readily understand that the above invention may be practiced with steps in a different order, and / or with hardware elements in configurations that are different than those which are disclosed. Therefore, although the application has been described based upon these preferred embodiments, it would be apparent to those of skill in the art that certain modifications, variations, and alternative constructions would be apparent.

[0057] While preferred embodiments of the present application have been described, it is to be understood that the embodiments described are illustrative only and the scope of the application is to be defined solely by the appended claims when considered with a full range of equivalents and modifications (e.g., protocols, hardware devices, software platforms etc.) thereto.

[0058] Moreover, the same or similar reference numbers are used throughout the drawings to denote the same or similar features, elements, or structures, and thus, a detailed explanation of the same or similar features, elements, or structures will not be repeated for each of the drawings. The terms "about" or "substantially" as used herein with regard to thicknesses, widths, percentages, ranges, etc., are meant to denote being close or approximate to, but not exactly. For example, the term "about" or "substantially" as used herein implies that a small margin of error is present. Further, the terms "vertical" or "vertical direction" or "vertical height" as used herein denote a Z-direction of the Cartesian coordinates shown in the drawings, and the terms "horizontal," or "horizontal direction," or "lateral direction" as used herein denote an X-direction and / or Y-direction of the Cartesian coordinates shown in the drawings.

[0059] Additionally, the term "illustrative” is used herein to mean "serving as an example, instance or illustration.” Any embodiment or design described herein is intended to be "illustrative” and is not necessarily to be construed as preferred or advantageous over other embodiments or designs.

[0060] It is to be understood that although this disclosure includes a detailed description on cloud computing, implementation of the teachings recited herein are not limited to a cloud computing environment. Rather, embodiments of the present invention are capable of being implemented in conjunction with any other type of computing environment now known or later developed.

[0061] For the avoidance of doubt, the term "comprising”, as used herein throughout the description and claims is not to be construed as meaning "consisting only of'.

[0062] A "destination ICAO” refers to the four-letter alphanumeric code designated by the International Civil Aviation Organization (ICAO) to identify specific airports around the world. These codes are used in flight planning, air traffic control, and airline operations to clearly and uniquely identify airports. For example: • John F. Kennedy International Airport (New York, USA): ICAO code is KJFK; • Dublin Airport (Ireland): ICAO is EIDW; • Heathrow Airport (London, UK): ICAO code is EGLL.

[0063] The ICAO code is different from the three-letter IATA code, which is more commonly used by the general public and airlines for ticketing and baggage handling.

[0064] In an airfield, a control area is defined as a specified region of airspace in which air traffic control (ATO) services are provided to ensure the safety and efficiency of air traffic. The control area is established to manage and separate aircraft operating within its boundaries, particularly in the vicinity of the airport and during critical phases of flight such as take-off, landing, and climbing or descending.

[0065] The control area has defined lower and upper altitude limits, and a defined horizontal boundary. Air traffic control services are provided to all aircraft within this area, regardless of their flight rules (Visual Flight Rules (VFR) or Instrument Flight Rules (IFR)). Flight rules are the rules under which pilots operate aircrafts in different weather and visibility situations.

[0066] Aircraft typically transmit telemetry information to ground stations and other aircraft. Data also exists about the destination ICAO.

[0067] Data sources include: a. Flight Data (for example, altitude, airspeed, direction, orientation), aircraft system status; landing time and the scheduled times for a flight to a destination ICAO. b. Navigation Data: Information from GPS, VHF Omnidirectional Range (VCR), Distance Measurement Equipment (DME), and other navigation aids. In particular, ADS-B (Automatic Dependent Surveillance-Broadcast), for real-time positioning and velocity data, regularly broadcasts the aircraft's position, velocity, and other information to air traffic control and other aircraft, using for example, VHF / UHF Radio, satellite communication; and AGARS (Aircraft Communications Addressing and Reporting System). c. Position Reports - two different flight data streams of aircraft Position Reports with different data frequencies, are stored for use by the image creation engine. d. Airport Data: for example, runway configurations at destination ICAOs. e. ASDO (Airport Surface Detection Equipment, Model X) is a surveillance system used by air traffic controllers to track ground movements of aircraft and vehicles on the airport surface. ASDE-X integrates data from various sources, including radar, multilateration, and Automatic Dependent Surveillance-Broadcast (ADS-B), to provide a comprehensive view of airport surface operations. This helps in enhancing safety and efficiency by preventing runway incursions and ensuring smooth ground operations. f. EFD (Electronic Flight Data) is a digital system used to manage flight data electronically within air traffic control environments. EFD systems replace traditional paper flight progress strips with digital displays, allowing air traffic controllers to access and update flight information more efficiently. These systems help improve the accuracy, speed, and reliability of flight data management, enhancing overall air traffic control operations. g. Visual I Camera identification of aircraft.

[0068] Transmission frequency varies based on the type of data, the systems in use, and the phase of flight.

[0069] ADS-B used for position, velocity, altitude, and identification information is transmitted every second. ACARS sends short messages about flight operations to ground stations at critical phases of flight such as landing. Flight Data Monitoring Systems (FDM) continuously records, but transmits every few seconds to minutes. Data includes speed, altitude, attitude, control surface positions, and system statuses.

[0070] Different aircraft and avionics systems have varying capabilities and requirements for telemetry transmission. Newer aircraft with advanced avionics may have more frequent and detailed telemetry updates.

[0071] The availability of communication networks (e.g., VHF / UHF, satellite) can impact the frequency of telemetry transmission. Bandwidth limitations may necessitate balancing the frequency and volume of data transmitted.

[0072] Many airfields use cameras to monitor aircraft movement. Real-Time Monitoring (RTM) cameras provide live feeds to air traffic control (ATC) and airport operations centres to assist in managing aircraft movements and responding to incidents. Video footage is recorded and stored for later analysis. Many types of cameras can be used, such as fixed, Pan-Tilt-Zoom (PTZ), Infrared and Thermal Cameras, and High-Definition (HD) Cameras.

[0073] The present invention, in accordance with a preferred embodiment, first creates an historical image archive showing the flight paths of the aircraft landing at a destination ICAO along with meta data for what type of landing occurred and times of scheduled and actual landings. Secondly an image classification model is created for each destination ICAO based on the image archive. Real-time images for flights arriving at a destination ICAO are created, and using an image classification engine and the model for the destination ICAO, an indication can be made on whether a flight has followed a successful flight path, or, for example, has had to implement a go-around.

[0074] FIG. 2, which should be read in conjunction with FIGS. 3 - 9 depicts a high-level exemplary schematic flow diagram 200 depicting operation methods steps identifying and actioning an aircraft path 504, according to a preferred embodiment of the present invention.

[0075] FIG. 3 depicts an aircraft system 300 comprising a control area 302 of a destination ICAO 304, according to a preferred embodiment of the present invention. A aircraft 306 is depicted landing on a runway configuration 308. Monitoring of the aircraft 306 is performed by ATC 310. Images are available through cameras 312, and communication between system elements made using transmitters / receivers 314, 316.

[0076] FIG. 4 depicts more detailed method steps of step 204 for creating an archive of historical images, according to an embodiment of the present invention.

[0077] FIG. 5 depicts images 506 of flight paths 504 of aircraft landing on the runway configuration 308, according to an embodiment of the present invention.

[0078] FIG. 6 depicts more detailed method steps of step 206 for analysing data according to an embodiment of the present invention. FIG. 7 depicts more detailed method steps of step 208 for creating a classification model 906 for a destination ICAO 304, according to an embodiment of the present invention.

[0079] FIG. 8 depicts more detailed method steps of step 210 for creating a real-time images, according to an embodiment of the present invention.

[0080] FIG. 9 depicts software components 201 used by the method 200 of FIG. 2, according to an embodiment of the present invention.

[0081] The method 200 starts at step 202. At step 204, an image creation engine 902 creates a historical image archive 904 showing journey paths of a single vehicle at a destination for use with model training for each destination ICAO 304. Alternatively, at step 204, the image creation engine 902 creates a historical image archive 904 showing journey paths of multiple vehicles operating under the same conditions (for example, under a level of turbulence) at the destination for use with model training for each destination ICAO 304.

[0082] Step 204 creates historical images back to X months and creates images and associated meta data for flight paths: At step 404, the image creation engine 204 obtains the runway configuration 308 for the destination ICAO 304. At step 406, the image creation engine 204 identifies a flight that is M miles from destination, or N minutes from scheduled arrival and obtains position reports for the associated time stamp. At step 408, the image creation engine 204 creates a timed image 502-1 at time Tx from the position reports, and timed image 502-2 at time Tx+i. At step 410, images 502-1, 502-2 are collated to trace a path 504-1 of the flight. A composite image 506 also includes an overlay 508 of the runway configuration 308. Processing continues until the aircraft 306 has landed, or taken an alternative path. At step 412, the composite image 506 is stored in the image archive 904. Images 502 of paths 504 are labelled with metadata, such as a classification of successful landing, or go-around, along with the scheduled time arrival and actual time of arrival. For example, a first path 504-2 depicts a successful landing, whereas a second path 504-3 depicts a go-around event. The skilled person would understand that many classifications could be defined, for example, a successful path, a go-around path, and a runway miss path. A runway miss path classification could assist ATC in the prevention of future air accidents

[0083] The data collection of step 204 gathers a large and diverse set of labeled images for training of a classification model 906, which is used in the invention. These images represent the categories the model is to recognize. The method returns to step 206.

[0084] At step 206 data associated with vehicle paths is analysed in preparation for creating the image classification model 906: At step 604 a real-time image repository of vehicle journey paths is created. At step 606,temporal vehicle telemetry data en route to a destination is collated and analysed. At step 608 temporal weather data along the journey route is collated and analysed. At step 610 temporal weather data at the destination is collated and analysed. The model can be further enhanced with relevant information that would affect the flight path. For example, other factors, such as wind speed, other traffic in the vicinity etc. would likely to make a difference as to whether there was a go around. Not only would these factors affect the predicted models, but these factors are also relevant in the determination of probability of whether a vehicle has followed a go-around path, The method returns to step 208.

[0085] At step 208 a classification model 906 is created for each destination ICAO 304.

[0086] An image classification model 906 is a type of Al model, designed to categorize images 504 into predefined classes or labels. Such a model is used to analyze an image and assign it to one or more categories based on its content.

[0087] The image archive 904 contains images 506 as previously defined along with the associated meta data 507 as it relates to an aircraft 306 landing at a destination ICAO 304. The invention is described with reference to a Convolutional Neural Network (CNN), which is one of the most commonly used architecture for image classification. A CNN can automatically and adaptively learn spatial hierarchies of features from input images. Typical CNN architectures include layers such as convolutional layers, pooling layers, fully connected layers, and activation functions (e.g., ReLU). CNN models. However, the skilled person would understand that other architectures for an Al model can be used for image classification.

[0088] The classification model 906 is created from the images 506 and the metadata 507 using the CNN engine 908. The CNN engine 908 comprises code and / or hardware used as a starting point. In addition, CNN engine 908 comprises pre-trained models 906 from other destination ICAOs that are suitable as a starting point to develop the model 906 of interest. Flight paths 504 are three-dimensional 3D models. Creating a CNN for 3D images involves extending the principles of 2D CNNs to handle three-dimensional data.

[0089] At step 704, data from the image archive 904 is gathered and pre-processed. Preprocessing transforms the images 506 into a format suitable for training. At step 706, the pre-processed data is built in preparation for training.

[0090] At step 708, the model 906 is trained using the data 506, 507 in the image archive. Training is performed by feeding the pre-processed images 506 / 507 into the model and using a labeled dataset to adjust the model's parameters. The model 906 learns to map input images 506 to the correct output categories by minimizing a loss function through optimization techniques like gradient descent.

[0091] At step 710 the performance of the model 906 is assessed using metrics such as accuracy and precision, on a separate test set that was not used during training.

[0092] At step 712, the model 906 can be enhanced. For example, augmentation techniques such as rotations, flips, and zooms can be applied to increase the diversity of the training set. Dropout and other regularization techniques can used to prevent overfitting. Experiments may also be made with different architectures, learning rates, and other hyperparameters to improve performance. The method returns to step 210.

[0093] At step 210 an identify component 912 identifies an aircraft 306 of interest. A vehicle arrival detection model is used to determine if the vehicle has arrived at its destination relative to vehicle telemetry data. Identification may be made using data received from the aircraft 306, or triggered externally, for example, from cameras 312. Identification may also be made based on expected arrival time, for example, from X minutes before the time of scheduled of arrival to the time of landing. A real-time image 910 is created for the aircraft 306 landing at the destination ICAO 304 using real-time images from the cameras 312, from X minutes before the time of scheduled of arrival to the time of landing the aircraft. Step 210 follows similar steps as those of step 204. At step 404, the image creation engine 204 obtains the runway configuration 308 for the destination ICAO 304. At step 406, the image creation engine 204 identifies the flight that is M miles from destination, or X minutes from scheduled arrival and obtains position reports for the associated time stamp. At step 408, the image creation engine 204 creates a timed image 502-1 at time Tx from the position reports, and timed image 502-2 at time Tx+i. At step 410, images 502-1, 502-2 are collated to trace a path 504-1 of the flight. A composite image 506 also includes an overlay 508 of the runway configuration 308. Processing continues until the aircraft 306 has landed, or taken an alternative path.

[0094] In parallel with steps 408 and 410, at step 810 an image classification engine 920 classifies the path 504. The image classification engine 920 returns a response indicating whether it found the image as indicating an unsuccessful arrival. A feedback loop is used to verify the arrival detection. In an embodiment, the image classification engine 920 is queried as to whether a successful landing has been made subsequent to collation step 410. The image classification engine 920 inputs the real-time image 910 and uses the latest model 906 for the destination ICAO 304 to categorise the path 504, for example, as successful. In this step 212, the trained model 906 classifies the real-time image 910 as being a new, unseen images. The model 906 outputs the probability or confidence scores for each category, and the category with the highest score is usually selected as the predicted label. At step 412, the composite image 506 is stored in the image archive 904. Images 502 of paths 504 are labelled with metadata, such as a classification of successful landing, or go-around, along with the scheduled time arrival and actual time of arrival. A real-time image is labeled with a tag to indicate that the image relates to a real-time event.

[0095] In an alternative embodiment, the image classification engine 920 is queried as to which category of path 504 the aircraft 306 is following. The image classification engine 920 inputs the real-time image 910 and uses the latest model 906 for the destination ICAO 304 to categorise the path 504, for example, as being on a successful path 504-2. In this step 212, the trained model 906 again classifies the real-time image 910 as being a new, unseen images. The model 906 outputs the probability or confidence scores for each category, and the category with the highest score is usually selected as the predicted label corresponding to determining the closest match within the image archive 904. Referring to FIG.5 again, the task to perform is identifying whether path 504-1 is a part of path 504-2, or a part of path 504-3. Parts of paths 504 are also referred to as partial paths. As speed of classification is advantageous, an Al ASIC and related hardware can be used. An example is a Groq Language Processing Unit (LPU), which can be used for inference, and NVIDIA Al accelerators, which can be used for training and inference.

[0096] The method returns to step 214.

[0097] At step 214 an action is taken based on the categorisation of the path 504. For example, in the case of a go-around, ATC 310 is alerted to continue control of the flight.

[0098] At step 216, the model 906 is enhanced by re-training with the details from the real-time image 910. This also allows an operator to manually verify the truth of the classification which can then be used to mark the real time image 910 as a user verified piece of data, and now as a historical image 506, 507. Model drift can be taking into account using monitoring techniques to indicate when retraining might be useful due to new runways being added for example.

[0099] At step 299, method 200 ends.

[00100] In an alternative embodiment, the invention is applied to road vehicle paths 504-4, 505-5.

[00101] FIG. 2, which should be read in conjunction with FIGS. 4, 6 -10 depicts a high-level exemplary schematic flow diagram 200 depicting operation methods steps identifying and actioning a vehicle path 504-4, 505-5, according to a preferred embodiment of the present invention.

[00102] FIG. 10 depicts a vehicle system 1000 comprising a control area 1002 of a road. Three lanes are depicted 1002,1004,1006. Monitoring of a first vehicle 1050, and a second vehicle 1060 is performed by road traffic control, or on board navigation equipment with each vehicle 1050,1060. Images are available through cameras 312, and vehicle sensors (not depicted), and communication between system elements made using transmitters / receivers 316.

[00103] In the left hand lane 1002 first vehicle 1050 is depicted at time Tx 1008, Tx+i1010, and Tn. First vehicle 1050 follows a first path 504-4, which is projected as continuing into a first extended path 1022.

[00104] In the right hand lane 1006 second vehicle 1060 is depicted at time Ty 1014, Ty+i 1016, and Tm. 1018. Second vehicle 1060 follows a second path 505-5, which is projected as continuing into a second extended path 1026.

[00105] In this embodiment, the steps 200 of FIG. 2 are followed to build images of paths 504-4, 505-5 of road vehicles using components relevant to the situation, for example, a road configuration 922 for that section of road. At step 212, the image classification engine 920 categorises the first vehicle 1050 as following first path 504-4, which is straight in the left hand lane 1002. Likewise the image classification engine 920 categorises the second vehicle as following second path 504-5, which is veering into a centre lane 1004. At step 212, no action needs to made for the first vehicle 1050, but braking or a steering correction may be required for the second vehicle 1060 if changing lanes is unintentional. Analysis of paths 504-4, 504-5 may also identify whether the paths 504-4, 504-5 match historic images of paths in the archive 904. In this way, vehicle paths 1022,1026 can be identified.

[00106] In an alternative embodiment time period Tx- Tn overlaps with time period Ty-Tm. In this embodiment, relative positions of vehicle paths 504-4 +1022, 504-5 +1026 is important. At step 212 actions can be carried out by either or both of the vehicles 1050,1060 to avoid an accident. One way of determining whether a collision is likely to occur is to create a composite path 1030 from individual paths 504-4 +1022, 504-5 +1026. The composite path 1030 can then be fed into the model 906 to determine whether the composite path 1030 represents a collision path by matching with historical images representing collision paths 1030. The skilled person would understand that many classifications could be defined, for example, a successful path, a lane veer path, and a collision path.

[00107] The skilled person would understand that the invention can be applied to many transport situations, including air transport, ground transport, and also sea transport.

[00108] In an alternative embodiment an alternative Al architecture is used. The skilled person will understand that other architectures are also applicable for image recognition and processing. For example, but not limited to, other architectures as well as CNNs include, Generative Adversarial Networks (GANs), Transformer-Based Models, Autoencoders, Recurrent Neural Networks (RNNs), Large Language Models, and specialized architectures can be used. For example, YOLO is used for real-time object detection.

Claims

1. A computer implemented method for managing vehicle paths, the method comprising: receiving a first dataset, the dataset comprising a first set of images of vehicle paths; training an Al model with the first set of images to determine a first model and a set of classifications for each of the images for a first location;identifying a first vehicle;creating a first image of a first path taken by the first vehicle;applying the first image to the model to determine a first classification of the set of classifications for the first image path; and based on the first classification, performing an action related to the first vehicle.

2. The method of claim 1, wherein receiving the first dataset comprises creating the first data set by gathering data, the data comprising at least one of vehicle position reports, vehicle images, and location data.

3. The method of either of claims 1 or 2, wherein the vehicle comprises one of air transport, ground transport, and sea transport.

4. The method of any of the preceding claims wherein the first image is added to the first dataset to create a second set of images, and the Al model is retrained with the second set of images.

5. The method of any of the preceding claims, further comprising: identifying a second vehicle;creating a second image of a second path taken by the second vehicle;comparing the first image with the second image to determine a composite image;applying the composite image to the model to determine a composite classification of the set of classifications for the composite image;based on the composite classification, performing an action related to at least one of the first vehicle and the second vehicle.

6. The method of any of the preceding claims, wherein the set of classifications comprises at least one of a successful path, a go-around path, a runway miss path, a lane veer path, a collision path.

7. The method of any of the preceding claims, wherein the Al model is taken from a list, the list comprising: a CNN; a GAN; a Transformer-Based Models; an Autoencoder; an RNN, a Large Language Models; and YOLO.

8. The method of any of the preceding claims, wherein creating a first image of a first path taken by the first vehicle comprises creating timed images for the first vehicle over successive time slices, and collating the timed images to determine the first image.

9. The method of any of the preceding claims wherein applying the first image to the model to determine a first classification comprises determining a closest match of the first image from the dataset.

10. The method of claim 9, wherein determining the closest match comprises matching the first image with partial images of the dataset.

11. A system for managing vehicle paths, the system operable for:receiving a first dataset, the dataset comprising a first set of images of vehicle paths;training an Al model with the first set of images to determine a first model and a set of classifications for each of the images for a first location;identifying a first vehicle;creating a first image of a first path taken by the first vehicle;applying the first image to the model to determine a first classification of the set of classifications for the first image path;based on the first classification, performing an action related to the first vehicle.

12. The system of claim 11, wherein receiving the first dataset comprises creating the first data set by gathering data, the data comprising at least one of vehicle position reports, vehicle images, and location data.

13. The system of either of claims 11 or 12, wherein the vehicle comprises one of air transport, ground transport, and sea transport.

14. The system of any of claims 11 to 13, wherein the first image is added to the first dataset to create a second set of images, and the Al model is retrained with the second set of images.

15. The system of any of claims 11 to 14, further operable for:identifying a second vehicle;creating a second image of a second path taken by the second vehicle;comparing the first image with the second image to determine a composite image;applying the composite image to the model to determine a composite classification of the set of classifications for the composite image; andbased on the composite classification, performing an action related to at least one of the first vehicle and the second vehicle.

16. The system of any of the preceding claims, wherein the set of classifications comprises at least one of a successful path, a go-around path, a runway miss path, a lane veer path, a collision path.

17. The system of any of the claims 11 to 16, wherein the Al model is taken from a list, the list comprising: a CNN; a GAN; a Transformer-Based Models; an Autoencoder; an RNN, a Large Language Models; and YOLO.

18. The system of any of claims 11 to 17, wherein creating a first image of a first path taken by the first vehicle comprises creating timed images for the first vehicle over successive time slices, and collating the timed images to determine the first image.

19. The system of any of claims 11 to 18, wherein applying the first image to the model to determine a first classification comprises determining a closest match of the first image from the dataset.

20. The system of claim 19, wherein determining the closest match comprises matching the first image with partial images of the dataset.

21. A computer program product for determining a metric at a second resolution at a location, the computer program product comprising: a computer readable storage medium readable by a processing circuit and storing instructions for execution by the processing circuit for performing a method according to any of claims 1 to 10.

22. A computer program stored on a computer readable medium and loadable into the internal memory of a digital computer, comprising software code portions for performing the method of any of claims 1 to 10, when said program is run on a computer.

Images