Device and method for flight preparation event detection and aircraft

Abstract

The invention relates to an apparatus and method for flight preparation event detection and an aircraft. An apparatus includes one or more processors configured to obtain bounding box data indicative of a first location history of a first bounding box associated with a first object detected in an image. The one or more processors are further configured to process at least the first location history using the trained model to detect an event associated with a flight preparation of the aircraft. The one or more processors are also configured to generate an output indicative of the event.

Description

Technical Field

[0001] The present disclosure generally relates to flight readiness event detection. Background Art

[0002] Turnaround delay is the amount of time an aircraft is delayed on the ground between flights. It is any unscheduled time included in the period of time an aircraft spends on the ground between the time it lands and the time it takes off again. Turnaround delays can occur due to a variety of factors, including weather conditions, mechanical problems, ground handling delays, air traffic congestion, and other operational issues.

[0003] Delays can have a cascading impact on an airline's scheduling and can result in missed connections, leading to unhappy customers, reduced revenue and increased costs. Reducing turnaround time is an important factor for airlines as it increases efficiency, reduces costs and improves the overall passenger experience. Summary of the invention

[0004] In a specific implementation, an apparatus includes one or more processors configured to obtain bounding box data indicating a first position history of a first bounding box associated with a first object detected in an image. The one or more processors are further configured to process at least the first position history using a trained model to detect an event associated with flight preparation of an aircraft. The one or more processors are further configured to generate an output indicating the event.

[0005] In another specific implementation, an aircraft includes one or more processors configured to obtain bounding box data indicating a first position history of a first bounding box associated with a first object detected in an image. The one or more processors are further configured to process at least the first position history using a trained model to detect an event associated with flight preparation of the aircraft. The one or more processors are further configured to generate an output indicating the event.

[0006] In another specific embodiment, a method includes obtaining, at a device, bounding box data indicating a first position history of a first bounding box associated with a first object detected in an image. The method also includes processing at least the first position history using a trained model to detect an event associated with flight preparation of an aircraft. The method also includes generating, at the device, an output indicating the event.

[0007] The features, functions, and advantages described herein can be achieved independently in various embodiments or may be combined in yet other embodiments further details of which can be found with reference to the following description and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1is a diagram illustrating a system configured to detect a flight preparation event.

[0009] Figure 2 It can be done by Figure 1 Schematic diagram of a specific example of flight preparation event detection performed by a system.

[0010] Figure 3 It can be done by Figure 1 Schematic diagram of another example of flight preparation event detection performed by a system.

[0011] Figure 4 It can be done by Figure 1 Schematic diagram of another example of flight preparation event detection performed by a system.

[0012] Figure 5 It can be done by Figure 1 Schematic diagram of another example of flight preparation event detection performed by a system.

[0013] Figure 6 is a schematic diagram illustrating a system configured to perform aircraft turnaround management based on detected flight readiness events.

[0014] Figure 7 is a diagram of a specific example of a flight preparation event.

[0015] Figure 8 is a diagram of a flow chart illustrating an example of a method of flight preparation event detection.

[0016] Fig. 9 is a flow chart illustrating an example of a life cycle of an aircraft including a flight preparation system.

[0017] Fig.10 is a block diagram of a particular implementation of an aircraft including a flight preparation system.

[0018] Fig.11 is a block diagram of a computing environment including a computing device configured to support aspects of computer-implemented methods and computer-executable program instructions (or code) according to the present disclosure. DETAILED DESCRIPTION

[0019] Aspects disclosed herein provide systems and methods for flight preparation event detection. Flight preparation delays may result in aircraft turnaround delays, which may have a significant impact on airline operations. Delays may result in missed connections, increased operating costs, and unhappy customers. Flight preparation delays may be caused by a variety of factors, such as delays in baggage handling, aircraft cleaning and maintenance, refueling, catering, passenger boarding, etc.

[0020] According to some aspects, a flight preparation event detector receives a sequence of images (e.g., a real-time stream) from a camera configured to capture a scene at a parking stand. The flight preparation event detector performs object detection to determine whether any objects of interest are detected in one or more images. Objects of interest may include aircraft, fuel trucks, catering trucks, air bridges, ground personnel, or any visually detectable resource used to perform an aircraft turnaround.

[0021] According to some aspects, a flight preparation event detector generates bounding box data indicating a position history of a bounding box of an object. For example, the flight preparation event detector detects an object in a first image and determines that the bounding box of the object has a first position in the first image. The flight preparation event detector detects the object in a second image and determines that the bounding box of the object has a second position in the second image. The position history includes a first position and a second position indicating a path of the object. The flight preparation event detector processes the position history (e.g., bounding box data) to determine whether a flight preparation event is detected. For example, the position history indicating a specific path of an aircraft corresponds to the aircraft arriving at an aircraft stand. The flight preparation event detector generates an output based on the detected event. For example, the output indicates that the aircraft has been detected arriving at a specific parking stand.

[0022] Early detection of predicted delays can enable the aircraft turnaround management system to initiate actions to prevent delays or to reduce any adverse effects of delays. For example, in response to determining that an aircraft arrival has been detected but a fuel truck arrival has not been detected, the aircraft turnaround management system can generate an alert indicating that a fuel truck is requested at the aircraft stand. As another example, based on determining that a fuel truck arrival at the aircraft stand is detected at a specific time, the aircraft turnaround management system determines a predicted delay based on an expected refueling duration and a target disconnect time. The aircraft turnaround management system can initiate reallocation of crew members who will miss subsequent flights, check availability of subsequent flights for passengers who will miss connecting flights, etc. based on the predicted delay.

[0023] The accompanying drawings and the following description illustrate specific exemplary embodiments. It should be understood that those skilled in the art will be able to design various arrangements that, although not explicitly described or shown herein, embody the principles described herein and are included within the scope of the claims appended to this description. In addition, any examples described herein are intended to help understand the principles of the present disclosure and are interpreted as not limiting. Therefore, the present disclosure is not limited to the specific embodiments or examples described below, but is defined by the claims and their equivalents.

[0024] Specific implementations are described herein with reference to the accompanying drawings. In the description, common features are represented by common reference numerals throughout the drawings. In some drawings, multiple instances of a particular type of feature are used. Although the features are physically and / or logically different, the same reference numeral is used for each, and different instances are distinguished by adding a letter to the reference numeral. When features are referred to herein as a group or type (for example, when a specific one of the features is not mentioned), the reference numeral is used without a distinguishing letter. However, when a specific feature among multiple features of the same type is referenced herein, the reference numeral is used with a distinguishing letter. For example, reference numerals are used in conjunction with distinguishing letters. Figure 1 , a plurality of images are shown and are associated with reference numerals 102A, 102B, and 102C. When referring to a particular one of these images, such as image 102A, a distinguishing letter "A" is used. However, when referring to any of these images or to these images as a group, reference numeral 102 is used without a distinguishing letter.

[0025] As used herein, various terms are used only for the purpose of describing particular implementations and are not intended to be limiting. For example, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. In addition, some features described herein are singular in some implementations and plural in other implementations. For illustration, Figure 1 Describes a system including one or more cameras ( Figure 1 160), indicating that in some implementations, the system 100 includes a single camera 160, and in other implementations, the system 100 includes multiple cameras 160. For ease of reference herein, such features are generally introduced as "one or more" features and are then referenced in the singular or optionally the plural, unless aspects relating to multiple features are being described.

[0026] The terms "include," "comprise," and "cover" are used interchangeably with "contain," "have," or "encompass." Additionally, the term "wherein" is used interchangeably with the term "in." As used herein, "exemplary" indicates an example, implementation, and / or aspect, and should not be interpreted as limiting or indicating a preference or preferred implementation. As used herein, ordinal terms (e.g., "first," "second," "third," etc.) used to modify an element (such as a structure, component, operation, etc.) do not themselves indicate any priority or order of the element relative to another element, but merely distinguish the element from another element with the same name (but for use of the ordinal term). As used herein, the term "set" refers to a grouping of one or more elements, and the term "plurality" refers to a plurality of elements.

[0027] As used herein, "generate", "calculate", "use", "select", "access" and "determine" are interchangeable unless the context indicates otherwise. For example, "generate", "calculate" or "determine" a parameter (or signal) may refer to actively generating, calculating or determining a parameter (or signal) or may refer to using, selecting or accessing an already generated parameter (or signal), such as by another component or device. As used herein, "coupling" may include "communicative coupling", "electrical coupling" or "physical coupling", and may also (or alternatively) include any combination thereof. Two devices (or components) may be directly or indirectly coupled (e.g., communicatively coupled, electrically coupled or physically coupled) via one or more other devices, components, wires, buses, networks (e.g., wired networks, wireless networks, or combinations thereof), etc. Two electrically coupled devices (or components) may be included in the same device or in different devices, and may be connected via electronic devices, one or more connectors, or inductive coupling, as illustrative, non-limiting examples. In some implementations, two devices (or components) that are communicatively coupled (e.g., in electrical communication) may directly or indirectly send and receive electrical signals (digital or analog), for example, via one or more wires, buses, networks, etc. As used herein, "directly coupled" is used to describe two devices that are coupled (e.g., communicatively coupled, electrically coupled, or physically coupled) without intervening components.

[0028] Figure 1 An example of a system 100 configured to detect a flight preparation event is depicted. The system 100 includes a flight preparation event detector 104 coupled to one or more cameras 160. Optionally, in some implementations, the flight preparation event detector 104 is also coupled to one or more sensors 170.

[0029] In some aspects, the flight preparation event detector 104, one or more cameras 160, one or more sensors 170, or a combination thereof are interconnected via one or more networks to achieve data communication. For example, the flight preparation event detector 104 is coupled to one or more cameras 160, one or more sensors 170, or both via one or more wireless networks, one or more wired networks, or any combination thereof. Two or more of the flight preparation event detector 104, one or more cameras 160, and one or more sensors 170 may be co-located with each other or geographically distributed.

[0030] The flight preparation event detector 104 may be coupled to (e.g., mounted or integrated into) an aircraft, a ground control system, an airline management system, or a combination thereof. One or more cameras 160 may be coupled to (e.g., mounted or integrated into) an airport building, an air bridge, an aircraft, a vehicle, or a combination thereof. In some aspects, the one or more cameras 160 are configured to capture a scene at a parking stand at an airport. For example, the one or more cameras 160 are configured to generate a sequence (e.g., a real-time stream) of images 102 capturing a scene at a parking stand. In some examples, the images 102 correspond to a video or a still image sequence.

[0031] Optionally, in some implementations, one or more sensors 170 may be installed on (e.g., mounted on or integrated into) an airport building, an air bridge, an aircraft, a vehicle, or a combination thereof. For example, one or more sensors 170 may include a door sensor that indicates whether a door of the aircraft is open. As another example, one or more sensors 170 may include a fuel sensor that indicates a fuel level of the aircraft.

[0032] The flight preparation event detector 104 includes a bounding box data generator 106 coupled to a trained model 108 (e.g., a trained neural network). The bounding box data generator 106 is configured to generate bounding box data 107 indicating a position history 130 of a bounding box 122 of an object 120 detected in the image 102. For example, the position history 130 indicates a path of the object 120. The bounding box data generator 106 is configured to provide the bounding box data 107 to the trained model 108. In an example, the flight preparation event detector 104 processes at least the position history 130 using the trained model 108 to detect an event 140 associated with flight preparation of the aircraft. The flight preparation event detector 104 is configured to generate an output 150 indicating the event 140.

[0033] In an illustrative and non-limiting example, the object 120 includes an aircraft, a fuel truck, a baggage truck, a de-icing truck, a catering truck, a push-back tractor, a snowplow, an airship, an air bridge, a protection bus, a belt loader, a container loader, a water truck, a lavatory service vehicle, a ground power unit, an air start unit, or a fire truck. In some examples, the object 120 may include any object or person associated with flight preparation of an aircraft. For illustration, in some aspects, the object 120 may include a baggage handler. The flight preparation event detector 104 may detect that the object 120 corresponds to a person having a specific role (e.g., a baggage handler) associated with flight preparation based on performing facial recognition, unified recognition, badge detection, etc. on the at least one image 102.

[0034] It should be appreciated that the flight preparation event detector 104 is provided as an illustrative example, which includes a bounding box data generator 106 that generates bounding box data 107 that indicates a path of the object 120. In other implementations, the flight preparation event detector 104 may include another type of path detector that processes the image 102 to generate path data that indicates the path of the object 120, and processes the path data using the trained model 108 to detect the event 140.

[0035] During operation, the flight preparation event detector 104 receives a sequence of images 102 from one or more cameras 160. In an example, an aircraft is expected to arrive at a parking stand at an airport at an expected in-block time. The one or more cameras 160 capture a scene at the parking stand. In a specific aspect, the flight preparation event detector 104 begins receiving a sequence of images 102 from the one or more cameras 160 at a first time, which is a first duration (e.g., 10 minutes) before the expected in-block time. In a particular aspect, the first duration corresponds to a pre-arrival event that is scheduled to occur before the aircraft arrives at the parking stand. In some implementations, the flight preparation event detector 104 determines the first duration based on a configuration setting, a user input, timing data associated with a pre-arrival event, or a combination thereof.

[0036] The flight preparation event detector 104 performs object detection on one or more images 102 to determine whether an object of interest is detected. In example 190, the flight preparation event detector 104 performs object detection on image 102A and determines that object 120 is detected. The flight preparation event detector 104 determines whether object 120 corresponds to an object of interest. For example, the flight preparation event detector 104 determines whether object 120 matches an object indicated by flight preparation data associated with one or more flight preparation events, matches an object of interest detected in a previous image 102, or both.

[0037] The bounding box data generator 106 determines a bounding box 122 for the object 120 in the image 102A. The bounding box data generator 106 determines a position 124A of the bounding box 122 in the image 102A. In some implementations, the position 124A of the bounding box 122 corresponds to the position of a midpoint 126 of the bounding box 122. In other implementations, the position 124A may correspond to another position associated with the bounding box 122. The bounding box data generator 106 updates the position history 130 associated with the object 120 to indicate that the bounding box 122 had the position 124A at a time associated with the image 102A (e.g., a reception time, a generation time, etc.).

[0038] Similarly, bounding box data generator 106 determines additional locations of bounding box 122 in one or more subsequent images of the sequence of images 102. For example, flight preparation event detector 104 performs object detection on image 102B to detect object 120 in image 102B, and bounding box data generator 106 determines that bounding box 122 for object 120 has location 124B in image 102B. As another example, flight preparation event detector 104 detects object 120 in image 102C, and bounding box data generator 106 determines that bounding box 122 for object 120 has location 124C in image 102C, and so on. In some cases, as object 120 moves closer to or further away from one or more cameras 160, bounding box 122 may have different sizes, different shapes, or both in different images (e.g., image 102A, image 102B, image 102C, etc.) in which object 120 is detected.

[0039] The bounding box data generator 106 generates bounding box data 107 indicating a position history 130 of the bounding box 122. The position history 130 is based on a sequence of positions 124 (e.g., position 124A, position 124B, position 124C, etc.) of the bounding box 122. The sequence of positions 124 indicates a path of the bounding box 122. For example, the sequence of positions 124 indicates a path of a midpoint 126 of the bounding box 122. The path of the bounding box 122 corresponds to an estimate of the path of the object 120. In some implementations, the bounding box data 107 includes the sequence of positions 124, an image 128 indicating the sequence of positions 124, or both. In some implementations, the bounding box data generator 106 determines (e.g., using a best fit analysis) a line indicating an estimated path of the bounding box 122 based on the sequence of positions 124. The bounding box data 107 includes data representing the line (e.g., the estimated path), an image, or both.

[0040] The bounding box data generator 106 provides the bounding box data 107 to the trained model 108. For example, the flight preparation event detector 104 processes at least the position history 130 using the trained model 108 to determine whether an event 140 is detected. In a specific aspect, the event 140 includes at least one of: an aircraft arrival, an aircraft departure, a truck arrival, a truck departure, a change in truck configuration, a refueling start, a refueling end, a baggage loading start, a baggage loading end, a de-icing start, a de-icing end, an aircraft door opening, an aircraft door closing, a meal start, a meal end, a passenger boarding, a passenger disembarking, a bridge extension, or a bridge retraction.

[0041] In a specific aspect, the trained model 108 is trained to detect whether the location history 130 indicates a location pattern that matches a historical location pattern associated with the event 140. For example, in some implementations, the flight preparation event detector 104 updates the trained model 108 (e.g., updates weights and biases of the trained model 108) based on labeled training data of a plurality of aircraft flight preparation events. To illustrate, the labeled training data includes a first location history of a first bounding box of a first object that indicates a first historical location pattern, and the first location history is labeled as being associated with a first event (e.g., an aircraft arrival). In some implementations, the flight preparation event detector 104 trains the trained model 108 to detect a location history that indicates a location pattern that matches a first historical location pattern associated with an event of the same event type as the first event (e.g., an aircraft arrival).

[0042] In some examples, an event may be associated with a location history of a bounding box of a plurality of objects. To illustrate, the labeled training data may include a second location history of a second bounding box of a second object indicating a second historical location pattern, and the second location history is also labeled as being associated with the first event. In some implementations, the flight preparation event detector 104 is configured to train the trained model 108 to determine that a first location history indicating a first location pattern that matches the first historical location pattern and a second location history indicating a second location pattern that matches the second historical location pattern are associated with an event of the same event type as the first event.

[0043] In some examples, the labeled training data includes location histories of multiple events labeled with the same event type. For example, the labeled training data includes a third location history of a third bounding box of a third object indicating a third historical location pattern, and the third location history is labeled as being associated with a second event (e.g., a second aircraft arrival), the second event having the same event type (e.g., aircraft arrival) as the event type of the first event (e.g., the first aircraft arrival). The flight preparation event detector 104 can train the trained model 108 to determine a first location history that indicates a first location pattern that matches the first historical location pattern or a third historical location pattern associated with an event of the same event type as the first event. In some implementations, the trained model 108 is trained on multiple location histories of multiple events labeled with the same event type so that the trained model 108 can give more weight to common features of multiple location histories associated with the event type.

[0044] In some examples, the labeled training data includes location histories of multiple events labeled with multiple event types. For example, the labeled training data includes a fourth location history of a fourth bounding box of a fourth object indicating a fourth location pattern, and the fourth location history is labeled as being associated with a third event (e.g., fuel truck arrival) having an event type (e.g., fuel truck arrival) of a first event type (e.g., airplane arrival) different from a first event (e.g., first airplane arrival). The trained model 108 is thus trained to detect multiple types of events.

[0045] In a specific aspect, the flight preparation event detector 104 processes at least the position history 130 using the trained model 108 to detect an event 140 associated with flight preparation of the aircraft. In some examples, the flight preparation event detector 104 processes the position history of the bounding boxes of multiple objects to determine that the event 140 is detected. For example, the flight preparation event detector 104 processes the position history 130 and a second position history of a second bounding box of a second object to determine that the event 140 is detected. In some examples, the flight preparation event detector 104 can process the same position history to detect multiple events. For example, the flight preparation event detector 104 processes at least the position history 130 to determine that the event 140 is detected, and processes the position history 130 and a third position history of a third bounding box of a third object to determine that a second event associated with flight preparation of the aircraft is detected.

[0046] The flight preparation event detector 104 generates an output 150 indicating that the event 140 was detected. For example, the output 150 indicates that the event 140 was detected at the first time at the parking stand. In some examples, the flight preparation event detector 104 generates a second output indicating that a second event was detected. The flight preparation event detector 104 provides the output 150 to one or more devices, such as an aircraft, an aircraft turnaround management system, a ground control system, an airline management system, or a combination thereof. For example, the flight preparation event detector 104 provides the output 150 to one or more devices when the event is detected. For illustration, the flight preparation event detector 104 provides the output 150 indicating that the event 140 was detected at the first time. The flight preparation event detector 104 provides the output 150 indicating that the second event was detected at the second time, and so on.

[0047] Optionally, in some implementations, the flight preparation event detector 104 receives sensor data 172 from one or more sensors 170. The flight preparation event detector 104 generates an output 150 based at least in part on determining whether the event 140 matches the sensor data 172. For example, the flight preparation event detector 104 selectively generates an alert based on determining whether the sensor data 172 indicates the event 140. For illustration, in response to determining that the sensor data 172 indicates that the door of the aircraft is closed, and the event 140 indicates that passengers have begun to board the aircraft, the output 150 may include an alert to verify whether the door sensor is faulty. As another example, in response to determining that the sensor data 172 indicates that the amount of fuel in the aircraft has not changed, and the event 140 indicates that the fuel truck is connected to the aircraft for at least a threshold duration and appears to have begun refueling of the aircraft, the output 150 may include an alert to verify that the fuel truck has begun refueling, the fuel sensor is functioning properly, or both.

[0048] It should be understood that the sensor data 172 indicating whether a door is open or the amount of fuel is provided as an illustrative example. In other examples, the sensor data 172 may indicate different types of information. For illustration, the sensor data 172 may indicate a detected position of an adaptable wing of the aircraft. For example, the adaptable wing may include a foldable wing tip, and the configuration of the foldable wing tip (e.g., having a folded position or an extended position) may be verified before the aircraft leaves the parking stand. The flight preparation event detector 104 may generate an alert in response to determining that the trained model 108 indicates that an event 140 corresponding to an extended position of the adaptable wing is detected and the sensor data 172 indicates that the adaptable wing has a folded position.

[0049] Technical advantages of automatically detecting event 140 based on the sequence of images 102 may include reducing human error and reducing delays in detecting and responding to event 140. In an example, the aircraft turnaround management system may determine that event 140 corresponds to a predicted delay, as shown in FIG. Figure 6 As further described, the aircraft turnaround management system can initiate actions to reduce (eg, prevent) the predicted delay, reduce the impact of the predicted delay, or both.

[0050] In some implementations, the flight readiness event detector 104 may be represented in hardware, such as via an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA), or the operations described with reference to the elements may be performed by a processor executing computer-readable instructions.

[0051] refer to Figure 2 , shows an example 200 of flight preparation event detection, which can be performed by Figure 1, such as when flight readiness event detector 104 performs object recognition to detect object 120A in a first image subset of the sequence of images 102 .

[0052] In the example, the first image subset includes image 102A, image 102B, image 102C, and image 102D. It should be understood that the first image subset containing four images is provided as an illustrative example. In other examples, the first image subset may include less than four images or more than four images.

[0053] The bounding box data generator 106 generates bounding box data 107 indicating a position history 130A of a bounding box 122A of the object 120A. For example, the position history 130A indicates a position 124A of the bounding box 122A in the image 102A, a position 124B of the bounding box 122A in the image 102B, a position 124C of the bounding box 122A in the image 102C, and a position 124D of the bounding box 122A in the image 102D. In an example, the bounding box data 107 indicates the position history 130A of the object 120A from a first time (e.g., a capture time, a reception time, or both) associated with the image 102A to a second time (e.g., a capture time, a reception time, or both) associated with the image 102D.

[0054] In a specific aspect, the bounding box data 107 indicates the locations 124A to 124D. In a specific aspect, the bounding box data 107 indicates a path based on the locations 124A to 124D. In a specific aspect, the bounding box data 107 includes an image of the path. Figure 2 As shown, as object 120A moves closer to or further away from one or more cameras 160 , the size and shape of bounding box 122A may change from one image to the next.

[0055] The flight preparation event detector 104 processes at least the position history 130A to detect an event 140A (e.g., an airplane arrival) using the trained model 108. For example, the flight preparation event detector 104 provides a first object type (e.g., airplane) of the object 120A and the position history 130A (e.g., a path, positions 124A to 124D, or both) as inputs to the trained model 108, and the trained model 108 generates an output indicating the event 140A (e.g., airplane arrival).

[0056] refer to Figure 3 , showing that Figure 1An example 300 of flight preparation event detection performed by the system 100 of FIG. The bounding box data generator 106 generates (e.g., updates) bounding box data 107 indicating a position history 130A of a bounding box 122A of an object 120A (e.g., an airplane) and a position history 130B of a bounding box 122B of an object 120B (e.g., a push-pull).

[0057] In an example, flight readiness event detector 104 performs object detection to detect object 120A and object 120B in a second image subset of the sequence of images 102. In a particular aspect, the second image subset corresponds to images from a second time to a third time.

[0058] In some examples, object 120A, object 120B, or both may be detected in one or more images of the second subset of images. To illustrate, the second subset of images may include at least one image in which both object 120A and object 120B are detected, at least one image in which a single one of object 120A or object 120B is detected, at least one image in which neither object 120A or object 120B is detected, or a combination thereof. In example 300, both object 120A and object 120B are detected in at least one of the same images of the second subset of images.

[0059] Bounding box data generator 106 updates bounding box data 107 to indicate a location history 130A of bounding box 122A of object 120A in the second image subset. Thus, location history 130A indicates a path of object 120A from the first time to the third time. Bounding box data generator 106 also updates bounding box data 107 to indicate a location history 130B of bounding box 122B of object 120B in the second image subset. Thus, location history 130B indicates a path of object 120B from the second time to the third time.

[0060] The flight preparation event detector 104 processes at least the position history 130A and the position history 130B using the trained model 108 to detect the event 140B (e.g., pushback start). For example, the flight preparation event detector 104 provides a first object type (e.g., aircraft) of the object 120A and the position history 130A associated with the first object type as a first input to the trained model 108, provides a second object type (e.g., pushback haul) of the object 120B and the position history 130B associated with the second object type as a second input to the trained model 108, and the trained model 108 generates an output indicating the event 140B (e.g., pushback start).

[0061] refer to Figure 4 , showing that Figure 14 is an example of flight preparation event detection performed by the system 100. The bounding box data generator 106 generates (eg, updates) the bounding box data 107 indicating the position history 130A of the bounding box 122A of the object 120A and the position history 130B of the bounding box 122B of the object 120B.

[0062] In an example, the flight readiness event detector 104 performs object detection to detect the object 120A and the object 120B in a third image subset of the sequence of images 102. In a particular aspect, the third image subset corresponds to images from the third time to a fourth time.

[0063] In some examples, object 120A, object 120B, or both may be detected in one or more images of the third subset of images. To illustrate, the third subset of images may include at least one image in which both object 120A and object 120B are detected, at least one image in which a single one of object 120A or object 120B is detected, at least one image in which neither object 120A or object 120B is detected, or a combination thereof. In example 400, both object 120A and object 120B are detected in at least one of the same images of the third subset of images.

[0064] Bounding box data generator 106 updates bounding box data 107 to indicate a position history 130A of bounding box 122A of object 120A in the third image subset. Thus, position history 130A indicates a path of object 120A from the first time to the fourth time. Bounding box data generator 106 also updates bounding box data 107 to indicate a position history 130B of bounding box 122B of object 120B in the third image subset. Thus, position history 130B indicates a path of object 120B from the second time to the fourth time.

[0065] The flight preparation event detector 104 processes at least the position history 130A and the position history 130B using the trained model 108 to detect the event 140C (e.g., pushback in progress). For example, the flight preparation event detector 104 provides a first object type (e.g., aircraft) of the object 120A and the position history 130A associated with the first object type as a first input to the trained model 108, provides a second object type (e.g., pushback haul) of the object 120B and the position history 130B associated with the second object type as a second input to the trained model 108, and the trained model 108 generates an output indicating the event 140C (e.g., pushback in progress).

[0066] refer to Figure 5 , showing that Figure 15 is an example of flight preparation event detection performed by the system 100. The bounding box data generator 106 generates (eg, updates) the bounding box data 107 indicating the position history 130A of the bounding box 122A of the object 120A and the position history 130B of the bounding box 122B of the object 120B.

[0067] In the example, the flight preparation event detector 104 performs object detection to detect the object 120A and the object 120B in a fourth image subset of the sequence of images 102. In a specific aspect, the fourth image subset corresponds to images from a fourth time to a sixth time, wherein the object 120A leaves the field of view of the one or more cameras 160 at a fifth time between the fourth time and the sixth time. In the example 500, the object 120A is not detected and the object 120B is detected in at least one image of the fourth image subset. For example, the image 102 is captured after the fifth time.

[0068] Bounding box data generator 106 updates bounding box data 107 to indicate a position history 130A of bounding box 122A of object 120A in the fourth image subset. Thus, position history 130A indicates a path of object 120A from the first time to the fifth time. Bounding box data generator 106 also updates bounding box data 107 to indicate a position history 130B of bounding box 122B of object 120B in the fourth image subset. Thus, position history 130B indicates a path of object 120B from the second time to the sixth time.

[0069] The flight preparation event detector 104 processes at least the position history 130A and the position history 130B using the trained model 108 to detect the event 140D (e.g., pushback completion). For example, the flight preparation event detector 104 provides a first object type (e.g., aircraft) of the object 120A and the position history 130A associated with the first object type as a first input to the trained model 108, provides a second object type (e.g., pushback haul) of the object 120B and the position history 130B associated with the second object type as a second input to the trained model 108, and the trained model 108 generates an output indicating the event 140D (e.g., pushback completion).

[0070] It should be understood that reference Figures 2 to 5 The specific location histories and corresponding events described are provided as illustrative examples. In other examples, the trained model 108 is trained to process different types of location histories to detect different types of events.

[0071] Figure 6 An example of a system 600 configured to perform aircraft turnaround management based on detected flight readiness events is depicted. In particular aspects, Figure 1System 100 includes one or more components of system 600 .

[0072] System 600 includes an aircraft turnaround management system 602 coupled to a flight readiness event detector 104. Although flight readiness event detector 104 is shown as being external to aircraft turnaround management system 602, in some examples, aircraft turnaround management system 602 may include flight readiness event detector 104.

[0073] The aircraft turnaround management system 602 has access to data indicating planned events 604 associated with the flight readiness of the aircraft, as shown in FIG. Figure 7 As further described. The aircraft turnaround management system 602 may also access planned timing data 606 indicating the planned timing of the planned events 604. In some examples, the planned timing data 606 indicates which planned events 604 must be performed sequentially. For illustration, the planned timing data 606 indicates that the baggage loading start event is dependent on the baggage cart arrival event having occurred. In some examples, the planned timing data 606 indicates a planned duration associated with a specific event or associated with a transition from one event to another event. For illustration, the planned timing data 606 indicates a planned duration between transitions from the baggage loading start event to the baggage loading end event.

[0074] Optionally, in some implementations, the aircraft turnaround management system 602 can access delay data 616 indicating typical delays associated with the unplanned events 614. To illustrate, the delay data 616 can indicate that the unplanned events 614 (e.g., a piece of luggage falling from a luggage cart prior to a baggage load end event) correspond to typical delays that are predicted to extend a typical duration associated with loading a baggage (e.g., extend the amount of time planned from a baggage load start event to a baggage load end event).

[0075] The aircraft turnaround management system 602 receives output 150 from the flight readiness event detector 104 indicating that the event 140 was detected, as shown in FIG. Figure 1 In a specific aspect, the output 150 also indicates the detection time 640 of the event 140. In another aspect, the reception time of the output 150 corresponds to the detection time 640 of the event 140.

[0076] The aircraft turnaround management system 602 tracks the planned events 604. Optionally, in some implementations, the aircraft turnaround management system 602 also tracks whether any unplanned events 614 occur. For example, the aircraft turnaround management system 602 updates the event status data 650 to indicate that the event 140 was detected at the detection time 640 in response to receiving the output 150 indicating the event 140. The event 140 can be included in the planned events 604 or the unplanned events 614.

[0077] The aircraft turnaround management system 602 determines the predicted delay 656 based on the event status data 650 and the planned timing data 606, the delay data 616, or both. In an example, the aircraft turnaround management system 602 can access a target out-of-block time 652 for the aircraft. According to some aspects, the target out-of-block time 652 refers to the time when the aircraft is expected to leave the parking stand for an on-time departure. The aircraft turnaround management system 602 determines the predicted out-of-block time 654 based on the event status data 650 and the planned timing data 606, the delay data 616, or both. For example, the aircraft turnaround management system 602 determines the predicted out-of-block time 654 based on the detection time 640 and the planned duration (e.g., the sum) in response to determining that the event 140 has a detection time 640 and the planned timing data 606 indicates a planned duration associated with one or more remaining events in the planned events 604. In another example, the aircraft turnaround management system 602 adjusts the predicted out-of-block time 654 based on the corresponding delay indicated by the delay data 616 in response to determining that at least one of the unplanned events 614 is detected. The aircraft turnaround management system 602 determines the predicted delay 656 based on a comparison of (eg, a difference between) the predicted out-of-block time 654 and the target out-of-block time 652 .

[0078] In some implementations, the aircraft turnaround management system 602 initiates one or more actions based on the detection time 640, the predicted out-of-block time 654, the predicted delay 656, or a combination thereof. In some implementations, the one or more actions may include at least one preventive action to reduce the predicted out-of-block time 654. In some implementations, the one or more actions may include at least one delay management action to reduce the consequences of the predicted delay 656.

[0079] In the example, the aircraft turnaround management system 602 determines an expected detection time of the second event in response to determining that the event 140 has a detection time 640 and the planned timing data 606 indicates a specific planned duration for transitioning from the event 140 to the subsequent second event. The aircraft turnaround management system 602 initiates one or more actions in response to determining that the event status data 650 indicates that the second event has not been detected at the expected detection time. For example, the one or more actions may include generating an alert, sending a request, reassigning a task associated with the second event, etc.

[0080] In an example, the aircraft turnaround management system 602 initiates one or more actions in response to determining that the predicted out-of-block time 654 is within a threshold duration of the target out-of-block time 652 or the predicted delay 656 is greater than a first delay threshold (e.g., greater than zero). For example, the one or more actions may include generating an alert, sending a request, reallocating one or more tasks associated with the remaining events in the planned events 604, or a combination thereof. In some implementations, the one or more actions may include changing crew assignments for subsequent flights, canceling passenger reservations on connecting flights that may be missed, adding passenger reservations on alternative connecting flights, alerting ground staff to prioritize the transfer of passengers on connecting flights within a threshold duration of the predicted delayed arrival time of the aircraft at the destination, or a combination thereof.

[0081] Technical advantages of automatically detecting events 140 based on the sequence of images 102 may include reducing human error and reducing delays in detecting and responding to events 140. In an example, aircraft turnaround management system 602, based on determining that event 140 corresponds to detection time 640, may initiate one or more actions to reduce predicted out-of-block time 654, reduce the impact of predicted delay 656 (if any), or both.

[0082] refer to Figure 7 , an example 700 of a flight preparation event is shown. In some aspects, the example 700 includes Figure 6 One or more of the planned events 604. For illustration, example 700 indicates some of the planned events 604 that must be completed before the aircraft departs.

[0083] Example 700 includes one or more pre-arrival events 604A, followed by an aircraft arrival 604B, followed by one or more post-arrival events 604C. The one or more pre-arrival events 604A are expected to occur before the aircraft arrives 604B. The aircraft arrival 604B corresponds to the aircraft arriving at a parking stand. The one or more post-arrival events 604C are expected to occur after the aircraft arrives 604B.

[0084] As an example, one or more pre-arrival events 604A may include a safety check of an area of ​​a parking stand by a dispatcher to verify that the area is free of debris. To illustrate, the trained model 108 detects an event 140 corresponding to the safety check in response to processing an image 102 representing a dispatcher, a person included in a dispatcher list, a marker indicating that the person is a dispatcher, or a combination thereof. As an example, one or more post-arrival events 604C may include placing a bearing gallery around a wheel of an aircraft.

[0085] Example 700 includes some events that may occur before, at the same time as, or after the aircraft arrives 604B. For illustration, example 700 includes a catering truck arriving 604D, a fuel truck arriving 604E, and a luggage cart arriving 604F.

[0086] The example 700 includes an air bridge extension 604G that is dependent on one or more post-arrival events 604C having occurred (e.g., bearing gallery placement). To illustrate, the trained model 108 detects the event 140 corresponding to the air bridge extension 604G in response to processing the image 102A representing a retracted air bridge and the image 102B indicating an at least partially extended air bridge. The example 700 also includes an air bridge retraction 604H following the air bridge extension 604G.

[0087] Example 700 includes some events that depend on multiple events that have occurred. For illustration, example 700 includes meal loading start 604I that depends on one or more post-arrival events 604C and catering vehicle arrival 604D having occurred. In some implementations, in response to processing image 102A indicating a first configuration of a catering vehicle (e.g., corresponding to a lowered cargo hold) and image 102B indicating a second configuration of a catering vehicle (e.g., corresponding to a raised cargo hold), the trained model 108 detects event 140 corresponding to meal loading start 604I. Meal loading start 604I is followed by meal loading end 604J, which is followed by catering vehicle departure 604K. In some embodiments, the trained model 108 detects event 140 corresponding to meal loading end 604J in response to processing image 102C indicating a first configuration of a catering vehicle (e.g., the cargo hold has returned to its lowered position). Thus, the trained model 108 can detect events based at least in part on the configuration (e.g., shape) of the detected object 120.

[0088] In an example, in response to determining that meal loading start 604I (eg, event 140) has a first detection time and the planned timing data 606 indicates a specific planned duration for transitioning from meal loading start 604I to meal loading end 604J, Figure 6 The aircraft turnaround management system 602 determines the expected detection time of the catering device end 604J. The aircraft turnaround management system 602 responds to the determination Figure 6 The event status data 650 indicates that the meal loading end 604J is not detected before the expected detection time, and one or more actions are initiated. For example, the one or more actions may include generating an alarm, sending a request, reallocating a task associated with the meal loading end 604J, etc.

[0089] In the example, the aircraft turnaround management system 602 performs a critical path analysis of the planned events 604 to identify one or more specific events included in one or more critical paths to the aircraft departure 604L. For example, a first planned duration between the air bridge extension 604G and the aircraft departure 604L may be less than a second planned duration between the catering loading start 604I and the aircraft departure 604L. Therefore, a first expected completion time of the first path from the air bridge extension 604G to the aircraft departure 604L is less than a second completion time of the second path from the catering loading start 604I to the aircraft departure 604L. If the second completion time is higher than the completion time of other paths to the aircraft departure 604L, the second path (e.g., catering loading start 604I, catering loading end 604J, and catering cart departure 604K) corresponds to the critical path to the aircraft departure 604L.

[0090] Non-critical paths have a tolerance for small delays. For example, a delay along the second path (e.g., meal loading start 604I) may cause a delay in the predicted out-of-block time 654, while the same delay in the first path (e.g., airbridge extension 604G) may not cause any delay in the predicted out-of-block time 654 if the expected completion time of the first path remains less than the expected completion time of the second path.

[0091] In some implementations, the aircraft turnaround management system 602 prioritizes preventing delays of events included in one or more critical paths. For example, in response to determining that the catering truck arrival 604D is included in the critical path and the fuel truck arrival 604E is included in the non-critical path, the aircraft turnaround management system 602 initiates one or more operations that prioritize the catering truck arrival 604D over the fuel truck arrival 604E. To illustrate, if a first driver is available to drive the catering truck or the fuel truck to the parking stand, the aircraft turnaround management system 602 assigns the first driver to drive the catering truck and assigns the fuel truck to the next available driver.

[0092] refer to Figure 8 , showing a specific implementation of the method 800 for flight preparation event detection. In a specific aspect, one or more operations of the method 800 are performed by the flight preparation event detector 104, Figure 1 System 100, Figure 6 The system may be implemented by at least one of the aircraft turnaround management system 602 , the system 600 , or a combination thereof.

[0093] The method 800 includes, at 802, obtaining bounding box data indicating a first position history of a first bounding box associated with a first object detected in an image. For example, the bounding box data generator 106 obtains bounding box data 107 indicating a position history 130 of a bounding box 122 associated with an object 120 detected in one or more of the sequence of images 102, as described with reference to FIG. Figure 1 as described.

[0094] Method 800 also includes, at 804, processing at least the first position history using the trained model to detect events associated with flight preparation of the aircraft. Figure 1 As depicted, flight readiness event detector 104 processes at least position history 130 using trained model 108 to detect events 140 associated with flight readiness of the aircraft.

[0095] The method 800 further includes generating an output indicating an event at 806. For example, the flight readiness event detector 104 generates an output 150 indicating the event 140, as shown in FIG. Figure 1 as described.

[0096] Technical advantages of automatically detecting events 140 based on a sequence of images 102 may include reducing human error and reducing delays in detecting and responding to events 140. In an example, Figure 6 The aircraft turnover management system 602 may determine that the event 140 corresponds to the predicted delay 656, as shown in FIG. Figure 6 The aircraft turnaround management system can initiate actions to reduce (eg, prevent) the predicted delay 656 , reduce the impact of the predicted delay 656 , or both.

[0097] refer to Fig. 9 , a flow chart 900 is shown illustrating the life cycle of an aircraft configured to perform flight readiness event detection. During pre-production, the exemplary method 900 includes, at 902, specification and design of the aircraft. During specification and design of the aircraft, the method 900 may include specification and design of a flight readiness system 920. The flight readiness system 920 includes the flight readiness event detector 104, the aircraft turnaround management system 602, or both. At 904, the method 900 includes material procurement, which may include obtaining materials for the flight readiness system 920.

[0098] During production, method 900 includes component and subassembly manufacturing at 906, and system integration of the aircraft at 908. For example, method 900 may include component and subassembly manufacturing of a flight preparation system 920 and system integration of the flight preparation system 920. At 910, method 900 includes certification and delivery of the aircraft, and at 912, placing the aircraft in service. Certification and delivery may include certification of the flight preparation system 920 to place the flight preparation system 920 in service. When placed in service by a customer, the aircraft may be scheduled for routine maintenance and servicing (which may also include modification, reconfiguration, refurbishment, etc.). At 914, method 900 includes performing maintenance and servicing on the aircraft, which may include performing maintenance and servicing on the flight preparation system 920.

[0099] Each process of method 900 may be performed or implemented by a system integrator, a third party, and / or an operator (e.g., a customer). For the purposes of this specification, a system integrator may include, but is not limited to, any number of aircraft manufacturers and main system subcontractors; a third party may include, but is not limited to, any number of vendors, subcontractors, and suppliers; and an operator may be an airline, a leasing company, a military entity, a service organization, etc.

[0100] Aspects of the present disclosure may be described in the context of an example of a vehicle. Specific examples of vehicles are Fig.10 Aircraft 1000 is shown.

[0101] exist Fig.10 In the example of FIG. 1 , aircraft 1000 includes airframe 1018 having plurality of systems 1020 and interior 1022. Examples of plurality of systems 1020 include one or more of propulsion system 1024, electrical system 1026, environmental system 1028, hydraulic system 1030, and flight preparation system 920. Any number of other systems may be included.

[0102] Fig.11 is a block diagram of a computing environment 1100 including a computing device 1110 configured to support various aspects of computer-implemented methods and computer-executable program instructions (or codes) according to the present disclosure. For example, the computing device 1110 or a portion thereof is configured to execute instructions to initiate, execute, or control the Figures 1 to 10 Describes one or more operations.

[0103] The computing device 1110 includes one or more processors 1120. The processor 1120 is configured to communicate with a system memory 1130, one or more storage devices 1140, one or more input / output interfaces 1150, one or more communication interfaces 1160, or any combination thereof. The system memory 1130 includes volatile memory devices (e.g., random access memory (RAM) devices), non-volatile memory devices (e.g., read-only memory (ROM) devices, programmable read-only memory, and flash memory), or both. The system memory 1130 stores an operating system 1132, which may include a basic input / output system for booting the computing device 1110 and a complete operating system that enables the computing device 1110 to interact with users, other programs, and other devices. The system memory 1130 stores system (program) data 1136, such as one or more of a sequence of images 102, sensor data 172, bounding box data 107, events 140, output 150, location history 130, or a combination thereof. For example, program data 1136 may include any data used or generated by system 100 , system 600 , or both, as described herein.

[0104] The system memory 1130 includes one or more applications 1134 (eg, instruction sets) that can be executed by the processor 1120. In a specific aspect, the one or more applications 1134 include the flight preparation system 920. As an example, the one or more applications 1134 include the one or more applications that can be executed by the processor 1120 to initiate, control, or execute the flight preparation system 920. Figures 1 to 10 For illustration, one or more applications 1134 include instructions executable by processor 1120 to initiate, control, or perform one or more operations described with reference to flight preparation system 920 .

[0105] In some implementations, processor 1120 may be implemented as a single processor or multiple processors, such as in a multi-core configuration, a multi-processor configuration, a distributed computing configuration, a cloud computing configuration, or any combination thereof. In some implementations, one or more portions of flight preparation system 920 are implemented by processor 1120 using dedicated hardware, firmware, or a combination thereof.

[0106] In a specific implementation, system memory 1130 includes a non-volatile computer-readable medium storing instructions that, when executed by processor 1120, cause processor 1120 to initiate, execute, or control operations to perform flight preparation event detection. The operations include obtaining bounding box data (e.g., bounding box data 107) indicating a first position history (e.g., position history 130) of a first bounding box (e.g., bounding box 122) associated with a first object (e.g., object 120) detected in an image (e.g., one or more of a sequence of images 102). The operations also include processing at least the first position history using a trained model (e.g., trained model 108) to detect an event (e.g., event 140) associated with flight preparation of an aircraft. The operations also include generating an output (e.g., output 150) indicating the event.

[0107] The one or more storage devices 1140 include non-volatile storage devices, such as magnetic disks, optical disks, or flash memory devices. In a specific example, the storage devices 1140 include both removable and non-removable memory devices. The storage devices 1140 are configured to store operating systems, images of operating systems, applications (e.g., one or more applications 1134), and program data (e.g., program data 1136). In a specific aspect, the system memory 1130, the storage devices 1140, or both contain tangible computer-readable media. In a specific aspect, the one or more storage devices 1140 are external to the computing device 1110.

[0108] One or more input / output interfaces 1150 enable the computing device 1110 to communicate with one or more input / output devices 1170 to facilitate user interaction. For example, one or more input / output interfaces 1150 may include a display interface, an input interface, or both. For example, the input / output interface 1150 is suitable for receiving input from a user, receiving input from another computing device, or a combination thereof. In some implementations, the input / output interface 1150 complies with one or more standard interface protocols, including a serial interface (e.g., a universal serial bus (USB) interface or an Institute of Electrical and Electronics Engineers (IEEE) interface standard), a parallel interface, a display adapter, an audio adapter, or a custom interface ("IEEE" is a registered trademark of the Institute of Electrical and Electronics Engineers of Piscataway, New Jersey.). In some implementations, the input / output device 1170 includes one or more user interface devices and a display, including some combination of buttons, keyboards, pointing devices, displays, speakers, microphones, touch screens, and other devices.

[0109] The processor 1120 is configured to communicate with a device or controller 1180 via one or more communication interfaces 1160. For example, the one or more communication interfaces 1160 may include a network interface. The device or controller 1180 may include, for example, a flight readiness event detector 104, one or more cameras 160, one or more sensors 170, an aircraft turnaround management system 602, an aircraft, a vehicle, a computing device, a communication device, a ground control device, one or more other devices, or any combination thereof.

[0110] In conjunction with the described systems and methods, an apparatus for performing flight preparation event detection is disclosed, comprising means for obtaining bounding box data indicating a first position history of a first bounding box associated with a first object detected in an image. In some implementations, the means for obtaining corresponds to the bounding box data generator 106, the flight preparation event detector 104, Figure 1 The system 100, the computing device 1110, the processor 1120, one or more other circuits or devices configured to obtain bounding box data, or a combination thereof.

[0111] The apparatus also includes means for processing at least the first position history using the trained model to detect an event associated with flight readiness of the aircraft. In some implementations, the means for processing corresponds to the trained model 108, the flight readiness event detector 104, Figure 1 The system 100, the computing device 1110, the processor 1120, one or more other circuits or devices configured to process the location history using the trained model, or a combination thereof.

[0112] The apparatus further includes means for generating an output indicative of the event. In some implementations, the means for generating corresponds to the flight readiness event detector 104, Figure 1 system 100, computing device 1110, processor 1120, one or more other circuits or devices configured to generate an output, or a combination thereof.

[0113] In some implementations, a non-transitory computer-readable medium stores instructions that, when executed by one or more processors, cause the one or more processors to initiate, execute, or control operations to perform some or all of the functions described above. For example, these instructions may be executable to implement Figures 1 to 11 In some embodiments, Figures 1 to 11Part or all of one or more of the operations or methods of may be implemented by one or more processors (e.g., one or more central processing units (CPUs), one or more graphics processing units (GPUs), one or more digital signal processors (DSPs)) that execute instructions, by dedicated hardware circuits, or any combination thereof.

[0114] Specific aspects of the present disclosure are described in the following set of related examples:

[0115] According to Example 1, an apparatus includes one or more processors configured to obtain bounding box data indicating a first position history of a first bounding box associated with a first object detected in an image; process at least the first position history using a trained model to detect an event associated with flight preparation of an aircraft; and generate an output indicating the event.

[0116] According to Example 2, the apparatus of Example 1 is included, wherein the bounding box data comprises a first image of a first path indicating a first midpoint of the first bounding box.

[0117] According to Example 3, the apparatus of Example 1 or Example 2 is included, wherein the bounding box data comprises a first sequence of positions indicating a first path of a first bounding box.

[0118] According to Example 4, the apparatus of any one of Examples 1 to 3 is included, wherein the one or more processors are configured to receive an image from a camera that captures a scene at a parking stand at an airport.

[0119] According to Example 5, the apparatus of any of Examples 1 to 4 is included, wherein the one or more processors are configured to receive the image from a camera coupled to the aircraft.

[0120] According to Example 6, the apparatus of any one of Examples 1 to 5 is included, wherein the one or more processors are configured to perform object detection on the image to: detect the first object; and generate the first bounding box.

[0121] According to Example 7, the apparatus of any of Examples 1 to 6, wherein the event corresponds to one of a plurality of events tracked by the aircraft turnaround management system, and wherein the aircraft turnaround management system updates a status of the event based on the output.

[0122] According to Example 8, including the apparatus of Example 7, wherein the aircraft turnaround management system determines the predicted delay based on a status of the event and scheduled timing data associated with the plurality of events.

[0123] Example 9 includes an apparatus according to any one of Examples 1 to 8, wherein the bounding box data includes a second position history of a second bounding box associated with a second object detected in the image, and wherein the one or more processors are configured to process the first position history and the second position history using the trained model to detect the event.

[0124] Example 10 includes an apparatus according to any one of Examples 1 to 9, wherein the bounding box data includes a third position history of a third bounding box associated with a third object detected in the image; and wherein the one or more processors are configured to process the first position history and the third position history using the trained model to detect a second event associated with flight preparation of the aircraft; and generate a second output indicative of the second event.

[0125] According to Example 11, the apparatus of any of Examples 1 to 10, wherein the one or more processors are configured to update the trained model based on labeled training data of a plurality of aircraft flight readiness events.

[0126] According to Example 12, the apparatus of any one of Examples 1 to 11 is included, wherein the trained model is trained to detect specific location patterns corresponding to historical location patterns associated with events.

[0127] Example 13 includes an apparatus according to any one of Examples 1 to 12, wherein the event includes at least one of the following: arrival of an aircraft, departure of an aircraft, arrival of a truck, departure of a truck, change of configuration of a truck, start of refueling, end of refueling, start of luggage loading, end of luggage loading, start of deicing, end of deicing, opening of an aircraft door, closing of an aircraft door, start of catering loading, end of catering loading, passenger boarding, passenger disembarking, extension of a bridge, or retraction of a bridge.

[0128] According to Example 14, the apparatus of any of Examples 1 to 13 is included, wherein the one or more processors are configured to obtain additional sensor data; and selectively generate an alert based on determining whether the additional sensor data indicates an event.

[0129] Example 15 includes the apparatus of any of Examples 1 to 14, wherein the first object comprises an aircraft, a fuel truck, a baggage truck, a de-icing truck, a catering truck, a push-back tractor, a snowplow, an airship, an air bridge, a protection bus, a belt loader, a container loader, a water truck, a washroom service vehicle, a ground power unit, an air start unit, or a fire truck.

[0130] According to Example 16, a method includes: obtaining, at a device, bounding box data indicating a first position history of a first bounding box associated with a first object detected in an image; processing at least the first position history using a trained model to detect an event associated with flight preparation of an aircraft; and generating, at the device, an output indicating the event.

[0131] Example 17 includes the method of Example 16, wherein the bounding box data comprises a first image of a first path indicating a first midpoint of a first bounding box.

[0132] According to Example 18, including the method of Example 16 or Example 17, wherein the bounding box data comprises a first sequence of positions indicating a first path of a first bounding box.

[0133] The method according to Example 19 including any one of Examples 16 to 18, further comprising receiving an image from a camera capturing a scene at a parking stand of an airport.

[0134] The method according to Example 20 including any of Examples 16 to 19, further comprising receiving an image from a camera coupled to the aircraft.

[0135] The method according to Example 21 including any one of Examples 16 to 20, further comprising performing object detection on the image to: detect a first object; and generate a first bounding box.

[0136] According to Example 22, including the method of any of Examples 16 to 21, wherein the event corresponds to one of a plurality of events tracked by the aircraft turnaround management system, and wherein the aircraft turnaround management system updates a status of the event based on the output.

[0137] According to example 23 including the method of example 22, further comprising determining, by the aircraft turnaround management system, a predicted delay based on a status of the event and scheduled timing data associated with the plurality of events.

[0138] The method according to Example 24 includes any one of Examples 16 to 23, further comprising processing the first position history and the second position history using the trained model to detect an event, wherein the bounding box data includes a second position history of a second bounding box associated with a second object detected in the image.

[0139] The method according to Example 25 includes any one of Examples 16 to 24, further comprising: processing the first position history and the third position history using the trained model to detect a second event associated with flight preparation of the aircraft, wherein the bounding box data includes a third position history of a third bounding box associated with a third object detected in the image; and generating a second output indicating the second event.

[0140] According to Example 26, the method of any one of Examples 16 to 25 further comprises updating the trained model based on labeled training data of a plurality of aircraft flight readiness events.

[0141] The method according to Example 27 includes any one of Examples 16 to 26, further comprising training the trained model to detect specific location patterns corresponding to historical location patterns associated with events.

[0142] Example 28 includes a method according to any one of Examples 16 to 27, wherein the event includes at least one of the following: arrival of an aircraft, departure of an aircraft, arrival of a truck, departure of a truck, change of configuration of a truck, start of refueling, end of refueling, start of baggage loading, end of baggage loading, start of de-icing, end of de-icing, opening of an aircraft door, closing of an aircraft door, start of catering loading, end of catering loading, passenger boarding, passenger disembarking, extension of a bridge, or retraction of a bridge.

[0143] Example 29 includes the method of any of Examples 16 to 28, wherein the one or more processors are configured to obtain additional sensor data; and selectively generate an alert based on determining whether the additional sensor data indicates an event.

[0144] Example 30 includes the method of any of Examples 16 to 29, wherein the first object comprises an aircraft, a fuel truck, a baggage truck, a de-icing truck, a catering truck, a push-back tractor, a snowplow, an airship, an air bridge, a protection bus, a belt loader, a container loader, a water truck, a washroom service vehicle, a ground power unit, an air start unit, or a fire truck.

[0145] According to Example 31, an apparatus includes: a memory configured to store instructions; and a processor configured to execute the instructions to perform the method of any one of Examples 16 to 30.

[0146] According to Example 32, a non-transitory computer readable medium stores instructions that, when executed by a processor, cause the processor to perform the method of any one of Examples 16 to 30.

[0147] According to Example 33, an apparatus includes means for performing the method of any one of Examples 16 to 30.

[0148] According to Example 34, an aircraft includes one or more processors configured to: obtain bounding box data indicating a first position history of a first bounding box associated with a first object detected in an image; process at least the first position history using a trained model to detect an event associated with flight preparation of the aircraft; and generate an output indicative of the event.

[0149] According to Example 35, including the aircraft of Example 34, wherein the one or more processors are configured to obtain additional sensor data; and selectively generate the alert based on determining whether the additional sensor data indicates an event.

[0150] According to Example 36 including Example 34 or Example 35, the aircraft further includes a door, wherein: the additional sensor data indicates whether the door is open.

[0151] According to Example 37, including the aircraft of Example 35 or Example 36, further comprising at least an adaptable wing, wherein the additional sensor data indicates a detected position of the adaptable wing.

[0152] The illustrations of the examples described herein are intended to provide a general understanding of the structures of different implementations. These illustrations are not intended to serve as a complete description of all elements and features of the devices and systems utilizing the structures or methods described herein. After reviewing this disclosure, many other embodiments may be apparent to those skilled in the art. Other implementations may be utilized and derived from this disclosure so that structural and logical replacements and changes may be made without departing from the scope of this disclosure. For example, method operations may be performed in an order different from the order shown in the figure or one or more method operations may be omitted. Therefore, this disclosure and the accompanying drawings are considered to be illustrative and not restrictive.

[0153] In addition, although specific examples have been shown and described herein, it should be understood that any subsequent arrangement designed to achieve the same or similar results may replace the specific implementation shown. This disclosure is intended to cover any and all subsequent adaptations or variations of different implementations. Combinations of the above implementations and other implementations not specifically described herein will be apparent to those skilled in the art upon reviewing this description.

[0154] The abstract of the present disclosure is submitted with the understanding that it is not used to interpret or limit the scope or meaning of the claims. In addition, in the aforementioned specific embodiments, different features may be combined together or described in a single implementation for the purpose of streamlining the present disclosure. The above examples illustrate but do not limit the present disclosure. It should also be understood that many modifications and variations are feasible according to the principles of the present disclosure. As reflected in the attached claims, the subject matter claimed for protection may involve features that are less than all the features of any disclosed example. Therefore, the scope of the present disclosure is limited by the attached claims and their equivalents.

Claims

1. A device for flight readiness event detection, the device comprising: One or more processors configured to: obtaining bounding box data indicating a first position history of a first bounding box associated with a first object detected in the image; processing at least the first position history using the trained model to detect events associated with flight preparation of the aircraft; as well as Output is generated indicative of the event.

2. The device according to claim 1, wherein: The bounding box data includes a first image of a first path indicating a first midpoint of the first bounding box.

3. The device according to claim 1, wherein: The bounding box data includes a first sequence of positions indicative of a first path of the first bounding box.

4. The device according to claim 1, wherein: The one or more processors are configured to receive the image from a camera capturing a scene at a parking stand at an airport.

5. The device according to claim 1, wherein: The one or more processors are configured to receive the image from a camera coupled to the aircraft.

6. The device according to claim 1, wherein: The one or more processors are configured to perform object detection on the image to: detecting the first object; and The first bounding box is generated.

7. The device according to claim 1, wherein: The event corresponds to one of a plurality of events tracked by an aircraft turnaround management system, and wherein the aircraft turnaround management system updates a status of the event based on the output.

8. The device according to claim 7, wherein: The aircraft turnaround management system determines a predicted delay based on a status of the event and scheduled timing data associated with the plurality of events.

9. The device according to claim 1, wherein: The bounding box data includes a second position history of a second bounding box associated with a second object detected in the image, and wherein the one or more processors are configured to process the first position history and the second position history using the trained model to detect the event.

10. The device according to claim 1, wherein: The bounding box data comprises a third position history of a third bounding box associated with a third object detected in the image; and wherein the one or more processors are configured to: processing the first position history and the third position history using the trained model to detect a second event associated with the flight preparation of the aircraft; and A second output is generated indicative of the second event.

11. The device according to claim 1, wherein: The one or more processors are configured to update the trained model based on labeled training data for a plurality of aircraft flight readiness events.

12. The device according to claim 1, wherein: The trained model is trained to detect specific location patterns that correspond to historical location patterns associated with events.

13. The device according to claim 1, wherein: The event includes at least one of the following: arrival of the aircraft, departure of the aircraft, arrival of a truck, departure of the truck, change of configuration of the truck, start of refueling, end of refueling, start of baggage loading, end of baggage loading, start of de-icing, end of de-icing, opening of the door of the aircraft, closing of the door of the aircraft, start of catering loading, end of catering loading, passenger boarding, passenger disembarkation, extension of a bridge, or retraction of the bridge.

14. The device according to claim 1, wherein: The one or more processors are configured to: obtaining additional sensor data; and An alert is selectively generated based on determining whether the additional sensor data is indicative of the event.

15. An aircraft comprising: One or more processors configured to: obtaining bounding box data indicating a first position history of a first bounding box associated with a first object detected in the image; processing at least the first position history using the trained model to detect events associated with flight preparation of the aircraft; as well as Output is generated indicative of the event.

16. The aircraft of claim 15, wherein: The one or more processors are configured to: obtaining additional sensor data; and An alert is selectively generated based on determining whether the additional sensor data is indicative of the event.

17. The aircraft of claim 16, further comprising: A door, wherein the additional sensor data indicates whether the door is open.

18. The aircraft of claim 16, further comprising: At least an adaptable wing, wherein the additional sensor data is indicative of a detected position of the adaptable wing.

19. A method for flight readiness event detection, the method comprising: At the device, obtaining bounding box data indicating a first position history of a first bounding box associated with a first object detected in the image; processing at least the first position history using the trained model to detect events associated with flight preparation of the aircraft; as well as At the device, an output is generated indicative of the event.

20. The method according to claim 19, wherein: The first object includes the aircraft, fuel truck, baggage truck, de-icing truck, catering truck, push-back tractor, snowplow, airship, air bridge, protection bus, belt loader, container loader, water truck, washroom service vehicle, ground power unit, air start unit or fire truck.

Images