Object monitoring system for an aircraft

Abstract

The title of the invention is Object Monitoring System for an Aircraft. A method, apparatus, system, and computer program product (822) for monitoring a platform (202). An image (224) is received by a computer system (216) from a sensor system positioned to monitor movement (212) of a person (204) relative to the platform (202). A group of persons (204) in the image (224) is identified by the computer system (216). Movement (212) of the group of persons (204) relative to the platform (202) is determined by the computer system (216) using the image (224). A count of persons (204) on the platform (202) is determined by the computer system (216) based on the determined movement (212) for the group of persons (204). A set of actions (228) is performed by the computer system (216) based on the count (226) of persons (204) on the platform (202).

Description

Technical Field

[0001] This disclosure generally relates to aircraft, and more specifically to methods, apparatus, systems, and computer program products for monitoring the entry into and exit of objects from an aircraft. Background Technology

[0002] An aircraft flight includes the boarding process for passengers to enter the aircraft. During this process, boarding passes are checked at the airport gates to ensure that passengers have boarded correctly. Before flight, airline personnel count the passengers on board to determine how many passengers are on board. This count can be compared to the number of boarding passes used for passenger check-in.

[0003] Once the flight arrives at the gate at the destination airport, disembarkation will begin. During disembarkation, passengers leave the aircraft with their luggage and other belongings. Before the crew leaves the aircraft, they conduct a check to ensure all passengers and their belongings have been unloaded. For example, when the crew leaves the aircraft to search for passengers or any items they may have left behind, they will check the cabin up and down and inspect overhead bins.

[0004] Between flights or after the last flight, cleaning staff conduct further checks on the aircraft. This cleaning helps identify items that passengers may have left behind. These processes are labor-intensive and error-prone; passengers, their belongings, or both may be accidentally left on the aircraft without being noticed during these checks.

[0005] Therefore, it is desirable to have methods and apparatus that take into account at least some of the problems discussed above, as well as other possible problems. For example, it is desirable to have methods and apparatus that overcome the technical problems of tracking passengers and objects entering and leaving aircraft. Summary of the Invention

[0006] Embodiments of this disclosure provide a method for monitoring aircraft traffic. Images are received by a computer system from a sensor system positioned to monitor human movement relative to the aircraft. The computer system identifies a group of people in the images. The computer system uses the images to determine the movement of this group of people relative to the aircraft. Based on the determined movement for this group of people, the computer system determines a count of people in the aircraft. The computer system performs a set of actions based on the count of people in the aircraft.

[0007] Another embodiment of this disclosure provides a method for monitoring a platform. A computer system receives images from a sensor system positioned to monitor the movement of people relative to the platform. The computer system identifies a group of people in the images. The computer system uses the images to determine the movement of this group of people relative to the platform. Based on the movement determined for this group of people, the computer system determines a count of people on the platform. The computer system performs a set of actions based on the count of people on the platform.

[0008] Another embodiment of this disclosure provides an aircraft monitoring system including a computer system and a controller within the computer system. The controller is configured to receive images from a sensor system located for monitoring human movement relative to an aircraft. The controller is configured to identify a group of people in the images. The controller is configured to use the images to determine the movement of the group of people relative to the aircraft. The controller is configured to determine a count of people in the aircraft based on the determined movement for the group of people. The controller is configured to perform a set of actions based on the count of people in the aircraft.

[0009] Another embodiment of this disclosure provides a platform monitoring system including a computer system and a controller within the computer system. The controller is configured to receive images from a sensor system located to monitor human movement relative to the platform. The controller is configured to identify a group of people in the images. The controller is configured to use the images to determine the movement of the group of people relative to the platform. The controller is configured to determine a count of people on the platform based on the determined movement for the group of people. The controller is configured to perform a set of actions based on the count of people on the platform.

[0010] Another embodiment of this disclosure provides a computer program product for monitoring an aircraft, the computer program product including a computer-readable storage medium having first program code, second program code, third program code, fourth program code, and fifth program code stored thereon. The first program code is executable by a computer system to cause the computer system to receive images from a sensor system positioned to monitor the movement of people relative to the aircraft. The second program code is executable by the computer system to cause the computer system to identify a group of people in the images. The third program code is executable by the computer system to cause the computer system to use the images to determine the movement of the group of people relative to the aircraft. The fourth program code is executable by the computer system to cause the computer system to determine a count of people in the aircraft based on the movement determined for the group of people. The fifth program code is executable by the computer system to cause the computer system to perform a set of actions based on the count of people in the aircraft.

[0011] These features and functions may be implemented independently in various embodiments of this disclosure, or may be combined in other embodiments, in which more details may be seen with reference to the following description and drawings. Attached Figure Description

[0012] The appended claims set forth novel features that are considered characteristic of the illustrative embodiments. However, the illustrative embodiments and their preferred modes of use, further objects and features, will be best understood by referring to the following detailed description of the illustrative embodiments of this disclosure when read in conjunction with the accompanying drawings, wherein:

[0013] Figure 1 It is a graphical representation of a network of a data processing system in which illustrative implementation methods can be carried out;

[0014] Figure 2 This is a block diagram illustration of a platform monitoring environment according to an illustrative implementation method;

[0015] Figure 3 It is a flowchart illustration of a process for a monitoring platform according to an illustrative implementation method;

[0016] Figure 4 It is a flowchart illustration of a process for monitoring aircraft traffic according to an illustrative embodiment;

[0017] Figure 5 It is a flowchart illustration of a process for identifying an object according to an illustrative embodiment;

[0018] Figure 6 It is a flowchart illustration of a process for identifying restricted articles according to an illustrative embodiment;

[0019] Figure 7 It is a diagram of a flowchart illustrating a more detailed process for monitoring aircraft traffic according to an illustrative embodiment;

[0020] Figure 8 This is another illustration of a flowchart of a more detailed process for monitoring aircraft traffic according to an illustrative embodiment; and

[0021] Figure 9 This is a block diagram illustration of a data processing system according to an illustrative embodiment. Detailed Implementation

[0022] The illustrative embodiments recognize and take into account one or more different considerations. For example, the illustrative embodiments recognize and take into account that it may be difficult to consistently monitor the entry and exit of passengers and their belongings by crew members and other personnel. For example, the illustrative embodiments recognize and take into account that sleeping passengers, luggage, or other abandoned items may be missed during checks conducted by crew members, maintenance personnel, and other personnel. The illustrative embodiments recognize and take into account that these types of inconsistencies may occur at the end of a multi-day set of flights conducted by the crew.

[0023] Therefore, illustrative embodiments provide methods, apparatus, systems, and computer program products for monitoring aircraft traffic. In one illustrative example, the method monitors a platform, such as an aircraft. Images are received by a computer system from a sensor system positioned to monitor human movement relative to the aircraft. The computer system identifies a group of people in the images. Using the images, the computer system determines the movement of this group of people relative to the platform. Based on the determined movement for this group of people, the computer system determines a count of people on the aircraft. Based on the count of people on the platform, the computer system performs a set of actions.

[0024] In illustrative examples, a person may be situated on the platform from within some other interior space, such as a cabin, room, skybridge, office, lobby, aircraft cabin, anteroom, atrium, loft, auditorium, cargo area, passageway, corridor, or platform. Alternatively, a person may be situated on the platform from some other external area, such as a deck, balcony, or platform.

[0025] Now referring to the attached diagram, specifically, referring to... Figure 1 This image depicts a graphical representation of a network in which illustrative embodiments of a data processing system can be implemented. The network data processing system 100 is a computer network in which illustrative embodiments can be implemented. The network data processing system 100 includes a network 102, which is a medium for providing communication links between various devices and computers connected together within the network data processing system 100. The network 102 may include connections such as wired, wireless communication links, or fiber optic cables.

[0026] In the depicted example, server computers 104 and 106 are connected to network 102 along with storage unit 108. Additionally, client device 110 is connected to network 102. As depicted, client device 110 includes client computers 112, 114, and 116. Client device 110 can be, for example, a computer, workstation, or network computer. In the depicted example, server computer 104 provides client device 110 with information such as boot files, operating system images, and applications. Furthermore, client device 110 may also include other types of client devices, such as mobile phones 118, tablets 120, and smart glasses 122. In this illustrative example, server computer 104, server computer 106, storage unit 108, and client device 110 are network devices connected to network 102, where network 102 is the communication medium for these network devices. Some or all of the client devices 110 can form an Internet of Things (IoT), where these physical devices can connect to network 102 and exchange information with each other through network 102.

[0027] In these examples, client device 110 is a client of server computer 104. Network data processing system 100 may include additional server computers, client computers, and other devices not shown. Client device 110 is connected to network 102 using at least one of wired, fiber optic, or wireless connections.

[0028] The program code located in the network data processing system 100 can be stored on a computer-readable storage medium and downloaded to the data processing system or other devices for use. For example, the program code can be stored on a computer-readable storage medium on the server computer 104 and downloaded to the client device 110 via the network 102 for use on the client device 110.

[0029] In the depicted example, network data processing system 100 is an Internet with network 102, representing a global collection of networks and gateways communicating with each other using a protocol suite of Transmission Control Protocol / Internet Protocol (TCP / IP). The core of the Internet is a backbone of high-speed data communication lines between major nodes or hosts consisting of thousands of commercial, government, educational, and other computer systems routing data and messages. Of course, network data processing system 100 can also be implemented using many different types of networks. For example, network 102 can consist of at least one of the following: the Internet, intranet, local area network (LAN), metropolitan area network (MAN), or wide area network (WAN). Figure 1 This is intended as an example, not as an architectural limitation for different illustrative implementations.

[0030] As used in this article, when referring to items, "many" means one or more items. For example, "many different types of networks" means one or more different types of networks.

[0031] Furthermore, when used with a list of items, the phrase "at least one of..." means that different combinations of one or more of the listed items can be used, and it is possible that only one of each item in the list is required. In other words, "at least one of..." means that any combination of items and the number of items from the list can be used, but not all items in the list are required. The item can be a specific object, thing, or category.

[0032] For example, but not limited to, "at least one of item A, item B, or item C" can include item A, item A and item B, or item B. The example can also include item A, item B, and item C, or item B and item C. Of course, any combination of these items can exist. In some illustrative examples, "at least one of..." can be, for example, but not limited to, two of item A; one of item B; and ten of item C; four of item B and seven of item C; or other suitable combinations.

[0033] In this illustrative example, client computer 114 communicates with a sensor system having sensors located in aircraft 130 and passenger boarding bridge 132. In some illustrative examples, the sensors may be located in aircraft 130 instead of passenger boarding bridge 132. Client computer 114 and sensor system are part of an aircraft monitoring system. The aircraft monitoring system also includes controller 134 located in server computer 104. In this illustrative example, controller 134 operates to determine how many people are on aircraft 130. This determination can be made at multiple different times. For example, it can be made before boarding, after boarding, before disembarking, after disembarking, before takeoff, after takeoff, or at other times or phases of flight.

[0034] The controller 134 receives sensor information 136 from the client computer 114. In this illustrative example, the sensor information 136 sent by the client computer 114 is generated by sensors in a sensor system located in the aircraft 130 and the passenger boarding bridge 132. As depicted, the sensor information 136 includes images 138 that can be analyzed to determine how many people are in the aircraft 130.

[0035] In this illustrative example, controller 134 can identify objects in image 138. Furthermore, controller 134 also identifies the motion of the objects in image 138. Controller 134 can classify objects into object types based on the type of motion identified for the objects. For example, the motion type can indicate that the object is a person, an animal, a piece of luggage, a trolley, a wheelchair, and some other type of object. Controller 134 can also analyze the features of the objects in image 138 to classify the objects.

[0036] Furthermore, in addition to or instead of image 138, controller 134 can also analyze other sensor information to identify the object type. For example, sensor information 136 may also include range measurement 140. Range measurement 140 can be generated from a LiDAR sensor using a pulsed laser beam to measure the distance to an object. These distances can be used to identify the object's size, position, movement, or other suitable information about the object.

[0037] In this illustrative example, controller 134 can analyze image 138 and distance measurement 140 to identify people who may be present in aircraft 130 and passenger boarding bridge 132. In this illustrative example, image 138 can be analyzed to determine the movement of the identified object relative to aircraft 130. The movement of the identified object and sensor information 136 can be used to determine where the object is located, where the object is moving, or whether the object is inside or outside aircraft 130, or in passenger boarding bridge 132, or can be used for some other location determination to monitor traffic on aircraft 130. In this example, traffic is the movement of objects in and out of aircraft 130.

[0038] The controller 134 can perform a set of actions based on the presence of a person in the aircraft 130. As used herein, when referring to objects, "set" means one or more objects. For example, "a set of different types of networks" is one or more different types of networks.

[0039] For example, once aircraft 130 has completed passenger boarding and before disembarking from passenger bridge 132, a count of the people present on aircraft 130 can be determined. This count can be used to determine whether aircraft 130 is ready to depart from passenger bridge 132. In this example, this set of actions can be at least one of the following: indicating whether all passengers have boarded aircraft 130, indicating the number of passengers present on aircraft 130, indicating the number of pieces of baggage present on aircraft 130, or other suitable actions.

[0040] In this illustrative example, controller 134 can monitor other aircraft and other passenger boarding bridges, such as aircraft 141 and passenger boarding bridge 142. In this illustrative example, aircraft 130 and aircraft 141 can be from the same airline or different airlines. In another illustrative example, aircraft 130 and aircraft 141 can be from the same airport or different airports. In this way, aircraft traffic can be monitored for airlines, entire airports, multiple airlines, multiple airports, or other suitable aircraft groups for which monitoring of inbound and outbound traffic is desired.

[0041] Now for reference Figure 2 This illustration depicts a block diagram of a platform monitoring environment according to an illustrative embodiment. In this illustrative example, the platform monitoring environment 200 includes hardware that can be used, such as... Figure 1 The hardware implementation components shown in the network data processing system 100.

[0042] In this illustrative example, platform monitoring environment 200 is an environment in which platform 202 can monitor a person 204 relative to platform 202. In this illustrative example, platform 202 is aircraft 206. Aircraft 206 can be, for example, an airplane, a commercial aircraft, a commercial jet, a rotorcraft, a tiltrotor aircraft, a vertical takeoff and landing (VTOL) aircraft, or some other type of aircraft.

[0043] In this illustrative example, monitoring of platform 202 can be performed by platform monitoring system 208. When platform 202 is aircraft 206, platform monitoring system 208 takes the form of aircraft monitoring system 210.

[0044] In this illustrative example, the aircraft monitoring system 210 can monitor traffic, such as the movement 212 of people 204 and objects 214 relative to the aircraft 206. Objects 214 can be categorized into many different object types. For example, object 214 is selected from at least one of the following: luggage, backpack, rolling suitcase, bag, box, bottle, handbag, briefcase, duffel bag, wallet, belt, animal, or some other object type.

[0045] In this illustrative example, the aircraft monitoring system 210 comprises many different components. As depicted, the aircraft monitoring system 210 includes a computer system 216 and a controller 218 within the computer system 216.

[0046] Controller 218 can be implemented as software, hardware, firmware, or a combination thereof. When using software, the operations performed by controller 218 can be implemented in program code configured to run on hardware (such as a processor unit). When using firmware, the operations performed by controller 218 can be implemented in program code and data and stored in persistent memory to run on a processor unit. When hardware is used, the hardware may include circuitry that operates to perform the operations in controller 218.

[0047] In illustrative examples, the hardware may take the form of at least one of the following: a circuit system, an integrated circuit, an application-specific integrated circuit (ASIC), a programmable logic device, or some other suitable type of hardware configured to perform a number of operations. Using a programmable logic device, the device can be configured to perform a number of operations. The device can be reconfigured later or can be permanently configured to perform said many operations. Programmable logic devices include, for example, programmable logic arrays, programmable array logic, field-programmable logic arrays, field-programmable gate arrays, and other suitable hardware devices. Additionally, the process can be implemented in organic components integrating inorganic components and can consist entirely of organic components other than humans. For example, the process can be implemented as a circuit of organic semiconductors.

[0048] Computer system 216 is a physical hardware system and includes one or more data processing systems. When more than one data processing system exists in computer system 216, those data processing systems communicate with each other using a communication medium. The communication medium can be a network. The data processing system can be selected from at least one of the following: a computer, a server computer, a tablet computer, or some other suitable data processing system.

[0049] In this illustrative example, the aircraft monitoring system 210 includes a sensor system 220. The sensor system 220 is a hardware system and may also include software.

[0050] In some illustrative examples, sensor system 220 may be considered a component separate from aircraft monitoring system 210. These components may be used to perform the operations described in the figure to monitor aircraft 206 when used to monitor other types of platforms. Sensor system 220 may consist of sensors 222 selected from at least one of the following: visible light camera, infrared (IR) sensor, panchromatic camera, terahertz camera, X-ray system, thermal infrared (IR) sensor, light detection and ranging (LiDAR) system, or other suitable types of sensor devices.

[0051] In this illustrative example, sensor 222 in sensor system 220 may be positioned relative to aircraft 206 to monitor the movement 212 of at least one of person 204 or object 214 relative to aircraft 206. For example, sensor 222 may be located in at least one of the following: cabin, aircraft door, bulkhead, aircraft sidewall, runway, aircraft floor, galley, lavatory, passenger comfort unit, passenger bridge door, cargo hold, airport gate, jet bridge, passenger bridge, or some other suitable location.

[0052] In this illustrative example, controller 218 is configured to perform a number of different operations. For example, controller 218 may receive image 224 from sensor information 225 received from sensor 222 in sensor system 220. In this illustrative example, sensor 222 and sensor system 220 are positioned to monitor the movement 212 of person 204 relative to aircraft 206. Controller 218 may identify a group of people 204 in image 224.

[0053] Furthermore, controller 218 can use image 224 to determine the movement 212 of the group of people 204 relative to aircraft 206. Controller 218 can determine a count 226 of people 204 in aircraft 206 based on the movement 212 determined for the group of people 204. For example, controller 218 can track the movement 212 of at least one person 204 or object 214 entering and leaving aircraft 206 to determine a count 226 of at least one person 204 or object 214 in aircraft 206.

[0054] In this illustrative example, the count 226 of people 204 in aircraft 206 can be determined in response to one or more events, based on movement 212 determined for the group of people 204. The events can be periodic or non-periodic. For example, for periodic events, the determination can be made every ten minutes, every five minutes, or after some other suitable time interval.

[0055] In another illustrative example, this determination can be made to determine how many people 204 are on aircraft 206 after a non-periodic event has occurred. A non-periodic event can be selected from at least one of the following: boarding the aircraft, disembarking from the aircraft, closing the aircraft door after disembarking, leaving the gate, partially disembarking the aircraft, takeoff, landing, or some other non-periodic event.

[0056] The controller 218 can perform a set of actions 228 based on the count 226 of people 204 in the aircraft 206. In the illustrative example, the set of actions 228 may be selected from at least one of the following: sending a message, generating an alarm, creating a log entry using a group of people 204, indicating preparation for takeoff, indicating no preparation for takeoff, indicating a missing passenger, indicating the presence of a passenger on board, indicating the presence of an object or animal left behind by a passenger, indicating a discrepancy between the passenger log of aircraft 206 and the passenger log of aircraft 206 (including during a stopover when a passenger who should have disembarked has not disembarked), indicating the presence of an unidentified object on aircraft 206, determining whether to restrict the flight of a person on aircraft 206, indicating the presence of an animal on aircraft 206, indicating when the number of baggage pieces exceeds the storage capacity of overhead bins and under-seat storage in the cabin, alerting the pilot, alerting the cabin crew, alerting the gate service personnel, preventing the aircraft engines from starting, preventing the aircraft doors of aircraft 206 from closing, instructing aircraft personnel to leave an object in the cargo hold, performing a cross-check of passenger counts during an emergency evacuation of aircraft 206, or other appropriate actions.

[0057] In an illustrative example, controller 218 may also use image 224 to identify a group of objects 214 associated with the group of people 204. Controller 218 may determine how many objects 214 are in aircraft 206 based on movement 212 determined for the group of objects 214. The identified group of objects 214 may be part of a count 226. In other words, count 226 may be how many people 204, how many objects 214, or both are present in aircraft 206. In this example, objects 214 may include animals. This determination may also be made in response to periodic or non-periodic events, such as those described for people 204.

[0058] Additionally, the count 226 of people 204, objects 214, or both can be used to perform various actions in action 228. For example, objects 214 can be analyzed to identify objects with dimensions corresponding to items that need to be stored in overhead bins or under-seat storage. The number of these types of objects can be used to determine whether the number of objects present exceeds the cabin's storage capacity for those items in aircraft 206.

[0059] In other words, it can be determined whether luggage, handbags, backpacks, and other items exceed the storage capacity of the overhead bins and under-seat storage in the cabin. If the number of items to be stored exceeds the cabin's storage capacity, action 228 can be taken, such as placing one or more items in the cargo area, to ensure that the remaining items can be properly stored in the cabin.

[0060] In addition to identifying a group of objects 214 associated with the group of people 204, the controller 218 can also determine whether the restricted item 230 exists within the group of objects 214. In an illustrative example, the restricted item 230 may be located among the objects in the group of objects 214 or may be an object within the group of objects 214.

[0061] In this illustrative example, the identification of the confined item 230 can be performed by analyzing a set of images 224 via controller 218. The set of images 224 may be generated by at least one of the following sensors: a panchromatic camera, a terahertz camera, a millimeter-wave scanner, an infrared (IR) camera, an X-ray system, or some other suitable type of sensor.

[0062] Using the identification of object 214, controller 218 can perform the group of actions 228 based on the movement 212 determined for at least one of the group of people 204 or the group of objects 214.

[0063] The controller 218 may use the artificial intelligence system 232 to identify at least one of the following using the image 224: person 204, object 214, person 204 count 226, or object 214 count 226.

[0064] As depicted, artificial intelligence system 232 is a system with intelligent behavior and may be based on human brain function. Artificial intelligence system 232 includes at least one of artificial neural networks, cognitive systems, Bayesian networks, fuzzy logic, expert systems, natural language systems, or some other suitable systems. Machine learning is used to train artificial intelligence system 232. Machine learning involves inputting data into the process and allows the process to adjust and improve the functionality of artificial intelligence system 232. These components can form one or more machine learning models within artificial intelligence system 232.

[0065] A machine learning model is an artificial intelligence model in an artificial intelligence system 232 that can learn without explicit programming. Machine learning models can learn based on training data input to them. They can learn using various types of machine learning algorithms. These algorithms include at least one of supervised learning, unsupervised learning, feature learning, sparse dictionary learning, anomaly detection, association rule processing, or other types of learning algorithms. Examples of machine learning models include artificial neural networks, decision trees, support vector machines, Bayesian networks, genetic algorithms, and other types of models. These machine learning models can be trained using data and processed with additional data to provide the desired output.

[0066] Alternatively, controller 218 may determine the identities of the group of people 204 in image 224 based on at least one of the following: motion type identified for objects 214 in image 224, object features, template comparison, or other suitable techniques for determining the identities 234 of a group of people 204. As depicted, controller 218 may use artificial intelligence system 232 to perform this determination.

[0067] For example, when determining the identities 234 of the group 204, the controller 218 can determine whether the group 204 consists of passengers, crew members, or other persons who should be present to allow the aircraft 206 to leave the passenger boarding bridge for takeoff. For example, the controller 218 can identify passengers within the group 204.

[0068] The controller 218 can also use facial recognition technology implemented in the artificial intelligence system 232 to determine who the passenger is from image 224. This process can be used to further verify passengers using boarding passes or checking in for a specific flight on aircraft 206. Therefore, additional redundancy can exist to verify passengers taking the flight.

[0069] Additionally, using the identification of person 204, controller 218 can determine whether the identified person 204 in aircraft 206 is on the restricted list. Furthermore, this determination can be made not only for person 204 in aircraft 206, but also for identified person 204 who has moved outside aircraft 206.

[0070] In an illustrative example, controller 218 can access identity repository 219. Identity repository 219 can be one or more collections of information, such as a database. Identity repository 219 can include a list or identification of persons 204 who may be present in or restricted from flying on aircraft 206. For example, identity repository 219 can identify persons 204 who may be restricted from flying, entering a country, or other types of restrictions. For example, identity repository 219 can include a terrorist screening database (TSDB) or other surveillance list. In this way, controller 218 in aircraft monitoring system 210 can operate to provide additional security beyond the checks and screenings conducted within the airport.

[0071] In one illustrative example, there are one or more technical solutions that overcome the technical problems of monitoring traffic of passengers and objects entering and leaving an aircraft. Therefore, one or more technical solutions can provide the technical effect of monitoring traffic, such as the movement of people, to determine how many people are on board an aircraft. One or more technical solutions can monitor the movement of people 204 entering and leaving aircraft 206 in a manner capable of counting how many people 204 are on board aircraft 206.

[0072] The process in the illustrative example can be used to monitor the entry and exit of passengers and passenger-associated objects to ensure that all passengers and their objects that board the aircraft disembark after the flight. Furthermore, the illustrative example can also be used to provide additional security checks for restricted objects. These checks can provide redundant information to the current screenings being conducted at the airport.

[0073] Computer system 216 can be configured to perform at least one of the steps, operations, or actions described in the various illustrative examples using software, hardware, firmware, or a combination thereof. Therefore, computer system 216 operates as a dedicated computer system, wherein controller 218 within computer system 216 is capable of monitoring traffic on the platform. Specifically, controller 218 transforms computer system 216 into a dedicated computer system compared to currently available general-purpose computer systems that do not have controller 218.

[0074] In an illustrative example, the use of controller 218 in computer system 216 integrates the process into a practical application for monitoring the platform, improving the performance of computer system 216. By positioning sensor system 220 to generate sensor information 225 regarding object movement 212, computer system 216 can more easily determine at least one of the following: a count 226 of people 204 or a count 226 of objects 214 present on platform 202, such as aircraft 206. Specifically, controller 218 in computer system 216 can determine the count 226 in a manner that does not require cameras or other sensors to completely cover the interior of aircraft 206.

[0075] In this illustrative example, the controller 218 in computer system 216 relates to a practical application of the process of integrating controller 218 into computer system 216 for monitoring traffic on platform 202, such as aircraft 206. In this illustrative example, controller 218 receives sensor information 225, including images 224, from sensor system 220 positioned to monitor the movement 212 of people 204 relative to platform 202. Controller 218 identifies a group of people 204 in images 224. Controller 218 uses images 224 to determine the movement 212 of this group of people 204 relative to platform 202. Further, controller 218 determines a count 226 of people 204 on platform 202 based on the determined movement 212 for this group of people 204. Controller 218 performs a set of actions 228 based on the count 226 of people 204 on platform 202. In this way, the controller 218 in the computer system 216 provides the practical application of the operation to monitor traffic relative to the platform 202, so that the count 226 of the number of people 204 present on the platform 202 can be determined.

[0076] Figure 2The illustration of the central platform monitoring environment 200 does not imply any physical or architectural limitations on the ways in which the illustrative implementation can be carried out. Other components besides those shown or in place of the shown components may be used. Some components may be unnecessary. Furthermore, boxes are presented to indicate some functional components. When implemented in the illustrative embodiment, one or more of these boxes may be combined, divided, or combined and divided into different boxes.

[0077] Although different instances of monitoring platform 202 have been described in the form of aircraft 206, the different operations performed by controller 218 can be applied to other platforms. For example, platform 202 can also take the form of: mobile platforms, fixed platforms, land-based structures, water-based structures, space-based structures, surface ships, tanks, personnel carriers, trains, spacecraft, space stations, satellites, submarines, automobiles, power plants, bridges, dams, houses, manufacturing facilities, buildings, and other suitable types of platforms other than aircraft 206 that require monitoring of personnel 204.

[0078] Furthermore, when the platform 202 takes on a form other than aircraft 206, the sensor 222 can be placed in other locations to monitor the movement 212 of at least one of the people 204 or objects 214. For example, these other locations could be corridors, rooms, conference rooms, reception areas, lobbies, doorways, foyer, or some other location.

[0079] As another example, in some illustrative instances, the artificial intelligence system 232 may be a component within the controller 218. As another example, in addition to or in place of image 224, other types of sensor information 225 may be used to determine the movement 212 of at least one of the person 204 or object 214.

[0080] For example, the distance in sensor information 225 can be used to determine movement 212. For example, the light detection and ranging (LiDAR) device in sensor 222 can provide the distance to an object. The LiDAR device can be used to identify objects such as person 204 or objects 214 associated with person 204. For example, the laser scanner in sensor 222 can generate the measurements in sensor information 225.

[0081] Then turn to Figure 3 The illustration depicts a flowchart of a process for a monitoring platform, based on an illustrative embodiment. Figure 3 The process can be implemented in hardware, software, or both. When implemented in software, the process can take the form of program code, which is executed by one or more processor units located in one or more hardware devices of one or more computer systems. For example, the process can be implemented in... Figure 2It is implemented in the controller 218 of the computer system 216.

[0082] The process begins by receiving images from a sensor system positioned to monitor the movement of people relative to the platform (operation 300). The process then identifies a group of people within the images (operation 302).

[0083] The process uses images to determine the movement of the group of people relative to the platform (operation 304). The process then determines the number of people on the platform based on the determined movement of the group of people (operation 306).

[0084] The process performs a set of actions (operation 308) based on the number of people on the platform. The process then terminates.

[0085] As previously stated, Figure 3 The process illustrated can be applied to many types of platforms. Furthermore, the count of a group of people can be determined without generating images of every single person on the platform. In other words, a localization sensor system can be used to determine the movement of people on the platform. This type of localization uses fewer sensors compared to generating images of everyone on the platform at a given time.

[0086] Then turn to Figure 4 The illustration depicts a flowchart of a process for monitoring aircraft traffic, based on an illustrative embodiment. Figure 4 The process can be implemented in hardware, software, or both. When implemented in software, the process can take the form of program code, which is executed by one or more processor units located in one or more hardware devices of the computer system. For example, the process can be implemented in... Figure 2 It is implemented in the controller 218 of the computer system 216.

[0087] The process begins by receiving images from a sensor system positioned to monitor the movement of people relative to the aircraft (operation 400). The process identifies a group of people in the images (operation 402). The process then identifies a group of objects in the images (operation 404). In this illustrative example, the group of objects is identified based on their movement. Further, the identification may include identifying the association between the object and the person. In other words, if the image shows an object being moved by a specific person, the object can be identified as associated with that specific person.

[0088] The process uses images to determine the movement of the group of people relative to the aircraft (operation 406). The process uses images to identify the movement of the group of objects relative to the aircraft (operation 408). In operations 406 and 408, the determination of movement may include the direction of movement of people and objects. The determination may also include determining which people entered the aircraft and whether people disembarked from the aircraft.

[0089] This process determines the number of people in the aircraft based on the movement of the group of people (operation 410). In other words, this process can determine which passengers are on the aircraft.

[0090] This process is based on a count of objects in the aircraft identified for movement against the group of objects (operation 412). In some illustrative examples, operation 412 may include identifying the type of object in addition to the presence of objects in the group of objects.

[0091] The process involves a series of actions (operation 414) based on the number of people on board the aircraft. The process then terminates.

[0092] This process can be performed any number of times to determine the number of people and objects present on the aircraft. It can also be performed to determine the number of people and objects present in other locations near the aircraft, such as passenger boarding bridges connected to the aircraft.

[0093] Then turn to Figure 5 The illustration depicts a flowchart of a process for identifying an object, according to an illustrative embodiment. Figure 5 The process can be implemented in hardware, software, or both. When implemented in software, the process can take the form of program code, which is executed by one or more processor units located in one or more hardware devices of a computer system. For example, the process can be implemented in... Figure 2 The process is implemented in the controller 218 of the computer system 216. This process can be combined with... Figure 2 The process shown is used together to add additional features to the monitoring personnel relative to the aircraft.

[0094] The process begins by identifying a group of people in the image (operation 500). The process then identifies the object type of a group of objects in the image (operation 502). The process then terminates.

[0095] In this illustrative example, at least one of the following techniques can be used for object identification in an image: motion type, object features, template comparison, or other suitable techniques. Figure 5 The identification of a person's identity and the identification of an object's type.

[0096] In this illustrative example, machine vision processes, comparisons with images of non-humans and objects, machine learning models, and other mechanisms and techniques may be used. In one example, images of authorized persons may be used as comparison templates. These images may be obtained from sources such as driver's license photos, passport photos, or other authorized personnel. The identified identity may be a passenger, crew member, or other authorized person. Persons who do not match the images in the comparison template may have an identity set as "unauthorized" or "unknown."

[0097] As another example, Figure 2 The artificial intelligence system 232 may include one or more machine learning models that have been trained to recognize people. These machine learning models are trained to roughly recognize people and then determine the identity of the identified people.

[0098] By identifying people, passengers, crew members, maintenance personnel, security personnel, and other personnel can be identified in traffic moving relative to the aircraft. Furthermore, it is also possible to identify specific passengers and those who should be on board the aircraft.

[0099] Then turn to Figure 6 The illustration depicts a flowchart of a process for identifying restricted items, based on an illustrative embodiment. Figure 6 The process can be implemented in hardware, software, or both. When implemented in software, the process can take the form of program code, which is executed by one or more processor units located in one or more hardware devices of one or more computer systems. For example, the process can be implemented in... Figure 2 The process is implemented in the controller 218 of the computer system 216. This process can be combined with... Figure 2 The processes shown are used together to add additional features to the monitoring personnel relative to the aircraft.

[0100] This process determines a set of object types for a group of objects identified in an image (operation 600). In operation 600, this determination can be made using a set of images generated by at least one of a panchromatic camera, a terahertz camera, a millimeter-wave scanner, an infrared (IR) camera, or an X-ray system. These types of sensors can be used to generate images of objects within objects, such as an image of a can of hairspray inside a bag.

[0101] The identification of the object type of this group of objects can be performed using various systems, such as artificial intelligence systems, machine learning models, computer vision systems, template matching systems, and other suitable systems for object recognition.

[0102] The process determines whether any restricted items exist based on the type of the group of objects identified for that group (Operation 602). The process then terminates.

[0103] In this illustrative example, various object recognition technologies can be used to... Figure 2 The controller 218 in the middle performs Figure 6 The operations within. For example. Figure 2 The artificial intelligence system 232 may include one or more machine learning models trained to identify objects, such as items that a person may carry or move. These machine learning models are trained to identify whether a particular object is a restricted item.

[0104] refer to Figure 7 The illustration depicts a flowchart of a more detailed process for monitoring aircraft traffic, based on an illustrative embodiment. Figure 7 The process can be implemented in hardware, software, or both. When implemented in software, the process can take the form of program code, which is executed by one or more processor units located in one or more hardware devices of one or more computer systems. For example, the process can be implemented in... Figure 2 It is implemented in the controller 218 of the computer system 216.

[0105] The process begins with receiving sensor data from a sensor system positioned to monitor traffic entering and leaving the aircraft (operation 700). In this illustrative example, the sensor data received in operation 700 can take many forms. For example, images can be received from visible light cameras and infrared (IR) cameras in the sensor system. As another example, distance measurements can be received from a light detection and ranging (LiDAR) device in the sensor system. These distance measurements can be used to determine the size of various objects. Furthermore, the sensor system can include measurements of submillimeter radiation set to terahertz waves from terahertz sensors such as terahertz cameras.

[0106] This process uses sensor data to identify objects (operation 702). In operation 702, objects can be identified in a series of images generated by a visible light camera. As another example, objects can be identified based on heat measured from sensor data from an infrared (IR) camera. The heat measurement can be used to generate a thermal map for detecting objects and their motion. As another example, objects can be detected based on sensor data received from a LiDAR (Light Detection and Ranging) device. The sensor data includes measurements of distance and wavelength. This information can be used to generate a three-dimensional representation of the object and track its movement. Objects can also be identified from sensor data derived from terahertz wave measurements detected by a detector sensor. Terahertz wave measurements can be used to identify objects within objects. For example, terahertz waves can penetrate fabrics and plastics used in objects to detect the presence of objects within those objects. Terahertz wave measurements can be used to generate images of solid objects.

[0107] The process then uses sensor data to characterize the object (operation 704). In operation 704, some or all of the different types of sensor data can be used to characterize the object. For example, motion identified in an image from a visible light camera can be used to determine whether the type of motion is human. In this way, the object can be classified as a human or a non-human object. As another example, thermal measurements from an infrared (IR) camera can be used to identify the presence of a human or animal compared to other object types. As yet another example, distance measurements from a light detection and ranging (LiDAR) device can be used to characterize the object based on the dimensions measured for that object. Terahertz measurements can be used to identify objects that may be concealed within other objects, such as luggage.

[0108] The process then determines whether an object is entering or leaving the aircraft based on the movement identified in the image (operation 706). In operation 706, a line can be used to determine when an object enters or leaves the aircraft. Depending on the direction of movement, the object enters or leaves the aircraft when it crosses the line. This line can cross an entrance, such as an aircraft door. In this illustrative example, the line is a virtual line used for counting. This virtual line can be based on lines of pixels in the image, features, or some other mechanism.

[0109] The process updates the counts of people and objects associated with people based on whether an object is entering or leaving the aircraft (operation 708).

[0110] The process then counts the number of objects present on the aircraft based on object detection and object characterization (operation 710). The process then performs a set of actions (operation 712). For example, in operation 712, for objects that are not human, the actions may include determining whether the object is permitted on the aircraft. If the object is not permitted on the aircraft, another action may be generating an alarm. For example, the object could be a piece of luggage larger than the permitted size for carry-on baggage. As another example, the object could be a restricted item, such as a can of aerosol hairspray larger than a specified size.

[0111] Furthermore, the action may also include determining whether a specific person is permitted on the aircraft. For example, the identity of a person on the aircraft can be determined. This identity can then be used to verify whether the person on the aircraft is permitted to be on board. For example, even with a normal boarding process at the gate, a passenger may mistakenly board the wrong aircraft.

[0112] refer to Figure 8 The illustration depicts a flowchart of a more detailed process for monitoring aircraft traffic, based on an illustrative embodiment. Figure 8The process can be implemented in hardware, software, or both. When implemented in software, the process can take the form of program code, which is executed by one or more processor units in one or more hardware devices of one or more computer systems. For example, the process can be implemented in... Figure 2 Implemented in the controller 218 of the computer system 216. In this illustrative example, it can be achieved through... Figure 2 The controller 218 of the artificial intelligence system 232 is used to form the different operations shown in the figure.

[0113] In this illustrative example, sensor information 225 can be generated by a sensor system 220 having sensors 222 located within aircraft 206—for example, in the cabin. Sensors 222 in aircraft 206 may include cameras positioned in the cabin, enabling the detection of movement 212 of passengers, crew, or other persons. Sensors 222 may also include other types of sensors, such as light detection and ranging (LiDAR) systems, terahertz cameras, or other suitable types of sensors.

[0114] The process begins by receiving sensor information from within the aircraft (operation 800). In this illustrative example, the sensor information includes images. The images can be generated at a rate that allows the movement of objects such as passengers, crew members, or other people to be determined. The process then determines the movement of people within the aircraft (operation 802).

[0115] This process analyzes movement (Operation 804). In Operation 804, the analysis can determine traffic patterns in different areas of the aircraft. These areas may include aisles, galleys, areas outside lavatories, or other areas within the cabin. As another example, the analysis can identify clusters or locations of passengers or other people in areas of the aircraft. As another illustrative example, the analysis can determine whether a passenger is making a gesture requesting assistance. This gesture can be made in lieu of using a call button. Furthermore, the analysis can be used to determine a passenger's ability to reach overhead bins or other locations within the cabin.

[0116] The process involves a series of actions based on the analysis of human movement within the aircraft (Operation 806). The process then terminates.

[0117] Many actions in Operation 806 can take many different forms. For example, analysis from Operation 804 can include identifying unusual movement. Unusual movement could be, for example, a passenger consuming excessive alcohol, being sick, unruly, or in other abnormal or unwanted states or behaviors. Using this type of analysis, the process can alert pilots, flight attendants, law enforcement officials, or others. This alert can also include passenger identification. This identification could include photographs or images of passengers whose unusual movement has been detected.

[0118] As another example, based on airline rules, federal regulations, or other rules, traffic analysis in Operation 804 can indicate that there are too many people in front of a lavatory or near a flight deck door. This result can be used to generate an alert to notify flight attendants or crew members of the situation.

[0119] In another illustrative example, the analysis could indicate that a passenger has requested assistance from a flight attendant. In this case, the action in operation 806 could alert the flight attendant to the specific passenger who has requested assistance.

[0120] As another illustrative example, when the analysis assesses a passenger's ability to reach and place their luggage in the overhead bins, actions could include alerting flight attendants that the passenger may need assistance with their luggage. Thus, at the end of the flight, the process can predict when a passenger might need help retrieving their luggage from the overhead bins. These and other actions can be based on analysis of the movement of passengers and other people within the aircraft.

[0121] The flowcharts and block diagrams depicting different embodiments illustrate the architecture, functionality, and operation of some possible implementations of the apparatus and methods in the illustrative embodiments. In this regard, each block in the flowchart or block diagram may represent at least one module, segment, function, or part of an operation or step. For example, one or more blocks may be implemented as program code, hardware, or a combination of program code and hardware. When implemented in hardware, the hardware may, for example, take the form of an integrated circuit manufactured or configured to perform one or more operations in the flowchart or block diagram. When implemented as a combination of program code and hardware, the implementation may take the form of firmware. Each block in the flowchart or block diagram may be implemented using a dedicated hardware system that performs different operations or combinations of dedicated hardware and program code executed by the dedicated hardware.

[0122] In some alternative implementations of the illustrative embodiments, one or more functions marked in the boxes may occur in a different order than those shown in the figures. For example, in some cases, two boxes shown consecutively may occur substantially simultaneously, or depending on the functions involved, the boxes may sometimes occur in reverse order. Moreover, additional boxes may be added in addition to those shown in the flowchart or block diagram.

[0123] For example, the process involving monitoring an object, as described in some different flowcharts, can be applied to other types of platforms besides aircraft. For instance, the process can be applied to tracking the movement of objects on buildings, trains, buses, stadiums, or other suitable types of platforms.

[0124] Now go to Figure 9 A block diagram of a data processing system is illustrated according to an illustrative embodiment. The data processing system 900 can be used for implementation. Figure 1 The data processing system 900 includes server computer 104, server computer 106, and client device 110. It can also be used to implement... Figure 2 The computer system 216 is described above. In this illustrative example, the data processing system 900 includes a communication framework 902 that provides communication between a processor unit 904, a memory 906, a permanent storage device 908, a communication unit 910, an input / output (I / O) unit 912, and a display 914. In this example, the communication framework 902 takes the form of a bus system.

[0125] Processor unit 904 is used to execute instructions for software that can be downloaded to memory 906. Processor unit 904 includes one or more processors. For example, processor unit 904 may be selected from at least one of the following: multi-core processor, central processing unit (CPU), graphics processing unit (GPU), physical processing unit (PPU), digital signal processor (DSP), network processor, or some other suitable type of processor. Further, processor unit 904 may be implemented using one or more heterogeneous processor systems, wherein the main processor and secondary processors coexist on a single chip. As another illustrative example, processor unit 904 may be a symmetric multiprocessor system containing multiple processors of the same type on a single chip.

[0126] Memory 906 and permanent storage device 908 are examples of storage device 916. A storage device is any hardware capable of temporarily, permanently, or both temporarily and permanently storing information, such as, for example, but not limited to, data, program code in functional form, or at least other suitable information. In these illustrative examples, storage device 916 may also be referred to as a computer-readable storage device. In these examples, memory 906 may be, for example, random access memory or any other suitable volatile or non-volatile storage device. Depending on the specific implementation, permanent storage device 908 may take various forms.

[0127] For example, permanent storage device 908 may contain one or more components or devices. For example, permanent storage device 908 may be a hard disk drive, a solid-state drive (SSD), flash memory, a rewritable optical disk, a rewritable magnetic tape, or a combination of the above. The media used in permanent storage device 908 may also be removable. For example, a removable hard disk drive may be used in permanent storage device 908.

[0128] The communication unit 910, in these illustrative examples, provides communication with other data processing systems or devices. In these illustrative examples, the communication unit 910 is a network interface card.

[0129] The input / output unit 912 allows data to be input and output using other devices that can be connected to the data processing system 900. For example, the input / output unit 912 can provide a connection for user input via at least one of a keyboard, mouse, or some other suitable input device. Furthermore, the input / output unit 912 can send output to a printer. The display 914 provides a mechanism for displaying information to the user.

[0130] Instructions for at least one of an operating system, application program, or program may be located in storage device 916 and communicate with processor unit 904 via communication frame 902. Processes in different implementations may be performed by processor unit 904 using computer-implemented instructions, which may be located in memory, such as memory 906.

[0131] These instructions are referred to as program code, computer-usable program code, or computer-readable program code that can be read and executed by a processor in processor unit 904. In an illustrative example, these instructions can be executed to cause one or more processors in processor unit 904 to perform actions in components such as... Figure 2 The operations described in the flowchart embodied in controller 218 are as follows. The program code in different embodiments may be embodied on different physical or computer-readable storage media, such as memory 906 or permanent storage device 908.

[0132] Program code 918 is functionally located on a computer-readable medium 920, which is optionally removable and downloadable to or transferable to a data processing system 900 for execution by a processor unit 904. In these illustrative examples, program code 918 and the computer-readable medium 920 form a computer program product 922. In another illustrative example, the computer-readable medium 920 is a computer-readable storage medium 924.

[0133] In these illustrative examples, computer-readable storage medium 924 is a physical or tangible storage device for storing program code 918, and not a medium for propagating or transmitting program code 918. As used herein, computer-readable storage medium 924 should not be construed as a transient signal itself, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., optical pulses through fiber optic cables), or electrical signals transmitted through wires.

[0134] Optionally, program code 918 can be transmitted to data processing system 900 using a computer-readable signal medium. The computer-readable signal medium can be, for example, a propagated data signal containing program code 918. For example, the computer-readable signal medium can be at least one of electromagnetic signals, optical signals, or any other suitable type of signal. These signals can be transmitted via a connection, such as a wireless connection, fiber optic cable, coaxial cable, wire, or any other suitable type of connection.

[0135] Furthermore, as used herein, “computer-readable medium 920” can be singular or plural. For example, program code 918 may be located in computer-readable medium 920 in the form of a single storage device or system. In another instance, program code 918 may be located in computer-readable medium 920 distributed across multiple data processing systems. In other words, some instructions in program code 918 may be located in one data processing system, while other instructions in program code 918 may be located in a data processing system. For example, a portion of program code 918 may be located in computer-readable medium 920 in a server computer, while another portion of program code 918 may be located in computer-readable medium 920 in a group of client computers.

[0136] The different components shown for data processing system 900 do not imply any architectural limitation on the ways in which different implementations can be carried out. In some illustrative examples, one or more of the components may be incorporated into another component or otherwise form part of another component. For example, in some illustrative examples, memory 906 or a portion thereof may be incorporated into processor unit 904. Different illustrative implementations may be implemented in data processing systems that include components other than those shown for data processing system 900 or components that replace them. Figure 9 Other components shown may differ from the illustrative example shown. Different implementations can be implemented using any hardware device or system capable of running program code 918.

[0137] Therefore, illustrative examples provide methods, apparatus, systems, and computer program products for monitoring platform traffic. In one illustrative example, a method monitors a platform such as an aircraft. Images are received by a computer system from a sensor system positioned to monitor the movement of people relative to the aircraft. A group of people is identified in the images by the computer system. The movement of this group of people relative to the platform is determined using the images by the computer system. Based on the determined movement of this group of people, a count of people on the aircraft is determined by the computer system. Based on the count of people on the platform, a set of actions are performed by the computer system.

[0138] In this way, platform monitoring systems, such as aircraft monitoring systems, can be operated to determine the number of passengers and carry-on baggage present on board the aircraft. In an illustrative example, the movement of objects such as passengers and baggage can be monitored to determine their entry and exit relative to the aircraft. This entry and exit monitoring can be used to determine the count of passengers and baggage or other objects more quickly and / or more accurately than current technologies used by airlines or other personnel to count passengers and baggage.

[0139] Furthermore, illustrative examples can provide characteristics that can identify restricted items. This identification of restricted items can be performed in addition to the screening currently being conducted at various locations within the airport.

[0140] The descriptions of different illustrative embodiments are presented for purposes of illustration and description and are not intended to be exhaustive or limited to embodiments of the disclosed forms. Different illustrative examples describe components that perform actions or operations. In the illustrative embodiments, components may be configured to perform the described actions or operations. For example, a component may have a configuration or design for a structure that provides the component with the ability to perform the actions or operations described in the illustrative examples as being performed by the component. Furthermore, with regard to the terms “comprising,” “including,” “having,” “containing,” and variations thereof as used herein, such terms are intended to encompass in a manner similar to the term “comprising” as an open transitional term, without excluding any additional or other elements.

[0141] Furthermore, this disclosure includes implementation methods according to the following terms:

[0142] Clause 1. A method for monitoring traffic of an aircraft (206), the method comprising:

[0143] Images (224) are received by a computer system (216) from a sensor system (220) positioned as a monitor (204) moving (212) relative to the aircraft (206);

[0144] The computer system (216) identifies a group of people (204) in the image (224);

[0145] Using the image (224), the computer system (216) determines the movement (212) of the group of people (204) relative to the aircraft (206);

[0146] The computer system (216) determines the count (226) of the people (204) in the aircraft (206) based on the movement (212) determined for the group of people (204); and

[0147] Based on the count (226) of the person (204) in the aircraft (206), a set of actions (228) are performed by the computer system (216).

[0148] Clause 2. The method described in Clause 1 further includes:

[0149] Using the image (224), the computer system (216) identifies a group of objects (214) associated with the group of people (204).

[0150] Clause 3. The method described in Clause 2 further includes:

[0151] The computer system (216) determines the count (226) of the objects (214) in the aircraft (206) based on the movement (212) determined for the group of objects (214).

[0152] Clause 4. The method according to Clause 3, wherein performing the set of actions (228) via the computer system (216) based on the movement (212) determined for the group of people (204) comprises:

[0153] The set of actions (228) are performed by the computer system (216) based on the movement (212) determined for at least one of the group of people (204) or the group of objects (214).

[0154] Clause 5. The method described in Clause 2 further includes:

[0155] The computer system (216) determines whether the restricted item (230) exists in the group of objects (214).

[0156] Clause 6. The method according to Clause 5, wherein determining by the computer system (216) whether the restricted article (230) exists in the set of objects (214) includes:

[0157] The computer system (216) analyzes a set of images (224) generated by at least one of a panchromatic camera, a terahertz camera, a millimeter-wave scanner, an infrared (IR) camera, or an X-ray system.

[0158] Clause 7. The method described under any of the foregoing clauses further includes:

[0159] Based on at least one of motion type, object features, or template comparison identified for objects (214) in the image (224), the computer system (216) determines a set of identities (234) of the group of people (204) in the image (224).

[0160] Clause 8. The method according to any of the preceding clauses, wherein identifying the group of people (204) in the image (224) by the computer system (216) comprises:

[0161] The artificial intelligence system (232) is used to identify the group of people (204) in the image (224) through the computer system (216).

[0162] Clause 9. The method according to any of the preceding clauses, wherein identifying the group of people (204) in the image (224) by the computer system (216) comprises:

[0163] Identify the number of passengers from the group of people (204).

[0164] Clause 10. The method according to any of the preceding clauses, wherein identifying the group of people (204) in the image (224) by the computer system (216) comprises:

[0165] Determine a set of identities (234) for the group of people (204).

[0166] Clause 11. The method according to any of the preceding clauses, wherein determining the count (226) of the persons (204) in the aircraft (206) via the computer system (216) based on the movement (212) determined for the group of persons (204) comprises:

[0167] After boarding the aircraft (206), disembarking from the aircraft (206), closing the aircraft door after disembarking, leaving the gate, partially disembarking from the aircraft (206), taking off, or landing at least one of these actions, the number of people (204) in the aircraft (206) is determined by the computer system (216) based on the movement (212) determined for the group of people (204).

[0168] Clause 12. The method according to Clause 3, wherein determining how many of the objects (214) the aircraft (206) has based on the movement (212) determined for the set of objects (214) via the computer system (216) comprises:

[0169] After boarding the aircraft (206), disembarking from the aircraft (206), closing the aircraft door after disembarking, leaving the gate, partially disembarking from the aircraft (206), taking off, or landing, the number of the objects (214) in the aircraft (206) is determined by the computer system (216) based on the movement (212) determined for the group of objects (214).

[0170] Clause 13. The method according to any of the preceding clauses, wherein the sensor system (220) comprises a sensor (222) located in at least one of the following: cabin, aircraft door, bulkhead, sidewall of the aircraft (206), runway, floor of the aircraft (206), galley, lavatory, passenger comfort unit, passenger bridge door, cargo hold, airport gate, jet bridge or passenger bridge.

[0171] Clause 14. The method described under any of the preceding clauses, wherein the set of actions (228) is selected from at least one of the following: sending a message, generating an alarm, creating a log entry with the set of people (204), indicating readiness for takeoff, indicating not readiness for takeoff, indicating a missing passenger, indicating the presence of a passenger in the aircraft (206), indicating the presence of an object left behind by the passenger, indicating that the passenger present on the aircraft (206) is inconsistent with the passenger log of the aircraft (206), indicating the presence of an unidentified object on the aircraft (206), determining whether to restrict the flight of a person on the aircraft (206), indicating the presence of an animal in the aircraft (206), indicating when the number of baggage pieces exceeds the storage capacity of the overhead bins and under-seat storage in the cabin, alerting the pilot, alerting the cabin crew, alerting the gate service personnel, preventing the aircraft engine from starting, preventing the aircraft door of the aircraft (206) from closing, instructing the crew of the aircraft (206) to leave the object in the cargo hold or to cross-check the passenger count during an emergency evacuation of the aircraft (206).

[0172] Clause 15. The method according to Clause 2, wherein the group of objects (214) is selected from at least one of the following: luggage, backpack, rolling suitcase, bag, box, bottle, handbag, briefcase, duffel bag, wallet, belt or animal.

[0173] Clause 16. The method according to any of the preceding clauses, wherein the sensor system (220) comprises at least one of the following: a visible light camera, an infrared sensor (IR), a panchromatic camera, a terahertz camera, an X-ray system, a thermal infrared (IR) sensor, or a light detection and ranging (LiDAR) system.

[0174] Clause 17. A method for monitoring a platform (202), the method comprising:

[0175] Images (224) are received by a computer system (216) from a sensor system (220) positioned as a monitor (204) moving (212) relative to the platform (202);

[0176] The computer system (216) identifies a group of people (204) in the image (224);

[0177] Using the image (224), the computer system (216) determines the movement (212) of the group of people (204) relative to the platform (202);

[0178] Based on the movement (212) determined for the group of people (204), the computer system (216) determines the count (226) of the people (204) on the platform (202); and

[0179] The count (226) of the person (204) on the platform (202) is used to perform a set of actions (228) through the computer system (216).

[0180] Clause 18. The method described pursuant to Clause 17 further includes:

[0181] Using the image (224), the computer system (216) identifies a group of objects (214) associated with the group of people (204).

[0182] Clause 19. The method described pursuant to Clause 18 further includes:

[0183] The computer system (216) determines the count (226) of the objects (214) on the platform (202) based on the movement (212) determined for the group of objects (214).

[0184] Clause 20. The method according to Clause 19, wherein performing the set of actions (228) via the computer system (216) based on the movement (212) determined for the group of people (204) comprises:

[0185] The set of actions (228) are performed by the computer system (216) based on the movement (212) determined for at least one of the group of people (204) or the group of objects (214).

[0186] Clause 21. The method described pursuant to Clause 18 further includes:

[0187] The computer system (216) determines whether the restricted item (230) exists in the group of objects (214).

[0188] Clause 22. The method according to Clause 21, wherein determining by the computer system (216) whether the restricted article (230) exists in the set of objects (214) includes:

[0189] The computer system (216) analyzes a set of images (224) generated by at least one of a panchromatic camera, a terahertz camera, a millimeter-wave scanner, an infrared camera, or an X-ray system.

[0190] Clause 23. The method of any one of Clauses 17-22, wherein the platform (202) is selected from: mobile platform, fixed platform, land-based structure, water-based structure, space-based structure, aircraft, airplane, commercial aircraft, commercial aircraft, rotorcraft, tiltrotor aircraft, tilt-wing aircraft, vertical takeoff and landing aircraft, surface ships, tanks, personnel carriers, trains, spacecraft, space stations, satellites, submarines, automobiles, power plants, bridges, dams, houses, manufacturing facilities and buildings.

[0191] Clause 24. An aircraft monitoring system (210) comprising:

[0192] Computer system (216); and

[0193] The controller (218) in the computer system (216), wherein the controller (218) is configured to:

[0194] Images (224) are received from a sensor system (220) positioned to monitor the movement (212) of a person (204) relative to an aircraft (206);

[0195] Identify a group of people (204) in the image (224);

[0196] The image (224) is used to determine the movement (212) of the group of people (204) relative to the aircraft (206);

[0197] The count (226) of the persons (204) in the aircraft (206) is determined based on the movement (212) determined for the group of persons (204); and

[0198] A set of actions (228) are performed based on the count (226) of the person (204) in the aircraft (206).

[0199] Clause 25. The aircraft monitoring system (210) as described in Clause 24 further includes:

[0200] Using the image (224), the computer system (216) identifies a group of objects (214) associated with the group of people (204).

[0201] Clause 26. The aircraft monitoring system (210) described in Clause 25 further includes:

[0202] The count (226) of the objects (214) in the aircraft (206) is determined by the computer system (216) based on the movement (212) determined for the group of objects (214).

[0203] Clause 27. The aircraft monitoring system (210) according to Clause 26, wherein performing the set of actions (228) via the computer system (216) based on the movement (212) determined for the set of persons (204) includes:

[0204] The set of actions (228) are performed by the computer system (216) based on the movement (212) determined for at least one of the group of people (204) or the group of objects (214).

[0205] Clause 28. The aircraft monitoring system (210) described in Clause 25 further includes:

[0206] The computer system (216) determines whether the restricted item (230) exists in the group of objects (214).

[0207] Clause 29. The aircraft monitoring system (210) pursuant to Clause 28, wherein determining by the computer system (216) whether the restricted article (230) exists in the group of objects (214) includes:

[0208] The computer system (216) analyzes a set of images (224) generated by at least one of a panchromatic camera, a terahertz camera, a millimeter-wave scanner, an infrared (IR) camera, or an X-ray system.

[0209] Clause 30. The aircraft monitoring system (210) pursuant to any one of Clauses 24-29 further comprises:

[0210] Based on the motion type, object features, or template comparison identified for objects (214) in the image (224), the computer system (216) determines a set of identities (234) of the group of people (204) in the image (224).

[0211] Clause 31. An aircraft monitoring system (210) according to any one of Clauses 24-30, wherein identifying the group of people (204) in the image (224) by the computer system (216) comprises:

[0212] The artificial intelligence system (232) is used to identify the group of people (204) in the image (224) through the computer system (216).

[0213] Clause 32. An aircraft monitoring system (210) according to any one of Clauses 24-31, wherein identifying the group of people (204) in the image (224) by the computer system (216) comprises:

[0214] Identify the number of passengers from the group of people (204).

[0215] Clause 33. An aircraft monitoring system (210) according to any one of Clauses 24-32, wherein identifying the group of people (204) in the image (224) by the computer system (216) comprises:

[0216] Determine a set of identities (234) for the group of people (204).

[0217] Clause 34. An aircraft monitoring system (210) according to any one of Clauses 24-33, wherein determining the count (226) of the persons (204) in the aircraft (206) via the computer system (216) based on the movement (212) determined for the group of persons (204) comprises:

[0218] After boarding the aircraft (206), disembarking from the aircraft (206), closing the aircraft door after disembarking, leaving the gate, partially disembarking from the aircraft (206), taking off, or landing at least one of these actions, the count (226) of the persons (204) in the aircraft (206) is determined by the computer system (216) based on the movement (212) determined for the group of persons (204).

[0219] Clause 35. The aircraft monitoring system (210) according to Clause 26, wherein determining by the computer system (216) how many of the objects (214) are in the aircraft (206) based on the movement (212) determined for the group of objects (214) includes:

[0220] After boarding the aircraft (206), disembarking from the aircraft (206), closing the aircraft door after disembarking, leaving the gate, partially disembarking from the aircraft (206), taking off, or landing at least one of these actions, the computer system (216) determines how many of the objects (214) are in the aircraft (206) based on the movement (212) determined for the group of objects (214).

[0221] Clause 36. An aircraft monitoring system (210) according to any one of Clauses 24-35, wherein the sensor system (220) comprises a sensor (222) located in at least one of: a cabin, an aircraft door, a bulkhead, a side wall of the aircraft (206), an aircraft runway, the floor of the aircraft (206), a galley, a lavatory, a passenger comfort unit, a passenger boarding bridge door, an airport gate, a jet bridge, or a passenger boarding bridge.

[0222] Clause 37. An aircraft monitoring system (210) according to any one of Clauses 34-36, wherein the set of actions (228) is selected from at least one of: sending information, generating an alarm, creating a log entry with the set of people (204), indicating readiness for takeoff, indicating not ready for takeoff, indicating a missing passenger, indicating the presence of a passenger in the aircraft (206), indicating the presence of an object left behind by the passenger, indicating that the passenger present on the aircraft (206) is inconsistent with the passenger log of the aircraft (206), indicating that the aircraft (228) is not ready for takeoff, ... 06) The presence of an unidentified object, determining whether to restrict the flight of a person on the aircraft (206), indicating the presence of an animal on the aircraft (206), indicating when the number of baggage pieces exceeds the storage capacity of the overhead bins and under-seat storage in the cabin, alerting the pilot, alerting the cabin crew, alerting the gate service personnel, preventing the aircraft engine from starting, preventing the aircraft doors of the aircraft (206) from closing, instructing aircraft staff to leave the object in the cargo hold, or to conduct a cross-check of passenger counts during an emergency evacuation of the aircraft (206).

[0223] Clause 38. The aircraft monitoring system (210) pursuant to Clause 25, wherein the group of objects (214) is selected from at least one of the following: baggage, backpack, rolling suitcase, bag, box, bottle, handbag, briefcase, duffel bag, wallet, belt or animal.

[0224] Clause 39. An aircraft monitoring system (210) according to any one of Clauses 24-38, wherein the sensor system (220) comprises at least one of: a visible light camera, an infrared (IR) sensor, a panchromatic camera, a terahertz camera, an X-ray system, a thermal infrared (IR) sensor, or a light detection and ranging (LiDAR) system.

[0225] Clause 40. A platform monitoring system (208) comprising:

[0226] Computer system (216); and

[0227] The controller (218) in the computer system (216), wherein the controller (218) is configured to:

[0228] Images (224) are received from a sensor system (220) positioned to monitor the movement (212) of a person (204) relative to the platform (202);

[0229] Identify a group of people (204) in the image (224);

[0230] The image (224) is used to determine the movement (212) of the group of people (204) relative to the platform (202);

[0231] The count (226) of the people (204) on the platform (202) is determined based on the movement (212) determined for the group of people (204); and

[0232] A set of actions (228) is performed based on the count (226) of the person (204) on the platform (202).

[0233] Clause 41. The platform monitoring system (208) as described in Clause 40 further includes:

[0234] Using the image (224), the computer system (216) identifies a group of objects (214) associated with the group of people (204).

[0235] Clause 42. The platform monitoring system (208) as described in Clause 41 further includes:

[0236] The count (226) of the objects (214) on the platform (202) is determined by the computer system (216) based on the movement (212) determined for the group of objects (214).

[0237] Clause 43. The platform monitoring system (208) according to Clause 42, wherein performing the set of actions (228) via the computer system (216) based on the movement (212) determined for the group of people (204) includes:

[0238] The set of actions (228) are performed by the computer system (216) based on the movement (212) determined for at least one of the group of people (204) or the group of objects (214).

[0239] Clause 44. The platform monitoring system (208) described in Clause 41 further includes:

[0240] The computer system (216) determines whether the restricted item (230) exists in the group of objects (214).

[0241] Clause 45. The platform monitoring system (208) according to Clause 44, wherein determining by the computer system (216) whether the restricted item (230) exists in the group of objects (214) includes:

[0242] The computer system (216) analyzes a set of images (224) generated by at least one of a panchromatic camera, a terahertz camera, a millimeter-wave scanner, an infrared (IR) camera, or an X-ray system.

[0243] Clause 46. The platform monitoring system (208) according to any one of Clauses 40-45, wherein the platform (202) is selected from: mobile platforms, fixed platforms, land-based structures, water-based structures, space-based structures, aircraft (206), airplanes, commercial aircraft, commercial aircraft, rotorcraft, tiltrotor aircraft, tilt-wing aircraft, vertical takeoff and landing aircraft, surface ships, tanks, personnel carriers, trains, spacecraft, space stations, satellites, submarines, automobiles, power plants, bridges, dams, houses, manufacturing facilities and buildings.

[0244] Clause 47. A computer program product (822) for monitoring an aircraft (206), said computer program product (822) comprising:

[0245] Computer-readable storage medium (824);

[0246] First program code (818), stored on the computer-readable storage medium (824), is executable by the computer system (216) to cause the computer system (216) to receive images (224) from a sensor system (220) positioned as a monitor (204) moving (212) relative to the aircraft (206);

[0247] A second program code (818) is stored on the computer-readable storage medium (824) and is executable by the computer system (216) to enable the computer system (216) to identify a group of people (204) in the image (224);

[0248] A third program code (818), stored on the computer-readable storage medium (824), is executable by the computer system (216) to cause the computer system (216) to use the image (224) to determine the movement (212) of the group of people (204) relative to the aircraft (206);

[0249] A fourth program code (818), stored on the computer-readable storage medium (824), is executable by the computer system (216) to cause the computer system (216) to determine (226) a count (226) of the persons (204) in the aircraft (206) based on the movement (212) determined for the group of persons (204); and

[0250] A fifth program code (818), stored on the computer-readable storage medium (824), is executable by the computer system (216) to cause the computer system (216) to perform a set of actions (228) based on the count (226) of the person (204) in the aircraft (206).

[0251] Many modifications and variations will be apparent to those skilled in the art. Furthermore, different illustrative embodiments may provide different features compared to other desired embodiments. The selection and description of one or more embodiments are intended to best explain the principles and practical applications of the embodiments and to enable others skilled in the art to understand the disclosure of various embodiments with various modifications suitable for the particular purpose considered.

Claims

1. A method for monitoring traffic of an aircraft (206), the method comprising: Images (224) are received by a computer system (216) from a sensor system (220) positioned as a monitor (204) moving (212) relative to the aircraft (206); The computer system (216) identifies a group of people (204) in the image (224); Using the image (224), the computer system (216) determines the movement (212) of the group of people (204) relative to the aircraft (206); Based on the movement (212) determined for the group of people (204), the computer system (216) determines the count (226) of the people (204) in the aircraft (206); and Based on the count (226) of the persons (204) in the aircraft (206), a set of actions (228) is performed by the computer system (216), wherein the set of actions includes determining the identity of a specific person among the group of persons identified in the image, wherein the specific person is identified as a passenger. The set of actions further includes instructing the passenger to disembark from the aircraft.

2. The method according to claim 1, further comprising: Using the image (224), the computer system (216) identifies a group of objects (214) associated with the group of people (204); or optionally... The computer system (216) determines the count (226) of the group of objects (214) in the aircraft (206) based on the movement (212) determined for the group of objects (214).

3. The method of claim 2, wherein performing the set of actions (228) via the computer system (216) based on the movement (212) determined for the group of people (204) comprises: The set of actions (228) are performed by the computer system (216) based on the movement (212) determined for at least one of the group of people (204) or the group of objects (214).

4. The method of claim 2, further comprising: The computer system (216) determines whether the restricted item (230) exists in the group of objects (214).

5. The method of claim 4, wherein determining by the computer system (216) whether the restricted article (230) exists in the group of objects (214) comprises: The computer system (216) analyzes a set of images (224) generated by at least one of a panchromatic camera, a terahertz camera, a millimeter-wave scanner, an infrared (IR) camera, or an X-ray system.

6. The method according to any of the preceding claims, further comprising: Based on at least one of motion type, object features, or template comparison identified for objects (214) in the image (224), the computer system (216) determines a set of identities (234) of the group of people (204) in the image (224).

7. The method of any one of claims 1-5, wherein identifying the group of people (204) in the image (224) by the computer system (216) comprises: The artificial intelligence system (232) is used to identify the group of people (204) in the image (224) through the computer system (216); The number of passengers is identified from the group of people (204); or Determine a set of identities (234) for the group of people (204).

8. The method according to any one of claims 1-5, wherein determining the count (226) of the persons (204) in the aircraft (206) via the computer system (216) based on the movement (212) determined for the group of persons (204) comprises: After boarding the aircraft (206), disembarking from the aircraft (206), closing the aircraft door after disembarking, leaving the gate, partially disembarking from the aircraft (206), taking off, or landing, the number of the people (204) on the aircraft (206) is determined by the computer system (216) based on the movement (212) determined for the group of people (204).

9. The method of claim 2, wherein determining how many of the objects (214) are in the aircraft (206) via the computer system (216) based on the movement (212) determined for the group of objects (214) comprises: After boarding the aircraft (206), disembarking from the aircraft (206), closing the aircraft door after disembarking, leaving the gate, partially disembarking from the aircraft (206), taking off, or landing, the number of the objects (214) in the aircraft (206) is determined by the computer system (216) based on the movement (212) determined for the group of objects (214).

10. The method of any one of claims 1-5, wherein the sensor system (220) comprises a sensor (222) located in at least one of the following: a cabin, an aircraft door, a bulkhead, a side wall of the aircraft (206), an aircraft runway, the floor of the aircraft (206), a galley, a lavatory, a passenger comfort unit, a passenger boarding bridge door, a cargo hold, an airport gate, a jet bridge, or a passenger boarding bridge.

11. The method according to any one of claims 1-5, wherein the set of actions (228) further comprises at least one of the following: Send message, Generate an alert. Log entries were created using the aforementioned group of people (204). Instructions to prepare for takeoff Instructions indicated that the aircraft was not ready for takeoff. Indicates the presence of an object left behind by the specific person. The indication is that the specific person present as a passenger on the aircraft (206) is inconsistent with the passenger log of the aircraft (206). Indicates the presence of an unidentified object on the aircraft (206). To determine whether the specific person is restricted from flying on the aircraft (206), The presence of animals is indicated in the aircraft (206). Indicates when the number of luggage items exceeds the storage capacity of the overhead bins and under-seat storage in the cabin. Remind the pilots, Remind flight attendants, Remind the gate staff, To prevent aircraft engines from starting, To prevent the aircraft doors of the aircraft (206) from closing, The aircraft (206) crew indicated that the object was left in the cargo hold. This indicates that the passenger should have boarded the aircraft but has not yet boarded the aircraft. The instruction indicates that the passenger should have boarded the aircraft and has boarded the aircraft, or Cross-checking of passenger counts is performed during an emergency evacuation of the aircraft (206).

12. The method of claim 2, wherein the group of objects (214) is selected from at least one of the following: luggage, backpack, rolling suitcase, bag, box, bottle, handbag, briefcase, duffel bag, wallet, belt or animal.

13. The method according to any one of claims 1-5, wherein the sensor system (220) comprises at least one of the following: a visible light camera, an infrared sensor (IR), a panchromatic camera, a terahertz camera, an X-ray system, a thermal infrared (IR) sensor, or a light detection and ranging (LiDAR) system.

14. An aircraft monitoring system (210), comprising: Computer system (216); and The computer system (216) includes a controller (218) configured to perform the method according to any one of claims 1-13.

15. A computer program product (822) for monitoring an aircraft (206), said computer program product (822) comprising: Computer-readable storage medium (824); and Program code (818), stored on the computer-readable storage medium (824), is executable by the computer system (216) to cause the computer system (216) to perform the method of any one of claims 1-13.

Images