System and method for monitoring the flight of an aircraft with a personal item

Abstract

A system and method of monitoring carry-on items of a flight of an aircraft. The system includes one or more cameras positioned to capture images of carry-on items associated with passengers of the aircraft. The system includes an interface configured to receive image data from the one or more cameras. The system further includes one or more processors coupled to the interface. The one or more processors are configured to analyze the image data to determine characteristics of the carry-on items. The one or more processors are configured to determine that one or more alert conditions are satisfied based on a comparison of the characteristics of the carry-on items to one or more alert criteria and data associated with the aircraft. The one or more processors are further configured to send an output to one or more devices based on the one or more alert conditions being satisfied.

Description

Technical Field

[0001] This disclosure relates in general to monitoring the storage of personal belongings in the storage compartments of aircraft. Background Technology

[0002] Many airlines allow passengers to carry carry-on items that can be fitted into the aircraft's storage compartments (e.g., overhead bins) and one personal item (e.g., wallet, briefcase, lap bag, etc.) that can be fitted under the seat in front of the passenger. Airlines may have size restrictions on carry-on items to allow for proper operation of the storage compartments and to allow each passenger to carry carry-on items on board. For example, some airlines have a 22-inch by 14-inch by 9-inch size limit for carry-on items. Other airlines may have different size restrictions. One or more passengers carrying oversized carry-on items, improperly placing carry-on items in storage compartments, placing personal items in storage compartments that can hold carry-on items, or a combination thereof, may cause aisle congestion and boarding delays during the boarding process. Storage space in the storage compartments often fills up during the flight, resulting in the need to mark and store one or more carry-on items associated with some passengers on board as checked baggage. Summary of the Invention

[0003] In a specific implementation, a system for monitoring carry-on items carried by an aircraft includes: one or more cameras positioned to capture images of carry-on items associated with passengers of the aircraft. The system includes: an interface configured to receive image data from the one or more cameras. The system also includes: one or more processors coupled to the interface. The one or more processors are configured to analyze the image data to determine the characteristics of the carry-on items. The one or more processors are configured to determine whether one or more alarm conditions are met based on a comparison of one or more alarm criteria with the characteristics of the carry-on items and data associated with the aircraft. The one or more processors are further configured to send output to one or more devices based on the satisfaction of the one or more alarm conditions.

[0004] In a specific implementation, a method for monitoring carry-on items in flight includes: receiving image data from a camera on the aircraft at a monitoring device. At least a portion of the camera is positioned to capture images of the interior of storage compartments on the aircraft. The method includes: analyzing the image data at the monitoring device to determine first information. The first information includes characteristics of items in each storage compartment and available storage space in each storage compartment. The method includes: determining, at the monitoring device, whether one or more alarm conditions are met based on a comparison of one or more alarm criteria with the first information. The method further includes: sending an output from the monitoring device to one or more devices based on the satisfaction of one or more alarm conditions.

[0005] In a specific implementation, a method for monitoring carry-on baggage during an aircraft flight includes: receiving image data from one or more cameras in the boarding area of ​​the aircraft at a monitoring device. The one or more cameras are positioned to capture images of carry-on baggage associated with passengers preparing to board the aircraft. The method includes: analyzing the image data at the monitoring device to determine the characteristics of the carry-on baggage associated with the passenger. The method further includes: determining, at the monitoring device, that one or more alarm conditions are met based on a comparison of one or more alarm criteria with the characteristics of the carry-on baggage and flight-related data. The method also includes: sending an output to one or more devices based on the satisfaction of one or more alarm conditions.

[0006] The features, functions, and advantages described herein can be implemented independently in various implementations or in combination with other implementations, and further details can be found by referring to the following description and figures. Attached Figure Description

[0007] Figure 1 This is a diagram showing a system of personal belongings configured to monitor the flight of an aircraft.

[0008] Figure 2 This is a flowchart illustrating a first method for monitoring the flight of an aircraft using personal belongings.

[0009] Figure 3 This is a flowchart illustrating a method for monitoring storage compartments on an aircraft.

[0010] Figure 4 This is a flowchart illustrating a second method for monitoring the flight of an aircraft using personal items.

[0011] Figure 5 This is a flowchart illustrating the lifecycle of an aircraft configured to monitor the storage of personal belongings in the aircraft's storage compartments.

[0012] Figure 6This is an illustration of an aircraft configured to monitor the storage of personal belongings in the aircraft's storage compartments.

[0013] Figure 7 This is a block diagram of a computing environment according to the present disclosure, including aspects of a computing device configured to support computer-implemented methods and computer-executing program instructions (or code). Detailed Implementation

[0014] The aspects disclosed here present systems and methods for monitoring the storage of personal belongings in the storage compartments of aircraft.

[0015] Specifically, the monitoring equipment analyzes image data from cameras on or in the aircraft's boarding area to detect items associated with passengers (e.g., personal belongings and carry-on items). Based on the analysis of the image data, the monitoring equipment can determine whether an alarm condition is associated with one or more passengers. For example, the monitoring equipment can determine if a particular passenger is carrying oversized carry-on items, carrying too many items, or both. When the monitoring equipment detects an alarm condition associated with a passenger, it alerts the boarding agent or crew. Based on the alert, the boarding agent or crew can resolve the alarm condition associated with the passenger.

[0016] In addition, the monitoring equipment tracks the number of carry-on items allowed on the aircraft. Based on the aircraft's locker capacity and the number of carry-on items permitted on board, the monitoring equipment can determine when to stop allowing additional carry-on items and can send alerts to check-in agents or crew members, indicating that the additional carry-on items should be marked and stored as checked baggage, or indicating that other measures should be taken to determine how many passengers are allowed to bring their carry-on items onto the aircraft.

[0017] Specifically, the monitoring device analyzes image data from cameras on the aircraft. At least a portion of the camera is positioned to capture images of the interior of the aircraft's storage compartments. Based on the analysis of the image data, the monitoring device determines the status associated with each storage compartment. For example, based on the image data of the storage compartments, the monitoring device can determine whether one or more additional items are available in the storage compartment, whether the storage compartment is full, whether additional space has been created in the storage compartment by moving one or more items or by changing the orientation of one or more items, whether an item is stored in a prohibited location, and whether one or more items prevent the storage compartment from being closed. For each storage compartment, the monitoring device sends a signal to a status indicator device associated with the storage compartment, causing the status indicator device to display a specific status indicator (e.g., a color code) to indicate the status associated with the storage compartment during the boarding process. Furthermore, the monitoring device can send information about the status of each storage compartment to portable devices associated with the aircraft's crew. The crew can then resolve any issues detected by the monitoring device.

[0018] In addition, the monitoring device can send information about the remaining storage space in the lockers to a device associated with the boarding agent. When there is no available space for carry-on items, the information sent to the boarding agent allows the agent to warn remaining passengers that one or more passengers may have to mark and store their carry-on items as checked baggage. The information can also instruct the boarding agent when to stop allowing additional carry-on items onto the aircraft.

[0019] In another specific aspect, the monitoring equipment receives image data from a first set of cameras in the boarding area and a second set of cameras on the aircraft. When one or more alarm conditions are determined to be associated with items in the boarding area, the monitoring equipment sends output to one or more devices associated with the boarding agent, enabling the boarding agent to resolve the alarm conditions. The monitoring equipment also monitors the lockers in the aircraft based on image data from the second set of cameras and provides information about the status of each locker to locker status indicator devices and devices associated with the aircraft crew.

[0020] The system can store image data from the boarding area, information derived from that image data, image data from the aircraft, and information derived from that image data in a database. If a passenger issue arises before takeoff (e.g., a passenger disembarks before takeoff), or if a passenger-related item issue occurs (e.g., carry-on items are left on the aircraft after arrival at the destination), the information in the database can be accessed and analyzed to resolve the problem. For example, if a passenger disembarks before takeoff, the database information can be accessed to ensure that passenger-related items are removed from the aircraft before takeoff.

[0021] The figures and the following description illustrate specific exemplary embodiments. It should be understood that, although not explicitly described or shown herein, those skilled in the art will be able to conceive of various arrangements encompassing the principles described herein and the technical solutions following this description. Furthermore, any embodiments described herein are intended to aid in understanding the principles of this disclosure and should not be considered as unrestricted. Therefore, this disclosure is not limited to the specific embodiments or examples described below, but is limited by the technical solutions and their equivalents.

[0022] Specific implementations are described herein with reference to the accompanying drawings. Throughout the description, common reference numerals denote common features. As used herein, various terms are used only to describe specific implementations and are not intended to be limiting. For example, the singular forms “a,” “an,” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. Furthermore, in some implementations, some features described herein are singular, and in others, some features described herein are plural. For illustration, Figure 1 It describes a system that includes one or more processors ( Figure 1 The monitor device 108 (of which "processor" 140) is indicated in some implementations to include a single processor 140, and in other implementations to include multiple processors 140. For ease of reference herein, this feature is generally described as "one or more" features and is subsequently referred to in the singular unless aspects relating to multiple features are described.

[0023] The terms “comprise,” “comprises,” and “comprising” are used interchangeably with “include,” “includes,” or “including.” Furthermore, the terms “wherein” and “where” are used interchangeably. As used herein, exemplary indications refer to embodiments, implementations, and / or aspects and should not be construed as limiting or indicating preferred or preferred implementations. As used herein, ordinal terms (e.g., “first,” “second,” “third,” etc.) used to modify elements, such as structures, components, operations, etc., do not themselves indicate any priority or order of an element relative to another element, but only distinguish an element from another element having the same name (but used in ordinal terms). As used herein, the term “group” refers to a grouping of one or more elements, and the term “multiple” refers to multiple elements.

[0024] As used herein, “generate,” “calculate,” “use,” “select,” “access,” and “determine” are interchangeable unless the context otherwise indicates. For example, a “generate,” “calculate,” or “determine” parameter (or signal) can refer to actively generating, calculating, or determining a parameter (or signal) or can refer to using, selecting, or accessing a parameter (or signal) such as one already generated by another component or device. As used herein, “coupled” can include “communication coupling,” “electrical coupling,” or “physical coupling” and can also (or alternatively) include any combination thereof. Two devices (or components) can be directly or indirectly coupled (e.g., communication coupling, electrical coupling, or physical coupling) via one or more other devices, components, wires, buses, networks (e.g., wired networks, wireless networks, or combinations thereof). As an illustrative, non-limiting embodiment, two electrically coupled devices (or components) can be included in the same or different devices and can be connected via electronics, one or more connectors, or inductive coupling. In some implementations, two devices (or components) that are communication-coupled (such as electrical communication) can directly or indirectly, such as via one or more wires, buses, networks, etc., send and receive electrical signals (digital or analog signals). As used herein, "direct coupling" is used to describe two devices coupled (e.g., communication coupling, electrical coupling, or physical coupling) without any intermediate components.

[0025] In the following description, various functions performed by specific components or modules are depicted. However, this distinction between components and modules is for illustrative purposes only. Alternatively, the functions described herein may be distinguished as being performed by a specific component or module among a plurality of components or modules. Moreover, alternatively, two or more components or modules described as separate entities may be integrated into a single component or module. Components or modules of an electronic device may be implemented using hardware (e.g., field-programmable gate array (FPGA) devices, application-specific integrated circuits (ASICs), digital signal processors (DSPs), controllers, etc.), software (e.g., instructions executed by a processor), or any combination thereof.

[0026] Figure 1 An embodiment of a system 100 configured to monitor the storage of carry-on items 102 in a locker 104 of an aircraft 106 is described. The system 100 includes a monitor device 108 communicatively coupled to one or more electronic devices in a boarding area 110 of the aircraft 106 and communicatively coupled to one or more electronic devices in the aircraft 106. The boarding area 110 may be part of an airport terminal. The monitor device 108 performs analysis of items associated with passengers of the aircraft 106 and provides output to one or more devices, or both, in the aircraft 106 based on the analysis of one or more devices in the boarding area 110.

[0027] In one implementation, one or more electronic devices in boarding area 110 include a boarding pass scanning station 112 with a monitor 114 visible to the boarding agent, a first set of cameras 116, a location marker 118, and a status indicator device 120. In other implementations, the one or more electronic devices include additional devices, different devices, fewer devices, or combinations thereof. As a passenger approaches and arrives at boarding pass scanning station 112, the first set of cameras 116 is positioned to capture images of the passenger and items associated with the passenger. In some implementations, the first set of cameras 116 includes one or more video cameras, one or more still image cameras, one or more radar cameras, other types of cameras, or combinations thereof.

[0028] Location markers 118 serve as calibration references. For example, location markers 118 are configured such that each location marker 118 is within the field of view of one or more cameras of the first group of cameras 116. Furthermore, the monitoring device 108 can know or access the location of the location markers 118. The images of specific location markers 118 in the image data 112 from the first group of cameras 116, combined with the location data, enable the monitoring device 108 to calculate the dimensions of items (e.g., carry-on items 102) in the boarding area 110 from the image data 122 and the location data.

[0029] Status indicator device 120 includes visual or auditory outputs (such as light, buzzer, speaker, or display) to alert the boarding agent processing passengers entering aircraft 106. Status indicator device 120 is activated to indicate the status of items associated with the passenger based on output 124 received from monitor device 108. In one implementation, when monitor device 108 determines that there is no problem with any item associated with the passenger, status indicator device 120 generates a first alarm (e.g., displaying a first color of light, emitting a first sound, or both) based on output 124; when monitor device 108 determines that there is a problem with at least one item associated with the passenger, it generates a second alarm (e.g., displaying a second color of light, emitting a second sound, or both); and when monitor device 108 cannot determine whether there is a problem with one or more items associated with the passenger, it generates a third alarm (e.g., displaying a third color of light, emitting a third sound, or both). In other implementations, the status indicator device 120 generates other types of alerts, such as text alerts or images (e.g., an image of a problematic personal item).

[0030] When status indicator device 120 indicates that there is no problem with an item associated with the passenger, the boarding agent processes the passenger's boarding pass at boarding pass scanning station 112 and allows the passenger to proceed to aircraft 106. When status indicator device 120 indicates that there is a problem with one or more items associated with the passenger, output 124 sent by monitoring device 108 includes a notification causing monitoring device 114 to display details associated with the problem and optional options for the boarding agent. The boarding agent can select specific optional options to instruct monitoring device 108 on how to resolve the problem. For example, monitoring device 108 can determine that two items are associated with a specific passenger and that both items are of a size that qualifies as carry-on item 102. Because only one carry-on item 102 is allowed per passenger on aircraft 106, monitoring device 108 provides output 124 that sends a signal causing status indicator device 120 to indicate a problem and sends a notification to monitoring device 114. The notification indicates that a specific passenger has two items that qualify as carry-on baggage 102 and provides a first optional option that allows the specific passenger to carry carry-on baggage 102 on the aircraft 106 and a second optional option that disallows the specific passenger from carrying carry-on baggage 102 on the aircraft.

[0031] When the status indicator device 120 indicates that the monitor device 108 cannot determine whether there is a problem with the carry-on item associated with the passenger, the monitor device 108 sends an output 124 to the monitor 114, including an optional option indicating the response the boarding agent can take (e.g., allowing the passenger to carry carry-on item 102 on aircraft 106 or disallowing the passenger from carrying carry-on item 102 on aircraft 106). The boarding agent can visually inspect the carry-on item associated with the passenger to determine if there is a problem and take appropriate action. The boarding agent can select an appropriate optional option and continue checking in the remaining passengers. For example, when the position of the carry-on item associated with a particular passenger relative to the particular passenger prevents the monitor device 108 from determining from the image data 122 whether the carry-on item is too large, the monitor device sends an output 124 that causes the status indicator device 120 to indicate that no decision has been made regarding the carry-on item 102 associated with the particular passenger and sends a notification to the monitor 114. The notification indicates that the monitoring device 108 cannot determine whether the carry-on item 102 is too large and includes a first optional option indicating that a passenger is allowed to carry the carry-on item 102 on the aircraft 106 and a second optional option indicating that a specific passenger is not allowed to carry the carry-on item 102 on the aircraft 106. When calculations performed by the monitoring device 108 (e.g., determining the available storage space in the locker 104) require the size of the carry-on item 102 associated with a specific passenger, the monitoring device 108 sets the undetermined size as the maximum permissible value for the carry-on item 102.

[0032] In some implementations, one or more electronic devices on the aircraft 106 include a second set of cameras 126, a case status indicator device 128 associated with each storage compartment 104 of the aircraft 106, and one or more portable devices 130 (e.g., tablets, smartphones, etc.) associated with the crew of the aircraft 106. Each portable device 130 includes an application configured to interface with the monitor device 108. In other implementations, one or more electronic devices in the aircraft 106 include additional devices, different devices, fewer devices, or combinations thereof. In some implementations, the second set of cameras 126 includes video cameras, still image cameras, radar cameras, other types of cameras, or combinations thereof. In some implementations, a data connection is physically connected to the aircraft 106 to power the second set of cameras 126 and the case status indicator device 128 and to enable communication between the monitor device 108, the second set of cameras 126, and the case status indicator device 128. In other implementations, the second set of cameras 126 and the container status indicator device 128 are powered from the power system of the aircraft 106, and communication between the second set of cameras 126 and the container status indicator device 128 and the monitor device 108 is provided wirelessly or via a physical data connection to the aircraft 106.

[0033] One or more cameras in the second group of cameras 126 are positioned to capture images of items stored in the storage compartments 104 of the aircraft 106 and to provide image data 132 to the monitoring device 108. For example, one or more cameras in the second group of cameras 126 are coupled to the compartment partitions, support structures, upper surfaces, storage compartment doors, other surfaces, or combinations thereof of the storage compartment 104. In some implementations, as a passenger stores their belongings and proceeds to their seat, one or more other cameras in the second group of cameras 126 are positioned to capture images of the passenger and the items associated with the passenger. In some implementations, a specific camera in the second group of cameras 126 is configured to capture images of the interior of the storage compartments 104 located in the aisle. The known dimensions of the storage compartments 104 and the known space reserved in specific storage compartments for safety equipment or other items facilitate the determination of available space and the orientation of items within the storage compartments 104. In some implementations, the locker 104 may include one or more reference markers located at known locations, may include at least two cameras for each locker 104 providing stereoscopic vision, or both, to determine the size of the items, the space in the locker 104, and the orientation of the items in the locker 104.

[0034] The monitoring device 108 determines the status of each locker 104 and sends an output 134, including a status signal, to the locker status indicator device 128. Each locker status indicator device 128 generates a visual display of the locker in response to the status signal from the output 134 of the monitoring device 108. The visual display indicates the status of the locker 104 associated with the locker status indicator device 128. For example, the visual display can include text (e.g., "full"), color codes (e.g., red indicating that locker 104 is full), or both.

[0035] Portable device 130 receives from monitor device 108 an output 134 including information indicating the status of storage compartments 104 in aircraft 106. This information enables portable device 130 to display visual information (e.g., text, graphics, and images) and generate audio information (e.g., specific sounds indicating a problem associated with storage compartment 104), or both. In some implementations, portable device 130 displays an illustration of a portion of the fuselage of aircraft 106 with storage compartments 104, and displays each storage compartment 104 in a color indicating the status associated with it. Crew members associated with portable device 130 can scroll, zoom in, zoom out, and select specific storage compartments. When a crew member selects a specific storage compartment, an image of that specific storage compartment, captured by a camera in a second set of cameras 126 displaying the interior of that particular storage compartment, is sent from monitor device 108 to portable device 130 for display.

[0036] The monitoring device 108 includes one or more interfaces 136, a memory 138, one or more processors 140, or a combination thereof. The one or more interfaces 136 enable communication between the monitoring device 108 and one or more electronic devices in the boarding area 110, one or more electronic devices on the aircraft 106, other electronic devices, or combinations thereof. The memory 138 includes a non-volatile computer-readable medium (e.g., a computer-readable storage device) storing instructions 142 executable by the processor 140. The instructions 142 are executed to initiate, execute, and control operations that facilitate monitoring of items associated with passengers on the aircraft 106, including carry-on items 102.

[0037] Storage 138 includes aircraft data 144. Aircraft data 144 includes compartment information about aircraft 106. In some implementations, aircraft data 144 includes data about different types of aircraft 106 or different internal configurations of aircraft. Compartment information includes the number of compartments, the total available space in each compartment, the size of each compartment, identifiers of specific compartments with space constraints, and the location of specific compartments within confined spaces, etc.

[0038] The memory includes alarm criteria 146. Alarm criteria 146 includes information used by processor 140 to determine whether one or more alarm conditions are met. Alarm criteria 146 includes thresholds for specific conditions. For example, alarm criteria 146 includes a per-passenger item count threshold indicating the maximum number of items (carry-on items) allowed on aircraft 106, a carry-on item count threshold indicating the number of carry-on items 102 allowed on aircraft 106 per flight, size thresholds for carry-on items 102 (e.g., length, width, and height), other thresholds associated with restrictions on passengers on aircraft 106, or combinations thereof.

[0039] The memory includes flight data 148. In some implementations, flight data 148 is stored in a database of monitor device 108 or in a database of one or more external devices that allow monitor device 108 to access flight data 148. Flight data 148 includes flight identifiers 150 (e.g., flight numbers) for each passenger permitted on aircraft 106, first image data 152, second image data 154, carry-on data 156, and passenger data 158. First image data 152 is a portion of image data 122 captured by one or more cameras in the first set of cameras 116 during the boarding process of aircraft 106. Second image data 154 is a portion of image data 132 from one or more cameras in the second set of cameras 126. Second image data 154 includes image data of passengers in the aircraft when passengers have loaded passenger-associated items and are seated, as well as images of the interior of each storage compartment 104 captured by one or more cameras in the second set of cameras 126. In some implementations, flight data 148 does not include the portions of first image data 152, second image data 154 corresponding to the images of passengers in the aircraft, portions of second image data 154 corresponding to the images of each storage compartment 104, or combinations thereof.

[0040] Carry-on data 156 includes accumulated information about carry-on items 102 on aircraft 106. In one implementation, carry-on data 156 indicates the total number of carry-on items on aircraft 106. In other implementations, carry-on data 156 refers to the sum of the total number of carry-on items and at least one dimension (e.g., height) of the carry-on items on the aircraft. Monitor device 108 can use the sum of the total number of carry-on items and at least one dimension to determine an estimate of the number of additional carry-on items that can be stored in locker 104. Monitor device 108 can provide the estimate of the number of additional carry-on items that can be stored in locker 114 to monitor 114 to notify the boarding agent of the available storage space for remaining carry-on items 102 on aircraft 106.

[0041] Passenger data 158 includes information about each passenger who has boarded aircraft 106. Passenger data 158 includes passenger identification information 160 and item information 162 about items associated with the passenger. In some implementations, passenger identification information 160 includes data included on a boarding pass scanned by boarding pass scanning station 112 (e.g., name and seat number), images of the passenger in image data 132, and images of items associated with the passenger, or both. For passengers located at or near boarding pass scanning station 112, item information 162 is determined from image data 122.

[0042] Item information 162 includes the number of items 164 associated with the passenger, including any carry-on item 102, the number of carry-on items 166, and carry-on characteristics 168 when carry-on items 102 are present (e.g., an estimate of the size of carry-on item 102). In some implementations, an item is considered a carry-on item 102 based on its size. For example, an item is considered a carry-on item 102 if it is larger than the size associated with personal items 178, such as being too large to fit in a seatback pocket in front of the passenger or under the seat in front of the passenger. In some implementations, each item carried by the passenger is considered a carry-on item 102. In some implementations, if the passenger has more than one item, the largest item is considered a carry-on item 102. In some implementations, alarm criteria 146 specifies a threshold used by the monitoring device 108 to characterize an item as either personal item 178 or carry-on item 102. The processor 140 in the monitoring device 108 uses the number of items 164, the number of personal items 166, and personal carry characteristics 168 to determine, through alarm criteria 146, whether one or more alarm conditions 170 exist for items associated with the passenger.

[0043] Processor 140 includes an item analyzer 172, a box analyzer 174, or both, that can be implemented at least partially by processor 140 executing instructions 142. Processor 140 can be implemented as a single processor or multiple processors, such as a multi-core configuration, a multi-processor configuration, a distributed computing configuration, a cloud computing configuration, or any combination thereof. In some implementations, one or more portions of item analyzer 172, box analyzer 174, or both are implemented by processor 140 using dedicated hardware, firmware, or a combination thereof.

[0044] The item analyzer 172 is configured to identify items associated with a passenger near or located at the boarding pass scanning station 112 based on image data 122 from the first set of cameras 116, determine whether the items include carry-on items 102, and determine one or more characteristics (e.g., length, width, and height) associated with the carry-on items 102. Carry-on items 102 may have different sizes and shapes, and may be located in different places relative to the passenger associated with them. For example, some carry-on items 102 may be carried by a trolley case pulled via an extendable / retractable handle or by another handle of the trolley case. As another embodiment, some carry-on items 102 may be a backpack on the passenger's back or a backpack, handbag, or other type of carrier carried by handles or straps. The item analyzer provides the status of the items associated with the passenger as an alarm condition 170. In some implementations, the item analyzer 172 includes machine learning models such as neural networks, random forests, gradient boosting decision trees, statistical models, or combinations thereof.

[0045] Based on image data 132 from a second set of cameras 126 positioned to capture images of the interior of storage compartments 104 within the capture aircraft 106, a box analyzer 174 is configured to determine the location of items within the storage compartments 104, identify the amount of available space in each storage compartment 104, and determine whether there is a storage problem in any given storage compartment 104. The box analyzer 174 provides the status of each storage compartment 104 as an alarm condition 170. In some implementations, the box analyzer 174 includes a machine learning module, such as a neural network, random forest, gradient boosting decision tree, statistical model, or a combination thereof.

[0046] During operation, the monitoring device 108 determines that boarding has commenced on the aircraft 106. This determination is based on factors such as activating the scanner at the boarding pass scanning station 112 and receiving input from the boarding agent. In response to the commencement of boarding, the first set of cameras 116 in the boarding area 110 is activated to capture images of the boarding area 110. Image data 122 from the first set of cameras 116 is provided to the monitoring device 108 for analysis by the item analyzer 172. Furthermore, in response to the commencement of boarding, the locker status indicator device 128 in the aircraft 106 and the cameras in the second set of cameras 126 are activated. In this implementation, the locker status indicator device 128 associated with a closed locker 104 and the cameras in the second set of cameras 126 remain deactivated until the corresponding locker 104 is opened.

[0047] As each passenger approaches or arrives at the boarding pass scanning station 112, the item analyzer 172 in the monitor device 108 analyzes the image data 122 from the first set of cameras 116 to determine whether one or more items are associated with the passenger and creates an entry for passenger data 158 associated with the passenger in the flight data 148. When one or more items are associated with a passenger, the monitor device 108 determines and saves item information 162, including carry-on characteristics 168 to determine whether one or more items are carry-on items 102. Carry-on characteristics 168 can include the size and other information of the carry-on item 102. For example, if the monitor device 108 determines that there is a security alarm regarding a specific carry-on item, or the color of one or more parts of the specific carry-on item, the additional information will include data indicating one or more colors of the carry-on item 102.

[0048] Based on carry-on characteristics 168, aircraft data 144, and alarm criteria 146, monitoring device 108 determines whether one or more alarm conditions 170 are met. If monitoring device 108 determines that one or more alarm conditions 170 are met, determines that specific information associated with carry-on item 102 (e.g., the size of carry-on item) cannot be determined from image data 122, or both, monitoring device 108 provides output 124 to one or more electronic devices in boarding area 110. Output 124 includes specific signals from the boarding agent via status indicator device 120 to monitor 114 indicating one or more alarm conditions 170, indicating what characteristics monitoring device 108 cannot determine, or both. When the message is associated with carry-on item 102, the message includes optional options associated with the boarding agent responding to the message (e.g., a first optional option instructing the boarding agent to allow carry-on item 102 on aircraft 106 and a second optional option instructing the boarding agent not to allow carry-on item 102 on aircraft 106). Selecting appropriate optional options allows the monitoring device 108 to maintain and save appropriate carry-on data 156. In response to specific signals, the boarding agent can visually inspect carry-on items 102 and process the passenger and carry-on items 102 accordingly (e.g., allowing carry-on items 102 on aircraft 106 or marking carry-on items 102 as checked baggage for storage). The boarding agent can select optional options that instruct the boarding agent how to respond to messages.

[0049] When the monitoring device 108 determines specific information associated with the carry-on item 102 and determines that the alarm condition 170 is in order to approve the passenger to board the aircraft 106, the monitoring device 108 provides a specific signal to the status indicator device 120 to notify the boarding agent that the boarding agent is able to process the passenger and allow the passenger to board the aircraft 106 with the carry-on item 102.

[0050] When a boarding agent processes a passenger (e.g., scans the passenger's boarding pass), passenger data 158 is stored by the monitoring device 108. Additionally, if a passenger is associated with carry-on items 102, the carry-on data 156 is updated.

[0051] Furthermore, during operation, the monitoring device 108 receives image data 132 from the second set of cameras 126 in the aircraft 106. The bin analyzer 174 in the monitoring device 108 processes the image data 132 from the second set of cameras 126 to determine information about the items stored in the bins 104 of the aircraft 106 and the available space in the bins 104. When the boarding process begins and the bin 104 is opened, the image data 132 includes data corresponding to the contents inside the opened bin 104. In some implementations, when the corresponding bin 104 is closed, no operating power is supplied to the cameras in the second set of cameras 126 that provide image data 132 of the contents of the bin 104.

[0052] Based on portions of image data 132 and aircraft data 144, the monitoring device determines first information 176 for each storage box 104 and stores it in memory 138. The first information 176 for each storage box 104 includes item characteristics of each item in the storage box 104 and data regarding the storage space in the storage box 104. Item characteristics of the items in the storage box 104 include the item's dimensions. The dimensions of the items in the storage box 104 are used to determine the type of item (e.g., personal belongings, clothing, or carry-on items, or other types of items), the item's orientation, remaining available storage space, and other information associated with the item. In some implementations, the space in the storage box 104 corresponds to a linear space along the length of the storage box 104 available for additional items in the storage box 104.

[0053] For each locker 104, the monitoring device 108 determines the status associated with the locker 104 based on information determined by the locker analyzer 174 from image data 132, aircraft data 144, and one or more thresholds from alarm criteria 146. A first status of the locker 104 indicates that there is space in the locker 104 for items. A second status indicates that there is an opportunity to free up space in the locker 104 by repositioning one or more items, relocating one or more items (e.g., placing items close together), or by the passenger associated with the items removing one or more personal items or clothing that could be stored in the seat pocket or floor from the locker 104. A third status indicates that the locker 104 is full. A fourth status indicates that there is a problem with the locker 104. When a passenger places items in the locker 104, the monitoring device 108 analyzes the image data 132 of the locker 104 and updates the status associated with the locker 104 when the status changes.

[0054] If the monitoring device 108 determines that for a period of time longer than a threshold time (e.g., one minute, two minutes, or some other time period), the placement of one or more items in the locker prevents the locker 104 from closing, or that a portion of one or more items (e.g., a strip) protrudes outside the locker 104, the monitoring device 108 changes the state of the locker 104 to a fourth state indicating a problem with the locker 104. The threshold time is the amount of time that provides passengers or crew with time to correct the problem and changes the state to the fourth state only when the locker 104 continues to have a problem.

[0055] The information determined by the monitoring device 108 includes characteristics of the items in the locker 104. These characteristics may include a first dimension corresponding to the longest length of the item in a direction parallel to the bottom of the locker 104 and a second dimension corresponding to the longest length of the item in a direction parallel to the side of the locker 104. The characteristics are compared with one or more thresholds from alarm criteria 146 to determine whether an item placed on the bottom of the locker 104 qualifies as a personal item 102, personal belongings, or another type of item. If the locker analyzer determines that the item placed on the bottom of the locker 104 is a personal item, the state associated with the locker 104 is changed to a second state indicating that additional space can be freed up in the locker 104. If sufficient space exists, additional space can be freed up by removing the personal item from the locker 104 or by storing the personal item on top of the personal item 102 in the locker 104.

[0056] The monitoring device 108 also compares a first dimension with a second dimension. When the first dimension is larger than the second dimension and the ratio of the first dimension to the second dimension is greater than a threshold, the state of the storage box 104 is changed to a second state indicating that additional space can be made in the storage box 104 by repositioning one or more items in the storage box 104. The ratio of the first dimension to the second dimension being greater than the threshold prevents state changes associated with the orientation of items having contours that cannot be changed upon repositioning (e.g., a nearly square or circular contour).

[0057] The monitoring device 108 determines the distance between items placed on the bottom of the storage box 104 and the available space in the storage box 104. When the sum of the distances exceeds a threshold (e.g., the allowable width of personal items 102), the state associated with the storage box 104 is changed to a second state indicating that additional space can be made in the storage box 104. When the available space in the storage box 104 is less than the threshold (e.g., the allowable width of personal items), but does not indicate that items are improperly positioned or that personal items are present on the bottom of the storage box 104, the state is changed to a third state indicating that the storage box 104 is full.

[0058] The monitoring device 108 provides a first portion of output 134 as a signal to the locker status indicator device 128. The signal indicates the status of the locker 104 associated with the corresponding locker 104. In a specific implementation, the locker status indicator device 128 includes different colors of light (e.g., different colored LEDs) indicating the status associated with the locker 104. For example, green indicates the first status, blue indicates the second status, purple indicates the third status, and yellow indicates the fourth status. The locker status indicator device 128 displays a signal on the first portion of output 134 indicating which color controls each locker 104. In other implementations, different colors or different types of indicators (e.g., text displays) are used. The locker status indicator device 128 of the locker 104 is activated when the boarding process begins and image data 132 of the interior of the locker 104 is received by the monitoring device 108. The locker status indicator device 128 is deactivated when the boarding process ends (e.g., when the door of the aircraft 106 closes).

[0059] In addition to sending a first portion of output 134 to the locker status indicator device 128, the monitor device 108 sends a second portion of output 134 to one or more portable devices 130 associated with the crew of aircraft 106. The second portion of output 134 enables an application on the portable device 130 to display a representation of the lockers 104 and the corresponding status of each locker 104. When the status of a particular locker 104 is changed to a fourth state, the particular locker 104 can be highlighted (e.g., visually distinguished on a display) via the application, and the application can play an audio notification indicating a problem with the particular locker 104.

[0060] In some implementations, the state of a specific locker 104 is set by allowing the crew to rewrite the state determined by the monitoring device 108 via input from the crew. For example, the monitoring device 108 determines that the state of the specific locker 104 is a second state indicating that space can be made in the specific locker 104. Although personal belongings are present in the locker, the crew can determine that removing personal belongings will not create enough space for another carry-on item 102. The crew can use the portable device 130 to rewrite the indicated third state and change the state to another state (e.g., a first state indicating available space or a fourth state indicating that the specific locker 104 is full, as appropriate).

[0061] In some implementations, the monitoring device 108 periodically stores images of the interior of each locker 104 into second image data 154 in memory 138 during the boarding process. If it is necessary to identify items associated with a passenger or to identify passengers associated with items, it is possible to examine the images of the interior of the locker 104 from the second image data 154 and the information from passenger data 158 in memory 138 (e.g., passenger name, seat, and images of items associated with the passenger from passenger identification information 160).

[0062] In some implementations, the first set of cameras 116 remains active until the last passenger is processed by the boarding agent and the last passenger leaves the boarding area 110. In some implementations, cameras in the second set of cameras 126, which capture images of the interior of the lockers, remain active during the boarding process until the corresponding locker 104 is closed, and the locker status indicator device 128 remains active until all lockers 104 are closed.

[0063] Figure 2 A method 200 for monitoring the flight of an aircraft using personal carry items is shown. In a particular implementation, by... Figure 1 The monitoring device 108 of the system 100 performs one or more operations in method 200.

[0064] Method 200 includes, in block diagram 202, receiving image data 122, 132 from one or more cameras. The one or more cameras may include a first set of cameras 116 in boarding area 110, a second set of cameras 126 in aircraft 106, or both. When a passenger approaches or arrives at a location where a boarding agent processes the passenger before allowing the passenger to proceed to aircraft 106 (e.g., where the boarding agent scans boarding passes, visually inspects the passenger and items associated with the passenger), one or more cameras in the first set of cameras 116 are positioned to capture images of the passenger and items associated with the passenger. One or more cameras in the first set of cameras 116 can be coupled to a ceiling, wall, counter, boarding pass scanning station 112, or other structure within boarding area 110.

[0065] The cameras in the second group of cameras 126 are positioned within the aircraft 106 to capture images of the interior of the open storage compartment 104. In one implementation, the cameras in the second group of cameras 126 are coupled to a compartment partition wall, and each camera is positioned to capture images of the interior of the storage compartment across an aisle. In other implementations, each storage compartment 104 includes a camera in the second group of cameras 126 coupled to a compartment partition wall, a top surface of the storage compartment, a door of the storage compartment, structural members of the storage compartment, other surface support surfaces, or combinations thereof, and the camera is positioned to capture images of the interior of the storage compartment.

[0066] Method 200 includes: in block diagram 204, analyzing image data 122, 132 captured by one or more cameras to detect features 168 of carry-on item 102. For example, one or more cameras in the first group of cameras 116 in the boarding area 110 of aircraft 106 are radar cameras. Image data 122 from the radar cameras is analyzed to determine the size (e.g., length, width, and height) of carry-on item 102. Alternatively or additionally, the area of ​​boarding area 110 imaged by one or more cameras in the first group of cameras 116 may include one or more reference markers in location markers 118. Images of the reference markers and carry-on item in image data 122 captured simultaneously or nearly simultaneously by two or more cameras located at different known locations are analyzed to determine the size of carry-on item 102.

[0067] As another embodiment for determining one or more characteristics of the carry-on item 102, when the carry-on item is positioned in the storage compartment 104 of the aircraft 106, one or more cameras in the second set of cameras 126 capture images of the carry-on item 102. The storage compartment 104 may include one or more location markers, radar cameras, or both for determining the size of items in the storage compartment and the spacing between items in the storage compartment 104. Image data 132 from one or more cameras is analyzed to determine the size of the carry-on item 102 (e.g., width, height, length, or a combination thereof). The size and other information determined from the image data 132 can be stored in first information 176 of the memory 138 of the monitoring device 108.

[0068] Method 200 includes, in block diagram 206, determining, based on a comparison of one or more alarm criteria 146 with characteristics 168 of carry-on items 102 and aircraft data 144 associated with aircraft 106, whether one or more alarm conditions 170 are met. Alarm criteria 146 include thresholds and conditions associated with items containing carry-on items 102, the location of items in storage spaces, lockers 104, and other conditions based on the items. Other alarm conditions include security concerns related to items associated with the passenger, security alerts related to items associated with the passenger, numerous items associated with the passenger, and other conditions.

[0069] Method 200 further includes, in block diagram 208, sending outputs 124, 134 based on one or more alarm conditions 170 satisfying one or more devices 114, 120, 128, 130. One or more devices are associated with a boarding agent (e.g., a computer device having a monitor 114 at boarding pass scanning station 112, a portable device associated with the boarding agent, a speaker, a status indicator device 120 visible to the boarding agent, boarding pass scanning station 112, etc.), a locker status indicator device 128 associated with locker 104, a computer device (e.g., portable device 130) associated with aircraft crew, other devices, or combinations thereof.

[0070] For example, when monitoring device 108 determines that a particular carry-on item is too large, monitoring device 108 activates status indicator device 120 and sends a message to monitor 114 associated with the boarding agent. The message notifies the boarding agent that the particular carry-on item is too large and provides selectable options. Selectable options may include a first option instructing the particular carry-on item to be marked as checked baggage for storage, a second option (e.g., a rejection option) instructing the boarding agent to allow the particular carry-on item to be carried on aircraft 106, other options, or combinations thereof. The boarding agent selects a particular selectable option via an input device to notify monitoring device 108 of the decision made regarding the particular carry-on item, causing monitoring device 108 to reset status indicator device 120, and allowing monitoring device 108 to store information associated with the particular carry-on item if it is allowed to be carried on aircraft 106 as carry-on item 102. Preventing oversized carry-on items from being brought onto the aircraft 106 can increase the speed of passenger boarding and prevent aisle congestion due to problems associated with placing oversized items in the storage compartments 104.

[0071] In another embodiment, security personnel or other airport staff can input an alert describing specific attributes of a located specific item into a computer system. Monitor device 108 accesses the alert, determines specific alert criteria associated with the alert, and saves the specific alert criteria along with alert criterion 146. Monitor device 108 determines, based on image data 122, 132, whether the item associated with the passenger has a high probability of being a specific item (e.g., 90% or greater than some other percentage). If monitor device 108 determines that a threshold probability is met, monitor device 108 sends a message to monitor 114 and a signal to status indicator device 120 to alert the boarding agent of a potential problem. The message describes the alert and provides the boarding agent with selectable response options. Selectable response options can include an option to notify security, an option to remove the alert based on the boarding agent's determination that the item associated with the passenger is not a specific item, other options, or combinations thereof.

[0072] In another embodiment, the monitoring device 108 keeps track of multiple carry-on items 102 permitted to pass through the boarding agent to the aircraft 106 and stores the information in carry-on data 156. When an alarm condition based on the carry-on data 156 indicates that there is insufficient available storage space in the locker 104 (e.g., less than a threshold amount of available storage space), a signal is sent to the boarding agent to indicate that the remaining carry-on items 102 of the passenger have not yet passed through, and the boarding agent should mark them as checked baggage for storage. In some implementations, based on image data 132 from the second set of cameras 126, the monitoring device is able to provide the monitor 114 with a count of the number of carry-on items 102 permitted to pass through the boarding agent, a count of carry-on items 102 in the locker 104, and an estimated count of the space available for carry-on items 102 in the locker 104 of the aircraft 106. If the estimated count of space available for carry-on items 102 begins to approach zero (e.g., becomes six or less), and more passengers remain on board compared to the estimated count of space, the boarding agent is able to communicate with the aircraft crew to determine whether additional carry-on space can be created by rearranging the carry-on items 102 already positioned in the storage compartment 104, and to inform passengers that they are waiting to board, and passengers may have to mark their carry-on items 102 as checked baggage for storage.

[0073] As another embodiment, when one or more personal items 102 are in an inappropriate position in the storage box 104, or when a smaller item (e.g., personal belongings, clothing, etc.) is placed in the storage box 104 that could replace the storage box 104 containing the personal item 102, the monitoring device 108 determines that alarm condition 170 is met. The monitoring device 108 sends a first portion of output 134 to change the box status indicator device 128 of the storage box 104 from a first color (e.g., green) indicating available storage space in the storage box 104 to a second color (e.g., blue) indicating that additional space can be made in the storage box 104, and sends a second portion of output 134 to the portable device 130 indicating that additional space can be made in the storage box 104. Based on the first or second part of output 134, improperly positioned carry-on items 102 can be repositioned to a proper location, and smaller items can be positioned on top of other carry-on items 102 or removed from storage compartment 104 to be placed in a seat pocket or on the floor via the passenger's seat associated with the smaller item. Proper positioning of carry-on items 102 and removal of smaller items from locations where carry-on items 102 can be placed into storage compartment 104 maximizes the number of carry-on items 102 that can be loaded into the storage compartment.

[0074] In another embodiment, when image data 132 indicates that the space between items (e.g., personal belongings 102, security equipment, other items, or combinations thereof) in locker 104 is less than a threshold (e.g., a height threshold for allowed personal belongings), monitoring device 108 determines that alarm condition 170 is met. In response, monitoring device 108 sends a first portion of output 134 to change the color of locker status indicator device 128 of locker 104 from indicating available space in locker 104 to another color (i.e., purple) indicating that locker 104 is full, and sends a second portion of output 134 as a message to portable device 130. Locker 104 can be closed in response to the change in locker status indicator device 128 or the message.

[0075] As another embodiment, when image data 132 indicates a problem associated with locker 104, monitoring device 108 determines that alarm condition 170 is met. The problem might be items in locker 104 preventing it from closing, items placed on or in front of safety equipment or other reserved areas of locker 104, something hanging outside locker 104, another problem, or a combination thereof. In response, monitoring device 108 sends a first portion of output 134 to change the color of locker status indicator device 128 to a color indicating a problem (e.g., yellow), and sends a second portion of output 134 as a message to portable device 130. In response, aircraft crew members are able to access locker 104 and resolve the problem.

[0076] Figure 3 A method 300 for monitoring the flight of an aircraft 106 using carry-on items 102 is shown. In a specific implementation, by... Figure 1 The monitoring device 108 of the system 100 performs one or more operations in method 300.

[0077] Method 300 includes, in block diagram 302, receiving image data 122 from one or more cameras 116 located in a boarding area 110 of aircraft 106. The one or more cameras 116 are positioned to capture images of items associated with a passenger preparing to board aircraft 106, wherein the items include carry-on items 102. The one or more cameras 116 in the boarding area 110 are coupled to a ceiling, wall, counter, boarding pass scanning station 112, other structures, or combinations thereof. In some implementations, the one or more cameras 116 include one or more digital video cameras configured to stream content to a monitoring device 108, one or more radar cameras, one or more digital cameras controlled by one or more timers to capture images at periodic intervals (e.g., once per second), or combinations thereof.

[0078] Method 300 includes, in block diagram 304, analyzing image data 122 captured by one or more cameras 116 to determine features 168 of carry-on item 102 associated with a passenger. Features 168 may include the size of the carry-on item 102 and other information. For example, one or more cameras 116 located in boarding area 110 of aircraft 106 are radar cameras. Image data from the radar cameras is used to determine the size of the carry-on item 102 (e.g., length, width, and height). Alternatively or additionally, the area of ​​boarding area 110 imaged by one or more cameras 116 may include one or more markers in location markers 118. Location markers 118 identify one or more markers and the location of one or more cameras 116. Images of the markers in image data 122 captured simultaneously or nearly simultaneously by two or more of the one or more cameras 116 are used to determine the size of the carry-on item 102.

[0079] When determining the size of carry-on item 102, specific features of carry-on item 102 can be ignored. For example, many carry-on items 102 are trolley cases with extendable / retractable handles. When determining the length of a trolley case with an extendable / retractable handle, the length of the extendable / retractable handle is ignored. Furthermore, when determining the length of carry-on item 102, if the initial determination is that the length exceeds a length threshold, an additional determination is performed to determine whether carry-on item 102 consists of two or more items placed on top of each other. For example, a passenger might place personal belongings (i.e., a laptop bag) on ​​top of a trolley case. The combined length of the trolley case and personal belongings might exceed the threshold length. The additional determination results in the length of carry-on item 102 being only the length of the trolley case.

[0080] At boarding pass scanning station 112, the boarding agent processes the passenger. Passenger processing may include scanning the passenger's boarding pass and observing the passenger and items associated with the passenger to determine whether something associated with the passenger should be resolved before allowing the passenger to board aircraft 106. Passenger-related data, image data 122 of passenger-related items, data about the aircraft's boarding process (e.g., video data), additional data, or combinations thereof may be associated with the flight and stored in the database of monitoring device 108 or one or more databases external to the monitoring device.

[0081] If a problem occurs, information in the flight database can be accessed. For example, if a person who boarded aircraft 106 subsequently disembarks before takeoff, information in the flight database can be accessed to ensure that items associated with that person are removed from aircraft 106. As another embodiment, if an item is left on aircraft 106 and is subsequently discovered after the passenger has disembarked, information in the flight database can be accessed to identify the appropriate passenger and facilitate the return of the item to the appropriate passenger.

[0082] Information about the flight is stored in the database for a specific period of time, and after that period of time has elapsed, the information is deleted or allowed to be overwritten. This specific period of time could be one hour, one day, one week, or another period of time, starting from the arrival of aircraft 106 at its destination. If necessary, and before the specific period of time expires, authorized personnel can prevent the deletion or overwriting of information about the flight in the database, or create a copy of the information in the database and store the copy in a location that has not been deleted after the specific period of time has elapsed, so that the information is available after the specific period of time.

[0083] Method 300 includes, in block diagram 306, determining, based on a comparison of one or more alarm criteria 146 with characteristics 168 of carry-on items 102 and aircraft data 144 associated with aircraft 106, that one or more alarm conditions 170 are met. Alarm criteria 146 include size restrictions on carry-on items 102 (e.g., length threshold, width threshold, and height threshold), a threshold number of items associated with a passenger, and other thresholds regarding other conditions. For example, alarm criteria 146 may include a threshold number of carry-on items permitted on aircraft 106. Regarding flight, the type of aircraft 106 and the available storage space in the overhead bins 104 of aircraft 106 are known. The threshold number may be a set value (i.e., 112 carry-on items 102), or it may be dynamically calculated based on the dimensions of carry-on items 102 already permitted on aircraft 106 and the total storage space of overhead bins 104. Alternatively or additionally, the threshold number may be based on the assessment of remaining available space by the monitoring device 108 based on image data 132 received from the camera 126 on the aircraft 106 (which provides specific image data of the interior of the storage box 104).

[0084] Method 300 further includes, in block diagram 308, sending output to at least one device based on the satisfaction of one or more alarm conditions 170. For example, when the satisfied alarm condition 170 indicates that the carry-on item 102 is too large, the passenger is associated with an item exceeding a threshold quantity, or both, the monitor device 108 sends a first portion of output 124 as an alarm to the status indicator device 120 and sends a second portion of output 124 as a message that can be displayed by the monitor 114. In response to the satisfaction of alarm condition 170 indicating that the carry-on item is too large, the boarding agent can determine whether to mark the oversized carry-on item as checked baggage for transport or whether to allow the passenger to bring the oversized carry-on item onto aircraft 106. The boarding agent can input data regarding the status of the oversized carry-on item to the monitor device 108 via an input device associated with the monitor 114. In response to an alarm condition 170 indicating an excess of items associated with a passenger, the boarding agent acknowledges the alarm condition and is able to reduce the number of items the passenger has by placing one or more items in another item (e.g., placing the first item in carry-on baggage 102), or the boarding agent is able to ignore the alarm condition and allow the passenger to board the aircraft 106.

[0085] Figure 4 A method 400 for monitoring the flight of an aircraft 106 using personal carry items is shown. In a specific implementation, by... Figure 1 The monitoring device 108 of system 100 performs one or more operations in method 400. Initially, the monitoring device 108 receives input indicating that the boarding process for the flight has begun. In response to the start of the boarding process, the monitoring device 108 is able to activate the camera 126 in the aircraft 106 and activate the compartment status indicator device 128 of each compartment 104 in the aircraft 106. The first part of the camera is positioned to capture images of the interior of the compartment 104.

[0086] When the locker 104 associated with the camera is opened, the camera in the first part of the camera module can be activated. When the monitoring device 108 can determine the status of the locker 104 from image data 132 received from one or more cameras associated with the locker 104, the locker status indicator device 128 of the locker 104 is activated. In a specific implementation, the locker status indicator device 128 indicates the status of the locker 104 by emitting light of a specific color. For example, in one implementation, green light indicates available storage space, blue light indicates that additional space can be made in the locker 104 by rearranging the items in the locker 104, violet light indicates that the locker 104 is full, and yellow light (e.g., amber light) indicates that there is a problem with the locker 104. In other implementations, different color schemes, less light, additional light, or combinations thereof can be used to indicate other status conditions. In still other implementations, different indication methods are used, such as displaying text indicating the status.

[0087] Method 400 includes, in block diagram 402, receiving image data 132 from a camera 126 in aircraft 106. At least a first portion of the camera 126 is positioned to capture images of the interior of the storage compartment 104. In some implementations, another portion of the camera 126 is positioned to capture images of the passenger in aircraft 106 as the passenger loads items and proceeds to their seat in aircraft 106. The camera 126 may be a video camera, one or more still-image cameras controlled by one or more timers to periodically capture images, other types of cameras, or combinations thereof.

[0088] Method 400 includes: in block diagram 404, analyzing image data 132 to determine first information 176 in each locker 104. First information 176 includes characteristics of items in the locker 104 and storage space. The first information includes the type of items in the locker 104, the size of the items stored in the locker 104, the orientation information of the items stored in the locker, the remaining space of the items in the locker 104, the position of the items relative to the edges of the locker 104 and mounting equipment, the position of the items relative to moving parts and door fasteners of the locker 104, other features, or combinations thereof. For example, monitoring device 108 determines the width and height of items placed at the bottom of the locker 104 based on image data 132 from camera 126. Based on the width and height of the items, the width-to-height ratio, other factors, or combinations thereof, monitoring device 108 determines whether the items are personal items 102 or different types of items (e.g., personal belongings, coats, or other clothing) that do not need to be loaded into the locker 104.

[0089] In another embodiment, the monitoring device 108 determines the orientation information of the item in the storage box 104 based on a comparison of a first dimension of the item along the length of the storage box and a second dimension of the item along the height of the storage box. In yet another embodiment, the monitoring device 108 determines the available storage space in the storage box 104 as the sum of the length of the storage box, the reduced length of the unusable space (e.g., the length occupied by security equipment), and the reduced first dimension of the item in the storage box 104 along the length of the storage box 104.

[0090] Method 400 includes: in block diagram 406, determining that one or more alarm conditions 170 are met based on a comparison of one or more alarm criteria 146 with first information 176 for each locker 104. The monitoring device 108 determines that alarm condition 170 is met when one or more personal items 102 are in an inappropriate position in the locker 104, when a smaller item (e.g., personal belongings, clothing, etc.) is placed in the locker 104 that can replace the personal item 102, when sufficient space for an additional personal item 102 can be freed up by moving items in the locker 104 closer together, when there is no sufficient space in the locker 104 for another personal item 102 (e.g., the locker is full), etc.

[0091] Method 400 further includes, in block diagram 408, sending output 134 to one or more devices based on the fulfillment of one or more alarm conditions 170. Output 134 may be a signal to one or more box status indicator devices 128, information to portable device 130, other outputs, or a combination thereof.

[0092] refer to Figure 5 The diagram 500, shown at 500, illustrates a flowchart example of the lifecycle of an aircraft configured to monitor the storage of items in its storage compartments. During pre-production, in block diagram 502, the exemplary method 500 includes the aircraft's specifications and design, such as references... Figure 6 The described aircraft 106. During the specification and design process of the aircraft, method 500 may include the specification and design of a second set of cameras 126 to monitor passengers moving to their seats during the boarding process, to monitor the interior of the storage compartments 104, or both. During the specification and design process of the aircraft, in block diagram 502, method 500 may include the specification and design of a compartment status indicator device 128 for each storage compartment 104. In block diagram 504, method 500 includes material procurement, which may include procuring the second set of cameras 126 and the compartment status indicator device 128, as well as necessary accessories (e.g., wiring, power connections, etc.).

[0093] In the production process, in block diagram 506, method 500 includes the manufacturing of components and sub-assemblies, and in block diagram 508, it includes the system integration of the aircraft. For example, method 500 may include the manufacturing of components and sub-assemblies for the second set of cameras 126 and the container status indicator device 128. In block diagram 510, method 500 includes the certification and delivery of the aircraft, and in block diagram 512, it includes putting the aircraft into operation. Certification and delivery may include the certification operations of the second set of cameras 126 and the container status indicator device 128 with other systems of the aircraft. Once the aircraft is put into operation by the customer, routine maintenance and repairs (which may also include modification, reconfiguration, refurbishment, etc.) may be scheduled for the aircraft. In block diagram 514, method 500 includes performing maintenance and repairs on the aircraft, which may include performing maintenance and repairs on the second set of cameras 126 and the container status indicator device 128.

[0094] Each operation in method 500 may be performed or completed by a system integrator, a third party, and / or an operator (e.g., a customer). For the purposes of this description, a system integrator may include, but is not limited to, any number of aircraft manufacturers and main system subcontractors; a third party may include, but is not limited to, any number of retailers, subcontractors, and suppliers; and an operator may be an airline, leasing company, military entity, service organization, etc.

[0095] Various aspects of this disclosure can be described in the context of embodiments of the vehicle. Specific embodiments of the vehicle are... Figure 6 The aircraft 106 shown is shown.

[0096] exist Figure 6 In one embodiment, the aircraft 106 includes a fuselage 618 and an interior 622 having multiple systems 620. Embodiments of the multiple systems 620 include one or more propulsion systems 624, electrical systems 626, environmental systems 628, and hydraulic systems 630. Any number of other systems may be included, such as a second set of cameras 126, a container status indicator device 128, or both.

[0097] Figure 7 This is a block diagram of a computing environment 700 according to the present disclosure, including a computing device 710 configured to support aspects of a computer-implemented method and computer-executable program instructions (or code). For example, the computing device 710 or a portion thereof is configured to execute instructions to initiate, execute, or control references. Figures 1 to 6 The described one or more operations. In a specific aspect, the computing device 710 includes... Figure 1 The monitoring device 108, boarding pass scanning station 112, cameras in the first group of cameras 116, specific location markers in location markers 118, cameras in the second group of cameras 126, and portable device 130, one or more servers, one or more virtual devices, or combinations thereof.

[0098] The computing device 710 includes one or more processors 720. In a specific aspect, the processor 720 and... Figure 1 The processor 720 corresponds to processor 140. Processor 720 is configured to communicate with system memory 730, one or more storage devices 740, and one or more input / output devices 770 via one or more input / output interfaces 750, one or more communication interfaces 760, or any combination thereof. System memory 730 includes volatile memory devices (e.g., random access memory (RAM) devices), non-volatile memory devices (e.g., read-only memory (ROM) devices, programmable read-only memory, and flash memory), or both. System memory 730 stores operating system 732, which may include a basic input / output system and a full operating system for booting computing device 710 to enable computing device 710 to interact with users, other programs, and other devices. System memory 730 stores system (program) data 736, such as... Figure 1 The data includes aircraft data 144, alarm standards 146, flight data 148, and first information 176 (e.g., the characteristics of items and storage space in the overhead storage box 104).

[0099] System memory 730 includes one or more applications 734 (e.g., instruction sets) executed by processor 720. As an example, one or more applications 734 include instructions 142, an item analyzer 172, and a box analyzer 174 executed by processor 720 to initiate, control, or execute references. Figures 1 to 6 One or more operations described.

[0100] In a specific implementation, system memory 730 includes a non-volatile computer-readable medium (e.g., a computer-readable storage device) storing instructions 142, which, when executed by processor 720, cause processor 720 to initiate, execute, or control operations to monitor the storage of carry-on items in the aircraft's storage compartment. The operations include: receiving image data from one or more cameras; analyzing the image data to determine the characteristics of the carry-on items; determining that one or more alarm conditions are met based on a comparison of one or more alarm criteria with the characteristics of the carry-on items and data associated with the aircraft; and sending output to one or more devices based on the satisfaction of one or more alarm conditions.

[0101] One or more storage devices 740 include non-volatile storage devices, such as disks, optical disks, or flash memory devices. In a particular embodiment, storage device 740 includes removable and non-removable memory devices. Storage device 740 is configured to store an operating system, images of the operating system, applications (e.g., one or more applications 734), and program data (e.g., program data 736). In a particular aspect, system memory 730, storage device 740, or both include tangible computer-readable media. In a particular aspect, one or more storage devices 740 are located external to computing device 710.

[0102] One or more input / output interfaces 750 enable communication between computing device 710 and one or more input / output devices 770, facilitating user interaction. For example, one or more input / output interfaces 750 may include a display interface, an input interface, or both. For example, input / output interfaces 750 are adapted to receive input from a user, input from another computing device, or a combination thereof. In some implementations, input / output interfaces 750 conform to one or more standard interface protocols, including serial interfaces (e.g., Universal Serial Bus (USB) interfaces or IEEE interface standards), parallel interfaces, display adapters, audio adapters, or custom interfaces (“IEEE” is a registered trademark of the Organization of Electrical and Electronics Engineers, Inc. of Piscataway, New Jersey). In some implementations, input / output devices 770 include one or more user interface devices and displays, including buttons, keyboards, pointing devices, displays, speakers, microphones, touchscreens, and some combination of other devices.

[0103] The processor 720 is configured to communicate with the device or controller 780 via one or more communication interfaces 760. For example, the one or more communication interfaces 760 may include a network interface. For example, the device or controller 780 may include... Figure 1 The monitoring device 108, the monitor 114, the camera in the first group of cameras 116, the status indicator device 120, the camera in the second group of cameras 126, the box status indicator device 128, or a combination thereof.

[0104] Further illustrative and non-exclusive embodiments according to this disclosure are described in the following paragraphs:

[0105] In an embodiment according to this disclosure, a system (100) for monitoring carry-on items (102) of an aircraft (106) in flight includes:

[0106] One or more cameras (116, 126) are positioned to capture images of personal belongings (102) associated with passengers of the aircraft;

[0107] Interface (136) is configured to receive image data (122, 132) from one or more cameras; and

[0108] One or more processors (140) are coupled to the interface and configured to:

[0109] Image data is analyzed to determine the characteristics of personal items (168);

[0110] Based on a comparison of one or more alarm criteria (146) with the characteristics of personal belongings and data (144) associated with the aircraft, it is determined that one or more alarm conditions (170) are met; and

[0111] The output (124, 134) is sent to one or more devices (114, 120, 128, 130) based on the fulfillment of one or more alarm conditions.

[0112] Optionally, according to the system described in the preceding paragraph, one or more cameras include one or more cameras in the first group of cameras (116) positioned to capture images of personal belongings in the boarding area (110) of the aircraft.

[0113] Optionally, in the system according to any one of the preceding paragraphs, one or more cameras include one or more cameras in the second set of cameras (126) positioned to capture images of the interior of the aircraft's storage compartment (104).

[0114] Optionally, in the system according to any one of the preceding paragraphs, based on the size of carry-on items determined from image data, one or more alarm criteria correspond to an oversized carry-on item, and wherein, in response to an oversized carry-on item, an output is provided to a device (114, 120) associated with the boarding agent.

[0115] Optionally, in the system according to any one of the preceding paragraphs, based on information obtained from image data, one or more alarm criteria correspond to the available storage space in the aircraft's locker that is short of a threshold amount, and wherein, in response to the available storage space in the locker being short of a threshold amount, an output is provided to the device (114, 120) associated with the boarding agent.

[0116] Optionally, in any of the preceding paragraphs, the determination of the lack of available storage space in the locker based on the aircraft data (144) associated with the aircraft and the number of carry-on items allowed to be transported from the boarding area to the aircraft (156) is based on the system according to any of the preceding paragraphs.

[0117] Optionally, in any of the preceding paragraphs, the determination of the lack of a threshold amount of available storage space in the locker is further based on image data of the interior of the locker captured by a particular camera of one or more cameras (132).

[0118] Optionally, in the system according to any one of the preceding paragraphs, one or more alarm criteria are associated with a specific locker, and wherein the output includes information sent to a device (130) associated with the crew of the aircraft, a signal activating a locker status indicator device (128) associated with the specific locker, or both.

[0119] Optionally, in the system according to any one of the preceding paragraphs, the color of the locker status indicator device is associated with a specific status of a particular locker.

[0120] In another embodiment of this disclosure, a method (400) for monitoring the flight of a personal item (102) of an aircraft (106) includes:

[0121] Image data (132) from a camera (126) on the aircraft is received (402) at the monitoring device (108), at least a portion of the camera being positioned to capture images of the interior of a storage compartment (104) for carry-on items (102);

[0122] Image data is analyzed (404) at the monitoring device to determine first information (176), the first information including the characteristics of the items in each locker and the available storage space in each locker;

[0123] At the monitoring device, based on a comparison (406) of one or more alarm criteria (146) with the first information, it is determined that one or more alarm conditions (170) are met; and

[0124] The output (134) is sent (408) from the monitor device to one or more devices (128, 130) based on the fulfillment of one or more alarm conditions.

[0125] Optionally, according to the method described in the preceding paragraph, analyzing the image data further includes: determining the state associated with each locker, and wherein sending the output includes: sending a signal that changes the color of the locker state indicator device (128) to indicate the state associated with the locker.

[0126] Optionally, according to any one of the preceding paragraphs, analyzing the image data includes: responding that the storage box has available space, determining that the state is a first state, and wherein analyzing the image data includes: responding that the storage box has available storage space less than a threshold amount, determining that the state is a third state indicating that the storage box is full.

[0127] Optionally, according to any one of the preceding paragraphs, analyzing the image data includes: in response to the size of the item placed in the specific storage box corresponding to the size associated with the personal item (178), determining that the state is a second state.

[0128] Optionally, according to any one of the preceding paragraphs, analyzing the image data includes: responding to one or more items located in the storage box preventing the storage box from closing, and determining that the state is the fourth state.

[0129] Optionally, according to any one of the preceding paragraphs, analyzing the image data further includes: determining the available storage space in the locker, and wherein sending the output includes: sending a message indicating the number of additional carry-on items that can be stored in the locker to a device (114) associated with the boarding agent.

[0130] In another embodiment of this disclosure, a method (300) for monitoring the flight of an aircraft (106) using a personal item (102) includes:

[0131] The monitoring device (108) receives (302) image data (122) from one or more cameras (116) in the boarding area (110) of the aircraft, the one or more cameras being positioned to capture images of carry-on items associated with passengers preparing to board the aircraft;

[0132] Image data is analyzed (304) at the monitoring device to determine the characteristics (168) of personal belongings associated with the passenger;

[0133] At the monitoring device, based on a comparison of one or more alarm criteria (146) with the characteristics of personal belongings and data (144) associated with the aircraft, it is determined (306) that one or more alarm conditions (170) are met; and

[0134] The output (124) is sent from the monitor device to one or more devices (114, 120) based on the fulfillment of one or more alarm conditions.

[0135] Optionally, according to the method described in the preceding paragraph, the features include estimates of the length, width, and height of the personal items carried.

[0136] Optionally, according to any of the preceding paragraphs, the method outputs an indication that the carry-on item is too large, based on one or more of the length, width, or height exceeding one or more alarm criteria for one or more corresponding size thresholds for the permissible size of the carry-on item.

[0137] Optionally, according to any one of the preceding paragraphs, estimating the length includes: determining a first length of the carried item, and in response to the first length exceeding a threshold, determining whether the first length is associated with two or more individual items; and in response to determining that the first length is associated with two or more individual items, estimating the length as the length associated with the longest of the two or more individual items.

[0138] Optionally, according to any one of the preceding paragraphs, one or more alarm criteria correspond to a lack of storage space in the aircraft's storage compartment by a threshold amount, and wherein the satisfaction of one of the one or more alarm conditions related to the lack of storage space by a threshold amount is based on second image data (134) received from the aircraft.

[0139] In some implementations, a non-volatile computer-readable medium (e.g., a computer-readable storage device) stores instructions that, when executed by one or more processors, cause one or more processors to initiate, execute, or control the execution of some or all of the operations described above. For example, instructions can be executed to implement Figures 1 to 6 One or more operations or methods. In some implementations, this can be implemented by one or more processors executing the instructions (e.g., one or more central processing units (CPUs), one or more graphics processing units (GPUs), one or more digital signal processors (DSPs)), by dedicated hardware circuitry, or any combination thereof. Figures 1 to 6 Part or all of one or more operations or methods in the [system / method].

[0140] The illustrative embodiments described herein are intended to provide a general understanding of the structure of the various implementations. The illustrative examples are not intended as a complete description of all elements and features in an apparatus or system utilizing the structures or methods described herein. Many other implementations will become apparent to those skilled in the art upon review of this disclosure. Other implementations can be utilized and derived from this disclosure, allowing for structural and logical substitutions and changes without departing from the scope of this disclosure. For example, method steps may be performed in a different order than those shown in the figures, or one or more method operations may be omitted. Accordingly, this disclosure and the figures are to be considered illustrative and not restrictive.

[0141] Furthermore, although specific embodiments have been shown and described herein, it should be understood that any subsequent arrangements designed to achieve the same or similar results may replace the specific implementations shown. This disclosure is intended to cover any and all subsequent adaptations or variations of the various implementations. It will be apparent to those skilled in the art upon review of the description that combinations of the above-described implementations and other implementations are not specifically described herein.

[0142] An abstract of this disclosure is provided without being construed as limiting or interpreting the scope or meaning of the claims. Furthermore, in the following detailed description, features may be grouped together, or described in a single implementation for the purpose of streamlining this disclosure. The embodiments described above illustrate, but do not limit, this disclosure. It should also be understood that various modifications and variations are possible based on the principles of this disclosure. As reflected in the technical solutions, the claimed subject matter may address fewer features than all features in any disclosed embodiment. Accordingly, the scope of this disclosure is defined by the technical solutions and their equivalents.

Claims

1. A system (100) for monitoring carry-on items (102) of an aircraft (106) in flight, said system comprising: Cameras (116, 126) are positioned to capture images of personal belongings (102) associated with passengers of the aircraft; An interface (136) is configured to receive image data (122, 132) from the camera, wherein the camera includes a first set of cameras (116) and a second set of cameras (126), the second set of cameras being positioned to capture images of the interior of the aircraft's storage compartment; and One or more processors (140) are coupled to the interface and configured to: The image data is analyzed to determine first information, which includes features (168) of personal items. The first information for each item includes a first size, which corresponds to the longest length of the item in the direction parallel to the bottom of the storage box, and a second size, which corresponds to the longest length of the item in the direction parallel to the side of the storage box. Based on a comparison of one or more alarm criteria (146) with the characteristics of the personal carry item and data (144) associated with the aircraft, it is determined that one or more alarm conditions (170) are met, wherein a first alarm condition among the one or more alarm conditions indicates that, in response to the ratio of the first size to the second size of a particular item being greater than a threshold, additional space can be freed up in a particular storage compartment by reorienting the particular item; and Outputs (124, 134) are sent to one or more devices (114, 120, 128, 130) based on the fulfillment of one or more alarm conditions, wherein the outputs include sending a message to a portable device (130) associated with the crew of the aircraft in response to the fulfillment of the first alarm condition.

2. The system according to claim 1, wherein, The first set of cameras (116) is positioned to capture images of personal belongings in the boarding area (110) of the aircraft.

3. The system according to claim 1 or 2, wherein, Based on the size of the carry-on item determined from the image data, one of the one or more alarm criteria corresponds to the carry-on item being too large, and wherein, in response to the carry-on item being too large, the output is provided to the device (114, 120) associated with the boarding agent.

4. The system according to claim 1 or 2, wherein, Based on information obtained from the image data, one or more alarm criteria correspond to a threshold amount of available storage space missing in the aircraft's storage compartment, and wherein, in response to a threshold amount of available storage space missing in the storage compartment, the output is provided to a device (114, 120) associated with the boarding agent.

5. The system according to claim 4, wherein, The determination of whether the available storage space in the storage box is lacking the threshold amount is based on aircraft data (144) associated with the aircraft and the number of carry-on items allowed to be transported from the boarding area to the aircraft (156).

6. The system according to claim 4, wherein, The determination of whether the available storage space in the storage box is lacking the threshold amount is further based on image data (132) of the interior of the storage box captured by a specific camera among the one or more cameras.

7. The system according to claim 1 or 2, wherein, One or more alarm criteria are associated with a specific locker, and the output includes information sent to a device (130) associated with the crew of the aircraft, a signal activating a locker status indicator device (128) associated with the specific locker, or both.

8. The system according to claim 7, wherein, The color of the box status indicator device is associated with the specific status of the particular storage box.

9. A method (400) for monitoring carry-on items (102) of an aircraft (106) in flight, the method comprising: Image data (132) from a camera (126) on the aircraft is received (402) at the monitoring device (108), at least a portion of the camera being positioned to capture images of the interior of a storage compartment (104) for personal carry items (102); The image data is analyzed (404) at the monitoring device to determine first information (176), the first information including the characteristics of the items in each of the storage boxes and the available storage space in each of the storage boxes, wherein the first information for each item includes a first size corresponding to the longest length of the item in the direction parallel to the bottom of the storage box, and a second size corresponding to the longest length of the item in the direction parallel to the side of the storage box; At the monitoring device, a comparison (406) between one or more alarm criteria (146) and the first information determines whether the one or more alarm conditions (170) are met, wherein a first alarm condition among the one or more alarm conditions indicates that, in response to the ratio of the first size to the second size of a particular item being greater than a threshold, additional space can be freed up in a particular storage box by reorienting the particular item; and Based on the fulfillment of one or more alarm conditions, an output (134) is sent (408) from the monitor device to one or more devices (128, 130), wherein the output includes sending a message to a portable device (130) associated with the crew of the aircraft in response to the fulfillment of the first alarm condition.

10. The method according to claim 9, wherein, Analyzing the image data further includes determining the state associated with each locker, and wherein sending the output includes sending a signal that changes the color of a locker state indicator device (128) to indicate the state associated with the locker.

11. The method according to claim 10, wherein, Analyzing the image data includes: in response to the storage box having available space, determining that the state is a first state, and wherein analyzing the image data includes: in response to the storage box having less than a threshold amount of available storage space, determining that the state is a third state indicating that the storage box is full.

12. The method according to claim 10 or 11, wherein, The analysis of the image data includes: in response to the size of the item placed in the specific storage box corresponding to the size associated with the personal item (178), determining that the state is a second state.

13. The method according to claim 10 or 11, wherein, Analyzing the image data includes: responding to one or more items located in the storage box preventing the storage box from closing, and determining that the state is the fourth state.

14. The method according to any one of claims 9 to 11, wherein, Analyzing the image data further includes determining the available storage space in the locker, and wherein sending the output includes sending a message indicating the number of additional carry-on items that can be stored in the locker to a device (114) associated with the boarding agent.

Images