System and method for determining the status of a storage locker assembly within an interior cabin of a vehicle

Abstract

The present invention relates to a system and method for determining the state of a storage cabinet assembly within an interior cabin of a vehicle, and discloses a system (100) comprising one or more components (102) within an interior cabin (104) of a vehicle (106). An imaging device (108) is configured to obtain an image of the one or more components (102). A state determination control unit (112) includes a processor (300). The state determination control unit (112) communicates with the imaging device (108). The state determination control unit (112) receives image data (122) including an image from the imaging device (108). In addition, the state determination control unit (112) determines the state of the one or more components (102) based on the image data (122).

Description

Technical Field

[0001] Embodiments of the subject disclosure generally relate to systems and methods for determining the status of components, such as storage bin assemblies, within an interior cabin of a vehicle. Background Art

[0002] Aircraft are used to transport passengers and cargo between different locations. The interior cabin of an aircraft includes numerous components that can be moved between open and closed positions. For example, an overhead storage locker assembly in the interior cabin includes a locker that can be moved between open and closed positions. As another example, an exit door can be moved between open and closed positions. As another example, a lavatory door in the interior cabin can be moved between open and closed positions. As another example, a galley cart compartment can be positioned between open and closed positions, with the galley cart being moved into the compartment in the open position and the galley cart already inside the compartment in the closed position. Typically, the interior cabin of a commercial aircraft vehicle includes various components that can be moved between open and closed positions.

[0003] During different phases of flight, certain components are in specific states. For example, during the boarding process, the locker assemblies are in an open state to allow passengers to store luggage and other personal items. As takeoff approaches, flight attendants typically monitor the locker assemblies and ensure that each is closed before the aircraft moves to the taxiway and / or runway. In other words, individuals, such as flight attendants, check to see if any locker assemblies are open. It is understandable that this process of traversing the interior cabin and visually determining which locker assemblies are open and closed takes time. Furthermore, if the flight attendant is distracted, such as if a passenger needs assistance, open locker assemblies may be overlooked.

[0004] Some locker assemblies include or are otherwise coupled to sensors that are connected to monitoring equipment via wiring. However, sensors and wiring add weight to the aircraft. These sensors and wiring increase manufacturing complexity because they need to be positioned and / or routed. Furthermore, the added weight of these sensors and wiring reduces fuel efficiency. Summary of the Invention

[0005] There is a need for a system and method for automatically determining the status of components within an interior compartment of a vehicle. Additionally, there is a need for a system and method for automatically determining whether a storage locker assembly within an interior compartment of a vehicle is open or closed. Furthermore, there is a need for a system and method for monitoring the status of components within an interior compartment that does not substantially add weight to the vehicle.

[0006] In light of these needs, certain embodiments of the subject disclosure provide a system comprising one or more components within an interior cabin of a vehicle, an imaging device configured to obtain images of the one or more components, and a state determination control unit comprising a processor. The state determination control unit communicates with the imaging device. The state determination control unit receives image data, including an image, from the imaging device and determines a state of the one or more components based on the image data.

[0007] In at least one embodiment, the state includes one or both of an open state and a closed state.In at least one embodiment, the one or more components are one or more overhead storage bin assemblies.

[0008] In at least one embodiment, the state determination control unit determines a region of interest within the image data. For example, the state determination control unit determines the region of interest by determining a vanishing point within the image. As another example, the state determination control unit determines a bisecting line passing through the vanishing point. The region of interest is determined by the bisecting line.

[0009] In at least one embodiment, the state determination control unit corrects perspective within the image data. For example, the one or more components include multiple components in a row. The state determination control unit generates a first end line at a first end of the row and a second end line at a second end of the row opposite the first end line. The state determination control unit also generates an upper boundary line associated with the upper edge of the row and a lower boundary line associated with the lower edge of the row. The first end line and the upper boundary line intersect at a first corner. The first end line and the lower boundary line intersect at a second corner. The second end line and the upper boundary line intersect at a third corner. The second end line and the lower boundary line intersect at a fourth corner. The first corner, the second corner, the third corner, and the fourth corner are corners of a quadrilateral that defines the multiple components. The state determination control unit geometrically transforms the quadrilateral into a rectangle with corresponding corners to form a perspective-corrected image.

[0010] In at least one embodiment, the state determination control unit extrapolates the shape of each of the one or more components within the image data.

[0011] In at least one embodiment, the state determination control unit determines the state of one or more components by clustering.

[0012] In at least one embodiment, the state determination control unit determines the state of the one or more components by determining differences in image attributes.

[0013] In at least one example, the system further includes a user device. The state determination control unit outputs a state signal indicating a state of the one or more components to the user device.

[0014] In at least one embodiment, the state determination control unit outputs a component control signal that automatically operates one or more components based on the state of the one or more components.

[0015] Certain embodiments of the subject disclosure provide a method comprising: obtaining, by an imaging device, an image of one or more components within an interior cabin of a vehicle; receiving, by a state determination control unit comprising a processor in communication with the imaging device, image data comprising the image from the imaging device; and determining, by the state determination control unit, a state of the one or more components based on the image data.

[0016] Certain embodiments of the subject disclosure provide a non-transitory computer-readable storage medium comprising executable instructions that, in response to execution, cause a system comprising a processor to perform operations comprising: receiving image data comprising images of one or more components within the interior cabin of a vehicle from an imaging device within the interior cabin of the vehicle; and determining a status of the one or more components based on the image data. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 A schematic block diagram of a system for determining a status of one or more components within an interior cabin of a vehicle is shown, according to an embodiment of the subject disclosure.

[0018] Figure 2 A flow chart illustrating a method for determining a status of one or more components within an interior cabin of a vehicle, according to an embodiment of the subject disclosure.

[0019] Figure 3 A schematic block diagram of a state determination control unit according to an embodiment of the present subject disclosure is shown.

[0020] Figure 4 An image of a portion of an interior cabin is shown, according to an embodiment of the subject disclosure.

[0021] Figure 5 A perspective-corrected image of a region of interest within an interior cabin including a row of overhead storage bin assemblies is shown, according to an embodiment of the subject disclosure.

[0022] Figure 6 A perspective-corrected image is shown having an extrapolated cabinet shape corresponding to an overhead storage cabinet assembly, in accordance with an embodiment of the subject disclosure.

[0023] Figure 7 A perspective-corrected image is shown having a first cluster of cabinet shapes corresponding to a first set of overhead storage bin assemblies and a second cluster of cabinet shapes corresponding to a second set of overhead storage bin assemblies, in accordance with an embodiment of the subject disclosure.

[0024] Figure 8 An image of a portion of an interior cabin is shown, according to an embodiment of the subject disclosure.

[0025] Figure 9 The embodiment according to the present disclosure is shown Figure 8 The image of the part of the interior cabin is within the vanishing point.

[0026] Figure 10 The embodiment according to the present disclosure is shown Figure 8 An image of a portion of an interior cabin includes a first region of interest and a second region of interest.

[0027] Figure 11 A first region of interest of an image is shown, in accordance with an embodiment of the subject disclosure.

[0028] Figure 12 The embodiment according to the present disclosure is shown Figure 11 A perspective-corrected image of a row of overhead locker assemblies within a first region of interest.

[0029] Figure 13 A front perspective view of a carrier is shown, according to an exemplary embodiment of the subject disclosure. DETAILED DESCRIPTION

[0030] When read in conjunction with the accompanying drawings, the foregoing summary of the invention and the following detailed description of certain embodiments will be better understood. As used herein, an element or step recorded in the singular and preceded by the word "one" or "an" should be understood as not necessarily excluding plural elements or steps. In addition, reference to "one embodiment" is not intended to be interpreted as excluding the existence of additional embodiments that also include the features described. Furthermore, unless explicitly stated to the contrary, an embodiment that "includes," "contains," or "has" an element or multiple elements having a specific condition may include additional elements that do not have that condition.

[0031] Certain embodiments of the subject disclosure provide a system and method for determining the status (e.g., open and closed) of components (e.g., locker assemblies) within the interior cabin of a vehicle (e.g., an aircraft). The system and method are configured to detect the status of components in real time using an in-cabin video camera to automatically record these events, trigger further procedures, and alert the flight crew with the event information.

[0032] Certain embodiments of the subject disclosure provide a system and method for automatically detecting cabin events using video feeds available from onboard cameras to automatically record, trigger future events, and alert the crew of the event information. By processing multiple in-flight events, the system and method can detect unusual potential events.

[0033] In at least one embodiment, the system and method includes viewing in-cabin camera feeds, determining regions of interest, calculating vanishing points, applying algorithms to correct for perspective, extrapolating cabinet positions, clustering components to identify status, and alerting the crew of the status. By analyzing image data obtained from one or more imaging devices to detect the status of components, embodiments of the subject disclosure are configured to operate without the use of sensors and wiring, thereby reducing manufacturing cost and complexity and improving fuel efficiency.

[0034] Figure 1 A schematic block diagram of a system 100 for determining a state of one or more components 102 within an interior cabin 104 of a vehicle 106 is shown, according to an embodiment of the subject disclosure. In at least one embodiment, the component(s) 102 are overhead storage bin assemblies configured to move between an open state and a closed state. In some examples, the component(s) 102 are one or more doors within the interior cabin 104, such as a lavatory door or an exit door. In some examples, the component 102 is a compartment within the interior cabin 104, such as a galley cart assembly.

[0035] In at least one embodiment, vehicle 106 is an aircraft. In some examples, vehicle 106 is a land-based vehicle, such as a bus, a passenger train car, etc. In some examples, vehicle 106 is a ship or a spacecraft.

[0036] System 100 includes an imaging device 108 within interior cabin 104. Imaging device 108 has a field of view 110. Component(s) 102 are within field of view 110. In at least one embodiment, imaging device 108 is a video camera. In some examples, imaging device 108 is a night vision camera. In some examples, imaging device 108 is an infrared camera.

[0037] Imaging device 108 may be a fixed camera within interior cabin 104. For example, imaging device 108 may be fixed to a ceiling, a wall, a monument, etc. within interior cabin 104. In some examples, imaging device 108 may be a mobile camera, such as a handheld camera, such as part of a smartphone or smart tablet.

[0038] The system 100 also includes a state determination control unit 112 that communicates with the imaging device 108. For example, the state determination control unit 112 communicates wirelessly with the imaging device 108, such as via Bluetooth, Wi-Fi, or other such connection. Alternatively, the state determination control unit 112 can communicate with the imaging device 108 via a wired connection.

[0039] In at least one embodiment, the component(s) 102 may include an actuator 114, such as an electric motor, configured to automatically move the component(s) 102 between an open state and a closed state. Furthermore, the state determination control unit 112 wirelessly communicates with the actuator 114 to control the opening and closing of the component(s) 102. Alternatively, the state determination control unit 112 communicates with the actuator 114 via a wired connection. Furthermore, optionally, the state determination control unit 112 does not communicate with the actuator 114. Furthermore, optionally, the component 102 may not include the actuator 114.

[0040] In at least one embodiment, user device 116 is also within interior cabin 104. User device 116 includes a display 118 coupled to interface 120. For example, user device 116 is a computer workstation within interior cabin 104. In some examples, user device 116 is a handheld device, such as a smartphone or a smart tablet. Display 118 can be a monitor or screen. Interface 120 can include a keyboard, a mouse, etc. In at least one embodiment, display 118 and interface 120 are integrated into a touch screen interface. State determination control unit 112 communicates with user device 116, such as via a wired or wireless connection.

[0041] As described herein, system 100 includes one or more components 102 within an interior cabin 104 of a vehicle 106. An imaging device 108 is configured to obtain images of the one or more components 102. A state determination control unit 112 communicates with imaging device 108. State determination control unit 112 receives image data 122 including an image from imaging device 108. State determination control unit 112 determines a state (such as an open state or a closed state) of one or more components 102 based on image data 122.

[0042] In operation, imaging device 108 obtains an image of component(s) 102. The image may be a video image or a still photographic image. Imaging device 108 obtains the image of component(s) 102 as image data 122. State determination control unit 112 receives image data 122 including the image of component(s) 102 from imaging device 108.

[0043] The state determination control unit 112 analyzes the image data 122 to determine the state of the component(s) 102. For example, the state determination control unit 112 analyzes the image data 122 to determine whether the component(s) 102 (such as the overhead storage bin assembly) is in an open state (i.e., open) or a closed state (i.e., closed).

[0044] In at least one embodiment, the state determination control unit 112 analyzes the image data 122 by determining a region of interest within the image data 122. For example, in at least one embodiment, the region of interest includes a row of components 102, such as a row of overhead storage bin assemblies within the interior cabin 104.

[0045] After determining the region of interest, the state determination control unit 112 performs perspective correction on the image data 122 to provide a perspective-corrected image of the image data 122. For example, the state determination control unit 112 corrects the perspective of the image in a three-dimensional space. Alternatively, the state determination control unit 112 may not perform perspective correction on the image data 122.

[0046] Next, the state determination control unit 112 extrapolates the individual components 102 within the perspective-corrected image. For example, during initial calibration or setup, the state determination control unit 112 is programmed to identify the size and shape of each individual component 102 within the image data 122 (such as within the perspective-corrected image). In at least one embodiment, the state determination control unit 112 extrapolates the shape of each component 102 within the image data 122.

[0047] The state determination control unit 112 then determines the state of the components 102 by clustering them, such as by determining differences in image attributes (e.g., differences in color, color intensity, brightness, etc.). For example, the components 102 within the perspective-corrected image are clustered based on color differences between them. If a single cluster associated with the components 102 exists within the perspective-corrected image, the state determination control unit 112 determines that all components 102 are in the same state, such as an open state or a closed state. In at least one embodiment, the state determination control unit 112 is calibrated to determine image attributes specific to the closed state. For example, an image of components 102 having a first color is determined to be in the closed state. Therefore, if a single cluster of components 102 all sharing the same attribute (e.g., the same color or color intensity) is associated with the closed state, the state determination control unit 112 determines that all components 102 are closed. However, if the single cluster of components 102 differs from the predetermined image attribute for the closed state, the state determination control unit 112 determines that the components 102 are in the open state.

[0048] In addition, the image data 122 may show different clusters of images. For example, a first cluster corresponding to a first set of one or more components 102 has a first image attribute (such as a first color or intensity), and a second cluster corresponding to a second set of one or more components 102 has a second image attribute that is different from the first image attribute. Therefore, the state determination control unit 112 determines that one of the first set or the second set is in an open state and the other is in a closed state. The state determination control unit 112 determines which set is in the closed state or the open state based on predetermined image attributes. For example, the state determination control unit 112 may be calibrated to identify a first image attribute as being in the closed state and a second image attribute that is different from the first image attribute as being in the open state, or vice versa.

[0049] The state determination control unit 112 associates the first cluster and the second cluster with the locations of the components 102 within the interior cabin 104. For example, the state determination control unit 112 is calibrated, such as via an initial manual calibration or setup, to locate and mark each component 102 within the interior cabin 102. Thus, based on the image analysis, the state determination control unit 112 detects the state (such as open or closed) of the components 102 within the interior cabin 104.

[0050] In at least one embodiment, as described herein, the state determination control unit 112 detects the state of the component 102 through clustering. The state determination control unit 112 compares the clusters within the image data 122 to distinguish between the open state and the closed state. In such an embodiment, the state determination control unit 112 does not compare the image data 122 with historical image data regarding the open state and the closed state. As a result, the system 100 can operate using less memory (which would otherwise store historical data) and less computing power, thereby providing efficient operation.

[0051] The status determination control unit 112 then outputs a status signal 126 to the user device 116. The status signal 126 indicates the status of the component(s) 102. The status signal 126 includes status data of the components 102 within the interior cabin 104. For example, the status signal 126 indicates which of the components 102 are open or closed. The status signal 126 is received by the user device 116, which may provide video or graphical data on the display 118, provide audio signals, etc. to alert the crew regarding which of the components 102 are open or closed. In this manner, the status determination control unit 112 automatically detects the status of the components 102 and alerts the crew, who can then close or open the components 102 as needed.

[0052] In at least one embodiment, the state determination control unit 112 can automatically operate the components 102, such as via the actuators 114, based on the determined states of the components 102. For example, after the state determination control unit 112 analyzes the image data 122 to determine which of the components 102 are open and closed, the state determination control unit 112 outputs component control signals 128 to the actuators 114 to selectively close the open components 102 or open the closed components 102 as needed. Thus, based on the determination of the states of the components 102, the state determination control unit 112 can automatically operate the components 102, such as via the actuators 114, to selectively move the components 102 between the open state and the closed state. Alternatively, the state determination control unit 112 is not configured to automatically operate the component(s) 102.

[0053] Alternatively, the state determination control unit 112 may include or otherwise be coupled to a memory that stores image data regarding the component 102 in the open and closed states (and optionally, full, partially full, empty, latched, and unlatched states). In this embodiment, the state determination control unit 112 compares the image data 122 with the stored image data to determine whether the component(s) 102 are in the open or closed state.

[0054] Figure 2 A flow chart illustrating a method for determining a status of one or more components within an interior cabin of a vehicle according to an embodiment of the subject disclosure is shown. Figure 1 and Figure 2 At 200, the imaging device 108 obtains an image of one or more components 102 within the interior cabin 104. At 202, the state determination control unit 112 determines a region of interest within the image data 122 comprising the image. At 204, the state determination control unit 112 corrects the perspective of the image data 122 to provide a perspective-corrected image. At 206, the state determination control unit 112 extrapolates the individual components 102 within the perspective-corrected image. At 208, the state determination control unit 112 determines a state of each component 102 based on the clustering.

[0055] Figure 3 A schematic block diagram of a state determination control unit 112 according to an embodiment of the subject disclosure is shown. In at least one embodiment, the state determination control unit 112 includes at least one processor 300 in communication with a memory 302. The memory 302 stores instructions 304, received data 306, and generated data 308. Figure 3 The illustrated state determination control unit 112 is merely exemplary and non-limiting.

[0056] As used herein, the terms "control unit," "central processing unit," "unit," "CPU," "computer," and the like may include any processor-based or microprocessor-based system, including systems using microcontrollers, reduced instruction set computers (RISC), application-specific integrated circuits, logic circuits, and any other circuits or processors comprising hardware, software, or a combination thereof capable of performing the functions described herein. These are exemplary only and, therefore, are not intended to limit in any way the definition and / or meaning of such terms. For example, the state determination control unit 112 may be or include one or more processors configured to control its operation, as described herein.

[0057] The state determination control unit 112 is configured to execute an instruction set stored in one or more data storage units or elements (such as one or more memories) in order to process data. For example, the state determination control unit 112 may include or be coupled to one or more memories. The data storage unit may also store data or other information as desired or needed. The data storage unit may be in the form of a physical storage element within an information source or a processing machine. The one or more data storage units or elements may include volatile memory or non-volatile memory, or may include both volatile memory and non-volatile memory. As an example, the non-volatile memory may include a read-only memory (ROM), a programmable ROM (PROM), an electrically programmable ROM (EPROM), an electrically erasable PROM (EEPROM) and / or flash memory, and the volatile memory may include a random access memory (RAM), which may act as an external cache memory. The data storage of the disclosed systems and methods is intended to include, but is not limited to, these and any other suitable types of memory.

[0058] The instruction set may include various commands that direct the state determination control unit 112 as a processor to perform specific operations (such as the methods and processes of the various embodiments of the subject matter described herein). The instruction set may be in the form of a software program. The software may be in various forms such as system software or application software. In addition, the software may be in the form of a collection of separate programs, a subset of programs within a larger program, or a portion of a program. The software may also include modular programming in the form of object-oriented programming. The processing of input data by the processor may be in response to user commands, or in response to the results of previous processing, or in response to a request made by another processor.

[0059] The figures of the embodiments herein may illustrate one or more control or processing units, such as the state determination control unit 112. It should be understood that the processing or control unit may represent a circuit, circuit system, or portion thereof, which may be implemented as hardware with associated instructions (e.g., software stored on a tangible, non-transitory computer-readable storage medium (such as a computer hard drive, ROM, RAM, etc.)) that perform the operations described herein. The hardware may include a state machine circuit system that is hardwired to perform the functions described herein. Alternatively, the hardware may include an electronic circuit that includes and / or is connected to one or more logic-based devices, such as a microprocessor, a processor, a controller, etc. Alternatively, the state determination control unit 112 may represent a processing circuit system, such as one or more of a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), (one or more) microprocessors, etc. In various embodiments, the circuit may be configured to execute one or more algorithms to perform the functions described herein. The one or more algorithms may include aspects of the embodiments disclosed herein, whether or not explicitly identified in a flowchart or method.

[0060] As used herein, the terms "software" and "firmware" are interchangeable and include any computer program stored in a data storage unit (e.g., one or more memories), including RAM memory, ROM memory, EPROM memory, EEPROM memory, and non-volatile RAM (NVRAM) memory, for execution by a computer. The above data storage unit types are exemplary only and, therefore, do not limit the types of memories that may be used to store computer programs.

[0061] Figure 4 An image 400 of a portion of the interior cabin 104 is shown according to an embodiment of the subject disclosure. Figure 1 and Figure 4 , image 400 is image data 122 obtained by imaging device 108 or otherwise a portion thereof. Image 400 depicts a plurality of components 102. As shown, components 102 are overhead storage assemblies 402 arranged in row 404. Image 400 depicts a three-dimensional space within interior cabin 104. Thus, image 400 provides perspective, such that a first overhead storage assembly 402a, which is further away, appears smaller than a second overhead storage assembly 402b, which is closer.

[0062] Figure 5 A perspective corrected image 500 of a region of interest 502 within the interior cabin 104 including the row 404 of overhead storage bin assemblies 402 is shown in accordance with an embodiment of the subject disclosure. Figure 1 、 Figure 4 and Figure 5After determining the region of interest 502 from the image 400 , the state determination control unit 112 provides the perspective-corrected image 500 such that each overhead storage bin assembly 402 has a similar shape and size within the perspective-corrected image 500 .

[0063] Figure 6 A perspective-corrected image 500 is shown having extrapolated cabinet shapes 600 corresponding to overhead storage bin assemblies 402, according to an embodiment of the subject disclosure. For example, during calibration, rectangles 602 corresponding to the two-dimensional front faces of the overhead storage bin assemblies 402 are overlaid on the corresponding overhead storage bin assemblies 402. Adjacent rectangles 602 are then placed next to the initial rectangle 602, and so on, to associate each overhead storage bin assembly 402 within an associated cabinet shape 600. In this manner, each overhead storage bin assembly 402 is associated with an associated cabinet shape 600. Each cabinet shape 600 is then associated with the location of a specific overhead storage bin assembly 402. Thus, each cabinet shape 600 can be labeled with the location of a specific overhead storage bin assembly 402.

[0064] Figure 7 A perspective-corrected image 500 is shown having a first cluster 702a of cabinet shapes 602a corresponding to a first group of overhead storage bin assemblies 402a and a second cluster 702b of cabinet shapes 602b corresponding to a second group of overhead storage bin assemblies 402b, in accordance with an embodiment of the subject disclosure. The first group of overhead storage bin assemblies 402a and the second group of overhead storage bin assemblies 402b may each include one or more overhead storage bin assemblies 402a and 402b.

[0065] Reference Figure 1 and Figure 7The state determination control unit 112 distinguishes the first cluster 702a from the second cluster 702b based on one or more differences in image attributes. For example, the first cluster 702a has a first image attribute 703 (such as a first color or color intensity), and the second cluster 702b has a second image attribute 705 (such as a second color or color intensity different from the first color or color intensity). The state determination control unit 112 detects the difference between the first image attribute 703 and the second image attribute 705 and, therefore, determines that the first group of overhead storage assemblies 402a is in a different state than the second group of overhead storage assemblies 402b. In at least one embodiment, the state determination control unit 112 is programmed to associate the second image attribute 705 with an open state. Therefore, the state determination control unit 112 determines that the first group of overhead storage assemblies 402a is in a closed state, while the second group of overhead storage assemblies 402b is in an open state. The state determination control unit 126 may then output a state signal 126 to the user device 116 indicating the states of the associated overhead storage assemblies 402a and 402b. Additionally, the state determination control unit 126 may output a component control signal 128 to the actuator 114 of the overhead storage bin assembly(s) 402b to automatically close the overhead storage bin assembly(s) 402b.

[0066] Figure 8 An image 800 of a portion of the interior cabin 104 is shown according to an embodiment of the subject disclosure. Figure 1 and Figure 8 In at least one embodiment, to determine the region of interest, the state determination control unit 112 analyzes the image 800 to determine a plurality of edges 802 of various features within the image 800. The edges 802 extend toward a vanishing point of the image 800.

[0067] Figure 9 The embodiment according to the present disclosure is shown Figure 8 The image 800 of the portion of the interior cabin 104 is a vanishing point 804. The edges 802 converge toward and intersect the vanishing point 804. Figure 1 and Figure 9 , the state determination control unit 112 determines a vanishing point 804 at the intersection of the edge 802 .

[0068] Next, state determination control unit 112 determines a bisecting line 806 passing through vanishing point 804. Bisecting line 806 is parallel to side edge 808 of image 800 and perpendicular to bottom edge 810 and top edge 812 of image 800. Bisecting line 806 divides image 800 into a first half 814 and a second half 816, each half being associated with a separate region of interest. Thus, the region of interest is determined by bisecting line 806.

[0069] Figure 10 The embodiment according to the present disclosure is shown Figure 8 The first region of interest 818 and the second region of interest 820 of the image 800 of the portion of the interior cabin 104 are shown. Figure 1 、 Figure 9 Figure 10 , the state determination control unit 112 separates the first region of interest 818 from the second region of interest 820 by a bisecting line 806 passing through the vanishing point 804 .

[0070] Figure 11 A region of interest 818 of image 800 is shown according to an embodiment of the subject disclosure. Figure 1 and Figure 11 , the state determination control unit 112 determines a region of interest 818 to be analyzed, such as via calibration, user input, etc. The region of interest 818 includes a row 901 of components 102 (such as the overhead storage bin assembly 402).

[0071] The state determination control unit 112 generates a first end line 902 at one end of the row 901. For example, the first end line 902 may be formed at a predetermined distance from the vanishing point 804 (eg, Figure 9 (as shown). The state determination control unit 112 generates a second end line 904 opposite the first end line 902. The second end line 904 may be located at the opposite end of the row 901. The state determination control unit 112 generates an upper boundary line 906 associated with the upper edge of the row 901 and a lower boundary line 908 associated with the lower edge of the row 902. The first end line 902 and the upper boundary line 906 intersect at a first corner 920. The first end line 902 and the lower boundary line 908 intersect at a second corner 922. The second end line 904 and the upper boundary line 906 intersect at a third corner 924. The second end line 904 and the lower boundary line 908 intersect at a fourth corner 926. The first corner 920, the second corner 922, the third corner 924, and the fourth corner 926 are corners of a quadrilateral (such as a trapezoid) that defines the overhead storage bin assemblies 402 of the row 901.

[0072] Figure 12 The embodiment according to the present disclosure is shown Figure 11 The perspective corrected image 950 of the row 901 of overhead storage bin assemblies 402 within the first region of interest 818 is shown. Figure 1 、 Figure 11 and Figure 12, the state determination control unit 112 geometrically transforms the quadrilateral defined by corners 920, 922, 924, and 926 into a rectangle having corresponding corners 1020, 1022, 1024, and 1026 to form a perspective-corrected image 950. The perspective-corrected image 950 includes the overhead storage bin components 402 of the row 901, each of which has the same or substantially the same shape. The state determination control unit 112 then extrapolates the bin shape and associated labels of each overhead storage bin component 402, such as the bin shape of the overhead storage bin component 402. Figure 6 As stated.

[0073] The uniform size of each cabinet shape of the associated overhead storage cabinet assembly reduces the complexity of the analysis and, therefore, reduces computing power. In this manner, the state determination control unit 112 operates with increased efficiency. In general, the uniform size of the cabinet shapes enables improved feature extraction for cabinet state determination.

[0074] Embodiments of the subject disclosure provide systems and methods that allow computing devices to quickly and efficiently analyze large amounts of data. For example, the interior cabin of a vehicle may include many components that may move between open and closed states, and that may be overlooked by crew members and other individuals within the cabin. Thus, the state determination control unit 112 is tracking and analyzing a large amount of data. As described above, the large amount of data is efficiently organized and / or analyzed by the state determination control unit 112. The state determination control unit 112 analyzes the data in a relatively short period of time in order to quickly and efficiently output information about which specific components are open or closed. It would be impossible for a human to effectively analyze such a large amount of data in such a short period of time. Thus, embodiments of the subject disclosure provide increased and efficient functionality, as well as significantly superior performance with respect to the person analyzing the data.

[0075] In at least one embodiment, components of system 100 , such as state determination control unit 112 , provide a specialized computer system and / or enable a computer system to operate as a specialized computer system for detecting the state of components within the interior cabin.

[0076] Figure 13A front perspective view of a vehicle 106 is shown, according to an exemplary embodiment of the subject disclosure. In at least one embodiment, the vehicle 106 is an aircraft. The aircraft 106 includes a propulsion system 1112 that may include, for example, two engines 1114. Optionally, the propulsion system 1112 may include more engines 1114 than shown. The engines 1114 are carried by wings 1116 of the aircraft 106. In some embodiments, the engines 1114 may be carried by a fuselage 1118 and / or an empennage 1120. The empennage 1120 may also support a horizontal stabilizer 1122 and a vertical stabilizer 1124. The fuselage 1118 of the aircraft 106 defines an interior cabin that may include a cockpit 1130, one or more work areas (e.g., a galley, a carry-on luggage area, etc.), and / or one or more passenger areas.

[0077] Alternatively, instead of aircraft, embodiments of the subject disclosure may be used with various other vehicles, such as automobiles, buses, locomotive and train cars, ships, etc. Additionally, embodiments of the subject disclosure may be used with fixed structures, such as commercial and residential buildings.

[0078] refer to Figures 1-13 Embodiments of the subject disclosure provide systems and methods for automatically determining the status of components within the interior compartment of a vehicle. Additionally, embodiments of the subject disclosure provide systems and methods for automatically determining whether a storage locker assembly within the interior compartment of a vehicle is open or closed. Furthermore, the systems and methods operate through image analysis and may not require separate sensors and wiring. Thus, embodiments of the subject disclosure provide systems and methods that reduce manufacturing complexity and cost, reduce the overall weight of a vehicle, and increase the fuel efficiency of the vehicle.

[0079] Additionally, the present disclosure includes embodiments according to the following clauses:

[0080] Clause 1. A system comprising:

[0081] one or more components, the one or more components being within an interior compartment of the vehicle;

[0082] an imaging device configured to obtain images of the one or more components; and

[0083] a state determination control unit comprising a processor, wherein the state determination control unit communicates with the imaging device,

[0084] wherein the state determination control unit receives image data including the image from the imaging device, and

[0085] The state determination control unit determines the state of the one or more components based on the image data.

[0086] Clause 2. The system of clause 1, wherein the state comprises one or both of an open state or a closed state.

[0087] Clause 3. The system of clause 1 or 2, wherein the one or more components are one or more overhead storage bin assemblies.

[0088] Clause 4. The system of any one of clauses 1-3, wherein the state determination control unit determines a region of interest within the image data.

[0089] Clause 5. The system according to any one of clauses 1-4, wherein the state determination control unit determines the region of interest by determining a vanishing point within the image.

[0090] Clause 6. The system according to any one of clauses 1-5, wherein the state determination control unit determines a bisection line passing through the vanishing point, wherein the region of interest is determined by the bisection line.

[0091] Clause 7. The system of any one of clauses 1-6, wherein the state determination control unit corrects for perspective within the image data.

[0092] Clause 8. A system according to any one of clauses 1-7, wherein the one or more components include multiple components in a row, wherein the state determination control unit generates a first end line at a first end of the row and a second end line at a second end of the row opposite to the first end line, wherein the state determination control unit also generates an upper boundary line associated with the upper edge of the row and a lower boundary line associated with the lower edge of the row, wherein the first end line and the upper boundary line intersect at a first corner, wherein the first end line and the lower boundary line intersect at a second corner, wherein the second end line and the upper boundary line intersect at a third corner, wherein the second end line and the lower boundary line intersect at a fourth corner, and wherein the first corner, the second corner, the third corner and the fourth corner are corners of a quadrilateral that defines the multiple components.

[0093] Clause 9. The system of any one of clauses 1-8, wherein the state determination control unit geometrically transforms the quadrilateral into a rectangle having respective corresponding corners to form a perspective-corrected image.

[0094] Clause 10. The system of any of clauses 1-9, wherein the state determination control unit extrapolates a shape of each of the one or more components within the image data.

[0095] Clause 11. The system according to any one of clauses 1-10, wherein the state determination control unit determines the state of the one or more components by clustering.

[0096] Clause 12. The system of any one of clauses 1-11, wherein the state determination control unit determines the state of the one or more components by determining differences in image attributes.

[0097] Clause 13. The system of any one of clauses 1-12, further comprising a user device, wherein the state determination control unit outputs a state signal indicating a state of the one or more components to the user device.

[0098] Clause 14. The system of any one of clauses 1-13, wherein the state determination control unit outputs a component control signal that automatically operates the one or more components based on the state of the one or more components.

[0099] Clause 15. A method comprising:

[0100] obtaining, by an imaging device, an image of one or more components within an interior cabin of the vehicle; and

[0101] receiving, by a state determination control unit including a processor in communication with the imaging device, image data including the image from the imaging device; and

[0102] The status of the one or more components is determined by the status determination control unit based on the image data.

[0103] Clause 16. The method of clause 15, wherein the state comprises one or both of an open state or a closed state.

[0104] Clause 17. The method of Clause 15 or 16, wherein the one or more components are one or more overhead storage bin assemblies.

[0105] Clause 18. The method of any of clauses 15-17, wherein the determining comprises determining a region of interest within the image data.

[0106] Clause 19. The method of any of clauses 15-18, wherein the determining further comprises determining the region of interest by determining a vanishing point within the image.

[0107] Clause 20. The method of any one of clauses 15-19, wherein the determining further comprises determining a bisecting line passing through the vanishing point, and determining the region of interest via the bisecting line.

[0108] Clause 21. The method of any one of clauses 15-20, further comprising correcting, by the state determination control unit, perspective within the image data.

[0109] Clause 22. The method of any of clauses 15-21, wherein the one or more components include a plurality of components in a row, and wherein the determining comprises:

[0110] generating a first end line at a first end of the row and a second end line at a second end of the row opposite to the first end line;

[0111] An upper boundary line associated with the upper edge of the row and a lower boundary line associated with the lower edge of the row are generated, wherein the first end line and the upper boundary line intersect at a first corner, wherein the first end line and the lower boundary line intersect at a second corner, wherein the second end line and the upper boundary line intersect at a third corner, wherein the second end line and the lower boundary line intersect at a fourth corner, and wherein the first corner, the second corner, the third corner, and the fourth corner are corners of a quadrilateral that defines the plurality of components.

[0112] Clause 23. The method of any one of clauses 15-22, wherein the determining further comprises geometrically transforming the quadrilateral into a rectangle having respective corresponding corners to form a perspective-corrected image.

[0113] Clause 24. The method of any of clauses 15-23, wherein the determining further comprises extrapolating a shape of each of the one or more components within the image data.

[0114] Clause 25. The method of any one of clauses 15-24, wherein the determining comprises determining the status of the one or more components by clustering.

[0115] Clause 26. The method of any of clauses 15-25, wherein the determining comprises determining a difference in image attributes.

[0116] Clause 27. The method of any of clauses 15-26, further comprising outputting, by the status determination control unit, a status signal indicating a status of the one or more components to a user device.

[0117] Clause 28. The method of any one of clauses 15-27, further comprising outputting, by the state determination control unit, a component control signal that automatically operates the one or more components based on the state of the one or more components.

[0118] Clause 29. A non-transitory computer-readable storage medium comprising executable instructions that, in response to execution, cause a system comprising a processor to perform operations comprising:

[0119] receiving image data comprising images of one or more components within an interior cabin of a vehicle from an imaging device within the interior cabin; and

[0120] A status of the one or more components is determined based on the image data.

[0121] Clause 30. The non-transitory computer-readable storage medium of Clause 29, wherein the determining comprises determining a region of interest within the image data.

[0122] Clause 31. The non-transitory computer-readable storage medium of clause 29 or 30, further comprising correcting perspective within the image data.

[0123] Clause 32. The non-transitory computer-readable storage medium of any of clauses 29-31, wherein the determining further comprises extrapolating a shape of each of the one or more components within the image data.

[0124] Clause 33. The non-transitory computer-readable storage medium of any of clauses 29-32, wherein the determining further comprises determining the status of the one or more components by clustering.

[0125] Although various spatial and directional terms may be used to describe embodiments of the subject disclosure, such as top, bottom, lower, middle, lateral, horizontal, vertical, front, etc., it should be understood that these terms are used only with respect to the orientations shown in the accompanying drawings. The orientation may be reversed, rotated, or otherwise changed so that top becomes bottom and vice versa, horizontal becomes vertical, etc.

[0126] As used herein, a structure, limitation, or element that is "configured to" perform a task or operation is specifically structurally formed, constructed, or adapted in a manner corresponding to the task or operation. For clarity and the avoidance of doubt, an object that is merely capable of being modified to perform a task or operation is not "configured to" perform a task or operation as used herein.

[0127] It should be understood that the above description is intended to be illustrative, not restrictive. For example, the above embodiments (and / or aspects thereof) can be used in combination with each other. In addition, many modifications can be made to adapt specific situations or materials to the teachings of the various embodiments of the present disclosure without departing from its scope. Although the size and type of the materials described herein are intended to limit the parameters of the various embodiments of the present disclosure, the embodiments are by no means restrictive and are exemplary embodiments. After reviewing the above description, many other embodiments will be apparent to those skilled in the art. Therefore, the scope of the various embodiments of the present disclosure should be determined with reference to the appended claims and the full scope of equivalents to which such claims are entitled. In the appended claims and the detailed description herein, the terms "including" and "wherein" are used as the simple English equivalents of the corresponding terms "comprising" and "wherein". In addition, the terms "first", "second" and "third" etc. are used only as labels and are not intended to impose numerical requirements on their objects.

[0128] This written description uses examples to disclose various embodiments of the disclosure, including the best mode, and also to enable any person skilled in the art to practice the various embodiments of the disclosure, including making and using any devices or systems and performing any incorporated methods. The patentable scope of the various embodiments of the disclosure is defined by the claims, and may include other examples that occur to those skilled in the art. Such other examples are intended to be within the scope of the claims if the examples have structural elements that do not differ from the literal language of the claims, or if the examples include equivalent structural elements with insubstantial differences from the literal language of the claims.

Claims

1. A system (100), comprising: One or more components (102), the one or more components (102) being within an interior compartment (104) of a vehicle (106); an imaging device (108) configured to obtain an image of the one or more components (102); as well as a state determination control unit (112), the state determination control unit (112) comprising a processor (300), wherein the state determination control unit (112) communicates with the imaging device (108), wherein the state determination control unit (112) receives image data (122) including the image from the imaging device (108), and wherein the state determination control unit (112) determines the state of the one or more components (102) based on the image data (122), and wherein the one or more components (102) include a plurality of components (102) in a row, wherein the state determination control unit (112) generates a first end line at a first end of the row and a second end line at a second end of the row opposite to the first end line, wherein the state determination control unit (112) further generates an upper boundary line associated with an upper edge of the row and a lower boundary line associated with a lower edge of the row, wherein the first end line and the upper boundary line intersect at a first corner, wherein the first end line and the lower boundary line intersect at a second corner, wherein the second end line and the upper boundary line intersect at a third corner, wherein the second end line and the lower boundary line intersect at a fourth corner, and wherein the first corner, the second corner, the third corner, and the fourth corner are corners of a quadrilateral that defines the plurality of components (102).

2. The system (100) of claim 1, wherein the state comprises one or both of an open state and a closed state.

3. The system (100) of claim 1 or 2, wherein the one or more components (102) are one or more overhead storage bin assemblies (402).

4. The system (100) according to claim 1 or 2, wherein the state determination control unit (112) determines a region of interest within the image data (122).

5. The system (100) according to claim 4, wherein the state determination control unit (112) determines the region of interest by determining a vanishing point within the image.

6. The system (100) according to claim 5, wherein the state determination control unit (112) determines a bisection line passing through the vanishing point, wherein the region of interest is determined by the bisection line.

7. The system (100) of claim 1 or 2, wherein the state determination control unit (112) corrects for perspective within the image data (122).

8. The system (100) of claim 7, wherein the state determination control unit (112) geometrically transforms the quadrilateral into a rectangle (602) having corresponding corners to form a perspective-corrected image.

9. The system (100) of claim 1, wherein the state determination control unit (112) extrapolates a shape of each of the one or more components (102) within the image data (122).

10. The system (100) of claim 1, wherein the state determination control unit (112) determines the state of the one or more components (102) by clustering.

11. The system (100) of claim 1, wherein the state determination control unit (112) determines the state of the one or more components (102) by determining differences in image attributes.

12. The system (100) of claim 1, further comprising a user device, wherein the state determination control unit (112) outputs a state signal indicating a state of the one or more components (102) to the user device.

13. The system (100) of claim 1, wherein the state determination control unit (112) outputs a component control signal that automatically operates the one or more components (102) based on the state of the one or more components (102).

14. A method comprising: receiving, by a state determination control unit (112) including a processor (300) in communication with the imaging device (108), image data (122) from the imaging device (108), wherein the image data (122) includes at least one image of one or more components (102) within the aircraft; and determining, by the state determination control unit (112), the state of the one or more components (102) based on the image data (122), wherein the state comprises one or both of an open state or a closed state, and wherein the one or more components (102) are one or more overhead storage bin assemblies (402), and wherein the one or more components (102) comprise a plurality of components (102) in a row, and Wherein said determining comprises: generating a first end line at a first end of the row and a second end line at a second end of the row opposite to the first end line; An upper boundary line associated with the upper edge of the row and a lower boundary line associated with the lower edge of the row are generated, wherein the first end line and the upper boundary line intersect at a first corner, wherein the first end line and the lower boundary line intersect at a second corner, wherein the second end line and the upper boundary line intersect at a third corner, wherein the second end line and the lower boundary line intersect at a fourth corner, and wherein the first corner, the second corner, the third corner, and the fourth corner are corners of a quadrilateral that defines the plurality of components (102).

15. The method of claim 14, wherein the determining comprises: determining a region of interest within the image data (122); determining the region of interest by determining a vanishing point within the image; determining a bisector passing through the vanishing point; as well as The region of interest is determined by the bisection line.

16. The method of claim 14, further comprising correcting, by the state determination control unit (112), perspective within the image data (122).

17. The method of claim 14, wherein the determining further comprises: The quadrilateral is geometrically transformed into a rectangle having corresponding corners (602) to form a perspective-corrected image.

18. The method of claim 14, wherein the determining further comprises: extrapolating a shape of each of the one or more components (102) within the image data (122); determining a state of the one or more components (102) by clustering; Determine differences in image properties.

19. The method according to claim 14, further comprising: Outputting, by the state determination control unit (112), a state signal indicating the state of the one or more components (102) to a user device; as well as A component control signal is outputted by the state determination control unit (112), the component control signal automatically operating the one or more components (102) based on the state of the one or more components (102).

20. A non-transitory computer-readable storage medium comprising executable instructions that, in response to execution, cause a system comprising a processor (300) to perform operations comprising: receiving image data (122) including images of one or more components (102) within an interior cabin (104) of a vehicle (106) from an imaging device (108) within the interior cabin (104); and determining a state of the one or more components (102) based on the image data (122), and wherein the one or more components (102) include a plurality of components (102) in a row, and Wherein said determining comprises: generating a first end line at a first end of the row and a second end line at a second end of the row opposite to the first end line; An upper boundary line associated with the upper edge of the row and a lower boundary line associated with the lower edge of the row are generated, wherein the first end line and the upper boundary line intersect at a first corner, wherein the first end line and the lower boundary line intersect at a second corner, wherein the second end line and the upper boundary line intersect at a third corner, wherein the second end line and the lower boundary line intersect at a fourth corner, and wherein the first corner, the second corner, the third corner, and the fourth corner are corners of a quadrilateral that defines the plurality of components (102).

21. The non-transitory computer-readable storage medium of claim 20, wherein the determining comprises determining a region of interest within the image data (122).

22. The non-transitory computer-readable storage medium of claim 20, further comprising correcting perspective within the image data (122).

23. The non-transitory computer-readable storage medium of claim 20, wherein the determining further comprises extrapolating a shape of each of the one or more components (102) within the image data (122).

24. The non-transitory computer-readable storage medium of claim 20, wherein the determining further comprises determining the status of the one or more components (102) by clustering.

Images