System and method for determining the status of a storage bin assembly within the interior cabin of a vehicle

Abstract

To provide systems and methods for determining a state of components, for example, stowage bin assemblies, within an internal cabin of a vehicle.SOLUTION: A system 100 includes one or more components 102 within an internal cabin 104 of a vehicle 106. An imaging device 108 obtains an image of the one or more components 102. A state determination control unit 112 includes a processor. The state determination control unit 112 is in communication with the imaging device 108 and receives image data 122 including the image from the imaging device 108. Further, the state determination control unit 112 determines a state of the one or more components 102 based on the image data 122.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] SUMMARY OF THE INVENTION Embodiments of the present disclosure generally relate to systems and methods for determining the status of equipment within the interior cabin of a vehicle, such as a storage bin assembly. [Background technology]

[0002] Aircraft are used to transport passengers and cargo between various regions. The interior cabin of an aircraft includes numerous pieces of equipment that can be moved between an open state and a closed state. For example, an overhead luggage bin assembly in the interior cabin includes a shelf that can be moved between an open state and a closed state. As another example, an exit door can be moved between an open state and a closed state. As another example, a lavatory door in the interior cabin can be moved between an open state and a closed state. As another example, a galley cart compartment can be in an open state in which a galley cart can be moved into the compartment, and in a closed state in which a galley cart is already present in the compartment. In general, the interior cabin of a commercial aircraft vehicle includes various pieces of equipment that can be moved between an open state and a closed state.

[0003] During various stages of a flight, certain equipment is in a specific state. For example, during check-in, luggage bin assemblies are in an open state, allowing passengers to stow their carry-on luggage and other carry-on items. As departure time approaches, flight attendants typically monitor the luggage bin assemblies to ensure that each bin is securely closed before the aircraft moves to the taxiway and / or runway. That is, individuals, such as flight attendants, visually determine whether a luggage bin assembly is open. As can be appreciated, the process of walking through the interior cabin and visually determining which luggage bin assemblies are open is time-consuming. Furthermore, if a passenger needs assistance, for example, the flight attendant may become distracted and overlook an open bin assembly.

[0004] Some luggage rack assemblies include sensors that are connected to monitoring devices via wiring, or are otherwise connected to the sensors. However, the sensors and wiring add weight to the aircraft. They also increase manufacturing complexity because they must be positioned and / or routed. Furthermore, the additional weight from the sensors and wiring reduces fuel efficiency. Summary of the Invention

[0005] A need exists for a system and method for automatically determining the status of equipment within the interior cabin of a vehicle. Further, a need exists for a system and method for automatically determining whether a luggage rack assembly within the interior cabin of a vehicle is open or closed. Additionally, a need exists for a system and method for monitoring the status of equipment within the interior cabin that does not add substantial weight to the vehicle.

[0006] With these needs in mind, certain embodiments of the present disclosure provide a system that includes one or more pieces of equipment within an interior cabin of a vehicle, an imaging device configured to capture images of the one or more pieces of equipment, and a state determination control unit including a processor, the state determination control unit in communication with the imaging device, the state determination control unit receiving image data from the imaging device that includes the images, and determining a state of the one or more pieces of equipment based on the image data.

[0007] In at least one embodiment, the state includes one or both of an open state or a closed state.In at least one embodiment, the one or more pieces of furniture are one or more overhead bin assemblies.

[0008] In at least one embodiment, the state determination control unit determines a region of interest in the image data. For example, the state determination control unit determines the region of interest by determining a vanishing point in the image. As a further example, the state determination control unit determines a bisector passing through the vanishing point. The region of interest is determined by the bisector.

[0009] In at least one embodiment, the state determination control unit corrects perspective in the image data. For example, the one or more pieces of equipment include a plurality of pieces of equipment lined up in a row. The state determination control unit generates a first terminal line at a first end of the row and a second terminal line at a second end of the row opposite the first terminal line. The state determination control unit further generates an upper boundary line associated with an upper edge of the row and a lower boundary line associated with a lower edge of the row. The first terminal line and the upper boundary line intersect at a first corner. The first terminal line and the lower boundary line intersect at a second corner. The second terminal line and the upper boundary line intersect at a third corner. The second terminal line and the lower boundary line intersect at a fourth corner. The first corner, the second corner, the third corner, and the fourth corner are corners of a quadrangle that defines the plurality of pieces of equipment. A state determination control unit geometrically transforms the quadrilaterals into rectangles with corresponding corners to form a perspective-corrected image.

[0010] In at least one embodiment, the state determination control unit extrapolates the shape of each of the one or more pieces of equipment in the image data.

[0011] In at least one embodiment, a state determination control unit determines the state of one or more pieces of equipment through clustering.

[0012] In at least one embodiment, a state determination control unit determines the state of one or more pieces of equipment by determining differences in image attributes.

[0013] In at least one example, the system further includes a user device, and the state determination control unit outputs a state signal to the user device indicative of the state of the one or more pieces of equipment.

[0014] In at least one example, the state determination control unit outputs an equipment control signal that automatically operates one or more equipment items based on the state of the one or more equipment items.

[0015] Certain embodiments of the present disclosure are directed to a method, comprising: capturing, with an imaging device, an image of one or more items within an interior cabin of the vehicle; receiving, by a state determination control unit including a processor and in communication with the imaging device, image data including the image from the imaging device; determining, by a status determination control unit, a status of the one or more pieces of equipment based on the image data; The present invention provides a method comprising:

[0016] Certain embodiments of the present disclosure provide a non-transitory computer-readable storage medium that includes executable instructions, the non-transitory computer-readable storage medium comprising: The instructions, upon execution, cause a system including a processor to: receiving image data from an imaging device within the interior cabin of the vehicle, the image data including an image of one or more items within the interior cabin; Determining the status of one or more pieces of equipment based on image data A non-transitory computer-readable storage medium is provided for causing a computer to perform operations including: [Brief explanation of the drawings]

[0017] [Figure 1] 1 illustrates a schematic block diagram of a system for determining the status of one or more pieces of equipment within an interior cabin of a vehicle, according to one embodiment of the present disclosure. [Figure 2] 1 illustrates a flowchart of a method for determining the status of one or more pieces of equipment within an interior cabin of a vehicle according to one embodiment of the present disclosure. [Figure 3] FIG. 2 illustrates a schematic block diagram of a state determination control unit according to an embodiment of the present disclosure. [Figure 4] 13 shows an image of a portion of an interior cabin according to one embodiment of the present disclosure. [Figure 5] 1 illustrates a perspective-corrected image of an area of ​​interest including a row of overhead bin assemblies within an interior cabin, according to one embodiment of the present disclosure. [Figure 6]1 illustrates a perspective-corrected image in which the shape of a shelf corresponding to an overhead luggage bin assembly has been extrapolated, according to one embodiment of the present disclosure. [Figure 7] 1 illustrates a perspective-corrected image having a first cluster of shelf shapes corresponding to a first group of overhead luggage bin assemblies and a second cluster of shelf shapes corresponding to a second group of overhead luggage bin assemblies, according to one embodiment of the present disclosure. [Figure 8] 13 shows an image of a portion of an interior cabin according to one embodiment of the present disclosure. [Figure 9] 9 illustrates vanishing points within an image of the portion of the interior cabin of FIG. 8 according to one embodiment of the present disclosure. [Figure 10] 9 illustrates a first region of interest and a second region of interest of the image of the portion of the interior cabin of FIG. 8 according to one embodiment of the present disclosure. [Figure 11] 1 illustrates a first region of interest of an image according to one embodiment of the present disclosure. [Figure 12] 12 illustrates a perspective-corrected image of a row of overhead bin assemblies within a first area of ​​interest of FIG. 11 according to one embodiment of the present disclosure. [Figure 13] 1 is a front perspective view of a vehicle according to an exemplary embodiment of the present disclosure. FIG. DETAILED DESCRIPTION OF THE INVENTION

[0018] The foregoing summary and the following detailed description of certain embodiments will be better understood when read in conjunction with the accompanying drawings. As used herein, elements or steps described in the singular and preceded by the terms "a" or "an" do not necessarily exclude a plurality of elements or steps. Furthermore, references to "one embodiment" are not intended to be interpreted as excluding the existence of additional embodiments that also incorporate the recited features. Furthermore, unless expressly stated to the contrary, embodiments "comprising" or "having" an element or elements having particular conditions may include additional elements that do not have those conditions.

[0019] Certain embodiments of the present disclosure provide systems and methods for determining the status (e.g., open and closed) of equipment (e.g., bin assemblies) within the interior cabin of a vehicle (e.g., an aircraft). The systems and methods are configured to detect equipment status using in-cabin video cameras in real time, automatically log these events, trigger further procedures, and alert the flight crew with the event information.

[0020] Certain embodiments of the present disclosure provide systems and methods that use video images from on-board cameras to automatically detect and log cabin events, trigger future events, and alert the crew about event information. By processing multiple in-flight events, the systems and methods can detect potential abnormal events.

[0021] In at least one embodiment, the system and method includes viewing an in-cabin video camera input, determining a region of interest, calculating a vanishing point, applying algorithms to correct perspective, extrapolating shelf locations, clustering equipment to identify conditions, and alerting the crew of the conditions. By analyzing image data obtained from one or more imaging devices to detect equipment conditions, embodiments of the present disclosure are configured to function without sensors and wires, thereby reducing manufacturing costs, complexity, and fuel efficiency.

[0022] 1 illustrates a schematic block diagram of a system 100 for determining the status of one or more pieces of furniture 102 within an interior cabin 104 of a vehicle 106, according to one embodiment of the present disclosure. In at least one embodiment, the furniture 102 is an overhead bin assembly configured to move between an open and a closed state. In some examples, the furniture 102 is one or more doors within the interior cabin 104, such as a lavatory door or an exit door. In some examples, the furniture 102 is a compartment within the interior cabin 104, such as a galley cart compartment.

[0023] In at least one implementation, the vehicle 106 is an aircraft. In some examples, the vehicle 106 is a land-based vehicle such as a bus, car, etc. In some examples, the vehicle 106 is a watercraft or spacecraft.

[0024] The system 100 includes an imaging device 108 within an interior cabin 104. The imaging device 108 has a field of view 110. The equipment 102 is present within the field of view 110. In at least one embodiment, the imaging device 108 is a video camera. In some examples, the imaging device 108 is a night vision camera. In some examples, the imaging device 108 is an infrared camera.

[0025] The imaging device 108 may be a fixed camera within the interior cabin 104. For example, the imaging device 108 may be fixed to a ceiling, a wall, a monument, or the like within the interior cabin 104. In some examples, the imaging device 108 may be a handheld camera, such as a small camera, part of a smartphone, or a smart tablet.

[0026] The system 100 further includes a state determination control unit 112 that communicates with the image capture device 108. For example, the state determination control unit 112 communicates wirelessly with the image capture device 108, such as via Bluetooth, Wifi, or other such connection. Alternatively, the state determination control unit 112 may communicate with the image capture device 108 via a wired connection.

[0027] In at least one embodiment, the accessory 102 may include an actuator 114, such as an electric motor, configured to automatically move the accessory 102 between an open state and a closed state. Further, the state determination control unit 112 communicates wirelessly with the actuator 114 to control the opening and closing of the accessory 102. Optionally, the state determination control unit 112 communicates with the actuator 114 via a wired connection. Also, optionally, the state determination control unit 112 does not communicate with the actuator 114. Also, optionally, the accessory 102 may not include the actuator 114.

[0028] In at least one embodiment, a user device 116 is also present within the interior cabin 104. The user device 116 includes a display 118 connected to an interface 120. For example, the user device 116 is a computer workstation within the interior cabin 104. In some examples, the user device 116 is a mobile device such as a smartphone or smart tablet. The display 118 may be a monitor or screen. The interface 120 may include a keyboard, a mouse, or the like. In at least one embodiment, the display 118 and the interface 120 are both incorporated into a touch panel interface. The state determination control unit 112 communicates with the user device 116, for example, via a wired or wireless connection.

[0029] As described herein, the system 100 includes one or more pieces of equipment 102 within an interior cabin 104 of a vehicle 106. An imaging device 108 is configured to capture images of the one or more pieces of equipment 102. A state determination control unit 112 is in communication with the imaging device 108. The state determination control unit 112 receives image data 122 including the images from the imaging device 108. The state determination control unit 112 determines a state (e.g., open or closed) of the one or more pieces of equipment 102 based on the image data 122.

[0030] During operation, the imaging device 108 captures images of the equipment 102. The images may be video images or still images. The imaging device 108 captures the images of the equipment 102 as image data 122. The state determination control unit 112 receives the image data 122 from the imaging device 108, including the images of the equipment 102.

[0031] The state determination control unit 112 analyzes the image data 122 to determine the state of the equipment 102. For example, the state determination control unit 112 analyzes the image data 122 to determine whether the equipment 102 (e.g., an overhead bin assembly) is in an open state (i.e., open) or a closed state (i.e., closed).

[0032] In at least one embodiment, the state determination control unit 112 analyzes the image data 122 by determining an area of ​​interest within the image data 122. For example, in at least one embodiment, the area of ​​interest includes a row of equipment 102, such as a row of overhead bin assemblies within the interior cabin 104.

[0033] After determining the region of interest, the state determination control unit 112 performs perspective correction on the image data 122 to provide a perspective-corrected image of the image data 122. For example, the state determination control unit 112 corrects the perspective of the image in three-dimensional space. Alternatively, the state determination control unit 112 may not perform perspective correction on the image data 122.

[0034] The state determination control unit 112 then extrapolates the individual pieces of equipment 102 within the perspective-corrected images. For example, during initial calibration or setup, the state determination control unit 112 is programmed to sense the size and shape of each individual piece of equipment 102 within the image data 122 (e.g., within the perspective-corrected images). In at least one embodiment, the state determination control unit 112 extrapolates the shape of each piece of equipment 102 within the image data 122.

[0035] The state determination control unit 112 then determines the state of the equipment items 102 through clustering, for example, by determining differences in image attributes (e.g., differences in color, color intensity, and / or brightness). For example, the equipment items 102 in the perspective-corrected image are clustered based on their color differences. If there is a single cluster for the equipment items 102 in the perspective-corrected image, the state determination control unit 112 determines that all of the equipment items 102 are in the same state, for example, an open state or a closed state. In at least one embodiment, the state determination control unit 112 is calibrated to determine image attributes of the closed state. For example, an image of the equipment items 102 having a first color is determined to be in the closed state. In this manner, if a single cluster of equipment items 102 that all share the same attribute, such as the same color or the same color intensity, is associated with the closed state, the state determination control unit 112 determines that all of the equipment items 102 are closed. However, if the single cluster of the equipment 102 differs from the predetermined image attribute of the closed state, the state determination control unit 112 determines that the equipment 102 is in the open state.

[0036] Further, the image data 122 may indicate the presence of various clusters of images. For example, a first cluster corresponding to one or more pieces of equipment 102 in a first set has a first image attribute (e.g., a first color or intensity), and a second cluster corresponding to one or more pieces of equipment 102 in a second set has a second image attribute different from the first image attribute. Correspondingly, the state determination control unit 112 determines that one of the first set or the second set is in an open state and the other is in a closed state. The state determination control unit 112 determines which set is in a closed state or an open state based on the predetermined image attribute. For example, the state determination control unit 112 may be calibrated to detect a first image attribute as a closed state and, accordingly, detect a second image attribute different from the first image attribute as an open state, or vice versa.

[0037] The state determination control unit 112 associates the first cluster and the second cluster with the position of the equipment 102 within the interior cabin 104. For example, the state determination control unit 112 is calibrated, e.g., through an initial manual calibration or setup, to identify and label the position of each equipment 102 within the interior cabin 104. Correspondingly, based on image analysis, the state determination control unit 112 detects the state (e.g., open or closed) of the equipment 102 within the interior cabin 104.

[0038] In at least one embodiment, the state determination control unit 112 detects the state of the equipment 102 through clustering, as described herein. The state determination control unit 112 compares clusters of image data 122 to distinguish between open and closed states. In such an embodiment, the state determination control unit 112 does not compare the image data 122 with past image data for open and closed states. In this manner, the system 100 can operate using less memory (which would otherwise store past data) and less computing power, resulting in efficient operation.

[0039] The status determination control unit 112 then outputs a status signal 126 to the user device 116. The status signal 126 indicates the status of the furniture 102. The status signal 126 includes status data of the furniture 102 in the interior cabin 104. For example, the status signal 126 indicates which of the furniture 102 are open and closed. The status signal 126 is received by the user device 116, which can provide video or graphic data on the display 118, audio signals, etc., to alert the crew as to which furniture 102 are open and closed. In this manner, the status determination control unit 112 automatically detects the status of the furniture 102 and alerts the crew, who can then open or close the furniture 102 as desired.

[0040] In at least one embodiment, the state determination control unit 112 can automatically operate the equipment 102, e.g., via the actuator 114, based on the determined state of the equipment 102. For example, after the state determination control unit 112 analyzes the image data 122 to determine which equipment 102 is open and which is closed, the state determination control unit 112 outputs an equipment control signal 128 to the actuator 114 to selectively close the open equipment 102 or selectively open the closed equipment 102, as desired. Thus, based on the determination of the state of the equipment 102, the state determination control unit 112 can automatically operate the equipment 102, e.g., via the actuator 114, to selectively move the equipment 102 between the open state and the closed state. Alternatively, the state determination control unit 112 is not configured to automatically operate the equipment 102.

[0041] Alternatively, the state determination control unit 112 may include or be otherwise connected to a memory that stores image data regarding the equipment 102 in the open and closed states (and, optionally, full, partially full, empty, locked, and unlocked states). In this embodiment, the state determination control unit 112 compares the image data 122 against stored image data to determine whether the accessory 102 is in an open or closed state.

[0042] 2 illustrates a flowchart of a method for determining the state of one or more pieces of equipment 102 within an interior cabin of a vehicle, according to one embodiment of the present disclosure. Referring to FIGS. 1 and 2 , at 200, the imaging device 108 acquires an image of one or more pieces of equipment 102 within the interior cabin 104. At 202, the state determination control unit 112 determines an area of ​​interest within image data 122 that includes the image. At 204, the state determination control unit 112 corrects the perspective of the image data 122 to provide a perspective-corrected image. At 206, the state determination control unit 112 extrapolates individual pieces of equipment 102 within the perspective-corrected image. At 208, the state determination control unit 112 determines the state of each piece of equipment 102 based on clustering.

[0043] 3 illustrates a schematic block diagram of a state determination control unit 112 according to one embodiment of the present disclosure. In at least one embodiment, the state determination control unit 112 includes at least one processor 300 in communication with a memory 302. The memory 302 stores instructions 304, received data 306, and generated data 308. The state determination control unit 112 illustrated in FIG. 3 is merely exemplary and non-limiting.

[0044] As used herein, terms such as “control unit,” “central processing unit,” “unit,” “CPU,” “computer,” and the like may include any processor-based or microprocessor-based system, including systems that use microcontrollers, reduced instruction set computers (RISC), application specific integrated circuits (ASIC), logic circuits, and any other circuits or processors, including hardware, software, or a combination thereof, capable of performing the functions described herein. The above examples are illustrative only and thus are not intended to limit in any way the definition and / or meaning of such terms. For example, state determination control unit 112 may be or include one or more processors configured to control the operation of the state determination control unit as described herein.

[0045] The state determination control unit 112 is configured to execute sequences of instructions stored in one or more data storage units or elements (e.g., one or more memories) to process data. For example, the state determination control unit 112 may include or be connected to one or more memories. The data storage units may also store data or other information as desired or needed. The data storage units may be in the form of information sources or physical memory elements within the processing machine. The one or more data storage units or elements may include volatile memory or nonvolatile memory, or may include both volatile and nonvolatile memory. By way of example, nonvolatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable PROM (EEPROM), and / or flash memory, while volatile memory may include random access memory (RAM), which may function as external cache memory. The data stores of the disclosed systems and methods are intended to comprise, without being limited to, these and any other suitable types of memory.

[0046] The set of instructions may include various commands that instruct the state determination control unit 112, as a processing machine, to perform particular operations, such as the methods and processes of various embodiments of the inventive subject matter described herein. The set of instructions may be in the form of a software program. The software may be in various forms, such as system software or application software. Furthermore, the software may be in the form of a collection of separate programs, a subset of a program within a larger program, or a portion of a program. The software may also include modular programming in the form of object-oriented programming. The processing of input data by a processing machine may be in response to user commands, results of previous processing, or requests made by another processing machine.

[0047] Diagrams of embodiments herein may depict one or more control or processing units, such as the state determination control unit 112. It should be understood that a processing or control unit may be, or a portion thereof, a circuit that can be implemented as hardware containing associated instructions (e.g., software stored on a tangible, non-transitory computer-readable storage medium, such as a computer hard drive, ROM, RAM, etc.) that perform the operations described herein. The hardware may include a state machine circuit hardened to perform the functions described herein. Optionally, the hardware may include electronic circuitry including and / or coupled to one or more logic elements, such as a microprocessor, processor, controller, etc. Optionally, the state determination control unit 112 may be a processing circuit, such as one or more of a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), and / or a microprocessor. The circuitry in various embodiments may be configured to execute one or more algorithms to perform the functions described herein. The one or more algorithms may include aspects of the embodiments disclosed herein, whether or not explicitly identified in a flowchart or method.

[0048] As used herein, the terms "software" and "firmware" are used interchangeably and include any computer program for execution by a computer that is stored in a data storage unit (e.g., one or more memories), including RAM memory, ROM memory, EPROM memory, EEPROM memory, and non-volatile RAM (NVRAM) memory. The types of data storage units listed above are merely exemplary and are therefore not limiting as to the types of memory that may be used to store computer programs.

[0049] 4 illustrates an image 400 of a portion of the interior cabin 104, according to one embodiment of the present disclosure. Referring to FIGS. 1 and 4, the image 400 is image data 122 captured by the imaging device 108, or is otherwise a portion of the image data 122. The image 400 illustrates a plurality of pieces of equipment 102. As shown, the pieces of equipment 102 are overhead bin assemblies 402 arranged in a row 404. The image 400 illustrates a three-dimensional space within the interior cabin 104. Correspondingly, the image 400 provides a sense of perspective, such that a first overhead bin assembly 402a, which is farther away, appears smaller than a second overhead bin assembly 402b, which is closer.

[0050] 5 illustrates a perspective-corrected image 500 of an area of ​​interest 502 including a row 404 of overhead bin assemblies 402 within the interior cabin 104, according to one embodiment of the present disclosure. Referring to FIGS. 1, 4, and 5, after determining the area of ​​interest 502 from the image 400, the state determination control unit 112 provides the perspective-corrected image 500, such that each of the overhead bin assemblies 402 has a similar shape and size in the perspective-corrected image 500.

[0051] FIG. 6 illustrates a perspective-corrected image 500 in which shelf shapes 600 corresponding to overhead storage bin assemblies 402 have been extrapolated according to one embodiment of the present disclosure. For example, during calibration, a rectangle 602 corresponding to the two-dimensional front view of the overhead storage bin assembly 402 is superimposed on the corresponding overhead storage bin assembly 402. Next rectangles 602 are then placed next to the first rectangle 602, and so on, until each overhead storage bin assembly 402 is associated with its associated shelf shape 600. In this manner, each overhead storage bin assembly 402 corresponds to its associated shelf shape 600. In this case, each shape 600 is associated with the location of a particular overhead storage bin assembly 402. Thus, each shelf shape 600 can be labeled with a location on the overhead storage bin assembly 402.

[0052] 7 illustrates a perspective-corrected image 500 according to one embodiment of the present disclosure, with a first cluster 702a of shelf shapes 602a corresponding to a first group of overhead luggage bin assemblies 402a and a second cluster 702b of shelf shapes 602b corresponding to a second group of overhead luggage bin assemblies 402b. The first group of overhead luggage bin assemblies 402a and the second group of overhead luggage bin assemblies 402b may each include one or more overhead luggage bin assemblies 402a and 402b.

[0053] 1 and 7 , the state determination control unit 112 distinguishes between the first cluster 702a and the second cluster 702b through one or more differences in image attributes. As an example, the first cluster 702a has a first image attribute 703, such as a first color or color intensity, and the second cluster 702b has a second image attribute 705, such as a second color or color intensity that is different from the first color or color intensity. The state determination control unit 112 detects the difference between the first image attribute 703 and the second image attribute 705 and therefore determines that the first group of overhead bin assemblies 402a is in a different state from the second group of overhead bin assemblies 402b. In at least one embodiment, the state determination control unit 112 is programmed to associate the second image attribute 705 with an open state. In this manner, the state determination control unit 112 determines that the first group of overhead bin assemblies 402a is in a closed state and the second group of overhead bin assemblies 402b is in an open state. The state determination control unit 126 then outputs a state signal 126 indicating the state of the associated overhead bin assemblies 402a and 402b to the user device 116. Furthermore, the state determination control unit 126 can output an equipment control signal 128 to the actuator 114 of overhead bin assembly 402b to automatically close overhead bin assembly 402b.

[0054] 8 illustrates an image 800 of a portion of the interior cabin 104, according to one embodiment of the present disclosure. Referring to FIGS. 1 and 8, in at least one embodiment, to determine the region of interest, the state determination control unit 112 analyzes the image 800 to determine a number of edges 802 of various features within the image 800. The edges 802 extend toward a vanishing point of the image 800.

[0055] 9 shows a vanishing point 804 in an image 800 of a portion of the interior cabin 104 of FIG. 8 according to one embodiment of the present disclosure. Edges 802 point toward and intersect at the vanishing point 804. Referring to FIGS. 1 and 9, the state determination control unit 112 determines the vanishing point 804 at the intersection of the edges 802.

[0056] The state determination control unit 112 then determines a bisector 806 that passes through the vanishing point 804. The bisector 806 is parallel to a side edge 808 of the image 800 and perpendicular to a bottom edge 810 and a top edge 812 of the image 800. The bisector 806 divides the image 800 into a first half 814 and a second half 816, each of which is associated with a different region of interest. In this manner, the region of interest is determined by the bisector 806.

[0057] 10 illustrates a first region of interest 818 and a second region of interest 820 of the image 800 of the interior cabin 104 of FIG. 8 according to one embodiment of the present disclosure. Referring to FIGS. 1, 9, and 10, the state determination control unit 112 separates the first region of interest 818 from the second region of interest 820 via a bisector 806 passing through the vanishing point 804.

[0058] 11 illustrates an area of ​​interest 818 of an image 800 according to one embodiment of the present disclosure. Referring to FIGS. 1 and 11, the state determination control unit 112 determines the area of ​​interest 818 to analyze, for example, via calibration and / or user input. The area of ​​interest 818 includes a row 901 of equipment 102, such as overhead bin assemblies 402.

[0059] The state determination control unit 112 generates a first terminal line 902 at one end of the column 901. For example, the first terminal line 902 may be formed a predetermined distance from the vanishing point 804 (shown in FIG. 9 ). The state determination control unit 112 generates a second terminal line 904 on the opposite side of the first terminal line 902. The second terminal line 904 may be at the opposite end of the column 901. The state determination control unit 112 generates an upper boundary line 906 associated with the upper edge of the column 901 and a lower boundary line 908 associated with the lower edge of the column 901. The first terminal line 902 and the upper boundary line 906 intersect at a first corner 920. The first terminal line 902 and the lower boundary line 908 intersect at a second corner 922. The second terminal line 904 and the upper boundary line 906 intersect at a third corner 924. The second end line 904 and the lower boundary line 908 intersect at a fourth corner 926. The first corner 920, the second corner 922, the third corner 924, and the fourth corner 926 are corners of a quadrilateral, such as a trapezoid, that defines the overhead bin assembly 402 of the row 901.

[0060] 12 illustrates a perspective-corrected image 950 of the row 901 of overhead bin assemblies 402 within the first region of interest 818 of FIG. 11 in accordance with one embodiment of the present disclosure. Referring to FIGS. 1, 11, and 12, the state determination control unit 112 geometrically transforms the quadrilateral defined by corners 920, 922, 924, and 926 into a rectangle having corresponding corners 1020, 1022, 1024, and 1026 to form the perspective-corrected image 950. The perspective-corrected image 950 includes the row 901 of overhead bin assemblies 402, each of which has the same or nearly the same shape. In this case, the state determination control unit 112 extrapolates the bin shape and associated label for each overhead bin assembly 402, for example, as described with respect to FIG. 6.

[0061] The uniform size of each shelf shape of the associated overhead luggage bin assembly reduces the analysis complexity and therefore the computational power. In this manner, the state determination control unit 112 operates efficiently. In general, the uniform size of the shelf shapes allows for improved feature extraction for determining the shelf state.

[0062] Embodiments of the present disclosure provide systems and methods that allow a computing device to quickly and efficiently analyze large amounts of data. For example, the interior cabin of a vehicle may include numerous pieces of equipment that are movable between open and closed states and that may be overlooked by the crew and other individuals within the interior cabin. As such, a large amount of data is tracked and analyzed by the state determination control unit 112. The vast amount of data is efficiently organized and / or analyzed by the state determination control unit 112, as described above. The state determination control unit 112 analyzes the data in a relatively short amount of time to quickly and efficiently output information about which particular pieces of equipment are open or closed. A human would not be able to efficiently analyze such a large amount of data in such a short amount of time. Therefore, embodiments of the present disclosure provide improved efficiency and significantly superior performance relative to a human analyzing large amounts of data.

[0063] In at least one embodiment, components of the system 100, such as the state determination control unit 112, provide and / or enable a computing system that operates as a specialized computing system for detecting the state of equipment within the interior cabin.

[0064] 13 is a front perspective view of a vehicle 106, according to an exemplary embodiment of the present disclosure. In at least one embodiment, the vehicle 106 is an aircraft. The aircraft 106 includes a propulsion system 1112, which may include, for example, two engines 1114. Optionally, the propulsion system 1112 may include more engines 1114 than shown. The engines 1114 are carried by wings 1116 of the vehicle 106. In some embodiments, the engines 1114 may be carried by a fuselage 1118 and / or a tail section 1120. The tail section 1120 may also include a horizontal stabilizer 1122 and a vertical stabilizer 1124. The fuselage 1118 of the aircraft 106 defines an interior cabin, which may include a cockpit 1130, one or more work sections (e.g., a galley, a personal carry-on baggage area, etc.), and / or one or more passenger sections.

[0065] Optionally, instead of aircraft, embodiments of the present disclosure may be used for various other vehicles such as automobiles, buses, locomotives, carriages, watercraft, etc. Additionally, embodiments of the present disclosure may be used for fixed structures such as commercial buildings, residential buildings, etc.

[0066] 1-13 , embodiments of the present disclosure provide systems and methods for automatically detecting the status of equipment within the interior cabin of a vehicle. Furthermore, embodiments of the present disclosure provide systems and methods for automatically determining whether a luggage rack assembly within the interior cabin of a vehicle is open or closed. Additionally, the above systems and methods function via image analysis, eliminating the need for separate sensors and wiring. Thus, embodiments of the present disclosure provide systems and methods that reduce manufacturing complexity and costs, reduce the overall weight of the vehicle, and increase the fuel efficiency of the vehicle.

[0067] Furthermore, the present disclosure includes embodiments according to the following clauses:

[0068] Clause 1. A system comprising: one or more pieces of equipment within the interior cabin of the vehicle; an imaging device configured to capture images of one or more pieces of equipment; a state determination control unit including a processor and in communication with the imaging device; Equipped with a state determination control unit receiving image data including an image from an imaging device; The system, wherein the state determination control unit determines a state of one or more pieces of equipment based on the image data.

[0069] Clause 2. The system of clause 1, wherein the state includes one of an open state or a closed state, or both.

[0070] Clause 3. A system as described in clause 1 or 2, wherein the one or more pieces of equipment are one or more overhead bin assemblies.

[0071] Clause 4. A system according to any one of clauses 1 to 3, wherein the state determination control unit determines a region of interest within the image data.

[0072] Clause 5. A system according to any one of clauses 1 to 4, wherein the state determination control unit determines the region of interest by determining a vanishing point in the image.

[0073] Clause 6. A system according to any one of clauses 1 to 5, wherein the state determination control unit determines a bisector passing through the vanishing point, and the region of interest is determined by the bisector.

[0074] Clause 7. A system according to any one of clauses 1 to 6, wherein the state determination control unit corrects for perspective in the image data.

[0075] Article 8. One or more pieces of equipment may include a number of pieces of equipment arranged in a row, a state determination control unit generating a first terminal line at a first end of the row and a second terminal line at a second end of the row opposite the first terminal line; the state determination control unit further generates an upper boundary line associated with the upper edge of the column and a lower boundary line associated with the lower edge of the column; the first end line and the top boundary line intersect at a first corner; the first end line and the bottom boundary line intersect at a second corner; the second end line and the upper boundary line intersect at a third corner; The second end line and the bottom boundary line intersect at the fourth corner; 8. The system of any one of clauses 1 to 7, wherein the first corner, the second corner, the third corner, and the fourth corner are corners of a quadrilateral that defines a plurality of fixtures.

[0076] Clause 9. A system according to any one of clauses 1 to 8, wherein the state determination control unit geometrically transforms the quadrilaterals into rectangles with corresponding corners to form a perspective-corrected image.

[0077] Clause 10. A system as described in any one of clauses 1 to 9, wherein the state determination control unit extrapolates the shape of each of a plurality of pieces of equipment in the image data.

[0078] Clause 11. A system as described in any one of clauses 1 to 10, wherein the state determination control unit determines the state of one or more pieces of equipment through clustering.

[0079] Clause 12. A system as described in any one of clauses 1 to 11, wherein the state determination control unit determines the state of one or more pieces of equipment by determining differences in image attributes.

[0080] Clause 13. The user device further includes: 13. A system as described in any one of clauses 1 to 12, wherein the state determination control unit outputs a state signal to a user device indicating the state of one or more pieces of equipment.

[0081] Clause 14. A system according to any one of clauses 1 to 13, wherein the state determination control unit outputs an equipment control signal that automatically operates one or more pieces of equipment based on the state of the one or more pieces of equipment.

[0082] Clause 15. A method comprising: capturing, with an imaging device, an image of one or more items within an interior cabin of the vehicle; receiving, by a state determination control unit including a processor and in communication with the imaging device, image data including the image from the imaging device; determining, by a status determination control unit, a status of the one or more pieces of equipment based on the image data; A method comprising:

[0083] Clause 16. The method of clause 15, wherein the state includes one of an open state or a closed state, or both.

[0084] Clause 17. A method as set forth in clause 15 or 16, wherein the one or more pieces of equipment are one or more overhead bin assemblies.

[0085] Clause 18. The method of any one of clauses 15 to 17, wherein determining includes determining a region of interest within the image data.

[0086] Clause 19. The method of any one of clauses 15 to 18, wherein determining further comprises determining the region of interest by determining a vanishing point in the image.

[0087] Clause 20. The method of any one of clauses 15 to 19, wherein determining further comprises determining a bisector passing through the vanishing point and determining the region of interest by the bisector.

[0088] Clause 21. The method of any one of clauses 15 to 20, further comprising correcting for perspective in the image data by the state determination control unit.

[0089] Article 22. One or more pieces of equipment may include a number of pieces of equipment arranged in a row, To decide, generating a first terminal line at a first end of the row and a second terminal line at a second end of the row opposite the first terminal line; generating a top border associated with a top edge of the column and a bottom border associated with a bottom edge of the column; the first end line and the top boundary line intersect at a first corner; the first end line and the bottom boundary line intersect at a second corner; the second end line and the upper boundary line intersect at a third corner; The second end line and the bottom boundary line intersect at the fourth corner; generating an upper boundary line and a lower boundary line, wherein the first corner, the second corner, the third corner, and the fourth corner are corners of a quadrilateral that defines the plurality of fixtures; 22. The method of any one of clauses 15 to 21, comprising:

[0090] Clause 23. The method of any one of clauses 15 to 22, wherein determining further comprises geometrically transforming the quadrilateral into a rectangle with corresponding corners to form a perspective-corrected image.

[0091] Clause 24. The method of any one of clauses 15 to 23, wherein determining further comprises extrapolating the shape of each of the one or more pieces of equipment in the image data.

[0092] Clause 25. The method of any one of clauses 15 to 24, wherein determining further comprises determining the status of one or more pieces of equipment through clustering.

[0093] Clause 26. The method of any one of clauses 15 to 25, wherein determining includes determining differences in image attributes.

[0094] Clause 27. The method of any of clauses 15 to 26, further comprising outputting, by the status determination control unit, a status signal to the user device indicating the status of the one or more pieces of equipment.

[0095] Clause 28. A system according to any of clauses 15 to 27, further comprising outputting, by the state determination control unit, an equipment control signal for automatically operating one or more pieces of equipment based on the state of the one or more pieces of equipment.

[0096] Clause 29. A non-transitory computer-readable storage medium containing executable instructions, The instructions, upon execution, cause a system including a processor to: receiving image data from an imaging device within the interior cabin of the vehicle, the image data including an image of one or more items within the interior cabin; determining the status of one or more pieces of equipment based on the image data; A non-transitory computer-readable storage medium for causing a computer to perform operations including:

[0097] Clause 30. The non-transitory computer-readable storage medium of clause 29, wherein determining includes determining a region of interest within the image data.

[0098] Clause 31. The non-transitory computer-readable storage medium of clause 29 or 30, further comprising correcting perspective in the image data.

[0099] Clause 32. A non-transitory computer-readable storage medium according to any one of clauses 29 to 31, wherein determining further comprises extrapolating the shape of each of the one or more pieces of equipment within the image data.

[0100] Clause 33. The non-transitory computer-readable storage medium of any one of clauses 29 to 32, wherein determining further comprises determining the status of one or more pieces of equipment through clustering.

[0101] For purposes of describing the embodiments of the present disclosure, various spatial and directional terms may be used, such as top, bottom, lower, center, sideways, horizontal, vertical, front, etc., but it should be understood that such terms are used only with reference to the orientation shown in the drawings. The orientation may be flipped, rotated, or otherwise changed, such as top becoming bottom or vice versa, horizontal becoming vertical, etc.

[0102] As used herein, a structure, limitation, or element that is "configured to" perform a task or operation is specifically structurally formed, configured, or adapted in a manner corresponding to that task or operation. For clarity and avoidance of doubt, an object that is merely modifiable to perform a task or operation is not "configured to" perform that task or operation as that term is used herein.

[0103] It should be understood that the above description is intended to be illustrative, not limiting. For example, the above-described embodiments (and / or aspects thereof) can be used in combination with each other. Furthermore, many modifications can be made to the teachings of the various embodiments of the present disclosure to adapt to a particular situation or material without departing from its scope. While the dimensions and types of materials described herein are intended to define the parameters of the various embodiments of the present disclosure, they are by no means limiting and are exemplary. Other embodiments will be apparent to those skilled in the art upon review of the foregoing specification. Accordingly, the scope of the various embodiments of the present disclosure should be determined with reference to the appended claims, along with the full scope of equivalents to which such claims are entitled. In the appended claims and in this detailed description, the terms "including" and "in which" are used as express synonyms of the terms "comprising" and "wherein," respectively. Furthermore, the terms "first," "second," and "third," etc., are used merely as labels and are not intended to impose numerical requirements on their objects.

[0104] The written description uses examples to disclose various embodiments of the present disclosure, including the best mode, and to enable any person skilled in the art to practice various implementations of the present disclosure, including making and using any device or system, and performing any methods incorporated therein. The patentable scope of the various embodiments of the present disclosure is defined by the claims, and may include other examples that occur to those skilled in the art. Such other examples are intended to be within the scope of the claims if the examples have elements that do not differ from the literal language of the claims, or if the examples include equivalent elements that differ only insignificantly from the literal language of the claims.

Claims

1. A system (100), comprising: one or more pieces of equipment (102) within an interior cabin (104) of a vehicle (106); an imaging device (108) configured to capture images of the one or more pieces of equipment (102); a state determination control unit (112) including a processor (300) and in communication with the imaging device (108); Equipped with the state determination control unit (112) receives image data (122) including the image from the imaging device (108); the state determination control unit (112) determines a state of the one or more pieces of equipment (102) based on the image data (122); The state determination control unit (112) determines a region of interest within the image data (122); The state determination control unit (112) determines a bisector passing through a vanishing point; The system (100) wherein the region of interest is determined by the bisector.

2. A system (100), one or more pieces of equipment (102) within an interior cabin (104) of a vehicle (106); an imaging device (108) configured to capture images of the one or more pieces of equipment (102); a state determination control unit (112) including a processor (300) and in communication with the imaging device (108); Equipped with the state determination control unit (112) receives image data (122) including the image from the imaging device (108); the state determination control unit (112) determines a state of the one or more pieces of equipment (102) based on the image data (122); the one or more pieces of equipment (102) include a plurality of pieces of equipment (102) arranged in a row; the state determination control unit (112) generates a first terminal line at a first end of the row and a second terminal line at a second end of the row opposite the first terminal line; the state determination control unit (112) further generates an upper boundary line associated with an upper edge of the column and a lower boundary line associated with a lower edge of the column; the first end line and the upper boundary line intersect at a first corner; the first end line and the lower boundary line intersect at a second corner; the second end line and the upper boundary line intersect at a third corner; the second end line and the lower boundary line intersect at a fourth corner; The system (100), wherein the first corner, the second corner, the third corner, and the fourth corner are corners of a quadrilateral that defines the plurality of fixtures (102).

3. The system (100) of claim 1 or 2, wherein the states include one of an open state or a closed state, or both.

4. The system (100) of any one of claims 1 to 3, wherein the one or more pieces of equipment (102) are one or more overhead bin assemblies (402).

5. The system (100) of any one of claims 1 to 4, wherein the state determination control unit (112) corrects for perspective in the image data (122).

6. The system (100) of claim 2, wherein the state determination control unit (112) geometrically transforms the quadrilateral into a rectangle (602) having corresponding corners to form a perspective-corrected image.

7. The state determination control unit (112) extrapolating the shape of each of the one or more pieces of equipment (102) within the image data (122); determining the state of the one or more pieces of equipment (102) through clustering; or determining the state of the one or more pieces of equipment (102) by determining differences in image attributes; The system (100) of any one of claims 1 to 6, further comprising:

8. further comprising a user device (116); 8. The system (100) of claim 1, wherein the state determination control unit (112) outputs a state signal (126) to the user device (116) indicative of the state of the one or more pieces of equipment (102).

9. 9. The system (100) of claim 1, wherein the state determination control unit (112) outputs an equipment control signal to automatically operate the one or more equipment items (102) based on the state of the one or more equipment items (102).

10. 1. A method comprising: receiving, by a state determination control unit (112) including a processor (300) and in communication with the imaging device (108), image data (122) from the imaging device (108), the image data (122) including at least one image of one or more pieces of equipment (102) within the aircraft; determining, by the state determination control unit (112), a state of the one or more pieces of equipment (102) based on the image data (122); Including, the state includes one or both of an open state and a closed state, and the one or more pieces of furniture (102) are one or more overhead bin assemblies (402); The determining step comprises: determining a region of interest within the image data (122); determining the region of interest by determining a vanishing point within the image; determining a bisector passing through said vanishing point; determining the region of interest by the bisector; A method comprising:

11. A method comprising: receiving, by a state determination control unit (112) including a processor (300) and in communication with the imaging device (108), image data (122) from the imaging device (108), the image data (122) including at least one image of one or more pieces of equipment (102) within the aircraft; determining, by the state determination control unit (112), a state of the one or more pieces of equipment (102) based on the image data (122); Including, the state includes one or both of an open state and a closed state, and the one or more pieces of furniture (102) are one or more overhead bin assemblies (402); the one or more pieces of equipment (102) include a plurality of pieces of equipment (102) arranged in a row; The determining step comprises: generating a first terminal line at a first end of the row and a second terminal line at a second end of the row opposite the first terminal line; generating a top border associated with a top edge of the column and a bottom border associated with a bottom edge of the column; the first end line and the upper boundary line intersect at a first corner; the first end line and the lower boundary line intersect at a second corner; the second end line and the upper boundary line intersect at a third corner; the second end line and the lower boundary line intersect at a fourth corner; generating an upper boundary line and a lower boundary line, wherein the first corner, the second corner, the third corner, and the fourth corner are corners of a quadrilateral that defines the plurality of pieces of equipment (102); A method comprising:

12. 12. The method of claim 10 or 11, further comprising correcting perspective in the image data by the state determination control unit.

13. Geometrically transforming the quadrilaterals into rectangles (602) with corresponding corners to form a perspective-corrected image. The method of claim 11 , comprising:

14. outputting, by the state determination control unit (112), a state signal (126) indicative of the state of the one or more pieces of equipment (102) to a user device (116); outputting, by the state determination control unit (112), an equipment control signal for automatically operating the one or more equipment items (102) based on the state of the one or more equipment items (102); 14. The method of any one of claims 10 to 13, further comprising:

Images