Situation detection device, aircraft passenger compartment and method for monitoring aircraft passenger compartments

Abstract

Situation detection device (10), comprising: a monitoring processor (1) with an input interface (7) and an output interface (8), which is designed to receive visual and / or acoustic monitoring signals (E) from an aircraft passenger compartment (20) via the input interface (7); and an AI system (3) comprising an AI processor (4), a rule generator (5) based on self-learning algorithms and a reference rule memory (6), which is in bidirectional data communication with the monitoring processor (1), wherein the AI ​​processor (4) is designed to check, on request (Q) of the monitoring processor (1), data patterns in the received visual and / or acoustic monitoring signals (E) for deviations from data patterns in a reference rule set (R) stored in the reference rule set memory (6), and wherein the monitoring processor (1) is designed to output indicator signals (A) via the output interface (8) if the deviations determined by the AI ​​processor (4) exceed one or more predefinable deviation thresholds.

Description

TECHNICAL AREA OF INVENTION

[0001] The invention relates to a situation recognition device, an aircraft passenger compartment, and a method for the automated monitoring of processes and situations in aircraft passenger compartments. TECHNICAL BACKGROUND

[0002] In public transportation, such as passenger aircraft, onboard processes must be monitored to enable timely and appropriate countermeasures by the cabin crew in the event of deviations from standard travel or transport conditions. It is known that dynamically generated image or sound material inside the vehicle is evaluated by human review. Particularly in passenger aviation, it is desirable to be able to quickly identify and assess unusual travel or transport conditions. This typically requires monitoring by cabin crew, whose continuity and omnipresence cannot be constantly guaranteed given the multitude of spaces, situations, and processes to be monitored.

[0003] To provide human cabin crew with a preliminary assessment and initial evaluation of unusual travel or transport conditions, approaches exist for automating monitoring processes. This allows for the generation of machine-generated situation indicators, abstracted from the actual situation, which can then be subjected to further review by the cabin crew.

[0004] Documents US 7,868,912 B2, US 2008 / 0031,491 A1, and US 9,111,148 B2 disclose adaptively learning pattern recognition devices for video surveillance systems. Document CN 107,600,440 A discloses a video surveillance system for detecting non-compliant passenger behavior. Document DE 44 16 506 A1 discloses lower-deck passenger compartments for aircraft. SUMMARY OF THE INVENTION

[0005] The object of the invention is therefore to find improved solutions for the automated detection of situations or processes, such as in a passenger compartment of an aircraft, which enable human cabin crew of the aircraft to obtain information about the occurrence of potentially unusual situations or processes more quickly and efficiently.

[0006] This problem is solved by a situation recognition device with the features of claim 1, by an aircraft passenger compartment with the features of claim 7, and by a method for automated monitoring of processes and situations in aircraft passenger compartments with the features of claim 9.

[0007] According to a first aspect of the invention, a situation detection device comprises a monitoring processor with an input interface and an output interface, which is designed to receive visual and / or acoustic monitoring signals from an aircraft passenger compartment via the input interface. The situation detection device also includes an AI system comprising an AI processor, a rule generator based on self-learning algorithms, and a reference rule memory, which is in bidirectional data communication with the monitoring processor. The AI ​​processor is designed to check, upon request from the monitoring processor, data patterns in the received visual and / or acoustic monitoring signals for deviations from data patterns in a reference rule stored in the reference rule memory.The monitoring processor is designed to output indicator signals via the output interface if the deviations determined by the AI ​​processor exceed one or more predefined deviation thresholds.

[0008] According to a second aspect of the invention, an aircraft passenger compartment, in particular a lower deck passenger compartment (“lower deck passenger compartment”, LDC), comprises a situation detection device according to the first aspect of the invention, one or more surveillance cameras which are coupled to the input interface of the surveillance processor and which are designed to send visual and / or acoustic surveillance signals to the surveillance processor in real time, and a warning device which is coupled to the output interface of the surveillance processor and which is designed to display warnings to a user about a potential hazardous situation in the aircraft passenger compartment, depending on indicator signals received from the surveillance processor.

[0009] According to a third aspect of the invention, a method for the automated monitoring of processes and situations in aircraft passenger compartments comprises the steps of receiving visual and / or acoustic monitoring signals from an aircraft passenger compartment by a monitoring processor, checking the received visual and / or acoustic monitoring signals for deviations from a reference rule set stored in a reference rule set memory of an AI system and generated by a rule set generator based on self-learning algorithms by an AI processor of the AI ​​system, and outputting indicator signals by the monitoring processor if the deviations determined by the AI ​​processor exceed one or more predefinable deviation thresholds.

[0010] A key aspect of the invention is the use of algorithmic machine learning to monitor and evaluate observable information about situations and processes in an aircraft passenger compartment with regard to potential deviations from patterns and regularities in dynamically and continuously collected information data. This is achieved using an artificial intelligence (AI) system, which is initialized with appropriate training data on the patterns and regularities of situations and processes in the aircraft passenger compartment. Collected information data, such as video or audio recordings, is arbitrated by the AI ​​system in real time according to a reference set of rules.

[0011] If the AI ​​system concludes that situations and processes depicted in the information data deviate from the reference situations or processes specified in the reference rule set, and that these deviations exceed predefined threshold values, the AI ​​system can perform an automated assessment and transmit indicators regarding a suspected hazard situation to the body responsible for the safety of the aircraft passenger compartment. Based on such indicators, action can be taken either automatically by downstream systems or individually after assessment by human users.

[0012] A particular advantage of the solutions according to the invention is that hazardous situations can be recognized more easily, reliably, and quickly without the need for permanent storage of information data. Storing information data without cause could infringe on the personal rights of observed individuals due to the unpredictability of the observed situations and processes – this is advantageously avoided by the solutions according to the invention, since only indicators abstracted from the actually recorded information material regarding the presumed presence of hazardous situations need to be stored.

[0013] This advantageously allows for a compromise between protecting passenger privacy and the monitoring measures necessary to maintain flight safety. With the solutions according to the invention, various potentially problematic situations can be quickly distinguished from expected or unproblematic situations in aircraft passenger compartments. For example, persons or objects in areas that must be kept clear, objects blocking emergency exits, objects that endanger flight safety due to their nature, persons requiring assistance, persons acting improperly, or persons acting with criminal intent can be detected and automatically displayed to the cabin crew so that countermeasures can be initiated.

[0014] Advantageous designs and further developments result from the additional sub-claims as well as from the description with reference to the figures.

[0015] According to some embodiments of the situation recognition device, the rule generator can be designed to detect patterns and regularities in temporally and spatially resolved observation data about situations and processes in an aircraft passenger compartment, based on observation data received via the input interface of the monitoring processor, and to store the detected patterns and regularities in the observation data as a reference rule set in the reference rule set memory. This allows the reference rule set to be advantageously adapted to the specific environment of the aircraft passenger compartment by training the AI ​​system.

[0016] According to some further embodiments of the situation detection device, the device can include an indicator data store coupled to the monitoring processor and designed to store a multitude of indicator signal specifications. The monitoring processor is designed to retrieve one of these specifications from the indicator data store, depending on the type of deviations detected by the AI ​​processor, and output it as an indicator signal at the output interface. This allows the output data of the situation detection device to be abstracted from the detected visual and / or acoustic monitoring signals, thus avoiding the need to output sensitive data that could potentially infringe on the privacy rights of persons in the aircraft passenger compartment. Instead, a hazardous situation can be assessed based on the indicator signals.

[0017] According to some further embodiments of the situation recognition device, the rule generator can include a support vector classifier, a neural network, a random forest classifier, a decision tree classifier, a Monte Carlo network, or a Bayesian classifier.

[0018] According to some further embodiments of the situation detection device, the visual and / or acoustic monitoring signals can include real-time video recordings from one or more surveillance cameras.

[0019] According to some further embodiments of the situation recognition device, the monitoring processor can be designed to delete the visual and / or acoustic monitoring signals after review by the AI ​​processor. This advantageously enables the monitoring of an aircraft passenger compartment without the unnecessarily storing passenger data longer than required. In particular, the recorded visual and / or acoustic monitoring signals are used solely for automated situation recognition by the AI ​​system and cannot be disclosed to unauthorized third parties.

[0020] According to some embodiments of the method, the rule generator may include a support vector classifier, a neural network, a random forest classifier, a decision tree classifier, a Monte Carlo network, or a Bayesian classifier. In some embodiments of the method, the monitoring processor may output indicator signals depending on the current flight phase of an aircraft containing the passenger compartment.

[0021] According to some aircraft passenger compartment designs, the passenger compartment may also include a warning signal interface through which the warning system can transmit warning signals to the aircraft cockpit and / or a flight attendant console. This advantageously allows flight attendants or other crew members, even those working at more distant locations within the aircraft, to be informed of potential hazardous situations in the passenger compartment. Particularly for lower-deck passenger compartments, which are not easily accessible to cabin crew without considerable travel distances, AI-based automated monitoring can be a helpful tool for monitoring flight safety.

[0022] The above embodiments and further developments can be combined with one another as appropriate. Further possible embodiments, further developments, and implementations of the invention also include combinations of features of the invention described previously or subsequently with regard to the exemplary embodiments, even if not explicitly mentioned. In particular, the person skilled in the art will also add individual aspects as improvements or additions to the respective basic form of the present invention. BRIEF SUMMARY OF THE CHARACTERS

[0023] The present invention will be explained in more detail below with reference to the exemplary embodiments shown in the schematic figures. These figures show: Fig. 1 a schematic block diagram of a situation detection device in an aircraft passenger compartment according to an embodiment of the invention; Fig. 2 a flowchart of a method for monitoring traffic movements according to a further embodiment of the invention; and Fig. 3 A schematic illustration of an aircraft with a passenger compartment and a situation recognition device according to a further embodiment of the invention.

[0024] The accompanying figures are intended to provide a further understanding of the embodiments of the invention. They illustrate embodiments and, in conjunction with the description, serve to explain the principles and concepts of the invention. Other embodiments and many of the aforementioned advantages become apparent with reference to the drawings. The elements of the drawings are not necessarily shown to scale. Directional terminology such as "above," "below," "left," "right," "over," "below," "horizontal," "vertical," "front," "back," and similar terms are used for explanatory purposes only and are not intended to limit the general public to specific embodiments as shown in the figures.

[0025] In the figures of the drawing, identical, functionally equivalent and similarly acting elements, features and components - unless otherwise stated - are each provided with the same reference symbols. DESCRIPTION OF EXAMPLES OF EXECUTION

[0026] The following description refers to self-learning algorithms used in artificial intelligence (AI) systems. Generally speaking, a self-learning algorithm replicates cognitive functions that, according to human judgment, are attributed to human thinking. By incorporating new training information, the self-learning algorithm can dynamically adapt the insights gained from previous training data to changing circumstances, in order to recognize and extrapolate patterns and regularities within the entirety of the training information.

[0027] In self-learning algorithms according to the present invention, all types of training that contribute to human knowledge acquisition can be used, such as supervised learning, semi-supervised learning, independent learning based on generative, non-generative, or deep adversarial networks (AN), reinforcement learning, or active learning. Feature-based learning (representation learning) can be employed in each instance. In particular, the self-learning algorithms according to the present invention can perform iterative adjustments of the parameters and features to be learned via feedback analysis.

[0028] A self-learning algorithm according to the present invention can be based on a support vector network (SVN), a neural network such as a convolutional neural network (CNN), a Kohonen network, a recurrent neural network, a time-delayed neural network (TDNN), or an oscillatory neural network (ONN), a random forest classifier, a decision tree classifier, a Monte Carlo network, or a Bayesian classifier. A self-learning algorithm according to the present invention can employ property-hereditary algorithms, k-means algorithms such as Lloyd's or MacQueen's algorithms, or TD learning algorithms such as SARSA or Q-Learning.

[0029] Aircraft passenger compartments within the meaning of the present invention can, in particular, comprise all modularly constructed cabin structures designed for the transport and accommodation of passengers during an air journey. Such aircraft passenger compartments can, for example, be designed as lower-deck passenger compartments. Examples of such lower-deck passenger compartments, which do not limit the invention, are disclosed on the internet at https: / / www.safran-cabin.com / printpdf / media / airbus-and-zodiacaerospace-enter-partnership-new-lower-deck-sleeping-facilities-20180410.

[0030] Fig. Figure 1 shows an exemplary illustration of a situation detection device 10. The situation detection device 10 has a monitoring processor 1 with an input interface 7 and an output interface 8. Visual and / or acoustic monitoring signals E from an aircraft passenger compartment 20 can be received at the input interface 7, for example, from monitoring devices such as surveillance cameras 21 installed at strategic monitoring points in the aircraft passenger compartment 20. Conversely, temporally and spatially resolved observation data about situations and processes in an aircraft passenger compartment 20 can be sent to the monitoring processor 1 via the input interface 7, from which patterns and regularities can be derived.These temporally and spatially resolved observation data can, for example, also be data recorded by surveillance cameras in the actual operation of the aircraft passenger compartment 20, or data sent to the monitoring processor 1 as computer-generated training data from connected other systems.

[0031] The situation recognition device 10 also includes an AI system 3. The AI ​​system 3 comprises an AI processor 4, a rule generator 5 based on self-learning algorithms, and a reference rule memory 6. The AI ​​system 3 communicates bidirectionally with the monitoring processor 1 via the AI ​​processor 4. The monitoring processor 1 can initially provide the AI ​​system 3 with a large number of historical and / or current observational data about situations and processes in the aircraft passenger compartment 20. This observational data can serve as the basis for the detection of patterns and regularities regarding possible situations and processes by the rule generator 5. The rule generator 5 can, for example, include a support vector classifier, a neural network, a random forest classifier, a decision tree classifier, a Monte Carlo network, or a Bayesian classifier.

[0032] The detected patterns and regularities in the situations and processes are initially stored iteratively in a training rule set T, which is dynamically and continuously updated. An operational reference rule set R is generated from the training rule set T, which the rule set generator 5 stores in the reference rule set memory 6. When the AI ​​processor 4 receives a request Q from the monitoring processor 1 to check data patterns in received visual and / or acoustic monitoring signals E for deviations from expected and non-critical data patterns, the AI ​​processor 4 accesses the reference rule set R stored in the reference rule set memory 6. Against this reference, the AI ​​processor 4 checks whether deviations from expected or normal situations or processes have occurred within the monitored area of ​​the aircraft passenger component 20.The rule generator 5 can update the reference rule set R stored in the reference rule set memory 6 at periodic intervals based on newly added observation data or on the basis of new external specifications.

[0033] The results of the deviation analysis are transmitted back to the monitoring processor 1, which is then designed to output indicator signals A via the output interface 8 if the deviations determined by the AI ​​processor 4 exceed one or more predefinable deviation thresholds.

[0034] An indicator data memory 2 can be connected to the monitoring processor 1. This indicator data memory 2 serves to temporarily or permanently store a large number of indicator signal specifications L, which the monitoring processor 1 can access in order to retrieve one of the large number of indicator signal specifications L from the indicator data memory 2, depending on the type of deviations determined by the AI ​​processor 4, and output it as an indicator signal A at the output interface 8.

[0035] A warning device 22 is connected to output interface 8. This warning device 22 can be used to display warnings to a user about a potential hazardous situation in the aircraft passenger compartment 20, depending on indicator signals A received from the monitoring processor 1, without having to display actual image or sound recordings from the aircraft passenger compartment 20. This can help to protect the privacy of aircraft passengers in the monitored compartment 20.

[0036] As in Fig. As illustrated by example in Figure 3, the situation detection device 10 can be installed in a passenger compartment 20 of a passenger aircraft A. For this purpose, one or more surveillance cameras 21 in the passenger compartment 20 can be connected to the input interface 7 of the monitoring processor 1 to send visual and / or acoustic monitoring signals E to the monitoring processor 1 in real time. The passenger compartment 20 can also have a warning signal interface 24, via which the warning device 22 can output warning signals C to the cockpit of the aircraft A and / or to a flight attendant console 30.

[0037] Advantageously, all components of the situation detection system 10 and the warning system 22 can be installed in the aircraft passenger compartment 20. However, it may also be possible to install individual components or system parts outside the aircraft passenger compartment 20.

[0038] Fig. Figure 2 shows a method M for the automated monitoring of processes and situations in aircraft passenger compartments. The method M can be implemented, for example, in a situation recognition device 10, as exemplified in the Fig. 1 shown, implemented and can be used for monitoring aircraft passenger compartments 20 as in Fig. 1. can be used as an example, illustrated, for instance in a passenger aircraft A, as in Fig. 3 are illustrated and explained using examples.

[0039] The procedure M comprises, as a first step M1, the reception of visual and / or acoustic monitoring signals E from an aircraft passenger compartment 20 by a monitoring processor 1. The situation recognition device 10 can, for example, be part of an electronic data processing system in which the observation data on traffic movements are recorded, processed, and only temporarily stored. In a second step M2, the received visual and / or acoustic monitoring signals E are checked for deviations from a reference rule set R stored in a reference rule set memory 6 of an AI system 3 and generated by a rule set generator 5 based on self-learning algorithms by an AI processor 4 of the AI ​​system 3. This is done by means of a rule set generator 5 of an AI system 3 based on self-learning algorithms.Such a rule generator 5 can, for example, include a support vector classifier, a neural network, a random forest classifier, a decision tree classifier, a Monte Carlo network, or a Bayesian classifier.

[0040] In iterative and dynamically adaptive learning processes, a training rule set T can be generated in the rule set generator 5. This training rule set T can be generated once or continuously updated, particularly during the ongoing reception of current observational data about situations and processes in the aircraft passenger compartment 20. The rule set generator 5 can generate a reference rule set R from the training rule set T, which is based on the patterns and regularities detected by the rule set generator 5. The reference rule set R is stored by the rule set generator 5 in the reference rule set memory 6 of the AI ​​system 3. This reference rule set R represents the operational rule set, which is used to check data patterns in received visual and / or acoustic monitoring signals E for deviations from expected data patterns classified as non-critical.For this purpose, the AI ​​processor 4 accesses the reference rule set R stored in the reference rule set memory 6. Against this reference, the AI ​​processor 4 checks whether deviations from expected or normal situations or processes have occurred within the monitored area of ​​the aircraft passenger component 20.

[0041] In a third step M3 of the procedure M, indicator signals A are finally output by the monitoring processor 1 if the deviations determined by the AI ​​processor 4 exceed one or more predefined deviation thresholds. These indicator signals A serve to alert human users to the presence of situations or processes in the aircraft passenger component 20 that are considered unusual. The situations or processes classified as unusual can then be automatically or semi-automatically fed into a system for initiating appropriate response measures in order to react early to the occurrence of potential flight safety hazards. In particular, it may be possible to output the indicator signals A depending on the flight phase.For example, during takeoff or landing, it may be necessary to keep certain areas, such as emergency exits, clear of objects like bags or suitcases that may be temporarily stored there during flight or while the aircraft is on the ground. Therefore, the monitoring processor 1 can be connected to an aircraft network to receive operational status signals from the aircraft, which can then be used to inform the monitoring processor 1's decision on whether and which indicator signals A should be output.

[0042] In the preceding detailed description, various features have been summarized in one or more examples to improve the clarity of the presentation. However, it should be clear that the above description is merely illustrative and in no way limiting. It serves to cover all alternatives, modifications, and equivalents of the various features and embodiments. Many other examples will be immediately and directly clear to the person skilled in the art based on their technical knowledge, given the above description.

[0043] The exemplary embodiments were selected and described to best illustrate the principles underlying the invention and its practical applications. This enables those skilled in the art to optimally modify and utilize the invention and its various exemplary embodiments with regard to the intended purpose. In the claims and the description, the terms "including" and "comprising" are used as neutral language terms for the corresponding terms "comprehensive." Furthermore, the use of the terms "a," "a," and "an" is not intended to fundamentally exclude multiple features and components described in this way.

Claims

[1] Situation detection device (10), comprising: a monitoring processor (1) with an input interface (7) and an output interface (8), which is designed to receive visual and / or acoustic monitoring signals (E) from an aircraft passenger compartment (20) via the input interface (7); and an AI system (3) comprising an AI processor (4), a rule generator (5) based on self-learning algorithms and a reference rule memory (6), which is in bidirectional data communication with the monitoring processor (1), wherein the AI ​​processor (4) is designed to check, on request (Q) of the monitoring processor (1), data patterns in the received visual and / or acoustic monitoring signals (E) for deviations from data patterns in a reference rule set (R) stored in the reference rule set memory (6), and wherein the monitoring processor (1) is designed to output indicator signals (A) via the output interface (8) if the deviations determined by the AI ​​processor (4) exceed one or more predefinable deviation thresholds. [2] Situation recognition device (10) according to claim 1, wherein the rule generator (5) of the AI ​​system (3) is designed to detect patterns and regularities in the observation data on the basis of temporally and spatially resolved observation data about situations and processes in an aircraft passenger compartment (20) received via the input interface (7) of the monitoring processor (1), and to store the detected patterns and regularities in the observation data as a reference rule set (R) in the reference rule set memory (6). [3] Situation recognition device (10) according to one of claims 1 and 2, further comprising: an indicator data storage device (2) which is coupled to the monitoring processor (1) and which is designed to store a multitude of indicator signal specifications (L), wherein the monitoring processor (1) is designed to retrieve one of the multitude of indicator signal specifications (L) from the indicator data storage device (2) depending on the type of deviations determined by the AI ​​processor (4) and to output it as an indicator signal (A) at the output interface (8). [4] Situation recognition device (10) according to one of claims 1 to 3, wherein the rule generator (5) comprises a support vector classifier, a neural network, a random forest classifier, a decision tree classifier, a Monte Carlo network or a Bayesian classifier. [5] Situation recognition device (10) according to one of claims 1 to 4, wherein the visual and / or acoustic monitoring signals (E) comprise real-time video recordings from one or more surveillance cameras (21). [6] Situation recognition device (10) according to any one of claims 1 to 5, wherein the monitoring processor (1) is designed to delete the visual and / or acoustic monitoring signals (E) after verification by the AI ​​processor (4). [7] Aircraft passenger compartment (20), comprising: a situation recognition device (10) according to one of claims 1 to 6; one or more surveillance cameras (21) which are coupled to the input interface (7) of the surveillance processor (1), and which are designed to send visual and / or acoustic monitoring signals (E) in real time to the monitoring processor (1); and a warning device (22) which is coupled to the output interface (8) of the monitoring processor (1) and which is designed to to display warnings to a user about a potential hazardous situation in the aircraft passenger compartment (20) depending on indicator signals (A) received from the monitoring processor (1). [8] Aircraft passenger compartment (20) according to claim 7, comprising a warning signal interface (24) through which the warning device (22) can output warning signals (C) to the cockpit of an aircraft (A) and / or to a flight attendant console (30). [9] Method (M) for automated monitoring of processes and situations in aircraft passenger compartments (20), comprising: Receiving (M1) visual and / or acoustic monitoring signals (E) from an aircraft passenger compartment (20) by a monitoring processor (1); Checking (M2) the received visual and / or acoustic monitoring signals (E) for deviations from a reference rule set (R) stored in a reference rule set memory (6) of an AI system (3) and generated by a rule set generator (5) based on self-learning algorithms by an AI processor (4) of the AI ​​system (3); and Output (M3) of indicator signals (A) by the monitoring processor (1) if the deviations determined by the AI ​​processor (4) exceed one or more predefinable deviation thresholds. [10] Method (M) according to claim 9, wherein the rule generator (5) comprises a support vector classifier, a neural network, a random forest classifier, a decision tree classifier, a Monte Carlo network or a Bayesian classifier. [11] Method (M) according to claim 9 or 10, wherein the output (M3) of indicator signals (A) by the monitoring processor (1) is performed depending on a flight phase of an aircraft (A) having the aircraft passenger compartment (20).

Images